In this talk, we briefly introduce Alluxio, present several ways how Alluxio can help Spark be more effective, show benchmark results with Spark RDDs and DataFrames, and describe production deployments both Alluxio and Spark working together.
Tag: apache spark
Alluxio provides Spark with a reliable data sharing layer, enabling Spark to excel at performing application logic while Alluxio handles storage.
Tachyon presents two talks at Strata + Hadoop World Singapore: Interactive data analytics with Spark on Tachyon in Baidu, and Make Tachyon ready for next-gen data center platforms with NVM
Tachyon: A reliable memory-centric distributed storage system presentation by founder Haoyuan Li.
We introduce Tachyon, a memory centric fault-tolerant distributed file system, which enables reliable file sharing at memory-speed across cluster frameworks, such as Spark and MapReduce.
Shaoshan Liu (Baidu) presents how Tachyon can help improve big data analytics (ad-hoc query) efficiency within Baidu.
Spark Summit 2014 – Haoyuan Li introduces Tachyon, a distributed in-memory storage system. Along with how Tachyon can further improve Spark’s performance and the integration between the two systems.