Memory is the key to fast big data processing. This has been realized by many, and frameworks such as Spark and Shark already leverage memory performance. As data sets continue to grow, storage is increasingly becoming a critical bottleneck in many workloads. To address this need, we have developed Tachyon, a memory-centric fault-tolerant distributed storage … Continued
We introduce Tachyon, a memory centric fault-tolerant distributed file system, which enables reliable file sharing at memory-speed across cluster frameworks, such as Spark and MapReduce.
Tags: apache spark, big data, data, hadoop, mapreduce, performance, spark, storage
Shaoshan Liu (Baidu) presents how Tachyon can help improve big data analytics (ad-hoc query) efficiency within Baidu.
Tags: analytics, apache spark, big data, data, meetup, spark, storage
AMP Camps are Big Data training events organized by the UC Berkeley AMPLab about big data analytics, machine learning, and popular open-source software projects produced by the AMPLab.
Tags: alluxio engineering, analytics, big data, cloud, compute, machine learning