Calvin Jia and Jiri Simsa explain how the current Alluxio tiered storage can be easily configured to use memory, SSDs, and hard drives in different tiers.
Tags: aws s3, big data, caching, conference, data, hdfs, storage, tiered storage, unified namespace
DataDriven NYC 2016 – In the past year, the Alluxio project experienced a tremendous improvement in performance and scalability and was extended with key new features including tiered storage, transparent naming, and unified namespace. At the same time, the Alluxio ecosystem has expanded to include support for more under storage systems and computation frameworks.
Tags: alluxio engineering, big data, compute, compute storage separation, data, data engineering, distributed systems, performance, scale, storage, tiered storage
Strata+Hadoop World 2016 – Baidu deployed Alluxio to accelerate its big data analytics workload. Bin Fan and Haojun Wang explain why Baidu chose Alluxio, as well as the details of how they achieved a 30x speedup with Alluxio in their production environment with hundreds of machines. Based on the success of the big data analytics engine, Baidu is currently expanding the Alluxio and Spark infrastructure to accelerate other applications, such as machine learning.
Tags: analytics, big data, compute, data, data engineering, data orchestration, machine learning, performance, storage
Strata+Hadoop World 2016 – Tachyon, a memory-centric fault-tolerant distributed storage system. An introduction of architecture, performance evaluation, and real world use cases.
Tags: architecture, big data, data, distributed systems, performance, rdd, spark, storage
Tachyon presents two talks at Strata + Hadoop World Singapore: Interactive data analytics with Spark on Tachyon in Baidu, and Make Tachyon ready for next-gen data center platforms with NVM
Tags: analytics, apache spark, big data, caching, compute, conference, data, spark, storage, strata
ODSC West 2015 – Tachyon, a memory-centric fault-tolerant distributed storage system. An introduction of architecture, performance evaluation, and real world use cases.
Tags: architecture, big data, data, distributed systems, performance, rdd, spark, storage
Tachyon: A reliable memory-centric distributed storage system presentation by founder Haoyuan Li.
Tags: apache spark, big data, data, hadoop, mapreduce, performance, spark, storage
We introduce Tachyon, a memory centric fault-tolerant distributed file system, which enables reliable file sharing at memory-speed across cluster frameworks, such as Spark and MapReduce.
Tags: apache spark, big data, data, hadoop, mapreduce, performance, spark, storage
Shaoshan Liu (Baidu) presents how Tachyon can help improve big data analytics (ad-hoc query) efficiency within Baidu.
Tags: analytics, apache spark, big data, data, meetup, spark, storage