Check out our new blog post: “Internet of Things: Are We There Yet? (The 2016 IoT Landscape)”: The Internet of Things is all about data!
Alluxio meetups, conferences, events and more
The latest Alluxio meetups, webinars, conferences and more
Calvin Jia and Jiri Simsa explain how the current Alluxio tiered storage can be easily configured to use memory, SSDs, and hard drives in different tiers. Alluxio users and administrators do not have to manually migrate the data because data in Alluxio is managed transparently between all the configured tiers, similar to the way the CPU manages L1, L2, and lower-level caches. Meanwhile, Alluxio also provides users fine-grained control of manipulating data to plug in their own data-management strategies; users can also pin files in Alluxio to a specific storage or specify a TTL to files. Calvin and Jiri also describe the interface for managing heterogeneous data sources into the Alluxio namespace, which takes advantage of Alluxio’s ability to interoperate with different underlying storage systems such as HDFS, S3, GlusterFS, or Swift.
A few months ago, Baidu deployed Alluxio to accelerate its big data analytics workload. Bin Fan and Haojun Wang explain why Baidu chose Alluxio, as well as the details of how they achieved a 30x speedup with Alluxio in their production environment with hundreds of machines. Based on the success of the big data analytics engine, Baidu is currently expanding the Alluxio and Spark infrastructure to accelerate other applications, such as machine learning.
Tachyon is a memory-centric fault-tolerant distributed storage system, which enables reliable file sharing at memory-speed. It originated from AMPLab, UC Berkeley in 2012, the same lab produced Apache Mesos and Apache Spark. Soon later, it became an open source project and is deployed at many companies. Since then, Tachyon has attracted more than 200 contributors from over 50 institutions. In 2015, company Tachyon Nexus was founded to further accelerate the development of Tachyon. In this talk, we will review Tachyon’s new features, deployments, and developments in 2015, and look into 2016.
In this talk we will focus on how Tachyon can help improve big data analytics (ad-hoc query) efficiency within Baidu.
During the past several years, Spark has significantly changed the landscape of big data computing. It improves performance of various applications dramatically. However, in certain Spark use cases, the bottleneck is in the I/O stack. In this talk, we will introduce Tachyon, a distributed memory-centric storage system. In addition, we will talk about several production use cases where Tachyon further improves Spark applications’ performance by orders of magnitude.
In the presentation, we will explore several potential industry use cases enabled by the new features. One-click cluster deployment enables users to experiment and prototype with Tachyon on AWS, launching not only Tachyon but also the computation framework and storage system of their choice. Mounting of multiple under storage systems and transparent naming enables more exciting use cases for Tachyon users.
we introduce Tachyon, a memory centric fault-tolerant distributed file system, which enables reliable file sharing at memory-speed across cluster frameworks, such as Spark and MapReduce.