Alluxio is an open source software solution that connects analytics applications to heterogeneous data sources through a data orchestration layer that sits between compute and storage.
Tags: alluxio engineering, big data, compute storage separation, data, data engineering, data orchestration, overview, storage, unified namespace
Strata NY 2018 – Learn how to use Alluxio as a pluggable optimization component. Understand how JD.com uses Alluxio to provide support for ad hoc and real-time stream computing while ensuring consistency between Alluxio and HDFS.
Tags: apache hadoop, benchmark, case study, compute storage separation, hdfs, presto
Learn more about use cases with Alluxio leveraged in MOMO, JD.com, and TalkingData.
Tags: alluxio engineering, analytics, caching, cloud object storage, cloud storage, compute, compute storage separation, data, on-prem object storage, performance, storage
TalkingData’s largest data broker, provides data intelligence solutions and processes over 20 terabytes of data and more than one billion session requests per day. TalkingData deployed Alluxio to unify disparate cloud, on-premise, and hybrid data sources for a range of analytics applications. The architecture provides self-service data access for data scientists and engineers, eliminating the need for ETL or manual IT assistance.
Tags: analytics, architecture, case study, compute storage separation, hybrid cloud
The world is entering the data revolution era. Along with the latest advancements of the Internet, Artificial Intelligence (AI), mobile devices, autonomous driving, and Internet of Things (IoT), the amount of data we are generating, collecting, storing, managing, and analyzing is growing exponentially. To store and process these data has exposed tremendous challenges and opportunities. … Continued
Tags: architecture, beginner, compute storage separation, overview
The primary appeal of a coupled compute-storage architecture, an architecture where the computation is happening on the machines where the data resides, is the performance possible by bringing the compute engine to the data it requires; however, the costs of maintaining such tight-knit architectures are gradually overtaking the performance benefits. Especially with the popularity of cloud resources, being able to independently scale compute and storage results in large cost savings and cheaper maintenance. In addition, data has become the new oil, and all modern organizations are looking to capture as much data as possible.
Strata Singapore 2017 – Hear about how to decouple compute and storage with Alluxio, exploring the decision factors and considerations, along with production best practices and solutions.
Tags: alluxio engineering, compute storage separation, locality, performance
In this talk, we briefly introduce Alluxio, present several ways how Alluxio can help Spark be more effective, show benchmark results with Spark RDDs & DataFrames, and describe production deployments with both Alluxio and Spark working together.
Tags: alluxio engineering, apache spark, architecture, big data, cloud, compute storage separation, conference, data, performance, rdd, spark, storage
Strata+Hadoop 2016 – In the past year, the Alluxio project experienced a tremendous improvement in performance and scalability and was extended with key new features including tiered storage, transparent naming, and unified namespace. At the same time, the Alluxio ecosystem has expanded to include support for more under storage systems and computation frameworks.
Tags: alluxio engineering, architecture, big data, cloud, compute storage separation, conference, hadoop, performance, scale, storage, strata, tiered storage, unified namespace