How T3Go’s high-performance data lake using Apache Hudi and Alluxio shortened the time for data ingestion into the lake by up to a factor of 2. Data analysts using Presto, Hudi, and Alluxio in conjunction to query data on the lake saw queries speed up by 10 times faster.
welcome to the alluxio community
Founder Haoyuan Li was a Ph.D. student at UC Berkeley AMPLab when he built the beginnings of Alluxio (originally called Tachyon) with the mission to orchestrate data for all data driven applications. Today, in addition to running critical workloads for thousands of users across the world, Alluxio has a vibrant community that has made countless contributions to the open source project.
Welcome to Alluxio Community! We would invite you to read about, try out, use, and contribute to Alluxio, as well as to share your experience, feedback, suggestions and ideas!
Alluxio Community Day IV
Online (virtual) event held June 24, 2021
Hear Alluxio users from Facebook, TikTok, Intel, Tencent and more share their experiences and insights on speeding up large-scale data access.
Slack
Ask questions, get answers.
GitHub
Become a contributor.
MAILING LIST
Join the Google group.
Join our channel.
When applications are only reading and writing through Alluxio, the Alluxio file system provides strong consistency. However, when clients are writing data across both Alluxio and under storage, the consistency depends on the Alluxio write type and under storage type. This article discusses what to expect in each scenario.
This blog explores an innovative platform with Presto as the computing engine and Alluxio as a data orchestration layer between Presto and S3 storage, to support online services with instantaneous response within the gaming industry. The preliminary results show that Presto with Alluxio outperforms S3 significantly in all cases.Alluxio with metadata caching shows up to 5.9x performance gain when handling large numbers of small files.
This article described how engineers at datasapiens brought down S3 API costs by 200x by implementing Alluxio as a data orchestration layer between S3 and Presto.
As the third largest e-commerce site in China, Vipshop processes large amounts of data collected daily to generate targeted advertisements for its consumers. In this article, Gang Deng from Vipshop describes how to meet SLAs by improving struggling Spark jobs on HDFS by up to 30x, and optimize hot data access with Alluxio to create … Continued
In this blog, Derek Tan, Executive Director of Infra & Simulation at WeRide, describes how engineers leverage Alluxio as a hybrid cloud data gateway for applications on-premises to access public cloud storage like AWS S3.
Join an Alluxio community event Near You
Don’t see an event in your area? Want to start a local Alluxio meetup? Drop us a note!
ACADEMIC PAPERS
Tachyon: Reliable, Memory Speed Storage for Cluster Computing Frameworks
ACM Symposium on Cloud Computing 2014
Alluxio: A Virtual Distributed File System
Berkeley EECS Ph.D. Dissertation
Tachyon: Reliable, Memory Speed Storage for Cluster Computing Frameworks
ACM Digital Library
join the community
4,000+
Stars
1,000+
Contributors
1 Million+
Downloads
apache 2.0 licensed
Alluxio Contributors
The Alluxio Open Source Contributors and Project Management Committee members come from a diverse and experienced background. The project members includes committers with decades of experience from Tencent, Google, Palantir, UC Berkeley, Carnegie Mellon, IBM, Intel and JD.com.
