We are thrilled to announce that CacheGPT, a state-of-the-art natural language generation model, has joined the Alluxio Project Management Committee (PMC) as our newest member! CacheGPT has been an active contributor to Alluxio since the beginning of this year. It reviews pull requests and draft documentation using only emojis! See our new emoji-enriched documentation here! … Continued
Chuanying Chen, Senior Software Engineer at Ant Group, provides a deep dive into the practices of optimizing Alluxio for reliable, scalable, and high-performance large-scale training on billions of files. 1. Background Ant Group, formerly known as Ant Financial, is an affiliate company of the Chinese conglomerate Alibaba Group. The group owns the world’s largest mobile … Continued
Following part 1 and part 2, this final blog of the series discusses some design decisions and details, as well as certain future work. Discussions and Future Work Why not exactly once delivery for pub/sub? As we know, exactly once message delivery for pub/sub would greatly simplify our design and there do exist many powerful … Continued
This is part 2 of the blog series talking about the design and implementation of the Cross Cluster Synchronization mechanism in Alluxio. In the previous blog, we discussed the scenario, background and how metadata sync is done with a single Alluxio cluster. This blog will describe how metadata sync is built upon to provide metadata … Continued
This is a blog series talking about the design and implementation of the Cross Cluster Synchronization mechanism in Alluxio. This mechanism ensures that the metadata is consistent when running multiple Alluxio clusters. Part 1 of this blog series discusses the scenario and background. Alluxio lies in between the storage and compute layers in order to … Continued
Shopee is the leading e-commerce platform in Southeast Asia. In this blog, Tianbao Ding and Haoning Sun from Shopee’s data infrastructure team share their project on query acceleration and “Data Access as a Service.” They describe how Shopee leverages Alluxio to improve Trino query performance by ~55% and how Alluxio enhances developer experience by providing … Continued
This blog post discusses the synergy between Trino and Alluxio, and how to deploy Alluxio as the caching layer for Trino. You will learn Why should you choose Alluxio as a cache for Trino How do Trino and Alluxio work together How to configure Alluxio to point to S3 storage like MinIO How to query … Continued
As we close out the Year of Tiger and welcome the Year of Rabbit, we are filled with gratitude for the support and contributions of the members of Alluxio Open Source Community. Thanks to your dedication and trust, the Alluxio Open Source project and community has continued to thrive and grow in ways we never … Continued
Originally published on vmblog.com: https://vmblog.com/archive/2022/12/27/alluxio-2023-predictions-what-s-next-for-data-analytics-ai-and-cloud-in-2023.aspx As we enter 2023, the world of analytics, AI, and cloud is entering an exciting new phase, with a wide range of innovations and developments set to reshape the landscape. Below are some trends that will have the most impact in the coming year. Trend 1: Cloud cost optimization is … Continued