Alluxio Blog

A Deep Dive into Caching in Presto

This article was initially posted on InfoWorld. Understand the caching mechanisms for the popular distributed SQL engine and how to use them to improve query speed and efficiency. Presto is a popular, open source, distributed SQL engine that enables organizations to run interactive analytic queries on multiple data sources at a large scale. Caching is a typical optimization … Continued

Alluxio Kubernetes Operator Tutorial: Simplifying Deploying and Managing Alluxio Clusters

This blog provides a tutorial on using the Kubernetes operator to simplify deploying and managing Alluxio clusters on Kubernetes. Introduction The Alluxio Kubernetes operator makes deploying and managing Alluxio and the datasets on Kubernetes easier. With the operator, Alluxio clusters can be deployed and managed seamlessly like any other native Kubernetes application. The operator handles … Continued

Speed Trino Queries with These Performance-Tuning Tips

Originally published at The New Stack: In this article, we will discuss how data engineers and data infrastructure engineers can make Trino, a widely used query engine that’s faster and more efficient. An open source distributed SQL query engine, Trino is widely used for data analytics on distributed data storage. Optimizing Trino to make it faster … Continued

Top Tips and Tricks for PyTorch Model Training Performance Tuning [2023]

Get the latest and greatest tips to accelerate your PyTorch model training for machine learning and deep learning. PyTorch, an open-source machine learning framework, has become the de facto choice for many organizations to develop and deploy deep learning models. Model training is the most compute-intensive phase of the machine learning pipeline. It requires continuous … Continued

Trino Optimization With Distributed Caching on Data Lakes: Trino Fest 2023 Session Recap

Originally published on By 2025, there will be 100 zetabytes stored in the cloud. That’s 100,000,000,000,000,000,000,000 bytes – a huge, eye-popping number. But only about 10% of that data is actually used on a regular basis. At Uber, for example, only 1% of their disk space is used for 50% of the data they access … Continued

What’s New in Alluxio Enterprise 2.10: Radically Resource-efficient for Improved Speed at Lower Cost

We are pleased to unveil the latest version of Alluxio. This new release represents a significant milestone to enhance system reliability under different kinds of resource limitations or stress scenarios, particularly to get the most out of limited hardware resources to scale at manageable costs. Enhanced Functionality: Dramatic Improvements in High Availability (HA): Mission-critical applications … Continued

Building High-performance Data Access Layer for Model Training and Model Serving for LLM

Bringing a large language model from its initial training to deployment requires numerous systems and components. At Zhihu, we grappled with a multi-cloud, cross-region AI platform, requiring an efficient solution to facilitate the rapid training and delivery of models for production use cases. This led us to adopt Alluxio, the high-performance data access layer for … Continued