Get the latest and greatest tips to accelerate your PyTorch model training for machine learning and deep learning. PyTorch, an open-source machine learning framework, has become the de facto choice for many organizations to develop and deploy deep learning models. Model training is the most compute-intensive phase of the machine learning pipeline. It requires continuous … Continued
We are pleased to unveil the latest version of Alluxio. This new release represents a significant milestone to enhance system reliability under different kinds of resource limitations or stress scenarios, particularly to get the most out of limited hardware resources to scale at manageable costs. Enhanced Functionality: Dramatic Improvements in High Availability (HA): Mission-critical applications … Continued
Understand the data transfer and egress costs of Amazon Web Services and how to reduce them. The Challenges with Rising Cloud Costs Cloud storage has gained popularity recently, but managing AWS or other cloud service costs can be a significant challenge for enterprises. While AWS offers free data input to the cloud service, they charge … Continued
This blog post discusses the synergy between Trino and Alluxio, and how to deploy Alluxio as the caching layer for Trino. You will learn Why should you choose Alluxio as a cache for Trino How do Trino and Alluxio work together How to configure Alluxio to point to S3 storage like MinIO How to query … Continued
One of the key features of Alluxio Enterprise Edition is Transparent URI, which provides ease of integration of Alluxio with your existing data stack without any changes to the location metadata of the Hive Metastore. This article provides a tutorial on employing the Alluxio Transparent URI capability with Trino, Hive Metastore and Spark, and with … Continued
Today, we are thrilled to announce that Alluxio 2.9 is generally available (GA) for both the free open source Alluxio Community Edition and Alluxio Enterprise Edition! With GA, you can expect stability, support, and enterprise-readiness from Alluxio. In this blog post, we explore how Alluxio is enabling growth and agility for analytics and AI applications … Continued
The Alluxio 2.8 version focuses on the S3 API, enterprise-grade security, scalability and observability in data migration. Enhanced S3 API makes managing Alluxio easier than ever. Features such as encryption at rest and policy-driven data management further improve Alluxio’s functionality to support enterprise customers.
Alluxio is the data orchestration platform to unify data silos across heterogeneous environments. This is the last article in a series to give you the basics of Alluxio’s architecture and solution.
Alluxio 2.6 significantly improves the performance of data-intensive AI/ML workloads across any storage, and also improves the general maintainability and visibility of Alluxio clusters, especially for large-scale deployments. We have taken the feedback and contributions from the community and introduced features which simplify deployment, introduce new data management capabilities, optimize performance, and provide enhanced visibility into system behavior.