From Cache to Cash: Introducing NFT for Data Orchestration
Today, we are excited to announce the launch of Non-fungible token (NFT) as a new feature in our leading data orchestration platform.
Tag: data orchestration
Pair Spark with Alluxio to Modernize Your Data Platform
Alluxio is the data orchestration platform to unify data silos across heterogeneous environments. This is the last article in a series to give you the basics of Alluxio’s architecture and solution.
Spark + Alluxio Overview | Pair Spark with Alluxio to Modernize Your Data Platform
By bringing Alluxio together with Spark, you can modernize your data platform in a scalable, agile, and cost-effective way. In this post, we provide an overview of the Spark + Alluxio stack. We explain the architecture, discuss real-world examples, describe deployment models, and showcase performance and cost benchmarking.
Tags: analytics, cloud, data orchestration, data platform, overview, spark, storage
Alluxio Day 10
Community Virtual Event *
Join us for the 10th Alluxio Day virtual community event featuring speakers from Uber, BiliBili, and Alluxio.
Alluxio and Apache Ranger Best Practices
As data stewards and security teams provide broader access to their organization’s data lake environments, having a centralized way to manage fine-grained access policies becomes increasingly important. Alluxio can use Apache Ranger’s centralized access policies in two ways: 1) directly controlling access to virtual paths in the Alluxio virtual file system or 2) enforcing existing access policies for the HDFS under stores.
Architecting a Heterogeneous Data Platform Across Clusters, Regions, and Clouds
Data platform teams are increasingly challenged with accessing multiple data stores that are separated from compute engines, such as Spark, Presto, TensorFlow or PyTorch. Whether your data is distributed across multiple datacenters and/or clouds, a successful heterogeneous data platform requires efficient data access. Alluxio enables you to embrace the separation of storage from compute and use Alluxio data orchestration to simplify adoption of the data lake and data mesh paradigms for analytics and AI/ML workloads.
Tags: ai, analytics, cloud, compute, data orchestration, data platform, data stores, ml, storage
A Year with Alluxio Community 2021
2021 marked accelerated growth for the Alluxio Open Source Project. We could not be more grateful for what the community has achieved together in this past year. This blog provides a glimpse of the year long summary of our community growth.
Architecting a Heterogeneous Data Platform Across Clusters, Regions, and Clouds
Alluxio Product School *
Whether your data is distributed across multiple datacenters and/or clouds, a successful heterogeneous data platform requires efficient data access. Alluxio enables you to embrace the separation of storage from compute and use Alluxio data orchestration to simplify adoption of the data lake and data mesh paradigms for analytics and AI/ML workloads.
Accelerating Machine Learning / Deep Learning in the Cloud: Architecture and Benchmark
This whitepaper introduces how to speed up end-to-end distributed training in the cloud using Alluxio to accelerate data access. With the help of Alluxio, loading data from cloud storage, training and caching data can be done in a transparent and distributed way as a part of the training process. This whitepaper also demonstrates how to set up and benchmark the end-to-end performance of the training process, along with a comparison of other popular approaches.
Tags: benchmark, cache, cloud, data orchestration, deep learning, distributed training, machine learning, performance, storage