How does replication in Alluxio happen across worker nodes? Is the unit of replication a file or a block?

Alluxio is a data orchestration system which provides data locality with intelligent multi-tiering. The replication parameters are easily configured and once done, Alluxio handles replication transparently to the requesting compute framework. As always, there’s no changes required by the end user, it’s transparent: In the above diagram, data is stored in RAM, SSD, or HDD. … Continued

Tags: ,

Top 5 Performance Tuning Tips for Running Presto on Alluxio

Presto is an open source distributed SQL engine widely recognized for its low-latency queries, high concurrency, and native ability to query multiple data sources. Alluxio is an open-source distributed file system that provides a unified data access layer at in-memory speed. The combination of Presto and Alluxio is getting more popular in many companies like JD, NetEase to leverage Alluxio … Continued

Top 10 Tips for Making the Spark + Alluxio Stack Blazing Fast

The Apache Spark + Alluxio stack is getting quite popular particularly for the unification of data access across S3 and HDFS. In addition, compute and storage are increasingly being separated causing larger latencies for queries. Alluxio is leveraged as compute-side virtual storage to improve performance. But to get the best performance, like any technology stack, you need to follow the … Continued