Deep Dive into the Implementation of Alluxio Metadata Storage
This article introduces the design and implementation of metadata storage in Alluxio Master, either on heap and off heap (based on RocksDB).
This article introduces the design and implementation of metadata storage in Alluxio Master, either on heap and off heap (based on RocksDB).
Alluxio (alluxio.io) is an open-source data orchestration system that provides a single namespace federating multiple external distributed storage systems. It is critical for Alluxio to be able to store and serve the metadata of all files and directories from all mounted external storage both at scale and at speed.
This talk shares our design, implementation, and optimization of Alluxio metadata service (master node) to address the scalability challenges.
Tags: alluxio engineering, distributed systems, grpc, metadata service, office hour, raft, rocksdb
It is critical for Alluxio to be able to store and serve the metadata of all files and directories from all mounted external storage both at scale and at speed. This talk shares our design, implementation, and optimization of Alluxio metadata service (master node) to address the scalability challenges.