Scalable and Highly-available Distributed File System Metadata Service Using gRPC, RocksDB and RAFT
Alluxio (alluxio.io) is an open-source data orchestration system that provides a single namespace federating multiple external distributed storage systems. It is critical for Alluxio to be able to store and serve the metadata of all files and directories from all mounted external storage both at scale and at speed.
This talk shares our design, implementation, and optimization of Alluxio metadata service (master node) to address the scalability challenges.
Tags: alluxio engineering, distributed systems, grpc, metadata service, office hour, raft, rocksdb