ALLUXIO BAY AREA MEETUP

Scalable Filesystem Metadata Services

This talk was presented by Alluxio’s top contributor and PMC Maintainer Calvin Jia at the Alluxio bay area Meetup. 

This talk shares our design, implementation and optimization of Alluxio metadata service to address the scalability challenges, focusing on how to apply and combine techniques including tiered metadata storage (based on off-heap KV store RocksDB), fine-grained file system inode tree locking scheme, embedded state-replicate machine (based on RAFT), exploration and performance tuning in the correct RPC frameworks (thrift vs gRPC) and etc.

Questions? Slack with the speakers, users, and many other community members!
Welcome to join Alluxio Bay Area Meetup Group to attend online meetups like this!

Big Data Machine Learning Platform at Pinterest

This was presented by the Yongsheng Wu, head of big data and ML platform at Pinterest, at the Alluxio bay area meetup. 

Yongsheng shares Pinterest’s journey to build a fast and scalable big data and ML platform in AWS for Pinterest to handle the requests and complexity in data at scale. In this talk, he will cover different aspects from the requirements of the platform, the challenges encountered, the technologies chosen, and the tradeoffs that were made.