Using Alluxio as a Fault-tolerant Pluggable Optimization Component of JD.com’s Computation Frameworks

JD.com is China’s largest online retailer and its biggest overall retailer, as well as the country’s biggest internet company by revenue. Currently, JD.com’s BDP platform runs more than 400,000 jobs (15+ PB) daily, on a system with more than 15,000 cluster nodes and a total capacity of 210 PB.

Alluxio has run in JD.com’s production environment on 100 nodes for six months. See how JD.com uses Alluxio to provide support for ad hoc and real-time stream computing, using Alluxio-compatible HDFS URLs and Alluxio as a pluggable optimization component.

Tags: , , , , ,

Unify Data at Memory Speed – Alluxio Overview

Alluxio brings your data to compute, on demand. It is a data orchestration layer for big data and AI/ML workloads in the cloud, enabling data locality, data accessibility, and data elasticity.

With Alluxio, accelerate your Spark or Presto workloads on S3, simplify Hadoop for the hybrid cloud, and bring big data and AI workloads to any object store. Companies like Barclays, Two Sigma, China Unicom, DBS Bank, and 7 out of the 10 largest internet companies rely on Alluxio.

Alluxio in MOMO

See how MOMO accelerates ad hoc analysis with Spark SQL and Alluxio.

Alluxio in JD

See how JD.com uses Alluxio to provide support for ad hoc and real-time stream computing, using Alluxio-compatible HDFS URLs and Alluxio as a pluggable optimization component.

Alluxio in Talking Data

TalkingData, China’s largest data broker, provides data intelligence solutions and processes over 20 terabytes of data and more than one billion session requests per day. TalkingData deployed Alluxio to unify disparate cloud, on-premise, and hybrid data sources for a range of analytics applications. The architecture provides self-service data access for data scientists and engineers, eliminating the need for ETL or manual IT assistance.