In this tech talk, the Intel and Alluxio teams will present a proposed reference architecture using Alluxio as the in-memory accelerator for object stores to enable modern analytical workloads such as Spark, Presto, Tensorflow, and Hive.
Haoyuan Li presents at Beijing Meetup on open source data orchestration and the value of leveraging Alluxio with rising trends driving the need for a new architecture. Four big trends driving this need: Separation of compute & storage, hybrid-multi cloud environments, rise of object store and self-service data across the enterprise.
Tags: big data, cloud, cloud storage, compute storage separation, data, data orchestration, hybrid cloud, meetup, multi cloud, storage
Twitter SF is hosting 2019’s half yearly RocksDB Meetup with speakers from Twitter, Facebook and the community on July 11th.
Join us June 24 in Menlo Park for our next meetup! We’ll have 3 valuable talks, a delicious BBQ dinner and amazing summertime-themed raffle prizes! This free event is sponsored by GridGain Systems and Oracle.
Traditionally, if you want to run a single Spark job on EMR, you might follow the steps: launching a cluster, running the job which reads data from storage layer like S3, performing transformations within RDD/Dataframe/Dataset, finally, sending the result back to S3. You end up having something like this.
If we add more Spark jobs across multiple clusters, you could have something like this.
Alluxio is a proud sponsor and exhibitor of Spark+AI Summit in San Francisco.
What’s Spark+AI Summit? It’s the world’s largest conference that is focused on Apache Spark – Alluxio’s older cousin open source project from the same lab (UC Berkeley’s AMPLab – now RISElab).
Learn more about the practice of Alluxio in AVA deep learning platform, Ctrip big data platform, and Sogou.
Tags: big data, case study, hive, machine learning, spark
Alluxio is an open source software solution that connects analytics applications to heterogeneous data sources through a data orchestration layer that sits between compute and storage.
Tags: alluxio engineering, big data, compute storage separation, data, data engineering, data orchestration, overview, storage, unified namespace
Strata London 2018 – Learn how JD.com uses Alluxio as a pluggable optimization component to provide support for ad hoc and real-time stream computing.
Tags: big data, hdfs, performance, storage, unified namespace