Learn how to set up EMR Spark with Alluxio so Spark jobs can seamlessly read from and write to S3. See the performance comparison between Spark on S3 with Spark, and Alluxio on S3.
Tags: aws s3, cloud, emr, office hour, performance, spark
Bay Area Meetup which include presentations on the architecture of Presto, its separation of compute and storage, cloud-readiness, recent advancements in the project such as Cost-Based Optimizer and Kubernetes Support. Presto and Alluxio production use cases and more.
Tags: alluxio engineering, aws s3, cloud object storage, compute storage separation, meetup, presto
Alluxio’s first cloud, data & orchestration Austin meetup featuring talks and demos on efficient data engineering with Apache Spark, Hive and Alluxio on S3.
Tags: apache hive, aws s3, cloud, meetup, spark
This tech talk gives a quick overview of Alluxio and the use cases it powers for Spark/Presto in Kubernetes and how to set up to run in Kubernetes.
Tags: cloud, hybrid cloud, kubernetes, locality, presto, spark, webinar
This meetup presents an overview of the motivations and design decisions behind the major changes in the Alluxio 2.0 release, and Real-time Data Processing for Sales Attribution Analysis with Alluxio, Spark and Hive at VIPShop.
Tags: alluxio engineering, apache hadoop, apache spark, compute, compute storage separation, data, data orchestration, hadoop, hdfs, meetup, scale, spark, storage
Joint hosted Alluxio New York meetup with talks to include: Embracing hybrid cloud for data-intensive analytic workloads and Alluxio on AWS EMR (fast storage access and sharing for Spark).
Tags: alluxio engineering, analytics, compute, compute storage separation, data, data orchestration, hybrid cloud, meetup, performance, storage
Alluxio maintainer and founding engineer Calvin Jia presents on Scalable Filesystem Metadata Services with RocksDB at the RocksDB meetup at Twitter.
Tags: alluxio engineering, meetup, metadata management, performance, scale, storage, unified namespace
Haoyuan Li’s keynote at O’Reilly Beijing discusses open source data orchestration and the value of leveraging Alluxio with rising trends driving the need for a new architecture. Four big trends driving this need: Separation of compute & storage, hybrid-multi cloud environments, rise of object store and self-service data across the enterprise.
Tags: big data, cloud, cloud object storage, cloud storage, compute storage separation, conference, data, data orchestration, hybrid cloud, multi cloud, on-prem object storage, storage
This tech talk highlights a simple solution is to run Spark on Alluxio as a distributed cache for S3. Alluxio stores data in memory close to Spark, providing high performance, in addition to providing data accessibility and abstraction for deployments in both public and hybrid clouds.
Tags: aws, cloud, compute storage separation, data, data orchestration, emr, hybrid cloud, on-prem object storage, spark, webinar