Tag: compute storage separation

Beijing Meetup: Talks from Sogou, Qiniu, JD.com & Alluxio

Beijing Meetup *

The future is the era of data, and the abstraction of efficient management, storage, and access to data is undoubtedly the cornerstone of this era. Open source distributed virtual data system Alluxio is dedicated to providing simple and efficient data abstraction, convenient data sharing and high-speed I/O for big data, machine learning, and artificial intelligence, while keeping applications and data persistent and providing rich Storage system selection. After several years of development, Alluxio was developed from a prototype of a research project involving only a few Ph.D. students and researchers in the AMPLab at the University of California, Berkeley, to more than 800 code contributors (Alluxio 1.8 release data), and deployed in Tencent. Baidu, JD, Two-Sigma, Barclays Bank and other hundreds of Chinese and foreign industry leaders in the production environment, become an important part of the data platform and data infrastructure.

Shanghai Meetup: Talks from Ctrip, Qiniu, Intel & Alluxio

Shanghai Meetup *

The future is the era of data, and the abstraction of efficient management, storage, and access to data is undoubtedly the cornerstone of this era. Open source distributed virtual data system Alluxio is dedicated to providing simple and efficient data abstraction, convenient data sharing and high-speed I/O for big data, machine learning, and artificial intelligence, while keeping applications and data persistent and providing rich Storage system selection.
After several years of development, Alluxio was developed from a prototype of a research project involving only a few Ph.D. students and researchers in the AMPLab at the University of California, Berkeley, to more than 800 code contributors (Alluxio 1.8 release data), and deployed in Tencent. Baidu, JD, Two-Sigma, Barclays Bank and other hundreds of Chinese and foreign industry leaders in the production environment, become an important part of the data platform and data infrastructure.

Unified Big Data Analytics – Any stack, Any Cloud

Boston Meetup *

This presentation focuses on how Alluxio helps the big data analytics stack to be cloud-native. The trending Cloud object storage systems provide more cost-effective and scalable storage solutions but also different semantics and performance implications compared to HDFS. Applications like Spark or Presto will not benefit from the node-level locality or cross-job caching when retrieving data from the cloud object storage. Deploying Alluxio to access cloud solves these problems because data will be retrieved and cached in Alluxio instead of the underlying cloud or object storage repeatedly.

Alluxio 2.0 Deep Dive & A Case of Real-time Processing with Spark

Bay Area Meetup *

We are excited to present Alluxio 2.0 to our community. The goal of Alluxio 2.0 was to significantly enhance data accessibility with improved APIs, expand use cases supported to include active workloads as well as better metadata management and availability to support hyperscale deployments. Alluxio 2.0 Preview Release is the first major milestone on this path to Alluxio 2.0 and includes many new features.

Unify Data Analytics: Any Stack Any Cloud | Webinar | Big Data Demystified

Alluxio Tech Talk *

This webinar reviews: The observation and analysis of trends of separation of Storage and Compute in Big Data ecosystem; Why and how to build a new data access layer between compute and storage in this data stack; Alluxio open source: history, overview, design, and architecture; Production Use case with Spark, Presto, Tensorflow and etc; A demo of running Presto on Alluxio on S3

Two Sigma Open Source Meetup

New York Meetup *

TSOS meetups focus on the open source projects that Two Sigma cares most about, from projects we generated in-house then open sourced to large external open source projects that we depend on to do our work. This time, Wenbo Zhao (Two Sigma) and Bin Fan (Alluxio) will be presenting on how Two Sigma uses Alluxio to make data-intensive compute independent of the storage beneath.

Tech Talk: Accelerate and Scale Big Data Analytics and Machine Learning Pipelines with Disaggregated Compute and Storage

Enterprises are increasingly looking towards object stores to power their big data & machine learning workloads in a cost-effective way. The combination of SwiftStack and Alluxio together, enables users to seamlessly move towards a disaggregated architecture. Swiftstack provides a massively parallel cloud object storage and multi-cloud data management system. Alluxio is a data orchestration layer, which sits between compute frameworks and storage systems and enables big data workloads to be deployed directly on SwiftStack. Alluxio provides data locality, accessibility and elasticity via its core innovations. With the Alluxio and Swiftstack solution, Spark, Presto, Tensorflow and Hive and other compute workloads can benefit from 10X performance improvement and dramatically lower costs. In this tech talk, we will provide a brief overview of the Alluxio and SwiftStack solution as well as the key use cases it enables.

Tags: , , , ,

Achieving Separation of Compute and Storage in a Cloud World

Alluxio Tech Talk *

In this tech talk, we will discuss why leading enterprises are adopting hybrid cloud architectures with compute and storage disaggregated, the new challenges that this new paradigm introduces, and the unified data solution Alluxio provides for hybrid environments.