The rise of compute intensive workloads and the adoption of the cloud has driven organizations to adopt a decoupled architecture for modern workloads – one in which compute scales independently from storage. While this enables scaling elasticity, it introduces new problems – how do you co-locate data with compute, how do you unify data across multiple remote clouds, how do you keep storage and I/O service costs down and many more.
Tag: data orchestration
This talk describes a stack of open-source projects to serve high-concurrent and low-latency SQL queries using Presto with Alluxio on big data in the cloud. Deploying Alluxio as a data orchestration layer to access cloud storage object storage (e.g., AWS S3), this architecture greatly enhances the data locality of Presto with distributed and cross-query caching, thus avoids reading the same data repeatedly from the cloud storage.
In this panel, creators of open source projects share their stories from why they started the project to the challenges they encountered on the way.
In this talk, HY discussed the key challenges and trends impacting data engineering, and explores the concept of Data Orchestration.
This session talks about challenges associated with querying diverse data sources at Walmart and how those are tackled using Presto & Alluxio.
In this talk, we share our lessons in building and rebuilding our monitoring systems and data platforms at Electronic Arts (EA).
Best use cases for Presto from the Data Engineer’s perspective. Also hear about recent Presto advancements such as Cost-Based Optimizer, Kubernetes-native deployment and the project roadmap going forward.
Alluxio core maintainers and founding engineers share the latest innovations in Alluxio 2.
Get started with Presto and Alluxio – Hands-on experience launching the EC2 instance, explore the Alluxio filesystem and cluster status, and run queries with Presto on Alluxio