Modern Data Platforms – Thinking Data Flywheel on the Cloud
In this session, we share common design patterns AWS customers are applying as part of their Data and AI journey.
Tags: aws, cloud, conference, data orchestration, data orchestration summit
Slides from our latest talks
From Files to Tables: Alluxio Structured Data Management
Learn more about Alluxio’s structured data management, developer preview in Alluxio 2.1.0 and catch the demo.
Tags: conference, data engineering, data orchestration, data orchestration summit
Deep Learning and Gene Computing Acceleration with Alluxio in Kubernetes
Learn about Alibaba’s use case in deep learning and gene computing acceleration using Alluxio in Kubernetes.
Tags: conference, data engineering, data orchestration, data orchestration summit, kubernetes
Apache Iceberg – A Table Format for Huge Analytic Datasets
This talk includes why Netflix needed to build Iceberg, the project’s high-level design, and will highlight the details that unblock better query performance.
Tags: apache iceberg, conference, data orchestration, data orchestration summit, performance
tf.data: TensorFlow Input Pipeline
This talk covers an overview of the project and highlight best practices for creating performant input pipelines.
Tags: conference, data orchestration, data orchestration summit, tensorflow
Simplified Data Preparation for Machine Learning in Hybrid and Multi Clouds
ODSC WEST 2019 Cloud storage brings great flexibility in management and cost-efficiency to data scientists, but also introduces new challenges related to data accessibility and data locality for machine learning applications. For instance, when the input data is stored in a remote cloud storage like AWS S3 or Azure blob storage, direct data access is … Continued
Tags: cloud bursting, compute storage separation, data orchestration, hdfs, hybrid cloud, machine learning, multi cloud, spark
Online Meetup: Powering Data Science and AI with Apache Spark, Alluxio, and IBM
Learn why leading companies are moving towards a decoupled compute and storage architecture, and the associated challenges and requirements. Hear about how Spark and Alluxio together can solve the challenges.
Tags: analytics, compute storage separation, hdfs, meetup, performance, spark, use case
Tech Talk: From limited Hadoop compute capacity to increased data scientist efficiency
Want to leverage your existing investments in Hadoop with your data on-premise and still benefit from the elasticity of the cloud?
Like other Hadoop users, you most likely experience very large and busy Hadoop clusters, particularly when it comes to compute capacity. Bursting HDFS data to the cloud can bring challenges – network latency impacts performance, copying data via DistCP means maintaining duplicate data, and you may have to make application changes to accomodate the use of S3.
“Zero-copy” hybrid bursting with Alluxio keeps your data on-prem and syncs data to compute in the cloud so you can expand compute capacity, particularly for ephemeral Spark jobs.
Tags: cloud bursting, hadoop, hybrid bursting, tech talk, zero copy
Enabling big data & AI workloads on the object store at DBS
Vitaliy and Dipti dive into how DBS Bank built a modern big data analytics stack, leveraging an object store as persistent storage even for data-intensive workloads, and how it uses Alluxio to orchestrate data locality and data access for Spark workloads.
Tags: aws, big data, conference, hybrid cloud bursting, object stores, unified namespace