In this talk, HY discussed the key challenges and trends impacting data engineering, and explores the concept of Data Orchestration.
Tag: data engineering
Best use cases for Presto from the Data Engineer’s perspective. Also hear about recent Presto advancements such as Cost-Based Optimizer, Kubernetes-native deployment and the project roadmap going forward.
Alluxio core maintainers and founding engineers share the latest innovations in Alluxio 2.
Hear about the challenges and evolution of data orchestration at Rakuten data system with the collaboration of Alluxio.
Learn more about Alluxio’s structured data management, developer preview in Alluxio 2.1.0 and catch the demo.
Learn about Alibaba’s use case in deep learning and gene computing acceleration using Alluxio in Kubernetes.
we introduce Tachyon, a memory centric fault-tolerant distributed file system, which enables reliable file sharing at memory-speed across cluster frameworks, such as Spark and MapReduce.
Welcome to the first event of the Cloud, Data, & Orchestration Austin Meetup! This meetup will feature two talks and an opportunity to engage with other data engineers, developers, and Alluxio users. Thanks to Bazaarvoice for hosting!
Today, real-time computation platform is becoming increasingly important in many organizations. In this article, we will describe how ctrip.com applies Alluxio to accelerate the Spark SQL real-time jobs and maintain the jobs’ consistency during the downtime of our internal data lake (HDFS). In addition, we leverage Alluxio as a caching layer to dramatically reduce the workload pressure on our HDFS NameNode.