Kubernetes is widely used to orchestrate computation with improved flexibility and portability for computation in public or hybrid cloud environments across infrastructure providers. However, running data-intensive workloads introduces challenges such as efficiently moving data to compute frameworks, accessing data from multiple or remote clouds, and co-locating data with compute.
Alluxio solves these problems as a new data orchestration layer bridging the gap between data locality with improved performance and data accessibility for analytics workloads in Kubernetes, and enables portability across storage providers.
In this Office Hour:
- Overview of Alluxio and the cloud use case with Spark in Kubernetes
- How to set up Alluxio and Spark to run in Kubernetes