Running Tensorflow on Alluxio FUSE

With Unified Namespace, Alluxio serves as the single unified access point for all your Tensorflow training data, transparently connecting to your existing storage systems.

Co-locating with Tensorflow applications, Alluxio caches the remote data locally for future access, providing data locality. Without Alluxio, slow remote storage may result in bottleneck on I/O and leave GPU resources underutilized.

Why Tensorflow + Alluxio

Unified Data Access to all your training data.

With Alluxio POSIX API, users can access training data transparently through Alluxio FUSE with no application rewrite. This greatly simplifies the development process, without complex requirements for different integration setups and credential configurations for each under storage.

Intelligent caching tailored to deep learning workload.

By co-locating Tensorflow applications with Alluxio workers, Alluxio utilizes intelligent strategies tailored to the I/O patterns of AI/ML workload to cache the remote data locally for future access, providing 2x performance improvements with data locality.

No more underutilized GPUs.

When reading data remote from the computation, slow remote storage often results in bottleneck on I/O and leaves GPU resources underutilized. Caching frequently used data, Alluxio eliminates the I/O stall so that your GPUs are continuously fed with data, increasing your GPU utilization to 97%+.

Featured Resources

White Paper
Blog
Blog

Related Resources

documentation
Running Deep Learning Frameworks on Alluxio
Read more
blog
Turn cloud storage or HDFS into your local file system for faster AI model training with TensorFlow
Read more
on demand
Speeding up TensorFlow and PyTorch with Alluxio
Watch video
on demand
Speeding up I/O for Machine Learning ft Apple Case Study using TensorFlow, NFS, DC OS, & Alluxio
Watch video

Sign-up for a Live Demo or Book a Meeting with a Solutions Engineer