Hope Wang

Six Tips To Optimize PyTorch for Faster Model Training

By

Originally published at The New Stack: https://thenewstack.io/this-is-how-to-optimize-pytorch-for-faster-model-training/ PyTorch is one of the most popular deep learning frameworks in production today. As models become increasingly complex and dataset sizes grow, optimizing model training performance becomes crucial to reduce training times and improve productivity.  In this article, I’ll share the latest performance tuning tips to accelerate the training … Continued

How Can AI Platforms Adapt to Hybrid or Multi-Cloud Environments?

By

This article was originally published on Spiceworks. https://www.spiceworks.com/tech/artificial-intelligence/guest-article/adapting-ai-platform-to-hybrid-cloud/ This blog discusses the challenges of implementing AI platforms in hybrid and multi-cloud environments and shares examples of organizations that have prioritized security and optimized cost management using the data access layer. In recent years, AI platforms have undergone significant transformations as GenAI and AI continue to … Continued

Maximize GPU Utilization for Model Training

By

GPU utilization or GPU usage, is the percentage of GPUs’ processing power being used at a particular time. As GPUs are expensive resources, optimizing their utilization and reducing idle time is essential for enterprise AI infrastructure. This blog explores bottlenecks hindering GPU utilization during model training and provides solutions to maximize GPU utilization. 1. Why … Continued

IWD 2024: Empower Women Developers in the Open-Source Community

By

This article was originally published on ITBrief. The author is Hope Wang, Developer Advocate, Alluxio. As we celebrate International Women’s Day, it is important to reflect on the progress we have made toward gender equality in the tech industry, particularly in open-source software (OSS). While there is still much work to be done, I am … Continued

Setting the Stage for Alluxio Community to Soar in the Year of the Dragon: 2023 Recap and 2024 Outlook

By , , , , , , and

As we step into 2024, we look back and celebrate an incredible year of 2023 for the Alluxio community. First and foremost, thank you to all of our contributors and the broader community! Together, we have achieved remarkable milestones. 💖 📈 Highlights by Numbers Let’s take a look at the Alluxio in 2023 by numbers. … Continued

GPUs Are Fast, I/O is Your Bottleneck

By

This article was initially posted on ITOpsTimes. Unless you’ve been living off the grid, the hype around Generative AI has been impossible to ignore. A critical component fueling this AI revolution is the underlying computing power, GPUs. The lightning-fast GPUs enable speedy model training. But a hidden bottleneck can severely limit their potential – I/O. If … Continued

A Deep Dive into Caching in Presto

By and

This article was initially posted on InfoWorld. Understand the caching mechanisms for the popular distributed SQL engine and how to use them to improve query speed and efficiency. Presto is a popular, open source, distributed SQL engine that enables organizations to run interactive analytic queries on multiple data sources at a large scale. Caching is a typical optimization … Continued

Alluxio Kubernetes Operator Tutorial: Simplifying Deploying and Managing Alluxio Clusters

By , and

This blog provides a tutorial on using the Kubernetes operator to simplify deploying and managing Alluxio clusters on Kubernetes. Introduction The Alluxio Kubernetes operator makes deploying and managing Alluxio and the datasets on Kubernetes easier. With the operator, Alluxio clusters can be deployed and managed seamlessly like any other native Kubernetes application. The operator handles … Continued