Blog

Alluxio Blog

Accelerating Data Loading in Large-Scale ML Training With Ray and Alluxio

In the rapidly-evolving field of artificial intelligence (AI) and machine learning (ML), the efficient handling of large datasets during training is becoming more and more pivotal. Ray has emerged as a key player, enabling large-scale dataset training through effective data streaming. By breaking down large datasets into manageable chunks and dividing training jobs into smaller … Continued

The Best Content of 2023 – Our Favorite Things

2023 is over, so we’ve compiled a collection of 2023’s most popular content according to our readers. In case you missed anything, here’s your chance to catch up on best practices ebooks, technical blogs, hands-on videos, webinars and more. Enjoy! ALL THINGS AI Building High-performance Data Access Layer for Model Training and Model Serving for … Continued

Setting the Stage for Alluxio Community to Soar in the Year of the Dragon: 2023 Recap and 2024 Outlook

As we step into 2024, we look back and celebrate an incredible year of 2023 for the Alluxio community. First and foremost, thank you to all of our contributors and the broader community! Together, we have achieved remarkable milestones. 💖 📈 Highlights by Numbers Let’s take a look at the Alluxio in 2023 by numbers. … Continued

A Journey Towards Data Locality on Cloud for Machine Learning and AI

In this blog, we discuss the importance of data locality for efficient machine learning on the cloud. We examine the pros and cons of existing solutions and the tradeoff between reducing costs and maximizing performance through data locality. We then highlight the new-generation Alluxio design and implementation, detailing how it brings value to model training … Continued

Beyond the Hype: 10 Core Principles for AI Success

This article was initially posted on datanami. The paradigm shift ushered in by Artificial Intelligence (AI) in today’s business and technological landscapes is nothing short of revolutionary. AI’s potential to transform traditional business models, optimize operations, and catalyze innovation is vast. But navigating its complexities can be daunting. Organizations must understand and adhere to some foundational … Continued

AI Infra Day Sessions Recap

Alluxio, the data platform company for all data-driven workloads, hosted the community event “AI Infra Day” on October 25, 2023. This virtual event brought together technology leaders working on AI infrastructure from Uber, Meta, and Intel, to delve into the intricate aspects of building scalable, performant, and cost-effective AI platforms. Bin Fan, Alluxio’s Chief Architect … Continued

The Data-Driven Heartbeat of Artificial Intelligence

This article was initially posted on Solutions Review. Artificial Intelligence (AI) has consistently been in the limelight as the precursor of the next technological era. Its limitless applications, ranging from simple chatbots to intricate neural networks capable of deep learning, promise a future where machines understand and replicate complex human processes. Yet, at the heart of … Continued

GPUs Are Fast, I/O is Your Bottleneck

This article was initially posted on ITOpsTimes. Unless you’ve been living off the grid, the hype around Generative AI has been impossible to ignore. A critical component fueling this AI revolution is the underlying computing power, GPUs. The lightning-fast GPUs enable speedy model training. But a hidden bottleneck can severely limit their potential – I/O. If … Continued