On Demand Video

AI Infra Day | Composable PyTorch Distributed with PT2 @ Meta

In this talk, Wanchao Liang, Software Engineer at Meta Pytorch Team, explores the technology advancements of PyTorch Distributed, and dive into the details of how multi-dimensional parallelism is made possible to train Large Language Models by composing different PyTorch native distributed training APIs.

Wanchao Liang
is a Software Engineer at Meta, PyTorch Team; Tech Lead in PyTorch Distributed training; Author of DTensor, a fundamental distributed abstraction to perform distributed computation. Previously worked on the TorchScript compiler, ONNX.