This session is worth the wait.
We’re getting the recording ready. Check back soon to watch the full session.

Building the Data Flywheel for Frontier AI Model

Data Curation

Model Training

Multimodal Data

Curation, deduplication, feature generation, search, and pretraining each want different compute, and Microsoft AI's Jack Song shows how they all run against the same data layer without rebuilding it between stages.

Building the Data Flywheel for Frontier AI Model

Building the Data Flywheel for Frontier AI Model

Data Curation

Model Training

Multimodal Data

Curation, deduplication, feature generation, search, and pretraining each want different compute, and Microsoft AI's Jack Song shows how they all run against the same data layer without rebuilding it between stages.

Thank you! Your recording is ready.

Your submission was successful. The video will appear in a moment.

Suqiang (Jack) Song
Member of Technical Staff, Head of Data Platform
Microsoft AI

About This Talk

AI training goes beyond one process. The same multimodal data moves through repeated cycles of curation, deduplication, feature generation, search, pretraining, and iteration, with each stage having distinct compute and query needs.

In this session, Jack Song of Microsoft AI will share how Lance serves as a common data foundation across the flywheel, keeping large-scale training data in a single durable, evolvable layer while different workloads operate on it in ways that best fit the job.

The session will look at real workflows spanning multimodal dataset preparation, deduplication and decontamination, feature pipelines, generation of pretraining-ready data, high-throughput training reads, and low-latency search. Rather than coupling the data to a single processing or query engine, teams can bring the right compute to the same underlying data as requirements change.

The result is a tighter data flywheel to curate, enrich, search, train, learn, and feed those outputs back into the next model iteration without rebuilding the data layer at every step.

About The Speaker

Suqiang (Jack) Song

At Microsoft AI, Suqiang leads Data Platform and Data Engineering teams supporting frontier model development across pre-training, reasoning, and post-training. They work across training-data pipelines, code and multimodal data, synthetic data generation, research data platforms, and governed data flywheels.

Before Microsoft AI, he led Data Platform teams at Uber, working on multi-cloud infrastructure, large-scale data migration, and multimodal data platforms. At Airbnb, Suqiang led AI/ML Infrastructure, including feature and embedding platforms, MLOps, and generative AI initiatives. Earlier, as a VP at Mastercard, he led global Data Engineering and Data & AI Platform teams across the US and India.

Transcript

Show transcriptHide transcript

Related recordings

Welcome: What Will AI Dream Next?

Chang She
•
Keynote
opening-keynote-what-will-ai-dream-next

AI Research

Multimodal Data

AI Infrastructure

Cosmos: Omnimodal World Models for Physical AI

Max Li
•
Session
cosmos-omnimodal-world-models-for-physical-ai

Physical AI

Multimodal Data

Model Training

Scalable, Multimodal Data Curation

Ethan Rosenthal
•
Session
scalable-multimodal-data-curation

Data Curation

Multimodal Data

Search & Retrieval

Building the Data Flywheel for Frontier AI Model

Suqiang (Jack) Song
•
Session
building-the-data-flywheel-for-frontier-ai-model

Data Curation

Model Training

Multimodal Data

From Raw Driving Logs to RL Training: A Petabyte-Scale Lance Pipeline in Production

Shiyan Xu
•
Session
from-raw-driving-logs-to-rl-training-a-petabyte-scale-lance-pipeline-in-production

Autonomous Driving

Data Curation

Model Training

From Raw Delivery to the Training Loop: Luma's Data Platform on Lance

Haoran Cai
•
Session
from-raw-delivery-to-the-training-loop-luma-s-data-platform-on-lance

Multimodal Data

Data Curation

Model Training

How Adobe Built a Multimodal Lakehouse for AI-Native Experiences

Miao Wang
Jiayao Hu
•
Session
how-adobe-built-a-multimodal-lakehouse-for-ai-native-experiences

Multimodal Data

AI Infrastructure

Search & Retrieval

Data Infrastructure for Robotics Foundation Models

Zhipeng Yan
•
Session
data-infrastructure-for-robotics-foundation-models

Physical AI

AI Infrastructure

Model Training

What Does "Now" Mean to Your Model? Time Alignment for Multimodal AI Data

Vasundra Srinivasan
•
Session
what-does-now-mean-to-your-model

Multimodal Data

Physical AI

Model Training

Scaling the Data Foundation for XPENG's VLA 2.0

Bo Tian
•
Session
scaling-the-data-foundation-for-xpengs-vla-2-0

Autonomous Driving

Model Training

Data Curation

Keynote | Rethinking AI Infrastructure

Gerred Dillon
•
Keynote
rethinking-ai-infrastructure

AI Infrastructure

Model Training