This session is worth the wait.
We’re getting the recording ready. Check back soon to watch the full session.

What Does "Now" Mean to Your Model? Time Alignment for Multimodal AI Data

Multimodal Data

Physical AI

Model Training

Learn how to align video and inferred actions in multimodal AI training data so robots and computer-use agents learn from the right sequence of events.

What Does "Now" Mean to Your Model? Time Alignment for Multimodal AI Data

What Does "Now" Mean to Your Model? Time Alignment for Multimodal AI Data

Multimodal Data

Physical AI

Model Training

Learn how to align video and inferred actions in multimodal AI training data so robots and computer-use agents learn from the right sequence of events.

Thank you! Your recording is ready.

Your submission was successful. The video will appear in a moment.

Vasundra Srinivasan
Principal AI Engineer
Salesforce

About This Talk

A robot learning to make coffee and a computer-use agent learning to build a pivot table face the same problem: a video shows what happened, but it doesn’t come with a reliable record of each action and when it occurred.

Figure reported that pre-training on human behavior raised a humanoid’s zero-shot success in 30 unseen homes from 9% to 56%. VideoAgentTrek turned 39,000 screen-recorded tutorials into 1.5 million training steps for computer-use agents. In both settings, turning observation into training data requires inferring actions and aligning them with the video.

Get that timing wrong, and a training example can pair an action with frames that already show its result. That gives the model information it won’t have when choosing what to do next.

Drawing on patterns from my book, I’ll walk through two practical decisions: how to store inferred actions alongside their timestamps, confidence scores, and labeler versions, and how to measure the delay between an action and its visible effect before training. Attendees will leave with a concrete approach to checking temporal alignment before these errors become part of what a model learns.

About The Speaker

Vasundra Srinivasan

Vasundra Srinivasan, Principal AI Engineer at Salesforce, works at the intersection of applied AI and enterprise data, where she has engineered some of the largest agentic AI and data foundations in production. She is the author of Data Engineering for Multimodal AI (O'Reilly, 2026), a hands-on guide whose twelve chapters each end in a runnable playbook. Her research centers on agent runtime architecture, evaluation, and running AI agents reliably at scale, with published arXiv work on agent memory, context engineering, and production patterns for the Model Context Protocol. She studied artificial intelligence at Stanford and has spoken at AI Council, AI Infra Summit, TDX and Dreamforce. Vasundra leads the Women in Data chapter for SF and Silicon Valley and is a 2026 USA Triathlon Foundation Ambassador.

Transcript

Show transcriptHide transcript

Related recordings

Welcome: What Will AI Dream Next?

Chang She
•
Keynote
opening-keynote-what-will-ai-dream-next

AI Research

Multimodal Data

AI Infrastructure

Cosmos: Omnimodal World Models for Physical AI

Max Li
•
Session
cosmos-omnimodal-world-models-for-physical-ai

Physical AI

Multimodal Data

Model Training

Scalable, Multimodal Data Curation

Ethan Rosenthal
•
Session
scalable-multimodal-data-curation

Data Curation

Multimodal Data

Search & Retrieval

Building the Data Flywheel for Frontier AI Model

Suqiang (Jack) Song
•
Session
building-the-data-flywheel-for-frontier-ai-model

Data Curation

Model Training

Multimodal Data

From Raw Driving Logs to RL Training: A Petabyte-Scale Lance Pipeline in Production

Shiyan Xu
•
Session
from-raw-driving-logs-to-rl-training-a-petabyte-scale-lance-pipeline-in-production

Autonomous Driving

Data Curation

Model Training

From Raw Delivery to the Training Loop: Luma's Data Platform on Lance

Haoran Cai
•
Session
from-raw-delivery-to-the-training-loop-luma-s-data-platform-on-lance

Multimodal Data

Data Curation

Model Training

How Adobe Built a Multimodal Lakehouse for AI-Native Experiences

Miao Wang
Jiayao Hu
•
Session
how-adobe-built-a-multimodal-lakehouse-for-ai-native-experiences

Multimodal Data

AI Infrastructure

Search & Retrieval

Data Infrastructure for Robotics Foundation Models

Zhipeng Yan
•
Session
data-infrastructure-for-robotics-foundation-models

Physical AI

AI Infrastructure

Model Training

What Does "Now" Mean to Your Model? Time Alignment for Multimodal AI Data

Vasundra Srinivasan
•
Session
what-does-now-mean-to-your-model

Multimodal Data

Physical AI

Model Training

Scaling the Data Foundation for XPENG's VLA 2.0

Bo Tian
•
Session
scaling-the-data-foundation-for-xpengs-vla-2-0

Autonomous Driving

Model Training

Data Curation

Keynote | Rethinking AI Infrastructure

Gerred Dillon
•
Keynote
rethinking-ai-infrastructure

AI Infrastructure

Model Training