Latent actions are confounded — and world models inherit it
Latent action models (LAMs) let world models learn control from ordinary, action-unlabeled video. But trained only to reconstruct the next frame, a LAM encodes whatever is predictive — embodiment motion, and also background motion, camera shake, and scene appearance. The world model then receives those confounders as part of its action condition: rollouts stay visually plausible while control quietly fails.
Confounded action condition
Latents absorb whatever predicts the next frame:
- Background & camera motion leak into the latent action
- Zero action ≠ zero motion — the scene keeps moving
- Transferred actions fail to steer new scenes
Debiased action condition
Latents face embodiment dynamics; context stays in observations:
- Confounding path cut before world-model training
- Zero action ⇒ motion largely suppressed
- Same 32-D latent interface — drop-in for the ACWM
Debias the LAM once — everything downstream gets better
Three objectives — embodiment-centric reconstruction, action-centric contrastive learning, and latent space calibration — applied in a lightweight three-stage pipeline, with the downstream conditioning format completely unchanged.
Stage 1 · LAM debiased fine-tuning
A short fine-tune on action-unlabeled video focuses latents on embodiment dynamics, pulls action-consistent transitions together across scenes, and anchors identical frames to a zero-transition reference.
1k steps · unlabeled videoStage 2 · ACWM debiased fine-tuning
The action-conditioned world model trains on latent actions extracted by the debiased encoder — inheriting clean, motion-facing controllability at scale.
same architecture & interfaceStage 3 · robot action based training
A lightweight action-to-latent mapping aligns executable robot actions to the already-debiased space — so adaptation to real robot control is fast and cheap.
12× fewer updatesSee CD-LAM in action
A looping highlight reel of side-by-side rollouts — replay, zero-action intervention, and target-action transfer, labels burned in. A narrated overview is in production.
Auto-looping highlight reel · 8 episodes · sound-free. Full narrated overview coming soon.
Cleaner actions in, controllable rollouts out
Evaluated against DreamDojo on EgoDex (latent-action rollouts) and AgiBot (robot-action rollouts and interventions), with identical architecture, latent dimension, and conditioning format — every gain comes from the LAM-side debiasing stage alone.
FDCE, head to head Foreground Displacement Chamfer Error · px, lower is better · robot-action conditioned
Zero-action rows measure spurious embodiment motion when the command says “don’t move” — the direct fingerprint of confounding. At 14B, CD-LAM cuts it to less than a quarter.
| Model | FDCE (px) ↓ | PSNR (dB) ↑ |
|---|---|---|
| DreamDojo-2B | 12.63 | 19.85 |
| CD-LAM-2B | 8.24 −34.8% | 20.60 +0.75 |
| DreamDojo-14B | 11.11 | 20.01 |
| CD-LAM-14B | 7.73 −30.4% | 21.01 +1.0 |
Robot-action conditioned rollouts after robot action based training; lower FDCE = better action following.
Watch the debiasing — synced, side by side
Robot-action conditioned rollouts on AgiBot after Stage 3, at 2B and 14B scale. Within each row every tile starts from the same first frame and receives the same commanded actions; playback is synchronized. The left tile is always the reference — what the commanded actions ask for — followed by CD-LAM and DreamDojo. FDCE (Foreground Displacement Chamfer Error; px, lower = better action following) compares induced foreground displacement between generated and reference rollouts rather than raw pixel appearance; clips are 4.9 s at 10 fps.
Cite CD-LAM
@article{wei2026cdlam,
title = {Causally Debiased Latent Action Model for Embodied Action Conditioned World Models},
author = {Wei, Yufan and Zhou, Kun and Mao, Lingjun and Zhang, Zijun and Xu, Ziming and Xi, Ziqiao and Liang, Shuang and Han, Ruobing and Yan, Yuchen and Wang, Xinyue and Feng, Fan and Huang, Biwei},
journal = {arXiv preprint arXiv:2607.09185},
year = {2026}
}