WM Paper Log

1.Instructions for Use

This article is used for long-term recording of World Model (WM) paper reading, reproduction, experiments, and innovation point analysis in the directions of robot navigation, path planning, and embodied intelligence.

The current focus is not to first "piece together innovation points", but to establish the paper lineage of CV / 3D Vision → World Model → Simulator → Planner / Policy → Robot, prioritizing work from formal top conferences and high-level journals, and from these selecting baselines with complete code, affordable compute, and the ability to conduct closed-loop experiments.

Current key directions: Navigation World Model, Latent World Model, 3D Structured World Model, World Model + Planning, World Model + Policy, Model-Based RL, dynamic obstacle prediction, real-time implementation, Sim2Real, ROS2 / Nav2 real-machine deployment.

---

2.First, let's clarify a few concepts.

Sensor / Observation
传感器 / 当前观测
        ↓
Perception / Representation
感知 / 世界表示
        ↓
World Model
世界模型
        ↓
Future Prediction
未来预测
        ↓
Planner / Policy
规划器 / 策略
        ↓
Controller
控制器
        ↓
Robot Command
机器人执行命令

• Computer Vision（CV，计算机视觉）：现在世界是什么样？
• World Model（WM，世界模型）：如果执行动作 A，未来世界会怎样？
• Planner（规划器） / Policy（策略）：应该选择什么动作？
• Controller（控制器）：把动作真正变成速度、关节或力矩指令。

Therefore, the output of a World Model is not necessarily RGB, nor is it usually equal to cmd_vel. It can output Future RGB, Future Latent, Future Occupancy, Future 3DGS, Future Physical State, Reward / Risk / Value, etc.

---

3.Renderer / Simulator / Planner framework

3.1.Renderer

Answer:What does the future world "look" like?

Common outputs: RGB, Video, Novel View, Depth, 3D rendering.

3.2.Simulator

Answer:How does the world actually change under the action?

Common outputs: Future State, Future Latent, Future Geometry, Future Occupancy, Future 3DGS, Object Pose, Reward / Risk.

3.3.Planner

Answer:What should be done to achieve the goal?

Common outputs: Action, Action Sequence, Trajectory, Velocity Command.

Common methods: CEM (Cross-Entropy Method, cross-entropy method), MPC (Model Predictive Control), MPPI (Model Predictive Path Integral), Learned Policy, Actor-Critic (actor-critic).

---

4.Core paper catalog

---

5.Direction proportion of papers: where each paper actually leans

The table below is intended to help quickly build intuition for theestimated research focusand is not an official figure provided by the authors. The three percentages sum to 100%.

---

6.Renderer / Simulator / Planner Functional Strength

This table differs from the "direction proportion" above. A system can be both a strong Renderer and a strong Simulator, so the three items are not required to sum.

---

7.Classified by World Model output form

---

8.The three most noteworthy paper lineages at present

A. Visual Future → Planning
Drive-WM → NWM → One-Step WM / AR Forcing

B. Latent Future → Planning / Policy
DINO-WM → X-MOBILITY → World4Drive / NavWAM

C. Explicit / Physical World → Simulator → Decision
GWM / DreMa / MoSim

Currently, the most noteworthy for focused tracking is B: Latent Future → Planning / Policy.

Reason: It sits exactly between CV, World Model, and robot planning, so it does not need to spend the main compute on photorealistic RGB generation, while retaining the advantages of mobile robot Navigation, Planning, and real-machine closed loop.

A question worth pursuing over the long term is:

For mobile robot navigation, what kind of Future Representation is most beneficial for Planning? Future RGB, Future Latent, Future Occupancy, Future Geometry, Future Risk—which one can truly improve closed-loop navigation?

---

9.Master List of Paper Links

---

10.X-MOBILITY — ICRA 2025

10.1.Understand in one sentence

RGB + Robot State
        ↓
World Model
        ↓
Probabilistic Latent State
        ↓
Action Policy
        ↓
Velocity + Optional Local Path

Decouple the World Model from the Action Policy. The core value is not generating pretty future images, but enabling the latent state to learn environment and dynamics information useful for navigation through world modeling.

The original paper uses Isaac Sim Nova Carter data, including the Random Action Dataset and the Nav2 Teacher Dataset, and provides a deployment chain for checkpoints, datasets, ONNX / TensorRT / ROS2.

Current positioning: first primary baseline candidate.

Worthwhile research problems: dynamic obstacles, RGB-only geometry grounding, Navigation-oriented representation, lightweight design, uncertainty-aware dynamics.

---

11.Navigation World Models（NWM）— CVPR 2025

11.1.Understand in one sentence

Observation + Navigation Action
        ↓
Conditional Diffusion Transformer
        ↓
Future Visual Observation
        ↓
CEM / Trajectory Ranking
        ↓
Navigation

NWM is a representative of Visual World Model → Planning. It shows that video generation-based WMs are not just CV: the model can imagine different futures for different candidate actions, and then CEM selects better actions.

Main issues: inference cost, long-horizon drift, OOD mode collapse, pedestrian temporal dynamics.

Current positioning: Must be read carefully, but training a large model from scratch is too costly, so it is not prioritized as the first complete retraining baseline.

---

12.DINO-WM — ICML 2025

12.1.Understand in one sentence

Image
 ↓
DINOv2 Features
 ↓
World Model
 ↓
Future Visual Latent
 ↓
CEM / Gradient Planning

Core question:Why must a World Model reconstruct future RGB?

DINO-WM Predict the future directly in the pretrained visual feature space, reducing irrelevant pixel reconstruction and making the world representation more task-oriented.

Current positioning: essential reading for Latent World Model + Planning; highly suitable for inspiring Navigation-Oriented World Model.

---

13.World4Drive — ICCV 2025

13.1.Understand in one sentence

Camera / Scene Features
        ↓
Vision Foundation Model
        ↓
Physical Latent World Model
        ↓
Generate Candidate Trajectories
        ↓
Predict Intention-aware Future Latents
        ↓
World Model Selector
        ↓
Best Trajectory

The most noteworthy aspect of this paper is:the World Model directly participates in Trajectory Generation / Evaluation / Selection (trajectory generation / evaluation / selection).

Although the task is autonomous driving, the methodology is very easy to transfer to mobile robots:

MPPI / Policy
产生候选轨迹
        ↓
World Model
预测每条轨迹对应的 Future State / Risk
        ↓
Trajectory Selection

Current positioning: key reference for the WM → Planning bridge.

---

14.WMNav — IROS 2025

14.1.Understand in one sentence

WMNav 将 Vision-Language Model（视觉语言模型）、World Model Memory（世界模型记忆）与 Object Goal Navigation（目标物体导航）结合。

It maintains a Curiosity Value Map and reduces the impact of hallucination on navigation decisions based on the feedback difference between the world-model plan and real observations.

Current positioning: clearly leaning toward Planner / Navigation. This indicates that a WM is not necessarily a video generator; it can also be a memory / reasoning component within a navigation system.

---

15.Drive-WM — CVPR 2024

Current Multiview Images
+ Different Driving Maneuvers
        ↓
Drive-WM
        ↓
Multiple Future Videos
        ↓
Image-based Reward
        ↓
Choose Safer Trajectory

It is the classic work for understanding the Renderer / Simulator → Planner pipeline: first imagine the future visuals corresponding to multiple driving actions, then select a safer trajectory based on reward.

Current positioning: CV / Video WM is relatively heavy, but its connection to Planning is very clear.

---

16.GWM — ICCV 2025

GWM changes the World Representation to explicit 3D Gaussian.

RGB Image(s)
   ↓
3D Gaussian Scene
   ↓
3D VAE
   ↓
Gaussian Latent + Action
   ↓
Latent Diffusion Transformer
   ↓
Future 3D Gaussian Scene

The core is not "switching to a different sensor input," but rather making the world representation have an explicit three-dimensional geometric structure.

Current positioning: 3D Vision + Simulator, suitable for learning Geometry-aware World Model, not the first choice for the main baseline of mobile navigation.

---

17.DreMa — ICLR 2025

DreMa uses Gaussian Splatting + Physics Simulator to build a Learnable Digital Twin:

Few Real Demonstrations
        ↓
Learnable Digital Twin
        ↓
Imagination
        ↓
Synthetic Demonstrations
        ↓
Imitation Learning
        ↓
Robot Policy

Current positioning: Simulator + Robot Learning. The core value is understanding that WM is not only for online planning, but can also be used for imagination / data generation.

---

18.MoSim — CVPR 2025

MoSim is well suited to correcting the misconception that "WM = future video generation":

Current Physical State
+ Action
      ↓
MoSim
      ↓
Future Physical State
      ↓
Model-Based RL / Zero-shot RL

Physical State includes joint positions, velocities, spatial positions and velocities, etc.

Current positioning: Simulator / Dynamics / Model-Based RL, with almost no dependence on CV.

---

19.MaskGWM — CVPR 2025

MaskGWM uses Diffusion Transformer + Video Mask Reconstruction, focusing on solving the Driving World Model's Long-Horizon Prediction (long-term prediction), Generalization, and Multi-view Generation (multi-view generation).

Current positioning: clearly leaning toward CV / Generative World Model. The paper is strong, but it is further from the current mobile robot Planning mainline than World4Drive, DINO-WM, X-MOBILITY.

---

20.One-Step WM / AR Forcing / NavWAM: 2026 New Route

20.1.One-Step World Model

Focus on addressing the high latency of multi-step Diffusion + Autoregressive Generation, use One-Step Generation to improve the real-time performance of Navigation WM, then connect to optimization-based planning.

20.2.AR Forcing

重点解决训练时使用 Ground Truth Context、推理时使用 Model-generated Context 导致的 Train-Test Distribution Shift 和 Long-Horizon Error Accumulation。

20.3.NavWAM

Observation + Goal
        ↓
World Action Model
        ↓
Future Observation
+ Goal Progress
+ Action Chunk

It combines "predicting the future" and "deciding actions," reflecting the WM → WAM (World Action Model) trend.

---

21.Current Baseline Selection Criteria

21.1.Tier 1: Truly worth considering for modification

1. X-MOBILITY — ICRA 2025：最贴移动机器人、ROS2、Sim2Real、Policy。
2. DINO-WM — ICML 2025：最适合研究“Future Latent 是否比 Future RGB 更适合 Planning”。
3. NWM — CVPR 2025：Navigation WM 标志性工作，但完整训练成本高。
4. World4Drive — ICCV 2025：如果研究重点逐渐偏“WM 如何评价/选择轨迹”，价值很高。

21.2.Tier 2: Strong Related Work / Methodological Reference

WMNav、Drive-WM、GWM、DreMa、MoSim、One-Step WM、AR Forcing、NavWAM。

21.3.Not recommended as the main baseline at present.

• MaskGWM : It slides too easily toward pure video generation;
• Atlas : It has not publicly released a complete training pipeline, and its scale is also unsuitable as a first baseline for a master's student;
• DreamerNav : It has high engineering and learning value, but if the goal explicitly requires a high-level venue, it is no longer prioritized as the main baseline.

---

22.A Research Question Worth Pursuing Long-Term

Does the Navigation World Model really need to predict photorealistic Future RGB?

An experimental lineage can gradually be formed as follows:

Future RGB
vs
Future Latent
vs
Future Occupancy / BEV
vs
Future Geometry / 3DGS
vs
Future Risk
        ↓
同一个 Planning / Policy Framework
        ↓
比较：
Success Rate
Collision Rate
Generalization
Latency
VRAM
Real-Robot Performance

This is closer to a research problem that can be pursued continuously than simply "adding a module to a certain model."

---

23.Unified Paper Reading Template

For each new paper from now on, prioritize answering the following questions.

23.1.Basic Information

• Title:
• Venue / Year:
• Organization:
• Paper / Project / GitHub:
• Formally published:
• Code / Checkpoint / Dataset available:

23.2.Observation / Input

What does the model look at: RGB, Depth, LiDAR, Robot State, Camera Pose, Language Goal?

23.3.World Representation

How does the model represent the world in its head: Pixel / Video Latent, DINO Feature, RSSM Latent, BEV, Occupancy, 3D Gaussian, Physical State, Digital Twin?

23.4.World Model Output

Future RGB、Future Latent、Future Occupancy、Future 3DGS、Future Physical State、Reward / Risk / Value、Action Chunk？

23.5.Renderer / Simulator / Planner Positioning

Renderer：
Simulator：
Planner / Policy：

23.6.Proportion of CV ↔ Robotics directions

CV / 3D Vision：
World Modeling / Simulator：
Planning / Policy / Robotics：

23.7.Planner / Policy

CEM, MPC, MPPI, Gradient Planning, Learned Policy, Actor-Critic, or World Action Model?

23.8.Robot Output

Mobile robot:vx / vy / wz、trajectory、local path?

Robotic arm: joint position, joint velocity, end-effector pose, action chunk?

23.9.Training / Compute

Record GPU, quantity, VRAM, training time, Epoch / Steps, Precision, Multi-GPU Strategy.

23.10.Evaluation

Record Baselines, Main Results, Ablation, Efficiency, Generalization, Failure Cases, Real Robot, Sim2Real.

23.11.Is it suitable to be my baseline?

代码完整度：
Checkpoint：
Dataset：
算力可承受：
移动机器人相关性：
实机难度：
创新空间：

---

24.Current Stage Checklist

• Close reading X-MOBILITY
• Close reading NWM
• Close reading DINO-WM
• Close reading World4Drive
• Reading WMNav
• Reading Drive-WM
• Close reading GWM
• Reading DreMa
• Reading MoSim
• Reading MaskGWM
• Reading One-Step WM
• Reading AR Forcing
• Reading NavWAM
• Understanding Renderer / Simulator / Planner
• Understanding different WM outputs such as RGB / Latent / Occupancy / 3DGS / Physical State
• Get an official checkpoint of at least one formal top-conference baseline running
• Build a Failure Case Dataset
• Propose a Research Question from stable failures
• Make a minimal modification to validate the hypothesis
• Then proceed to formal innovation design