Embodied AI Pipeline Catalog¶
Each Pipeline composes multiple nodes from the 45-node Knowledge System. Resolve a target node when a prerequisite label is too broad; the Pipeline remains the source of truth for executable stages, artifacts, metrics, and promotion gates.
This catalog turns the repository's topic chapters into end-to-end engineering loops. Each track defines prerequisites, inputs, stages, artifacts, metrics, and a promotion gate. The commands are registered in the machine-readable pipelines/manifest.json.
Need a goal before a track? Start from the seven research routes, which combine these Pipelines into outcome-oriented experiment briefs.
Quick start¶
python scripts/run_pipeline.py --list
python scripts/run_pipeline.py --validate
python scripts/run_pipeline.py --show vla-policy
python scripts/run_pipeline.py --run vla-policy --dry-run
python scripts/run_pipeline.py --run vla-policy
Use the smoke command to verify interfaces and data flow. Use --full only after the smoke run passes and the expected compute budget is available.
A pipeline is a contract, not just a command: follow the artifact and its promotion gate.
Tracks¶
| Track | Closed loop | Current evidence | Guide |
|---|---|---|---|
| Simulation & data | task definition → simulator → expert → episodes → quality checks | smoke-tested | Open |
| VLA policy | multimodal data → policy training → closed-loop evaluation | smoke-tested teaching baseline | Open |
| World model | transitions → dynamics model → rollout → planning | smoke-tested model; planning is a separate stage | Open |
| RL post-training | MDP → reward → PPO → evaluation → regression | smoke-tested teaching baseline | Open |
| Robot foundation model | observation schema → adapter → action chunk → safety layer | interface-tested mock adapter | Open |
| Embodied reasoning | instruction → task plan → skills → feedback → replan | interface-tested rule planner | Open |
| Sim-to-real | simulation → robustness → HIL → shadow → guarded deployment | documented; hardware-dependent | Open |
| Dexterous retargeting | landmarks → geometry → IK/optimization → smoothing → evaluation | smoke-tested synthetic input | Open |
| Perception & state estimation | calibration → synchronization → fusion → uncertainty → validation | deterministic synthetic smoke test | Open |
| Navigation & locomotion | state → map/terrain → planning → control → recovery | deterministic grid-navigation smoke test | Open |
| Dexterous grasping & fine manipulation | state → pre-grasp → approach → contact → lift → hold/recover | abstract MuJoCo contact-dynamics smoke test | Open |
Chinese navigation: Pipeline 总览（中文）.
Evidence labels¶
- smoke-tested: the repository contains a lightweight executable path.
- interface-tested: local schemas/adapters can be checked without model weights or hardware.
- documented: the engineering gates are defined, but no universal local command can represent the real system.
- experimental: an exploratory path that should not be treated as a validated baseline.
Passing a smoke test verifies wiring, not research quality or real-robot safety. Report benchmark results with the configuration, seed, hardware, checkpoint, and result artifact.