humanoid-loco: Unitree G1 locomotion with reinforcement learning
Velocity-commanded walking trained with PPO and domain randomization in MuJoCo Playground (MJX), exported to ONNX and tested sim-to-sim in plain CPU MuJoCo on 500 seeded episodes per condition.
NVIDIA L4, simulation, no real robotsim-to-sim, not sim-to-realevery number from results/*.json0 / 500falls in plain CPU MuJoCo, nominal (20 s episodes)
42 vs 97falls with a 1-step action delay: DR vs no DR at the same training steps
130.7M stepstraining in 108 minutes on one NVIDIA L4
0.154 m/smean velocity-tracking error, nominal
Videos
The ONNX policy running in a different simulator from the one it was trained in (sim-to-sim).
Same policy in the training simulator, same scripted velocity commands.
Also walks in the nominal case; the difference shows up under latency and low friction (table below).
Did not learn to stand within its budget. Kept on purpose: failures are part of the result.
147.5M env steps in 48 minutes on one NVIDIA L4. Constant command 0.8 m/s forward; the camera follows one robot (Isaac Lab G1, 37 joints). Trained and shown in the same simulator: this is not a transfer test.
266.4M env steps in 110 minutes (stopped at wall-clock cap, 2710 of 3000 iterations). Within Isaac Sim, nominal: 4/500 falls on the task's own terrain mix.
Left: brax PPO + DR policy on the Menagerie G1 in plain CPU MuJoCo. Right: Isaac Lab RSL-RL policy on the Isaac Lab G1 in Isaac Sim. Different policies, robot models, commands and simulators; a visual impression, not a matched comparison.
Falls in plain CPU MuJoCo (out of 500 episodes, 20 s each)
| condition | PPO + DR (final) | PPO + DR (step-matched) | PPO, no DR | RSL-RL PPO + DR |
|---|---|---|---|---|
| nominal | 0/5000.0% | 4/5000.8% | 3/5000.6% | 500/500100.0% |
| floor friction x0.5 | 26/5005.2% | 19/5003.8% | 43/5008.6% | 500/500100.0% |
| floor friction x1.5 | 0/5000.0% | 3/5000.6% | 3/5000.6% | 500/500100.0% |
| torso mass +3 kg | 2/5000.4% | 3/5000.6% | 5/5001.0% | 500/500100.0% |
| torso mass -3 kg | 1/5000.2% | 1/5000.2% | 1/5000.2% | 500/500100.0% |
| random pushes (harsher than training) | 338/50067.6% | 440/50088.0% | 435/50087.0% | 500/500100.0% |
| 1-step action delay | 9/5001.8% | 42/5008.4% | 97/50019.4% | 500/500100.0% |
| 2-step action delay | 491/50098.2% | 500/500100.0% | 500/500100.0% | 500/500100.0% |
| sensor noise (training level) | 0/5000.0% | 4/5000.8% | 5/5001.0% | 500/500100.0% |
| more accurate solver | 0/5000.0% | 2/5000.4% | 2/5000.4% | 500/500100.0% |
Same seeds (start state, velocity command, pushes) for every policy. Wilson 95% intervals are in the full results.
Does domain randomization help? Paired comparison at the same training steps
| condition | falls, PPO + DR | falls, PPO no DR | exact McNemar p |
|---|---|---|---|
| nominal | 4 | 3 | 1 |
| floor friction x0.5 | 19 | 43 | 0.00072 |
| floor friction x1.5 | 3 | 3 | 1 |
| torso mass +3 kg | 3 | 5 | 0.69 |
| torso mass -3 kg | 1 | 1 | 1 |
| random pushes (harsher than training) | 440 | 435 | 0.65 |
| 1-step action delay | 42 | 97 | 1.3e-07 |
| 2-step action delay | 500 | 500 | 1 |
| sensor noise (training level) | 4 | 5 | 1 |
| more accurate solver | 2 | 2 | 1 |
Highlighted rows: p < 0.01. One training seed per policy, so this speaks to these two checkpoints, not to domain randomization in general.
Isaac Lab track: same task idea, Isaac Sim + RSL-RL
Isaac Sim 4.5.0.0 and Isaac Lab v2.1.1 installed on this host; Isaac-Velocity-Flat-G1-v0 trained with RSL-RL PPO, 4096 parallel envs, 1500 iterations, 147,456,000 env steps, 48 minutes, final mean training episode length 1000 of 1000 steps.
Evaluation within Isaac Sim (PhysX), not sim-to-sim; 500 episodes of 20 s per condition. Different robot model, fall definition and command ranges than the MuJoCo table above, so compare within this table only.
| condition | flat policy: falls (Wilson 95%) | rough-terrain policy (terrain mix) |
|---|---|---|
| nominal | 0/5000.0% [0.0%, 0.8%] lin-vel err 0.121 m/s | 4/5000.8% [0.3%, 2.0%] lin-vel err 0.108 m/s |
| robot contact friction x0.5 | 0/5000.0% [0.0%, 0.8%] lin-vel err 0.116 m/s | 9/5001.8% [0.9%, 3.4%] lin-vel err 0.112 m/s |
| robot contact friction x1.5 | 0/5000.0% [0.0%, 0.8%] lin-vel err 0.124 m/s | 3/5000.6% [0.2%, 1.7%] lin-vel err 0.108 m/s |
| torso mass +3 kg | 3/5000.6% [0.2%, 1.7%] lin-vel err 0.151 m/s | 8/5001.6% [0.8%, 3.1%] lin-vel err 0.120 m/s |
| torso mass -3 kg | 0/5000.0% [0.0%, 0.8%] lin-vel err 0.102 m/s | 4/5000.8% [0.3%, 2.0%] lin-vel err 0.098 m/s |
| random pushes (none in training) | 395/50079.0% [75.2%, 82.3%] lin-vel err 0.326 m/s | 269/50053.8% [49.4%, 58.1%] lin-vel err 0.299 m/s |
| 1-step action delay | 123/50024.6% [21.0%, 28.6%] lin-vel err 0.288 m/s | 50/50010.0% [7.7%, 12.9%] lin-vel err 0.163 m/s |
| 2-step action delay | 477/50095.4% [93.2%, 96.9%] lin-vel err 0.844 m/s | 432/50086.4% [83.1%, 89.1%] lin-vel err 0.660 m/s |
| sensor noise (training level) | 0/5000.0% [0.0%, 0.8%] lin-vel err 0.119 m/s | 6/5001.2% [0.6%, 2.6%] lin-vel err 0.109 m/s |
Isaac to MuJoCo sim-to-sim was not attempted: the Isaac Lab G1 has 37 action joints (with hands), the Menagerie G1 used above has 29 actuators, and only 18 joint names match, so no exact joint and observation mapping exists.
How it was built
- Training: brax PPO, MuJoCo Playground
G1JoystickFlatTerrain, 8,192 parallel environments, Playground default config with its domain randomizer. - Export: static ONNX graph (opset 17), max action difference to the training framework 8.6e-07.
- Evaluation: the ONNX policy runs closed-loop at 50 Hz in plain CPU MuJoCo, with the observation rebuilt and checked element-wise against MJX.
- Ablations: no domain randomization; an RSL-RL PPO run (failed to learn within its budget).
Limits
- Simulation only: nothing here has run on a real robot.
- Two control steps of action delay and the harsh push condition break every policy often.
- One training seed per configuration; budgets below the Playground default of 200M steps.
- Isaac Lab results are measured inside Isaac Sim, where the policy was trained; they are not a transfer test.
Pavan Yadav Annappa. Model: Unitree G1 from MuJoCo Menagerie (BSD-3-Clause). Code: MIT.