Reinforcement-learning environments for training XgoDuck, a small biped whose geometry and inertia differ from MicroDuck. Policies are trained with PPO on mjlab (MuJoCo Warp) and exported to ONNX.

This repository is a downstream project based on microduck_rl by Pollen Robotics. Training runs on mjlab. Joint actuation uses BAM (Better Actuator Models) from Rhoban.

Thank you to the authors of those projects.

The task code follows the MicroDuck recipes, but the robot does not:

• The MJCF and meshes use the XgoDuck model (src/mjlab_microduck/robot/xgoduck/).
• Actuators are the HLS1910 BAM model in src/mjlab_microduck/robot/xgoduck/params/1910_m6.json, not the MicroDuck XL330.

You need a CUDA GPU, Python 3.12, and uv.

cd xgoduck_rl
uv sync

On ARM machines (DGX Spark / GB10, Jetson), the first uv sync downloads a large CUDA wheel and uv's default 30 s HTTP timeout can abort it. Set UV_HTTP_TIMEOUT=600 for that first sync.

From the repository root:

uv run train Mjlab-Velocity-Flat-XgoDuck --env.scene.num-envs 4096
uv run train Mjlab-StandUp-Flat-XgoDuck --env.scene.num-envs 4096
uv run train Mjlab-Velocity-Swizzle-XgoDuck --env.scene.num-envs 4096

uv run list-envs prints every registered task. XgoDuck checkpoints and TensorBoard logs go under logs/rsl_rl/<experiment>/.

play with no checkpoint flag loads the newest model_*.pt in that task's experiment directory:

uv run play Mjlab-Velocity-Flat-XgoDuck
uv run play Mjlab-Velocity-Swizzle-XgoDuck

Pass --checkpoint-file path/to/model_XXXX.pt to choose a specific checkpoint.

Export bakes observation normalization into the graph. Run it from the repository root and point it at a trained checkpoint:

uv run python scripts/export.py Mjlab-Velocity-Flat-XgoDuck \
    --checkpoint-file logs/rsl_rl/xgoduck_velocity/<run>/model_XXXX.pt \
    --onnx-file logs/rsl_rl/xgoduck_velocity/<run>/<run>.onnx

If you omit --onnx-file, the file is written to output.onnx in the current directory. *.onnx is gitignored. Keep the export next to the run that produced it, for example logs/rsl_rl/xgoduck_velocity/<run>/<run>.onnx.

• src/mjlab_microduck/: training environments, robot assets, and actuator integration.
• scripts/: training workflows, playback, ONNX export, and policy inference tools.
• vendor/bam/: the pinned BAM runtime, including its upstream documentation and license.
• pyproject.toml and uv.lock: reproducible dependency configuration.

Training checkpoints, logs, recordings, and local validation tools are excluded from the source release. New training runs write to the ignored logs/ directory by default. To play archived checkpoints, pass their path with --checkpoint-file.

The code is licensed under Apache 2.0. See LICENSE.