# Robot Bench

A test bed for letting a robot work out its own control from incentives, in
simulation, before anything happens to real hardware.

You describe a robot as parts, joints, motors and sensors in a JSON spec. The
spec is the single source of truth: the trainer compiles it to MuJoCo and
learns a policy, and the browser bench builds the same robot in three.js so you
can watch what it learned.

The policy is never told the kinematics. It only ever sees what the sensors you
bolted on actually publish, at their real rate, with noise, bias and latency.

## Layout

```
robots/          robot specs (the thing an editor will eventually emit)
trainer/         MuJoCo env + NumPy PPO
  controller.py  THE control laws: balance, IK, trim, brace, filters.
                 One implementation - the trainer, the scorer, the live demo
                 and drive mode all import it. Five copies of the balance law
                 once existed and drifted (an inverted reflex sign among
                 them); a divergence is now an import error, not a Saturday.
  mjcf.py        spec -> MJCF
  env.py         sensors, actuators, reward, termination, domain randomisation
  nets.py        MLP, Adam, running observation normaliser
  ppo.py         PPO: diagonal-Gaussian policy, GAE, clipped surrogate
  vecenv.py      vectorised envs, in-process or across worker processes
  train.py       training entry point
  rollout.py     run a policy, write a replay for the bench
bench/           three.js viewer (no build step, plain ES modules)
runs/            checkpoints and per-iteration logs
```

## Setup

MuJoCo dropped Intel-Mac wheels after 3.2.7, and this machine is an Intel Mac
on macOS 12, so both the Python version and the MuJoCo version are pinned.

```sh
/usr/local/opt/python@3.12/bin/python3.12 -m venv .venv
./.venv/bin/pip install "mujoco==3.2.7" numpy
cd bench && npm install          # three.js only
```

## Use

```sh
# train (writes runs/<name>/policy_best.json and log.jsonl)
cd trainer
../.venv/bin/python train.py --steps 6000000 --envs 24 --workers 3 --run biped_v1

# produce a replay
../.venv/bin/python rollout.py --policy ../runs/biped_v1/policy_best.json \
    --out ../bench/replays/latest.json

# watch it
cd .. && ./serve.sh 8901
# open http://127.0.0.1:8901/bench/index.html
```

`rollout.py` with no `--policy` gives you the passive fall, which is the useful
baseline: it shows how unstable the machine is with the motors merely holding
their nominal pose.

## The spec

`robots/wheeled_biped.json` is the worked example: chassis, hip pitch, knee
pitch and a drive wheel per leg.

- **Links** nest as a tree. Each carries geoms (box / capsule / cylinder /
  sphere) with mass, and optionally the joint connecting it to its parent.
- **Joints** have a type, an axis, limits in degrees, damping, armature and
  friction loss. `continuous: true` marks an unlimited joint such as a wheel.
- **Actuators** pick the feedback loop per joint: `position` (a servo with kp
  and kv), `velocity`, or `torque` (raw current, what a BLDC wheel really
  takes). A position actuator on a joint with a nominal angle is centred on
  that angle, so a zero action means "hold the standing pose" rather than
  "middle of the mechanical travel".
- **Sensors** are the robot's whole world. Each has a rate, noise, bias and an
  `in_obs` flag. Flip `in_obs` off and the part is still fitted but the policy
  can no longer use it, which is how you answer "does it actually need the
  magnetometer".
- **Task** holds the reward weights, the command ranges and what counts as
  falling over.
- **domain_randomisation** perturbs mass, centre of mass, friction, actuator
  gain, control latency and random shoves. This is the part that decides
  whether the policy survives contact with the real robot.

Ground-truth channels (base pose, velocity) exist in the model but are wired
only to the reward, never to the observation. Reward is a training-time
construct and does not ship.

## What comes out

A policy JSON: two 64-unit tanh layers, plus the observation normaliser. It is
deliberately framework-free so it can be turned into fixed-point C for an MCU
without dragging a runtime along.

It is a neural network, not readable control code. If you want classical code
on the robot, the honest route is to use the learned policy as a reference
signal and fit an LQR or cascaded PID to it. The bench gives you the data to
do that; it does not do it for you yet.

## Known limits

- Learning speed is bounded by this machine: 4 cores, no CUDA, so no GPU-
  accelerated RL. The same spec and MJCF run unchanged on a rented GPU box when
  you want to go faster.
- The bench replays; it does not yet simulate interactively in the browser, and
  there is no drag-to-assemble editor. The spec is still hand-edited JSON.
- "Everyday floorspace" is currently a flat floor with tunable friction. Steps,
  slopes, rugs and thresholds are not modelled yet.
- Sim-to-real transfer is untested against hardware.