HCI Lab 1: Machine Learning in Unity with ML-Agents

Signal Chase starts with a simple task: move a cyan sphere towards a gold target. The extension makes the target move and reflect from the platform edges. Both tasks use learned PPO policies, not a scripted chasing rule.

Runtime camera capture of the moving-target task. The desktop control overlay is not included in this capture.

From a fixed target to a moving one Link to heading

The scene is a 10-by-10 platform. The agent observes the relative target position, its own velocity, the target velocity, and its position on the platform: eight values in total. Two continuous actions apply acceleration along X and Z.

A capture within 0.8 units gives +1 reward. Leaving the platform gives -1. A small distance-progress reward and time penalty guide learning between captures. Each attempt has a 10-second limit; falls and timeouts count as failures.

The moving target travels at 1.2 units per second. Including its velocity lets the policy react to motion rather than only follow a fixed location. In the desktop application, target control can also be enabled for WASD or arrow-key movement.

Setup and a failed first run Link to heading

The project uses Unity 6000.0.23f1, ML-Agents 4.0.0 and Python ML-Agents 1.1.0. Python runs in a separate environment on D:, without changing the existing Unity projects or Conda environments.

The first training run exposed a physics problem. Rotation was frozen, and default contact friction resisted small force commands. The agent often stayed still instead of making useful progress. More training alone was not the fix.

I gave the sphere and platform an explicit zero-friction material, keeping Rigidbody damping to control sliding. The second run used a new run ID, so its results stayed separate from the failed attempt.

Each policy then trained for about 200,000 steps, using 16 parallel arenas and a network with two hidden layers of 64 units. The CPU run took about five minutes on 15 October.

The second run’s TensorBoard data. Reward improves for both tasks; the right panel shows policy entropy, not test success rate.

Independent checks Link to heading

The final Windows player was tested twice: once with random actions and once with deterministic ONNX inference. Each run contained 100 attempts per task. Both used evaluation seed 20261015, different from training seed 742, with matching reset layouts and target headings.

All attempts, including failures, are included. Time is the learned policy’s mean capture time in simulated seconds. These are automatic trials with reflected target motion, not a usability study or a score against a human-controlled opponent. The result covers this evaluation seed and environment, not every possible target path.

The two exported models are about 24 KB each. They run inside Unity on the CPU, without a Python training process.

Project files Link to heading

• Download the Unity source, trained models and recorded checks
• Download the evaluation summary

Open the SignalChase folder in Unity 6000.0.23f1. The archive includes the saved scene, models, training configuration and reproduction scripts. The earlier Unity setup record covers the basic environment; this post adds the learning task and its extension.