Tell an AI-controlled robot arm to put a bowl on a plate “without hitting the mug,” and it will faithfully restate the rule in its reasoning. Then it will drive straight through the mug.
That paradox anchors a new preprint from USC, UCF and UCSB researchers. Coding agents, language models that write robot controllers as programs, need no robot-specific training. But benchmarks usually ask only whether the goal was reached, not what the robot touched on the way. On SafeLIBERO, which adds one obstacle per task, the agents struggled. With skill memory and a frozen vision-language-action policy, GPT-5.5 avoided collisions in only 31% of episodes, and GPT-6 in 59%.
The cause isn’t perception, since the agent identifies the obstacle, or the prompt, which already forbids contact. The authors blame planning and episode length: hundreds of steps of tool results and images bury the one sentence naming the obstacle. Their traces show the constraint mentioned early, then vanishing, while the plan takes the shortest line through the obstacle. This long-context explanation is an interpretation, not an isolated test. A stronger model helped mainly with local choices like which side to grasp, not with long routes.
Their fix, SafeHarness, rests on a simple principle: a rule that must hold at step 300 can’t depend on a sentence written at step 0. So the rule moves from the prompt into the tools.
For routes, a segmentation model boxes the target and obstacle, and the agent proposes waypoints. An external checker rejects any path crossing the obstacle’s box, forcing a replan. If the arm later stalls or strays too close, execution halts and planning restarts. For contact, the system checks before every grasp and release whether an obstacle is adjacent, then approaches from the far side. If no side is safe, it turns the gripper tangent to the obstacle and lowers the object vertically.
With GPT-6, SafeHarness reached 71.9% task success and 87.5% collision avoidance, versus 31% and 59% without it (2.3× and 1.5×). Against the previous best, AEGIS, a safety layer that filters a frozen policy’s actions, that is a relative gain of 6.5% and 27%. Notably, once harnessed, GPT-5.5 and GPT-6 were nearly equally safe (86.0% vs 87.5%), so safety became a property of the harness, not the model.
Caveats: the obstacle is still displaced in about one episode in eight, testing was simulated with one obstacle per scene, and the contact logic assumes parallel-jaw grippers.
The takeaway: telling a planner a rule isn’t enforcing it, and enforcement belongs where the robot actually moves.
Transparency Note: This article was prepared with the assistance of artificial intelligence technologies. Every piece is thoroughly reviewed and verified by our editorial team, who retain full responsibility for the final content.
Picture: Gemini AI