The problem
Domestic robots will be instructed by people who do not speak the same language — or speak it imprecisely. The study asked how well a general-purpose LLM, constrained by careful prompting, could turn free-form multilingual instructions into the exact commands a manipulator needs.
Approach
- 01GPT-4 interprets each English or German instruction with iterative prompt refinement and few-shot examples.
- 02The model's output is restricted to a fixed vocabulary of robot commands, so it cannot invent actions.
- 03Each command triggers a pre-taught skill on the Franka Emika Panda — gripper moves, joint motions and grasps — for steps such as fetching a cup, adding milk or sugar, and stirring.
Evidence
83%
Overall task accuracy
Source: University of Brighton research record
3%
Accuracy gap between English and German inputs
Source: University of Brighton research record
Publication
2025
Conference
Integration of Multilingual NLP with 7-DOF Robotic Manipulation for Enhanced Domestic Task Automation
IEEE International Conference on Advanced Robotics and Mechatronics (ICARM 2025) · IEEE ICARM 2025 · pp. 452–457 · Portsmouth, UK
GPT-4-based few-shot prompting converts English and German instructions into robot commands for beverage preparation on a Franka Emika Panda — 83% overall task accuracy, with a 3% gap between languages.