ACE: Sony AI's Safe, High-Speed Table Tennis Robot Outplays Elite Players

Overview

ACE, Sony AI Zurich's first physical AI system, is built to challenge elite table tennis players by combining reinforcement learning with an optimization-based controller. The project aims to demonstrate safe real-world interaction between humans and fast AI agents, using a purpose-built robotic arm and a high-fidelity perception stack.

• Reinforcement learning trained in simulation before real-world deployment
• Optimization-based safety guarantees to prevent collisions
• End-to-end latency around 20 milliseconds, faster than human reaction time
• Matches conducted on a level playing field with standard equipment and licensed umpires

Overview and Goals

In this talk, Sony AI Zurich presents ACE, a physical AI system designed to compete with elite table tennis players. The core idea is to fuse a learned control policy with an optimization-based controller to guarantee safe operation while achieving the speeds needed to return challenging shots. The system emphasizes measuring ball spin and trajectory as well as the physics behind fast table tennis, highlighting the potential of AI to operate at human-scale time frames in the real world.

“Safety is a really big concern.” - Peter Dur

System Architecture and Hardware

ACE is built around a six degree-of-freedom robotic arm plus two additional spatial DOFs to maximize reach and speed. The architecture follows a perception–control–hardware loop, with multiple cameras around the court operating at 200 frames per second to locate the ball in 3D space, and an event-based camera paired with special optics to measure spin in real time. The team also uses ITTF-certified balls with a visible logo to infer spin axis and magnitude, enabling precise spin-aware planning. Latency from observation to commanded torque is reported as about 20 milliseconds, roughly ten times faster than human reflexes.

“From ball flying in space to our robot generating a torque, this takes us around 20 milliseconds, which is roughly 10 times faster than the reaction time of human table tennis players.” - Peter Dur

Learning Pipeline and Safety

The robot’s control policy is learned through reinforcement learning in a tailored simulation that mirrors real physics. This learned component is then integrated with an optimization-based controller that enforces safety constraints so the robot never collides with the table or itself. This hybrid learning–control approach enables the system to operate robustly in the real world, including unforeseen events such as a ball bouncing off the net.

“Reinforcement learning with an optimization based part lets our robot control run in the real world in situations that we haven't seen in simulation.” - Peter Dur

Evaluation: Playing Against Humans on a Level Field

To validate performance, ACE faced professional players under standard competition conditions: players used their own rackets, ITTF-certified balls, and licensed umpires observed the matches. The warm-up and game format were preserved to ensure a fair compare against elite human players. The results demonstrated that the physically embodied AI agent could outperform elite performers in a physical sport setting, signaling a new frontier for human–AI collaboration at fast time scales.

"This robot is really one of the first examples of a robot that can interact with humans at the fastest time scales." - Peter Dur