Gemini Robotics 2: A Hybrid Architecture for Physical AI
Google DeepMind's Gemini Robotics 2 introduces an intelligence layer for physical AI, enabling more dexterous, multi-step tasks through a combination of vision-language-action (VLA) and embodied reasoning (ER) models. The architecture highlights a hybrid approach, integrating on-device processing for real-time safety and critical decisions with cloud-based models for deeper reasoning and long-term learning, crucial for robust robotic systems in unpredictable real-world environments.
Read original on The New StackOverview of Gemini Robotics 2 Architecture
Gemini Robotics 2 is designed to enhance physical AI capabilities, allowing robots to perform complex, multi-step tasks with greater dexterity and full-body control. The system is composed of three core models: the Vision-Language-Action (VLA) model, its lightweight On-Device version, and the Embodied Reasoning (ER) model. This integrated approach aims to bridge the gap between AI models and real-world physical execution, addressing challenges like continuous replanning and adaptation to dynamic environments.
Key Components and Their Roles
- Vision-Language-Action (VLA) Model: Converts visual and linguistic inputs into precise motor control commands, enabling full-body dexterity and fine motor skills. This is essential for delicate tasks.
- Gemini Robotics On-Device 2: A lightweight VLA model that runs locally on the robot. This component is critical for privacy, reliability, and determinism, ensuring robots can react instantly to critical situations (e.g., avoiding obstacles) without relying on constant internet connectivity.
- Embodied Reasoning (ER) Model: Acts as the robot's high-level brain. It processes task instructions, communicates with humans, devises multi-step plans, and coordinates with the VLA model. It also enables self-correction and adaptation to unforeseen changes in the environment, addressing the 'long-horizon autonomy' challenge.
Hybrid Cloud-Edge Architecture for Robotics
The article emphasizes a hybrid architecture where critical, real-time perception, motion planning, and safety decisions occur at the edge (on-device). Deeper reasoning, long-term learning, and more complex cognitive tasks are handled by cloud-based models. This distributed approach ensures both responsiveness and advanced intelligence, showcasing a common system design pattern for IoT and AI-driven physical systems.
Scalability and Adaptability
Gemini Robotics 2 is designed for adaptability across various robot embodiments. Developers can quickly adapt skills to new systems with different shapes, sensors, and degrees of freedom with minimal examples and adaptation time. Furthermore, the system supports robot collaboration, allowing multiple intelligent humanoids to work together on complex workflows, highlighting aspects of distributed task execution and coordination within a physical system.
The introduction of ASIMOV-Agentic, a new benchmark for agentic safety orchestration and uncertainty resolution, underscores the importance of safety and human oversight in AI-driven robotic systems. This includes the agent's ability to refuse unsafe actions and flag situations for human intervention.