Uber's uSpec: Automating Design Documentation with Local AI Agents
Uber developed uSpec, an agentic system that automates the generation of detailed component design specifications from visual designs. This system leverages local AI agents, the Figma Console Model Context Protocol (MCP), and an internal GenAI Gateway to ensure data privacy and integrate with Uber's AI platform. By keeping the processing local and encoding domain expertise in 'Agent Skills,' uSpec significantly reduces documentation time and ensures consistency across diverse platform stacks and accessibility frameworks.
Read original on InfoQ ArchitectureOverview of uSpec Architecture
Uber's uSpec is an innovative system designed to bridge the gap between visual design artifacts and detailed technical specifications. It operates as a "Visual-to-Technical Spec" compiler, drastically reducing the manual effort and time required for design documentation. The core architectural decision focuses on local processing to maintain data privacy and security, a critical concern for proprietary design information.
Key Components and Data Flow
- AI Agent in Cursor IDE: Initiates the process, connecting to a local Figma Desktop session.
- WebSocket Bridge: Facilitates communication between the AI agent and the local Figma application.
- Figma Console Model Context Protocol (MCP): An open-source protocol used by the agent to "crawl" the component tree, extracting design data like tokens and variant axes directly from the local Figma instance.
- Agent Skills: Structured Markdown files encoding Uber's domain-specific expertise, such as platform-specific accessibility mappings (VoiceOver, TalkBack, ARIA) and density logic for scaling padding and typography across different UI frameworks (SwiftUI, React, Android Compose). These skills are crucial for translating visual properties into precise technical contracts.
- GenAI Gateway: A Go-based proxy that acts as a secure intermediary for all agentic requests. It mirrors the OpenAI API and provides essential functions like PII Redaction, scrubbing internal identifiers before requests are sent to external Large Language Models (LLMs) such as Claude 3.5 or GPT-4o.
- Michelangelo: Uber's centralized AI platform, where uSpec is integrated for broader AI ecosystem access and management.
Security and Data Privacy
A cornerstone of uSpec's design is its commitment to data privacy. By keeping the pipeline local and leveraging the Figma Console MCP, proprietary design data never leaves the developer's machine or Uber's local network. This crucial architectural decision enables the use of AI-assisted documentation in a sensitive enterprise environment, mitigating risks associated with cloud-based API calls to external AI services.
Addressing Combinatorial Complexity
Uber manages designs across seven platform stacks and three accessibility frameworks, leading to a significant combinatorial workload for any design change. uSpec directly addresses this by automating the translation of a single visual design into numerous detailed technical specifications tailored for each stack and accessibility requirement. This automation ensures consistency and reduces the burden of manual translation, which previously took weeks.
The article also touches upon a broader industry trend where individual developers are increasingly using agentic workflows to generate production-ready code directly from requirements, potentially bypassing traditional visual design tools like Figma. However, enterprises still value Figma as a "visual source of truth" for cross-functional alignment, suggesting a hybrid future for design and development workflows.