Designing Authorization for AI Context Assembly in Distributed Systems
This article highlights a critical system design challenge: ensuring proper authorization during the 'context assembly' phase for AI models, especially when integrating data from multiple disparate sources. It argues against late-stage filtering, advocating for structural authorization where permissions dictate how context is assembled, preventing unauthorized data from ever reaching the model. The discussion covers architectural decisions by major cloud vendors and the inherent risks of delayed authorization checks, emphasizing the need for robust identity resolution across diverse enterprise data ecosystems.
Read original on The New StackThe Challenge of Context Assembly and Authorization
The core problem addressed is how to prevent AI models from accessing or summarizing information that a user is not authorized to see, especially when the model's context is assembled from various enterprise data sources (e.g., Salesforce, Slack, Google Drive). The article criticizes the common practice of applying permissions as a 'filter' *after* the context has been assembled, arguing this is a reactive approach that can lead to sensitive data leakage.
Authorization at the Wrong Stage
Applying authorization *after* data has been retrieved, reranked, summarized, or cached allows unauthorized content to cross a critical boundary, potentially exposing it to prompt injection attacks or bugs in filtering logic. The model may have already 'read' the sensitive information.
Structural Authorization: Permissions as a Property of Assembly
Instead of post-assembly filtering, the article proposes that permissions should be a *structural property* of the context assembly process itself. This means authorization should dictate *how* the context is composed for a specific identity, ensuring that only authorized information is included from the outset. This pre-emptive approach ensures the model never encounters data it shouldn't.
- Early Identity Resolution: Identity must be resolved at the 'assembly' stage, before data is fed to the model.
- Vendor Approaches: Major platforms like AWS and Microsoft are moving towards identity-aware retrieval, where permissions govern the data graph and are checked at query time.
- Distributed Permissions: A key complexity arises when context spans multiple systems (e.g., S3, Salesforce, Slack), each with its own permission model. The 'assembly' layer must orchestrate authorization across these disparate systems, not act as a gatekeeper but as an orchestrator that passes user context for downstream enforcement.
Key Design Considerations for Authorization in AI Context Retrieval
Building a secure AI context retrieval system requires addressing several critical design questions to ensure robust authorization:
- Authorization Timing: Is identity resolved during context assembly (proactive) or after retrieval (reactive)? The former is preferred.
- Context Scope: How much context resides outside centralized data lakes (e.g., in SaaS applications) where standard IAM policies may not apply directly?
- Staleness Window: What is the maximum acceptable delay for permission changes to propagate across all integrated systems?
- Granularity of Checks: Can authorization be re-checked at every hop or transition between components, or only at the entry point? Re-checking at every hop improves security composition.
Principle of Least Privilege at Context Assembly
Ensure that the AI model only ever 'sees' content that the specific querying user is authorized to access. This requires integrating and enforcing fine-grained permissions at the earliest possible stage in the data flow to the model, effectively making permissions a blueprint for context creation.