Designing Research Workflows with Bounded Web Search and Fetch for SaaS AI Agents

Building AI assistants that interact with the web requires careful architectural design, particularly concerning data provenance, security, and resource management. Unrestricted web access can lead to noisy prompts, security vulnerabilities, and unpredictable agent behavior. This article outlines a structured approach using Cloudflare's Agents SDK to create bounded, auditable web research workflows for SaaS AI agents.

Controlling Web Access with Bounded Search and Fetch

The recommended pattern for SaaS assistants involves a tightly controlled sequence: a focused query, a limited web search, selective page fetching, and presenting source provenance to the user. This approach ensures that the agent's context is manageable and reviewable. The Cloudflare Agents SDK provides `web_search` to return structured results (titles, URLs, descriptions) and `web_fetch` to retrieve content from selected pages within defined limits.

• Deliberate Reading List: Instead of broad queries, guide the model to state a specific question, then issue a search with a modest result limit. Prioritize authoritative sources (e.g., vendor documentation) and allow for human or automated inspection of search results before fetching.
• Bounded Fetch Operation: Configure `web_fetch` with explicit policies for allowed hosts (allowlisting), blocked hosts, response byte limits, timeouts, and redirect handling. This prevents unbounded resource consumption or access to unintended domains. Implement content windowing to fetch long pages incrementally, requesting more only when necessary to answer the question.
• Untrusted Input: Treat all retrieved page text as untrusted. Architecturally, this means keeping fetched content separate from trusted system instructions. Use clearly marked data fields for web content and instruct the model to summarize it as evidence, not to obey commands found within it. This is crucial for mitigating prompt injection risks and maintaining control over the agent's actions.

Architectural Considerations for Reviewability and Auditability

To make the AI agent's research workflow reliable and auditable, the system design must account for logging, error handling, and user transparency. For instance, if a fetch operation fails due to a timeout or blocked host, the application must preserve these structured failure details and present a clear status to the user, rather than silently producing an empty "source."

1. Comprehensive Logging: Log critical steps including the query, selected source URLs, final URLs (after redirects), response status, content type, and whether fetching completed or required continuations. This supports privacy and retention policies and allows retrospective analysis of sources used.
2. User Feedback & Control: Design the UI to show chosen URLs, allow users to remove irrelevant results, and present a compact evidence panel with read status and citations. This transparency helps users understand how an answer was derived and enables them to verify sources.
3. Policy Management: Provide a consolidated interface for team owners to review and configure host policies, byte limits, timeouts, and redirect rules. This transforms a basic capability into an auditable operational feature.