Architecting Resilient LLM Agents with Dual-Process (System 1 & 2) Thinking

The core challenge in building reliable LLM agents is their probabilistic nature, often leading to "hallucinations" or unexpected behavior in edge cases. The article argues against "vibe coding"—relying solely on raw LLM output—and instead proposes a structured architectural approach to integrate LLMs into robust applications.

Dual-Process Architecture for LLM Agents

Inspired by Daniel Kahneman's System 1 and System 2 cognitive models, this architecture divides responsibilities to leverage the strengths of both LLMs and traditional deterministic code:

• System 1 (The LLM): Handles tasks requiring natural language understanding, creative generation, and pattern matching. It's fast and intuitive but prone to errors.
• System 2 (The Orchestrator): Implemented with deterministic code, it provides validation, state management, tool execution, and critical "sanity checks." It's slower and logical, acting as a guardrail.

The Plan-Execute-Verify Cycle

This hybrid loop dictates the interaction between System 1 and System 2, ensuring that LLM outputs are systematically checked and controlled:

1. System 1 proposes a plan or action based on its understanding.
2. System 2 rigorously parses and validates the proposed plan against predefined schemas and business rules.
3. System 2 executes necessary external tools or functions in a controlled, sandboxed environment.
4. System 1 reviews the results for "logical drift" or unexpected outcomes, potentially triggering a refinement or retry.

Implementing the Verification Gate (System 2 Component)

The Verification Gate is a critical System 2 component responsible for ensuring the LLM's output meets technical criteria before execution. This involves several layers of checks:

• Strict Schema Validation: Ensures the LLM's output conforms to expected data structures (e.g., JSON schema).
• Security Check (Tool Whitelisting): Prevents the agent from executing unauthorized or potentially malicious tools.
• Logic Check (Parameter Integrity): Enforces business logic or performance best practices, such as adding `LIMIT` clauses to database queries.
• Error Handling & Fallback: If checks fail, System 2 can force a retry, halt execution, or invoke human intervention.

interface AgentAction { tool: string; params: Record<string, any>; reasoning: string; }

async function system2VerificationGate(
  rawOutput: string,
  allowedTools: string[]
): Promise<AgentAction> {
  try {
    // 1. Strict Schema Validation
    const action: AgentAction = JSON.parse(rawOutput);

    // 2. Security Check: Tool Whitelisting
    if (!allowedTools.includes(action.tool)) {
      throw new Error(`Security Violation: Unauthorized tool '${action.tool}'`);
    }

    // 3. Logic Check: Parameter Integrity
    if (action.tool === 'database_query' && !action.params.query.includes('LIMIT')) {
      console.warn("Performance Risk: Query missing LIMIT. Injecting safe constraint...");
      action.params.query += " LIMIT 100";
    }
    return action;
  } catch (e) {
    // Fallback: System 2 forces a retry or halts execution
    throw new Error(`System 2 Rejected Output: ${e.message}`);
  }
}

Mitigating Common Pitfalls in LLM Agent Design

• Token Exhaustion: System 2 should monitor the LLM's reasoning length and terminate sessions to prevent infinite loops.
• State Drift: Instead of passing the entire chat history, use a Vector Database to store and retrieve only contextually relevant historical facts, minimizing context window bloat and improving accuracy over long interactions.