Using Interrogatory LLMs for System Design Context and Requirements Gathering

The initial phases of system design often require a significant amount of context gathering. This includes understanding user requirements, implementation guidelines, and interactions with external systems. Traditionally, this is a human-intensive process, involving writing detailed specifications and conducting interviews. Martin Fowler's 'Interrogatory LLM' concept proposes an alternative where an LLM acts as an interviewer to extract this crucial information.

LLMs for Context Generation

An Interrogatory LLM can be prompted to ask a human user a series of questions to build a comprehensive context report. This report can then serve as input for subsequent design stages, potentially fed to another LLM for further processing or used directly by human designers. A key insight from Harper Reed's approach, cited in the article, is to instruct the LLM to ask only one question at a time, which can improve the quality and structure of the gathered information.

LLMs for Document Validation and Review

Beyond generating new context, Interrogatory LLMs can also be used to validate existing documentation, such as software specifications. Instead of asking a human expert to passively read and review a document, the LLM can interview the expert, prompting them with questions derived from the document to assess its accuracy and completeness. This interactive approach can be more effective, especially for poorly written or complex specifications, as it leverages conversation to clarify ambiguities.

The article suggests that while some may find the 'AI-writing' style unappealing, the ability to extract critical information from individuals who struggle with traditional writing methods outweighs the stylistic concerns. This technique has broad applicability beyond LLM-specific workflows, offering a novel method for knowledge transfer within engineering teams and for initial system conceptualization.