I was working on an agent with a complex prompt that requested numbered output and required the AI to do the counting as it went. Beyond 16 items, it faltered 50% of the time. By 18 items, the failure rate was 80%, and it would go no higher. To solve the problem, the number of items was precalculated and handed to the model to provide X items. This simple solution enabled the agent to produce 80 or more listed items with 0% errors.

In addition to informing the state, external input can relieve some of the processing burden and potential errors in AI results.

External input can relieve some of the processing burden and potential errors in AI results.

Being more fluid than canonical documents, external input clarifies the task, simplifies the processing, increases accuracy and provides real-time data. These inputs all contribute to the construction and maintenance of the current model state. These methods can be broadly grouped as processed inputs, retrieval, live system integration, and tool feedback.

The counting example in this article is a processed or calculated input. Examples include calculations, SQL, validation and parsing. This information is known or can be determined before executing a prompt.

Retrieval locates information that the system can use for reasoning. Examples are RAG, database queries, and searches.

Live systems provide ongoing or constantly updating inputs like email, CRM, calendars, APIs, sensors, etc. This goes beyond stored or retrievable knowledge to consider what is true “right now.”

Environment and system feedback is the base layer of information that encompasses what the possibilities are for task completion as well as evaluation and documentation of how the task output fits in the space. For instance, an agent changes code, the test suite fails, the agent uses the feedback to modify the code and repeat the process. The recorded failures and successes from this process can be made accessible to the AI’s state.

While the options are varied, the inputs all contribute to the refinement and focus of the state. Some define possibilities while others provide fences. This layer or approach is often more fluid than documentation. This is the layer where the system state can be continually updated and refined as the information is expanded, made more accurate, and ultimately aligned with the most current data.

External inputs can be used across the state, but are especially useful when used on the micro level, or individual prompts. They steer a single response with very explicit requirements before generation. Updates, new information, retrieval, and state reconciliation happen here at a more “real-time” level than documentation, providing current data and specificity. Together, they minimize the ambiguity that can result in drift, assignment misinterpretation, hallucinations and inconsistent or incompatible responses.