Start with the job, not the buzzword

Teams often begin an AI project by asking whether they need an agent. A better first question is what the system must reliably do. If the job is to answer questions from a controlled body of documents, retrieval-augmented generation is usually a cleaner starting point than a fully agentic workflow.

Agents become useful when the system must choose between tools, perform multiple dependent steps, inspect the result of one action before deciding the next action, or operate across several systems. That flexibility also adds more ways for the workflow to fail, so it should be earned by the problem.

When RAG is the right baseline

RAG works well when the value comes from finding the right evidence and keeping the answer grounded in that evidence.

  • Internal policy, product, support or engineering documentation
  • Technical manuals and standard operating procedures
  • Knowledge search where citations and traceability matter
  • Use cases where the answer can be produced from retrieved context without taking external actions

When agentic behavior becomes useful

An agent is more appropriate when the model must coordinate actions rather than simply generate an answer.

  • Research that requires searching, comparing and refining across multiple sources
  • Operational workflows that create tickets, update records or call business APIs
  • Tasks that need a planner-executor loop or explicit human approval between steps
  • Automation where tool results determine what the system should do next

The architecture I would usually recommend

For most business knowledge products, begin with strong ingestion, retrieval, permissions, reranking and evaluation. Add tool use only around workflows that clearly benefit from action-taking. This creates an architecture that is easier to test, cheaper to operate and simpler to explain to users.

The highest-leverage work is often not the final prompt. It is document quality, access control, retrieval quality, observability and a repeatable evaluation set that tells you whether the system is actually improving.