When not to build a knowledge graph

Short answer

Skip the knowledge graph when your questions are about single records, single documents or numbers in a table. A database, a search index or plain vector RAG will answer those faster and for less money. A graph earns its cost only in three cases: the answer depends on how things connect across several steps, every answer must show where it came from, or the domain is a dense web of many entity types whose cross-references a table or a list of facts would lose. If none of those holds, don't build one.

Five signs you don't need a graph

1. Your questions are lookups. "What was revenue in 2023?", "Which contract expires next month?", "What does the manual say about error 42?" Each answer lives in one row or one passage. A table or a search index returns it directly. Putting a graph in front of it adds a data model to maintain and changes nothing about the answer.

2. The relationships you need are joins you can already write. If the useful connections are one or two steps deep and their shape does not change (customer to order to product), SQL handles them well and your team already knows how to run it. A graph starts to pay off when the depth is unknown in advance, as in "does any product contain material from this supplier, however far upstream".

3. Nobody can name the questions. If you cannot write down five to ten concrete questions the graph must answer, you are not ready to model anything. The ontology is shaped by those questions. Without them you get a big diagram and no way to tell whether it works.

4. Nobody can check the answers. A graph pilot needs a person from the domain who can say whether an answer is correct, complete and useful. Without that person you can measure speed and cost, but you cannot show the graph is right.

5. The data can't be connected. Graphs link records through shared identifiers, names, codes and dates. If your sources have none of those, or the only way to link them is guessing, the graph will be exactly as unreliable as the guesses. Fix identifiers first. That work helps whatever you build next.

What to use instead

If your questions are mostly	Start with
Numbers, filters, aggregates	The database you have, or a well-modelled SQL view
"Find the passage that says X"	Full-text search, then vector RAG if the wording varies
Summaries of individual documents	An LLM over the document, no retrieval layer at all
A user's recent chat history (agent memory)	A structured store of facts with validity dates

The last row needs a note. Two write-ups from August 2026 put graph-based agent memory against simpler stores. In Ping Lin's benchmark of four memory designs, ingesting one long user history into an LLM-extracted graph cost about $14 against about $0.03 for an embedding-based store. At retrieval the hosted graph store (Zep Cloud) returned a median of 21.5k characters against 4.0k for a structured store of dated facts, more than five times as much, and still scored lower. In the second write-up, a team's temporal graph passed its static evals and then failed 41% of "what was true at time T" questions. The cause was a query that sorted by the newest date instead of filtering on the question's time, and the team shipped the graph once the fix passed. That second case is a lesson about testing time, whatever the store. The first one supports the table: for remembering one user's conversation, a graph is usually more machinery than the job needs. Neither study tested multi-hop questions over a connected domain, which is where a graph is supposed to pay off. I went through both in Do Knowledge Graphs Fail as Agent Memory?

When a graph does earn its cost

Multi-hop questions whose depth you don't know in advance. Supply-chain exposure, ownership chains, "which uses are left uncovered if this substance is withdrawn". In a graph each of these is one variable-length pattern. In SQL they need recursive queries, which get harder to write, read and keep fast as the paths get longer and cross more tables.

Answers that must carry their evidence. In regulated work the question after "what" is "how do you know". A graph can store the source, date and status of each fact on the relationship itself, so the answer and its trail come back in the same query. The guide on provenance for auditable AI answers shows how.

A densely connected domain. Crop protection is a good example: active substances, products, crops, pests, resistance groups, residue limits and country-by-country approvals, all cross-referenced, much of it already coded in published standards such as EPPO codes. Turning that into a flat list of facts drops the cross-references, and they carry most of the value.

How to decide in an afternoon

1. Write down ten real questions from the people who will use the system. Use questions they asked last month, not questions they might ask one day.
2. Mark each one: does the answer need more than one hop between different kinds of things?
3. Mark each one again: does the user need to see where the answer came from?
4. Check the sources: do they share identifiers you can join on?

If fewer than three of the ten need multiple hops or a traceable answer, use the simpler tool from the table above. If three or more do and the sources connect, a small, scoped pilot is worth considering, and the case gets stronger the more of the ten qualify. The pilot readiness checklist covers the same ground in eight questions and gives a next step for each score.

Example from a working graph

The 10-K graph I built from 25 annual reports shows both sides. Revenue by segment comes from the filings' structured XBRL data and never touches a language model. For those questions the graph adds nothing a table would not give you, and a table would be simpler. The graph earns its place on the connected layer: who names whom as a competitor (239 declared relationships, and only one pair, Alphabet and Microsoft, name each other), who depends on TSMC, which risk topics are new in the latest year. Each of those answers carries the filing and section it came from.

The plant-protection register graph is the other case. When the EU decided in 2025 not to renew the approval of flufenacet (Regulation (EU) 2025/910), one query showed that it was in 45 products registered in Poland, covering 226 crop-and-pest uses, and that 36 of those uses would be left with no other registered product, all in cereals. Getting that from spreadsheets means joining products, substances, crops and pests by hand. In the graph it is one traversal.