On the same incidents, Claude Fable 5.1 got 27 of 28 root cause diagnoses right querying Prometheus, Loki, and Tempo, at an estimated USD 72. Querying the same telemetry through a single GreptimeDB interface, it got 26 of 28 right at USD 37. Same model, same data, half the input.

Those are numbers from Agent RCA Bench. When people build LLM agents for root cause analysis (RCA), the model and prompt get most of the attention; we wanted to measure another variable: hold the model and prompt fixed, change only the query interface, and see what happens to the investigation. We had a stake in the answer. Our expectation was that a unified data model and query language would help, but we lacked the data to show it. The benchmark compares three interfaces: the native Prometheus, Loki, and Tempo APIs; GreptimeDB's SQL and PromQL; and GreptimeDB with a semantic layer of table semantics and entity relationships on top.

Six models, 168 end-to-end investigations per interface:

• Raw, the GreptimeDB query interface, produced 130 correct diagnoses versus 105 for Split, the three-pillar interfaces: 40% fewer wrong diagnoses. Cumulative input was about 48% lower, and estimated run costs about 45% lower.
• All six models used fewer input tokens; five produced more correct diagnoses.
• The semantic layer reduced the data returned in focused retrieval tasks. On service component and dependency-edge faults, correct diagnoses rose from 106 to 112, with four of the six models improving; on node faults they fell from 24 to 12. No end-to-end efficiency endpoint passed statistical correction.

The benchmark uses public data from OpenRCA, OpenRCA2, and RCA-100, and open-source features. The code, protocols, and artifacts are public. The six-model report has the full results.

Three interfaces, the same incidents

Arm	Interface available to the agent
Split	Tool wrappers over the native query APIs of Prometheus, Loki, and Tempo
Raw	Read-only SQL and PromQL over GreptimeDB's metric, log, and trace tables
Graph	Raw plus table-semantics search and Semantic Graph query tools

Each pair used the same model configuration, incident, source telemetry, and tool-call budget. The investigation method and diagnosis contract in the system prompt were byte-identical; interface notes varied with the tools. Raw versus Split compares complete interface bundles: storage, query language, and tool design change together. Corrections and execution differences are described at the end.

The end-to-end suite covers 14 incidents: ten from OpenRCA2 involving workload restarts, call-path delays, CPU saturation, and memory pressure; four from RCA-100 involving node CPU, memory, disk I/O, and host unavailability. Six models ran each incident twice per interface, giving 504 runs. Another 192 focused runs tested table discovery and direct-dependency retrieval.

Scoring uses no LLM judge. Correct counts cover all runs. Efficiency tests include only pairs where both diagnoses were correct, both cited a successful non-truncated query, neither had a runner error, and neither exhausted its budget. Eligible repetition-pair differences are reduced to a median per incident before cross-incident testing.

Correct diagnoses, input, and cost

The Fable numbers from the opening come from here: 27/28 correct under Split and 26/28 under Raw; cumulative input about 30.9 million versus 15.8 million tokens; estimated costs USD 72.37 versus USD 36.86.

Other models improved their correct counts as well:

Model	Split correct, of 28	Raw correct, of 28	Raw input reduction
GPT-5.6-Sol	13	22	53.00%
DeepSeek V4 Pro	13	18	21.97%
Claude Fable 5.1	27	26	48.78%
GLM-5.3	10	14	27.07%
Gemini 3.8 Flash	22	25	63.42%
Qwen3.8-Max	20	25	24.40%

These are descriptive totals, including incorrect investigations. Three of the 12 pre-specified Raw–Split endpoints passed Holm correction, all measuring input reduction: Fable, Gemini, and Qwen. Correction followed the separately frozen four-model and two-model cohorts (m = 8 and m = 4), not a pooled 12-endpoint family. The protocols were frozen in Git and bound to artifacts by hash, not registered with a third party.

Tool calls fell from 6,576 under Split to 6,129 under Raw, about 6.8%, and five of the six models reached correct diagnoses in fewer calls; cumulative input fell by about 48%. No model's tool-call endpoint passed correction.

Across 168 investigations per interface, estimated costs were USD 183.95–187.39 for Split and USD 98.37–100.50 for Raw: roughly USD 1.09–1.12 versus USD 0.59–0.60 per investigation. The intervals reflect missing Gemini cache-usage detail. The report gives the pricing and exchange-rate basis; the figures cover model API usage only.

What SQL did during an investigation

Cross-signal JOINs were rare. Across 336 Raw and Graph investigations, 192 successful JOIN calls paired spans within traces or combined tables, including metric tables. Only three joined different signal types.

Some queries aligned restart counts with readiness, or memory usage with limits, by time and entity. Others paired client and server spans. These queries returned comparisons the model could inspect directly. Without that work in the query, the model has to match records and compare values itself. Large results may also reappear in later requests as message history, accumulating input tokens.

The models mostly chose SQL: only 22 GreptimeDB-arm investigations successfully evaluated PromQL, versus 160 under Split. PromQL and LogQL support aggregation, and the tools included Tempo's TraceQL metrics endpoint. We have not quantified how much each query pattern contributes to the input gap. Case 007 below shows one concrete use of SQL pairing.

The semantic layer: fewer rows in retrieval, mixed end-to-end results

Graph adds search_table_semantics to find tables by telemetry concept and query_semantic_graph to retrieve entities, source-backed relationships, and windowed statistics, plus a snapshot of semantic coverage. All 168 Graph investigations successfully used the latter, making 369 calls. The semantic layer documentation describes the interface.

In focused retrieval, rows returned fell in all 35 eligible model–incident results for table discovery. For dependency retrieval, 11 of 12 reduced both rows and calls.

End to end, Graph got 124 correct and Raw 130; no Graph–Raw efficiency endpoint passed correction. GLM improved from 14 to 16 correct. DeepSeek stayed at 18, with fewer calls in eight and fewer rows in seven of its ten eligible incidents. GPT and Fable showed no consistent benefit. The semantic tools also added input: Fable's median discovery input delta was +7,129 tokens.

The fault-type breakdown was more striking:

Fault group	Raw correct	Graph correct
Service component and dependency-edge faults	106/120	112/120
Infrastructure node faults	24/48	12/48

This is a post-measurement breakdown. The groups come from OpenRCA2 and RCA-100 respectively, so dataset and fault level coincide and this split is descriptive only.

The node cases did have deployment relationships. Reviewing their 96 Raw/Graph trajectories showed that 39 of 48 Graph investigations received runs_on and part_of relationships. The node example below shows what the models did with them.

How models investigated the same incident

I picked two incidents and checked the queries, results, final diagnoses, and repeated runs. These are post-hoc observations about specific investigations.

accounting's error logs, or shipping's slow calls

Case 007 comes from OpenTelemetry Demo. shipping calls quote; accounting is another service. The source injection record identifies a NetworkDelay in the shipping-to-quote direction. The agent cannot see that record.

Under Split, GPT started with global ERROR logs and retrieved eight Order parsing failed: entries from accounting. It kept investigating that service's traces, PostgreSQL spans, and parse failures, ultimately blaming a data semantics error in accounting. The errors were real, but already existed before the alert and did not explain the shipping-to-quote delay.

Fable also used Split. It summarized errors and latency, grouped shipping spans by operation and client/server role, then retrieved one slow request and one pre-alert request:

Request	Shipping client span	Matching quote server span
Before the alert	About 1.51 ms	About 0.29 ms
Slow request	About 1,692 ms	About 0.30 ms

The client waited over a second longer while server processing barely changed. Fable checked CPU, memory, restarts, and logs, then correctly identified call-path delay on shipping → quote. It used 24 calls; GPT used 32. In the repeat, Fable was right again and GPT blamed accounting again, this time for a memory leak.

Under Raw, GPT took a different path. After inspecting the schema, it grouped span durations by service, operation, and role, compared time windows, and investigated shipping and quote. In its second run, a SQL self-join paired the spans by trace ID and parent–child relationship, then aggregated them into two rows: baseline and anomalous, each with client and server durations.

GPT got both Raw runs right, using 22 and 17 calls. Fable also got both right, using 20 and 19 calls versus 24 and 28 under Split. GPT's first Raw run used no JOIN and still succeeded. What JOIN did here was move the comparison into the query; other factors also shaped the outcome.

The node CPU data arrived; the diagnosis still named a service

Case 011 is node CPU saturation. Under Raw, Fable compared service and operation latency across windows, inspected deployment locations, paired spans, and queried node resources. The faulty node's CPU rose from about 6% to nearly 100%. After checking calls involving that node and its status, Fable correctly diagnosed node CPU saturation in 30 tool calls.

Gemini reached the same node. Its 28th Raw call ranked nodes by peak CPU; its 29th retrieved full CPU time series for two candidates without truncation. It then continued investigating recommendation's logs and traces and blamed that service's latency, finishing incorrectly after 47 calls. Its repeat also took 47 calls, but was correct. Fable was correct in both runs.

Gemini did better under Split: both runs were correct, at 43 and 44 calls. In the first, an all-node CPU query was truncated, so it later retrieved the faulty node's series separately, checked its Pods and traces, and diagnosed node CPU saturation.

Adding the semantic layer changed Fable's result too. Both Graph runs blamed checkout latency. In the second, query 21 had already returned the faulty node's CPU rise. Fable mentioned it in the final answer but ruled it out: checkout's slow calls appeared earlier, it reasoned, and checkout ran on another node with normal CPU.

Both cases made us look more closely at how tools influence the direction of an investigation. Under Split, GPT pursued accounting errors in both runs; under Raw, it compared shipping and quote durations in both. Fable found the faulty node under Raw, but with the semantic layer it focused on checkout latency in both runs.

What a tool returns may influence what the model examines next. Once it takes a direction, later queries add more information about it, and the model may increasingly interpret new evidence through that explanation. The node CPU anomaly had been retrieved, yet it was still ruled out.

We want to test whether query interfaces also shape the model's investigative attention: could making one kind of information easier to obtain lead the model to spend too long on one kind of explanation?

The four-model and two-model artifacts record the queries and diagnoses by case, model, and repetition.

What I would test in an RCA agent

When building agents, we often treat a successful tool call with a complete result as a normal execution. Several incorrect investigations here met both conditions. The model even found the faulty node and still chose another answer. Checking whether tool calls succeeded would not catch that.

What I want to inspect is why the agent chose its next query after each tool result. GPT found accounting errors and kept querying accounting. Fable focused on checkout, then used checkout's timing and deployment location to rule out the node fault. Later queries had a rationale, but the initial direction might not explain the alert. An evaluation that checks individual tool calls and final answers can miss what happened in between.

This also changes how I would design tools. One interface returns the slowest services; another returns how much the same operation changed before and during the incident. Even with the same underlying data, these results ask the model to make different judgments. Beyond field coverage and token count, I would ask: what is the model likely to query next after reading this result? Could this tool keep sending it toward the same kind of explanation?

I would start by adding a check to the evaluation: for each incorrect run, did the key evidence ever reach the model? If not, inspect the tools and queries. If it did, look at where it was ignored or ruled out. Test the corresponding fixes separately. Otherwise, we may give an agent that already found the evidence but interpreted it incorrectly more tools and a longer context, then pay more for the same wrong answer.

Corrections and limits

The original four-model cohort and the later Qwen/Gemini extension froze their protocols separately. Before publication, we found two defects: Tempo retention allowed historical traces to expire during measurement, and our Split wrappers repeated label names in every record, adding overhead absent from SQL's column headers.

We fixed retention and changed the three Split tools to put repeated field names in a header and records in arrays, preserving sample counts and truncation rules. All 168 Split runs were rerun; this article uses those results.

To control cost, Raw and Graph were retained. Their limited PromQL use still carries repeated-label overhead, estimated at 0.71% and 0.21% of their input. This makes GreptimeDB look more expensive. Holding queries and trajectories fixed, removing it would widen the Split–Raw input gap by about 0.8%. That is an offline estimate of input encoding.

The retained and replacement runs occurred at different times. Time effects and individual corrections are confounded; tool-description changes, rate limits, and temporary connection-failure retries are documented in the full report.

There are 14 end-to-end incidents, with only 2–14 eligible for each efficiency test. Model rankings describe the frozen configurations on this task set.

Greptime sponsors and maintains the benchmark. The repository provides commands to reproduce JSON and HTML from published artifacts without calling a model or starting GreptimeDB, and to compare the outputs byte for byte.