The short answer: during the seven full Pacific Time days ending September 3, 2026, ChatDiagram recorded 4,870 unique requests in which its router said a diagram should be delivered. Of those, 3,294 reached the server's accepted-artifact terminal state and 1,576 did not—a 32.4% delivery-failure rate. The failed requests ended with no artifact, empty model output, or an artifact rejected by deterministic validation.
This is finally a rate with a shared denominator: every row has one unique request ID, the same expectsArtifact=true eligibility rule, and one terminal outcome. It measures whether ChatDiagram accepted an artifact for the promised request. It still does not measure whether the diagram was visible in the browser, readable after export, or technically correct. An electrical single-line diagram can pass syntax validation while showing the wrong source path, protective-device data, or neutral bond.
Observation window: August 28, 2026 00:00 through September 4, 2026 00:00 America/Los_Angeles—seven complete days, excluding the partial query day. The query ran September 4, 2026 at 03:09 Pacific Time. It covered 2,519 authenticated accounts and excluded one known internal operator account. No email, prompt, diagram content, title, error text, or customer identifier was returned.
What the 32.4% measures
The eligible population is not every chat message and not every saved diagram. It contains only server-side generation records where the router set expectsArtifact=true: ChatDiagram had committed to producing a diagram in that request. The terminal kind then supplies the outcome.
| Terminal outcome | Requests | Share | What the log establishes |
|---|---|---|---|
| Accepted artifact | 3,294 | 67.6% | A diagram payload passed the server's terminal acceptance path |
| No artifact | 876 | 18.0% | The request promised a diagram but produced no artifact payload |
| Empty output | 503 | 10.3% | The model returned no usable output |
| Invalid artifact | 197 | 4.0% | An artifact was present but deterministic validation rejected it |
The 1,576 unsuccessful terminal outcomes touched 1,010 of the 2,519 accounts in the eligible cohort. That is an affected-account count, not 1,576 different people: one account can make several requests, and one person can hold several accounts.
The previous equal seven-day window contained 3,862 promised requests, of which 670 failed delivery, or 17.3%. The rise to 32.4% is an operational warning, not a causal finding. It was not produced by one isolated day: each of the seven dates in the current window exceeded the previous window's aggregate failure rate, with daily rates ranging from 27.6% to 39.6%. Traffic mix, model routing, product releases, and diagram-type mix all changed between the windows; this query does not identify which change caused the difference.
What can be said about single-line diagrams
Among the 279 requests whose terminal metadata named the single-line diagram engine, 273 reached an accepted artifact and six did not. That 97.8% accepted share is useful for checking the post-routing SLD path, but it is not a complete SLD success rate.
The reason is a classification boundary. In the same window, 1,382 failed requests had no terminal engine value. A request that fails before an engine is named cannot be assigned to SLD, flowchart, floor plan, or another type without reading customer input or adding an earlier privacy-safe intended-type field. This study did neither. It therefore refuses to move those untyped failures into or out of the SLD denominator.
The honest conclusion is narrower: once a request was logged as SLD, the accepted-artifact path usually completed in this window. ChatDiagram still lacks the metadata needed to measure all requests that intended to become SLDs, including failures before engine assignment.
Why the older render tables could not produce this rate
The first version of this study compared saved artifacts with client render-error events. In the 30 full days ending August 20, 685 authenticated accounts saved 1,140 SLD artifacts, while 43 authenticated accounts generated 58 SLD render-error events. Those were useful workload counts, but the tables represented different lifecycle stages and had no shared attempt ID.
The new request_id field fixes that specific denominator problem for the server's delivery stage. Google SRE frames a service indicator as good events divided by total eligible events from the same population. OpenTelemetry similarly recommends recording one operation population with errors represented as outcomes. The current query now satisfies that contract for accepted artifact delivery.
It does not make the older client log obsolete. A payload can be accepted by the server and still fail in the browser renderer. Mermaid's syntax reference is one example of the parsing layer that sits after generation. Client rendering, saving, export, and technical review each need their own terminal measure rather than being folded into one reassuring number.
The reliability ladder a technical diagram must climb
Use four separate gates instead of calling every accepted payload “successful”:
- Delivery: did a promised request produce an artifact the server accepted? This study measures that gate.
- Renderability: did the browser parse and display the artifact without truncation or a hard renderer error?
- Visual usability: are labels legible, connections visible, and the exported page free of clipping and overlap?
- Technical correctness: do topology, ratings, symbols, states, and relationships agree with the source information and applicable authority?
Passing a later gate implies the earlier ones were crossed for that reviewed artifact, but it does not make the gates interchangeable. Automated parsers are appropriate for XML completeness, schema validity, unknown node references, or impossible syntax. A domain reviewer is still needed for questions such as whether a generator neutral should be switched, whether an interlock is fail-safe, or whether a P&ID tag describes the intended instrument.
Worked review: one SLD, four closure records
Suppose an AI-generated one-line shows a utility source, generator, automatic transfer switch, 480-volt bus, transformer, and 208-volt panel.
| Gate | Evidence | Close only when |
|---|---|---|
| Delivery | Terminal generation record | The promised request has one accepted artifact outcome |
| Renderability | Browser render and parser result | The complete diagram renders with no hard error |
| Visual usability | Desktop, mobile, and export inspection | Labels, lines, and callouts remain readable without clipping |
| Technical correctness | Current one-line checklist, calculations, equipment data, and adopted code | Every source path, rating, fault-duty value, and bond is verified |
An accepted artifact closes only the first row. The reviewer should record the evidence for each remaining row rather than letting a green generation status stand in for the complete engineering review.
Reproduction query
The production query selected timestamps, opaque user IDs, terminal kinds, request IDs, and the three booleans or labels required for aggregation. It evaluated email only inside the internal-account exclusion predicate. The displayed result was grouped before publication.
select
count(*) as promised_requests,
count(distinct metadata->>'request_id') as request_ids,
count(*) filter (where kind = 'chat') as accepted_artifacts,
count(*) filter (where kind in ('chat_no_artifact', 'chat_empty'))
as requests_without_accepted_artifact,
count(*) filter (where metadata->>'failure_reason' = 'diagram_no_artifact')
as missing_artifact,
count(*) filter (where metadata->>'failure_reason' = 'empty_output')
as empty_output,
count(*) filter (where metadata->>'failure_reason' = 'diagram_invalid_artifact')
as invalid_artifact,
count(distinct ai_usage.user_id) as accounts,
count(distinct ai_usage.user_id) filter (
where kind in ('chat_no_artifact', 'chat_empty')
) as affected_accounts
from public.ai_usage
where created_at >= '2026-08-28 00:00 America/Los_Angeles'::timestamptz
and created_at < '2026-09-04 00:00 America/Los_Angeles'::timestamptz
and coalesce((metadata->>'expectsArtifact')::boolean, false) = true
and not exists (
select 1 from public.users
where users.id = ai_usage.user_id
and lower(users.email) = :excluded_internal_email
);All 4,870 eligible rows had a non-null request ID, and all 4,870 request IDs were distinct. That one-to-one check is required before treating rows as requests. The daily check used the same eligibility and exclusion predicates, grouping only by Pacific Time calendar date.
Limitations and next step
- “Accepted” is a server state, not proof that the browser displayed or saved the artifact.
- The terminal outcome does not establish visual or technical correctness.
- Failed requests without an engine value cannot be attributed to SLD or another diagram type.
- The account cohort excludes the named internal account, but an account is only a privacy-safe proxy for a person.
- The seven-day comparison is descriptive; it does not isolate the effect of a model or release.
The next instrumentation step is to carry the same opaque request ID through client render, save, and export outcomes, and to add a privacy-safe intended-type field before generation begins. Until then, use 32.4% as the accepted-artifact delivery SLI for this exact window—not as a universal AI diagram failure rate.
For an actual electrical project, use ChatDiagram's AI single-line diagram generator for an unissued draft, then close the separate topology, ratings, fault-duty, grounding, and document-control checks before anyone relies on it.