When the Machine Becomes a Witness

2026-09-18 · 4,314 words · Singular Grit Substack · View on Substack

Why an authentic AI audit log can still be incomplete, and why silence in a trace should never be mistaken for proof that nothing happened

Keywords: AI agents; electronic evidence; audit logs; provenance; authenticity; completeness; legal evidence; explainability; contestability

Artificial-intelligence systems are moving from answering questions to taking actions. An agent can retrieve a customer record, consult persistent memory, call an external service, delegate a task, modify a database, send an instruction and return a short sentence stating that the work is complete. The visible output may be only the last line in a distributed act, which means that the evidential problem begins before anyone asks whether the output is accurate. The first question is what, exactly, the machine record is capable of proving.

This distinction matters because digital evidence often acquires an aura of precision that human recollection does not. A timestamp, a hash, a signed export or a structured JSON trace looks objective, and in one sense it may be: the bytes can be preserved exactly and the time of capture can be documented. Yet a perfectly preserved record can still omit the event class that matters, preserve a transformed summary rather than source data, or stop at the boundary of a third-party provider. Authentication protects a record against one category of challenge; it does not establish that the record contains everything needed for the inference being made.

The problem becomes sharper with autonomous agents because the system is not one program executing one closed sequence on one machine. Modern agent architectures combine models, memory, retrieval, tools, orchestration layers, provider infrastructure and external applications, while work on ReAct, Reflexion, MemGPT and related systems shows how behaviour emerges through repeated interactions rather than a single response (Yao et al., 2023; Shinn et al., 2023; Packer et al., 2024). Provenance research is therefore moving towards execution graphs that record instructions, observations, memory, tool use and effects rather than treating the final answer as the whole event (Souza et al., 2025; Wang et al., 2026). The legal significance follows directly: a machine can produce several authentic records that are authoritative for different propositions and inadequate for others.

The output is not the act

A short machine output often proves less than readers assume. Suppose an automated lender records only loan rejected; that record may reliably establish that a rejection was communicated, but it does not establish which customer record was retrieved, which policy version was applied, whether a memory object influenced the decision, whether an external search occurred, or whether an override was authorised. The output is therefore evidence of an outcome, not automatically evidence of the path that produced it. Treating the output as the act collapses several distinct factual questions into one convenient file.

Agent systems make that collapse particularly dangerous because the relevant act can cross multiple technical and organisational boundaries. A parent agent can issue a request to a sub-agent, the sub-agent can call a provider API, and the provider can alter an external account even though the parent application records only a success message. The parent trace can be complete for the fact of delegation while incomplete for the external effect, just as a bank instruction can be proved without proving that settlement occurred. The evidential unit must therefore be defined by the proposition in dispute rather than by whichever log is easiest to export.

This proposition-specific view is familiar outside artificial intelligence. A sent-email record can establish that a message left one system, while a delivery receipt can speak to receipt and a downstream business record can establish that the recipient acted on it. The mistake with AI is to assume that a rich execution trace changes the logic of proof merely because it contains more fields. More detail can improve evidence, but volume does not convert one event class into another.

Authenticity and completeness are different questions

Authenticity asks whether the thing offered is what the proponent says it is. If a log export was created by a specified system, preserved without alteration and tied to a known source, authentication can be strong even when the logging configuration excluded a critical category of events. That is why cryptographic integrity, chain of custody and digital signatures should be treated as necessary controls rather than magical proof of completeness. A hash can tell us that captured bytes have not changed; it cannot tell us whether the system captured the relevant event in the first place.

Completeness asks a different question: whether the surviving record is adequate for the inference sought. A trace that contains every tool call recorded by an application may still omit provider-side retries, external state changes, expired temporary logs, suppressed memory reads or actions performed by a delegated service. The omission can be lawful, accidental, architectural or deliberate, and those possibilities cannot be distinguished merely by inspecting the surviving file. Silence in the record is therefore ambiguous until the capture boundary is known.

This point is easy to miss because digital systems often fail in ways that preserve a tidy record. A disabled logger produces no malformed event; it produces no event at all, which can make the trace look cleaner rather than more suspicious. A pipeline that stores only successful calls can create a sequence with perfect internal consistency while suppressing failed attempts that explain the final outcome. The absence of contradiction inside a log is consequently weak evidence when the system could not, or did not, record the conflicting class of event.

Figure 1. An agentic act becomes evidence only after capture, classification and proposition-specific selection.

The record is a projection, not a mirror

Every audit record is a projection of activity through an observability design. Engineers decide which events exist, which fields are emitted, which payloads are redacted, which providers expose telemetry, how long events persist and which transforms occur before export. Those decisions can be sensible and lawful, yet they determine what later investigators are able to reconstruct. A trace should therefore be read as the product of an observation system rather than a transparent window into the execution itself.

The distinction between observable events and internal computation is especially important for language models. Research on chain-of-thought faithfulness has shown that verbalised reasoning can be incomplete or unfaithful to the process that generated an answer, while other work shows that under some conditions it can still convey useful information (Turpin et al., 2023; Lanham et al., 2023; Barez et al., 2025; Zaman & Srivastava, 2026). The correct evidential response is not to discard generated reasoning, but to classify it honestly. A generated rationale is generated material that may support some inferences; it should not silently become a transcript of inaccessible internal computation.

The same classification discipline applies to retrieval and memory. If an agent records that it retrieved document X, that event can establish a retrieval event if the logger is founded and the identifier is meaningful, but it does not necessarily establish that every relevant part of X influenced the later action. A stored memory summary can be authentic as a stored artefact while still being a transformation of prior events rather than the events themselves. Preserving the epistemic status of each record prevents later readers from merging observation, inference and explanation into one synthetic narrative.

The legal lesson from Horizon is broader than Horizon

The Post Office Horizon litigation demonstrates why apparently regular computer records can become dangerous when error information, support history and operational limitations sit outside the record being relied upon. In Bates v Post Office Ltd (No 6: Horizon Issues) [2019] EWHC 3408 (QB), the High Court examined the operation and reliability of Horizon in extensive technical detail and rejected the simple proposition that apparent accounting discrepancies could be treated as self-explanatory proof. The later Court of Appeal decision in Hamilton and others v Post Office Ltd [2021] EWCA Crim 577 concerned convictions in which the reliability of Horizon formed part of the background to whether prosecutions and convictions were safe. The lesson is not that computer evidence is uniquely unreliable, but that evidence about a computer cannot be separated from evidence about what the system was capable of recording and how defects were surfaced.

English law has long contained a common-law presumption that a mechanical instrument or computer was operating properly unless there is evidence to the contrary, a doctrine associated with DPP v McKeown; DPP v Jones [1997] 1 WLR 295. The difficulty is informational: a party challenging the output may need access to logs, defect records, configuration data or provider material before there is enough evidence to rebut the presumption. If the evidence needed to reveal the defect is controlled by the proponent, the presumption can become stronger precisely because the opposing party lacks observability. That asymmetry suggests that any presumption of regular operation should be bounded by a disclosed record boundary rather than attached to the machine as an undifferentiated object.

The same concern appears in American evidence law through different doctrinal routes. Cases such as United States v Lamons, 532 F.3d 1251 (11th Cir. 2008), and United States v Lizarraga-Tirado, 789 F.3d 1107 (9th Cir. 2015), show courts dealing with machine-generated material through authentication and foundation rather than treating every automated record as human hearsay. In 2025 and 2026, the US Advisory Committee on Evidence Rules examined proposed Rule 707 for machine-generated evidence that would be subject to Rule 702 standards if presented through a witness, and the May 2026 report expressly treated the proposal as a developing response rather than a settled rule. The doctrinal paths differ, but the practical question remains the same: what foundation justifies the inference being asked of the machine output?

Purpose-relative authority is the missing discipline

The useful question is not whether a log is authoritative in the abstract. The useful question is whether a particular record is authoritative for a particular proposition, because one record can be excellent evidence of local execution and poor evidence of external effect. This is a more demanding approach than simply asking whether the file came from the system. It forces the proponent to state what is being proved and then identify the event class needed to prove it.

Consider a tool call recorded by an agent application. The event can be strong evidence that the application attempted to invoke a named tool with specified parameters, but it may not prove that the provider received the request, that the provider accepted it, or that the downstream state changed. A provider receipt can supply the next link, and a third-party business record can supply another. Evidential authority is therefore distributed across sources rather than concentrated in the most visually impressive trace.

This approach also avoids the opposite error of trace maximalism. A platform may export thousands of events, including model text, memory operations, tool calls, retries and telemetry, yet a larger dataset is not automatically a more complete account of the disputed act. Completeness is not a percentage of all possible fields; it is adequacy for a defined inference. The right record can be small, provided that it captures the relevant event class and the opposing party can test the foundation.

A public agent trace shows why the distinction matters

A public SWE-agent demonstration provides a concrete example without requiring trust in a private production system. The published marshmallow-code__marshmallow-1867 trajectory records a local reproduction of a TimeDelta precision defect, inspection of fields.py, an attempted edit, a corrected one-line edit using round(...), a second reproduction with a changed result, and submission of a one-file patch. Those events are meaningful evidence of what happened inside the recorded SWE-agent environment. The trace is therefore useful, but only if the proposition stays within that boundary.

Independent GitHub records establish different facts. Marshmallow issue 1865 records the underlying defect, while pull request 1867 records the upstream fix that was actually merged; the merged implementation used integer microsecond arithmetic, changed additional files and added tests. The public trajectory patch and the upstream merged implementation are not the same artefact. It follows that the agent trace can support the proposition that a local patch was produced and submitted, but it cannot by itself prove that the patch was merged or that the agent authored the project fix.

This reconciliation matters because the mistake would be easy to make from the trace alone. A reader who sees an agent reproduce a bug, edit the relevant file, obtain a better result and submit a patch may slide from local success to external authorship without noticing that the final proposition requires evidence from another custodian. The GitHub records do not discredit the trajectory; they define its proper evidential scope. Two records can be authentic and reliable while proving different things.

The example also exposes a design lesson for enterprise agents. If a system is expected to prove that an external effect occurred, it should not rely only on the internal request log; it should preserve a receipt, callback, transaction identifier or independent downstream state where proportionate. Where that evidence is unavailable, the system should say so instead of letting a generic success field stand in for external completion. Good provenance is not maximal logging; it is deliberate capture of the evidence needed for foreseeable consequential claims.

The completeness statement

A practical response is to require a short completeness statement whenever a party wants a machine record to carry a presumption of regular operation. The statement would not certify truth and would not claim that every internal state was observed. It would identify the proposition, the system components involved, the event classes expected to be captured, the schemas and configurations examined, the relevant custodians, known gaps, clock and ordering limits, transformations, retention rules and any independent store used for reconciliation. The document would therefore operate as evidence about the evidence.

The most important field is the capture boundary. A statement should disclose, for example, that application-level tool calls were recorded but provider-side execution logs were not, or that memory retrieval metadata survived while retrieved payloads were deleted after seven days. It should also distinguish unknown from absent: a missing provider record is not equivalent to a record proving that no provider event occurred. By turning observability assumptions into explicit assertions, the statement gives the opposing party something concrete to test.

A completeness statement also has to be founded rather than ceremonial. A signed form prepared by a person who did not inspect the configuration, schema or relevant source stores does not solve the problem merely because it looks formal. Foundation should identify the preparer, basis of knowledge, systems inspected, reconciliation performed and any unverified provider information. The burden rule works only when the statement can itself be challenged.

The burden rule: no statement, no special presumption

The central procedural rule should be modest: no founded completeness statement means no special presumption that the relevant machine process operated correctly for the disputed proposition. That does not mean automatic exclusion of the evidence, and it does not mean that every machine output requires an expert. The proponent can still authenticate the record, call a knowledgeable witness, offer corroboration and persuade the court through ordinary evidential foundations. The point is to prevent an information-poor party from receiving a shortcut that the record architecture does not justify.

Where a founded statement exists, any presumption should remain narrow. It should attach only to the identified component, event class and time period, and it should remain rebuttable by evidence of material gaps, conflicting records or known failure modes. A statement saying that the application recorded all tool-call events during a defined period cannot create a presumption about unlogged provider actions or model computation. Bounded presumptions are safer because they track the actual evidential foundation instead of converting system identity into system infallibility.

This burden allocation also creates the right engineering incentive. Organisations that want strong evidential reliance have a reason to design observable event classes, reconcile across custodians and preserve known-gap indicators, while organisations can still choose narrower logging where privacy, cost or proportionality requires it. The trade is explicit: less observability can be lawful and sensible, but it should reduce the strength of later claims that depend on what was not recorded. Evidence law should not reward opacity by turning missing observability into presumed regularity.

Figure 2. A completeness statement controls the availability and scope of any bounded presumption; it is not a condition of admission.

Contestability is part of reliability

A machine record becomes materially stronger when another party can challenge how it was created. Kaminski and Urban (2021) frame contestation as a central requirement for accountable AI, while reviewability and decision-provenance research similarly emphasises the ability to reconstruct and interrogate process rather than merely receive an explanation (Singh et al., 2019; Cobbe, Lee, & Singh, 2021). For evidence, contestability means access to enough schema information, configuration context, custody history and knowledgeable human testimony to test the claimed inference. Reliability that cannot be challenged is difficult to distinguish from confidence asserted by the party that controls the system.

The relevant human witness may vary with the issue. A custodian can explain collection and preservation, a systems engineer can explain event generation and failure modes, a domain expert can evaluate substantive validity, and an organisational witness can address authority and governance. No single witness needs omniscience, but the record should connect these roles so that generated narrative is not presented as personal knowledge. The machine does not eliminate witnesses; it changes what witnesses need to establish.

Timing also matters. Producing a complex trace shortly before a hearing can make nominal disclosure functionally useless when interpretation requires proprietary schemas, specialised tools or provider cooperation. Effective challenge may require early notice, export documentation and a method for inspecting the record without exposing live credentials or unnecessary trade secrets. Contestability is therefore a procedural property as much as a technical one.

What organisations should build now

Consequential agent systems should be designed with evidential propositions in mind before a dispute occurs. Teams already model threats, permissions and service dependencies; the same engineering process can identify foreseeable propositions such as the request was issued, the provider accepted the request, the customer record used was version X, the policy file was version Y, or the external state changed at time T. Each proposition can then be mapped to the event class and custodian that can actually prove it. This reduces the common mistake of discovering after an incident that the system retained an attractive trace but not the evidence needed for the critical inference.

Identifiers should cross boundaries wherever feasible. A local tool-call identifier that also appears in a provider receipt and downstream transaction record can make reconciliation far stronger than three independent timestamps, particularly when clocks differ or retries occur. Event ordering should record clock source, uncertainty and transformation rather than pretending that millisecond timestamps from separate providers create a single perfect chronology. Integrity controls should protect the record while still preserving enough metadata to explain how the record was produced.

Known gaps should be first-class records. If logging was disabled, retention expired, a provider API failed, or a privileged payload was intentionally withheld, the system should preserve a gap object recording what is known about the absence without fabricating the missing content. This turns silence from an unexplained void into an evidential fact about the capture process. A gap marker will not prove what happened, but it can prevent the much worse inference that nothing happened because nothing appears in the trace.

Retention should be tiered rather than uniform. High-consequence external actions may justify longer preservation of event metadata and integrity evidence, while transient prompts or sensitive payloads may require shorter periods, encryption, segmentation or deletion subject to legal hold. The design should distinguish preservation from broad accessibility, because storing material for a legal purpose does not require exposing it to every operator or analyst. Proportionality works best when retention, access and evidential function are designed together.

The rule that matters

The machine witness is not one witness and does not speak through one record. It is a set of components and custodians that leave different traces of a distributed act, with each trace carrying a particular relationship to the proposition being asserted. A final output may prove the communicated result, a local log may prove a request, a provider receipt may prove acceptance, and a downstream record may prove effect. The discipline is to stop asking whether the machine record is trustworthy in the abstract and start asking what this record can prove.

That discipline also changes how absence should be treated. No event in a log can mean no event occurred, but it can also mean the event class was disabled, the logger failed, the provider controlled the data, the record expired, the payload was lawfully minimised or the export excluded the relevant source. Without a founded statement of the capture boundary, those possibilities remain unresolved. Silence is therefore not proof of non-occurrence merely because the file is authentic.

A defensible approach is narrower and stronger. Authenticate the record, identify the proposition, disclose the capture boundary, preserve the distinction between direct events and generated explanations, reconcile independent stores where the claim crosses custody boundaries, and require a founded completeness statement before granting any special presumption of correct operation. This does not make machine evidence harder to use; it makes the evidential claim correspond to what the system actually recorded. When the machine becomes a witness, the question is not whether it can speak, but whether the record allows another party to test what its silence and its statements really mean.

References and source links

Barez, F., et al. (2025). Chain-of-thought is not explainability. Proceedings of the ACM Conference on Fairness, Accountability, and Transparency. https://aigi.ox.ac.uk/wp-content/uploads/2025/07/Cot_Is_Not_Explainability.pdf

Bates v Post Office Ltd (No 6: Horizon Issues) [2019] EWHC 3408 (QB). https://www.bailii.org/ew/cases/EWHC/QB/2019/3408.html

Cobbe, J., Lee, M. S. A., & Singh, J. (2021). Reviewable automated decision-making: A framework for accountable algorithmic systems. Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency, 598–609. https://doi.org/10.1145/3442188.3445921

Cobbe, J., Veale, M., & Singh, J. (2023). Understanding accountability in algorithmic supply chains. Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency, 1186–1197. https://doi.org/10.1145/3593013.3594073

European Parliament & Council of the European Union. (2016). Regulation (EU) 2016/679 (General Data Protection Regulation). https://eur-lex.europa.eu/eli/reg/2016/679/oj/eng

European Parliament & Council of the European Union. (2024, consolidated 27 July 2026). Regulation (EU) 2024/1689 (Artificial Intelligence Act). https://eur-lex.europa.eu/eli/reg/2024/1689/2026-07-27/eng

Grimm, P. W., Grossman, M. R., & Cormack, G. V. (2021). Artificial intelligence as evidence. Northwestern Journal of Technology and Intellectual Property, 19(1), 9–106. https://scholarlycommons.law.northwestern.edu/njtip/vol19/iss1/2/

Hamilton and others v Post Office Ltd [2021] EWCA Crim 577. https://www.bailii.org/ew/cases/EWCA/Crim/2021/577.html

Judicial Conference of the United States, Advisory Committee on Evidence Rules. (2026, May 17). Report of the Advisory Committee on Evidence Rules. https://www.uscourts.gov/sites/default/files/document/advisory_committee_on_evidence_rules_may_2026.pdf

Kaminski, M. E., & Urban, J. M. (2021). The right to contest AI. Columbia Law Review, 121(7), 1957–2048. https://columbialawreview.org/content/the-right-to-contest-ai/

Lanham, T., et al. (2023). Measuring faithfulness in chain-of-thought reasoning. arXiv. https://arxiv.org/abs/2307.13702

Marshmallow Project. (2021). TimeDelta serialization precision, issue 1865. https://github.com/marshmallow-code/marshmallow/issues/1865

Marshmallow Project. (2021). Fix: TimeDelta precision errors, pull request 1867. https://github.com/marshmallow-code/marshmallow/pull/1867

Packer, C., et al. (2024). MemGPT: Towards LLMs as operating systems. arXiv. https://arxiv.org/abs/2310.08560

Shinn, N., et al. (2023). Reflexion: Language agents with verbal reinforcement learning. Advances in Neural Information Processing Systems, 36, 8634–8652. https://doi.org/10.52202/075280-0377

Singh, J., Cobbe, J., & Norval, C. (2019). Decision provenance: Harnessing data flow for accountable systems. IEEE Access, 7, 6562–6574. https://doi.org/10.1109/ACCESS.2018.2887201

Souza, A. A., et al. (2025). PROV-AGENT: Unified provenance for tracking AI agent interactions in agentic workflows. 2025 IEEE International Conference on eScience. https://arxiv.org/abs/2508.02866

SWE-agent Project. (n.d.). Public v0.7.0 demonstration trajectory: marshmallow-code__marshmallow-1867. https://raw.githubusercontent.com/SWE-agent/SWE-agent/main/trajectories/demonstrations/replay__marshmallow-code__marshmallow-1867__default__t-0.20__p-0.95__c-2.00__install-1___install_from_source/marshmallow-code__marshmallow-1867.traj

Turpin, M., Michael, J., Perez, E., & Bowman, S. R. (2023). Language models do not always say what they think: Unfaithful explanations in chain-of-thought prompting. Advances in Neural Information Processing Systems, 36, 74952–74965. https://proceedings.neurips.cc/paper_files/paper/2023/hash/ed3fea9033a80fea1376299fa7863f4a-Abstract-Conference.html

United States v Lamons, 532 F.3d 1251 (11th Cir. 2008). https://media.ca11.uscourts.gov/opinions/pub/files/200614427.pdf

United States v Lizarraga-Tirado, 789 F.3d 1107 (9th Cir. 2015). https://www.govinfo.gov/content/pkg/USCOURTS-ca9-13-10530/pdf/USCOURTS-ca9-13-10530-0.pdf

Wang, Y., et al. (2026). From agent traces to trust: A survey of evidence tracing and execution provenance in LLM agents. arXiv. https://arxiv.org/abs/2606.04990

Yao, S., et al. (2023). ReAct: Synergizing reasoning and acting in language models. International Conference on Learning Representations.

https://openreview.net/forum?id=WE_vluYUL-X

Zaman, K., & Srivastava, S. (2026). Is chain-of-thought really not explainability? Chain-of-thought can be faithful without hint verbalization. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics. https://doi.org/10.18653/v1/2026.acl-long.2217

← Back to Substack Archive