Building an adversarial testbed for AI verification in Europe
From verification primitives to a red-teamed reference architecture
What we are doing
Lucid is building and operating an adversarial testbed for AI verification inside a government-funded research institute that runs clusters for domestic workloads. The testbed consists of four parts:
An operating cluster. Live workloads, with an operator that has agreed to host verification infrastructure in the production path.
An evidence plane. Hardware-rooted mechanisms that can be composed against a given claim.
An independent red team. The host institute holds technical authority over adversarial testing. It sets the test plan, and Lucid does not review it in advance.
A publication protocol. Agreed before testing begins. Results are published either way, negative findings included.
Verification claims are the input to the testbed (for example, “this cluster ran only inference during the reporting period”). This determines which mechanisms are composed, what evidence leaves the facility, and what the red team tries to break. The output of a cycle is an attacked pilot: a composed system installed on operating hardware, exercised against live workloads, and subjected to independent adversarial testing whose results are public.
The first configuration retrofits two eight-GPU H100 servers against three claims: where compute runs (hardware-anchored identity), how it is used (job manifest checked against telemetry), what model is loaded (TEE attestation). We do this through “structured transparency,”1 a system in which agreed-upon collections of facts can be rigorously checked without disclosing model weights, training data, customer data, evaluation sets, or sensitive infrastructure details.
The output will be an open reference architecture for evidence-backed coordination over frontier AI compute, together with the assets that can be utilised across alternative architectures: claim and receipt schemas, threat models, and a red-team methodology that has been run by world class security experts.
Builder vs. breaker
Lucid builds and remediates. The govt funded institute holds technical authority over adversarial testing: it sets the test plan independently, and Lucid does not review it in advance. Results are published under a protocol agreed before testing begins, including negative results. The red team targets JTAG and SPI-flash substitution, attestation spoofing, power-telemetry fraud, covert-channel network routing, flooding, side channels, and physical tamper paths.
Acceptance targets: ≤10% error against H100 power-to-compute benchmarks; detection of an unauthorised location change of GPUs, detection of network traffic above approved thresholds and remediation of all High and Critical CVEs found in the test cycle.
As we outline below, the builder vs breaker set up is one of our key contributions to advancing verification technology (rather than one specific component or architecture). Amodo’s Verification SitRep grades adversarial validation as among the least developed parts of the field, so we are excited to use these 18 months to help close this gap and to generate as many reusable insights as possible.
Configuration one: the design
An FPGA DC-SCM boots before the host, verifies component firmware, anchors server identity, against which physical location is audited, and signs power telemetry. CPU and GPU TEEs attest the runtime and the loaded model. A programmable FlexNIC and a traffic shaper constrain and observe the I/O path, enforcing network restrictions and randomised inference routing, and committing network telemetry rather than traffic contents. An Assurance Node, deployed inside the cluster, cross-checks these signals against a signed Verifiable Job Manifest, evaluates agreed policy, and emits a chained receipt. Full traffic is never mirrored or retained.
Below is a diagram to visualise the design:
We use a defense in depth approach, meaning that systems are protected by multiple overlapping layers of security, so that a breach of one layer is not sufficient to compromise the system as a whole. This ensures we are not reliant on a single root of trust, and increases the chance that cheating should be detected.
What our configuration one aims to verify
Below we map the three claim families to specific claims an actor may wish to make as part of a governance regime, domestic or international. Two reading notes. The claim families and example claims are the input layer: they are the part of this document we most want challenged. The tooling column is the configurable layer: for several claim families, alternative evidence sources exist (power fingerprinting, network telemetry, TEE attestation, recomputation), and part of the testbed’s job is to establish which compositions survive attack, and at what cost
Why it matters for the ecosystem
Amodo’s Verification SITREP tracks progress on technological primitives that support recompute-based workload classification. The algorithmic primitives have made significant progress; almost everything that turns a primitive into a deployed system has not. Recomputation-server security, verification reporting, TAP installation assurance and physical security and audits are all graded not on track, and recomputation red-teaming has not started.
We seek to advance key components of verification tech from ‘not started’ or ‘not on track’ to ‘red teamed’ within 18 months. This means having an end-to-end system, installed in a production grade datacenter, with a competent team having been paid to break it. In addition to the verification solutions Lucid is currently developing (location verification, power telemetry) we intend to rapidly develop end-to-end proofs of concept for recomputation that can be red-teamed in our partners’ research clusters.
An attacked pilot is a specific claim: the whole system installed on operating hardware, exercised against live workloads, and subjected to external testing under the publication protocol above. It is only one rung on the ladder, but an important one. If the architecture stands up to red teaming, it is ready to be developed into a reference design and scaled; if it does not, the failure modes are published and the field learns what needs to be patched.
What is deployed at project close
Platform assets, reusable beyond this configuration:
A published Verifiable Job Manifest schema
A signed, hash-chained receipt schema with a working implementation
The red-team methodology and publication protocol as actually run, with results, negative findings included
Installation and assurance documentation for in-path devices
An open-source reference architecture
Configuration-one artifacts:
An operational 16 × H100 demonstrator with the evidence plane installed alongside live workloads
FPGA DC-SCM and FlexNIC prototypes
A power-to-compute estimator validated against H100 benchmarks
Vulnerability reports, and patched firmware and software
What this means
First, some caveats: sixteen GPUs is not a frontier cluster. Scaling from two servers to a hundred thousand accelerators is unsolved, and in-path device count and installation assurance are the hardest parts of it. We cannot verify what happens with undeclared, non-retrofitted compute (which is a risk in any governance regime). Side channels are bounded and measured, not eliminated. In short: this is not a treaty-grade verification stack.
However, it is a step forward for the field to verifiably pace AI development internationally: a machine for finding out, in public, which verification claims survive contact with a competent adversary. The assets that outlive configuration one (schemas, evidence chains, threat models, red-team methods, and a public account of which claims held, which assumptions failed, and what has to be engineered next) are useful whichever claims a future agreement settles on. This moves us forward (incrementally) on optionality for companies and governments before a crisis rather than during one.
As we kickstart this project, we are seeking input: Table 1 is configuration one’s verifiable claims. These are our current best guess at the claims most worth attacking first. However, the set up is (by design) re-configurable, so if you are a lab, a government, or a security researcher and there is a claim you think it is urgent to work on - please get in touch!
You can reach our Europe-based Director at: connor@lucidcomputing.ai
‘A system exhibits structured transparency to the extent that it conforms with a set of predetermined standards about who should be able to know what, when they should be able to know it, and what they should be able to do with this knowledge; in other words, if it enforces a desired information flow.’ https://arxiv.org/pdf/2012.08347v1