When you evaluate a deepfake detector, a demo and a single accuracy figure are close to useless. A detector that scores 99% on a vendor's chosen dataset can collapse on the compressed, unfamiliar, real-world content you will actually feed it, and the glossy number tells you nothing about which generators it has seen, how it performs across different people, or what it cannot do. There is a document designed to answer exactly those questions: the model card. Demanding one, and knowing how to read it, is the most efficient due diligence a buyer can do.
This piece explains what a model card is, why it has moved from a research nicety to a governance requirement, and, section by section, what a good one should tell you before you buy a deepfake detector. It is written for the fraud, risk, security, and procurement teams who have to choose a detector and defend the choice.
The underlying point is that a model card turns a sales conversation into a disclosure. What a vendor is willing to put in writing, and what they leave vague, tells you most of what you need to know.
- A model card is the structured disclosure document for an AI model, and it is the single most useful due-diligence artifact to demand before buying a deepfake detector.
- Model cards began as a research norm in 2019 and are now partly mandated: the EU AI Act's Annex IV requires technical documentation for high-risk systems, including performance by demographic group.
- The training-data section is the one that predicts generalization: which generators, how diverse, how recent, and how often refreshed.
- Performance should never be a single accuracy number; demand true and false positive rates at a stated threshold, plus results on unseen generators and in-the-wild data.
- Fairness disclosure matters, because detectors can underperform for some demographic groups, and the EU AI Act requires testing that surfaces disparate impact.
- A credible model card states clear limitations and failure modes; a card that claims to catch everything is a warning sign.
- Versioning and update cadence belong on the card too, because an unmaintained detector degrades and models can change silently after deployment.
- Learn to read the green and red flags: named training generators, honest metrics, subgroup performance, and versioning are good signs; a lone accuracy figure is not.
What a Model Card Is, and Why It Matters Now
A model card is a short, structured document that describes a machine-learning model: what it is for, what it was trained on, how it performs, and where it falls short. The format originated in 2019 research on model reporting and has since become a standard way to communicate a model's capabilities and limitations honestly, in place of marketing claims. The major AI labs now publish them for their flagship models, and the same discipline applies to a detector you are about to depend on.
What has changed recently is that this is no longer optional for many systems. The EU AI Act's Annex IV mandates structured technical documentation for high-risk AI, including the model's purpose, training-data sources, performance metrics broken down by demographic group, risk assessments, and human-oversight mechanisms, with enforcement arriving in 2026, and ISO 42001 adds AI management-system expectations such as version control. In the US, the NIST AI Risk Management Framework pushes the same direction of documented, accountable models. Against a backdrop where a large majority of organizations admit they cannot fully validate or trace their AI training data, the model card is becoming the common artifact that makes a model auditable, and a vendor's willingness to provide a thorough one is itself a signal.
Intended Use, Scope, and Training Data
The first thing a model card should make unambiguous is what the detector is actually for. That means the modalities it covers, image, video, audio, the manipulation types it detects, face-swap, full synthesis, lip-sync, the attack vectors it addresses, and, just as importantly, what it is not designed to do. A card that defines scope precisely lets you check the tool against your real threats; a card that gestures at "detecting deepfakes" without specifics is hiding the gaps.
The training-data section is the one experienced buyers read most closely, because it quietly predicts the single most important property of a detector: generalization. Which generators was the model trained on, how diverse and recent is that data, and, critically, how is it refreshed as new generators appear? A detector trained on a fixed snapshot of last year's models will degrade as generation advances, so the card should describe not just the data but the process for keeping it current. Data provenance belongs here too, where the data came from and on what basis, which both the EU AI Act and responsible-AI practice increasingly expect, often in a companion datasheet alongside the model card.
Performance: Beyond a Single Accuracy Number
The performance section is where marketing and reality diverge most, so it deserves scrutiny. A single accuracy percentage is nearly meaningless on its own, because it hides the trade-off that actually matters operationally: the true positive rate, how many real fakes the detector catches, and the false positive rate, how often it wrongly flags genuine users, measured at a stated decision threshold that you should be able to tune. A card that gives you both rates at a defined threshold, ideally with a curve across thresholds, is being honest; one that gives you a lone headline figure is not.
Two further conditions separate a credible card from a flattering one. First, generalization: the card should report performance on generators that were not in the training set, because that is the test that predicts real-world durability, and a vendor that only shows results on familiar models is avoiding the hard question. Second, realism: results on compressed, resized, and in-the-wild media, not just pristine benchmark images, since that is what production traffic looks like. Best of all is independent, third-party validation, an ISO/IEC 30107-3 result for presentation attacks, a CEN/TS 18099 evaluation for injection, or an in-the-wild benchmark, rather than numbers the vendor generated for itself.
Fairness, Limitations, and Upkeep
Three sections round out a card that can actually be trusted, and their presence or absence is revealing. Fairness comes first, because detection accuracy can vary across people: a detector may perform worse for some skin tones, ages, or genders, and a model that is accurate on average can still fail a subgroup badly. A serious card reports performance broken down by demographic group, which is not just good practice but, as one 2026 compliance analysis put it, something the EU AI Act requires performance testing to surface, naming disparate impact directly. A card with no fairness breakdown is leaving a known risk undocumented.
The limitations section should read like an honest confession, not a disclaimer. It should state what the detector does not catch, whether that is injection attacks, audio, or the newest generators, and under what conditions it is weakest. A card claiming the model catches everything is a red flag, because every detector has blind spots and the useful ones are named. Finally, upkeep: because an unmaintained detector degrades and because models can be updated silently after deployment without anyone being told, the card should document the version, the retraining and update cadence, and how changes are communicated. Explainability and data-handling details, how the model surfaces its reasoning and how it treats biometric data under rules like the GDPR, belong here as well.
Reading a Model Card: Green Flags and Red Flags
Put together, reading a model card comes down to a few contrasts. A green flag is a card that names the specific generators in its training data; a red flag is a vague "trained on diverse datasets." A green flag reports true and false positive rates at a stated threshold; a red flag offers a single accuracy number. A green flag shows unseen-generator and in-the-wild results; a red flag shows only benchmark-dataset performance. A green flag breaks results down by demographic group and states clear limitations; a red flag has no fairness testing and claims to catch everything. And a green flag documents versioning and update cadence; a red flag has none. In short, the quality of the disclosure is a proxy for the quality and the honesty of the product.
This transparency is the posture DuckDuckGoose, based in Delft, is built around. Its explainable approach to detection, surfacing which regions drove a verdict rather than emitting an unexplained score, reflects the same commitment a good model card embodies, and its ISO 27001, SOC 2, and GDPR compliance aligns with the governance frameworks, from the EU AI Act to NIST, that now expect documented, auditable models. The broader lesson for any buyer is to treat the model card as a gate: if a vendor cannot or will not tell you what their detector was trained on, how it performs on unseen generators and across demographics, and what it cannot do, that silence is your answer. For the wider set of questions to ask a vendor, see our guide on choosing a deepfake detection SDK, and for the strategic decision behind it, deepfake detection build versus buy.
Frequently Asked Questions
What is a model card?
It is a short, structured document that describes a machine-learning model: its intended use, the data it was trained on, how it performs, its limitations, and fairness and oversight considerations. Originating in 2019 research, model cards have become the standard way to communicate a model's real capabilities honestly, and they are the most useful due-diligence artifact to request before buying a detector.
Why should I ask for a model card before buying a deepfake detector?
Because it forces disclosure of the things a demo and a headline accuracy figure hide: which generators the model was trained on, how it performs on unseen models and degraded media, how accuracy varies across demographic groups, and what it cannot detect. A vendor's willingness to provide a detailed card, and what they leave vague, tells you most of what you need to know.
What performance information should a model card include?
Not a single accuracy number. It should give the true positive rate and false positive rate at a stated, tunable decision threshold, results on generators not in the training set, performance on compressed and real-world media, and ideally independent third-party validation such as an ISO/IEC 30107-3 or CEN/TS 18099 evaluation rather than only vendor-generated figures.
Why does the training-data section matter so much?
Because it predicts generalization, the most important real-world property of a detector. Which generators the model saw, how diverse and recent the data is, and how it is refreshed determine whether the detector keeps working as new generators appear. A model trained on a fixed snapshot degrades over time, so the card should describe the process for staying current, not just the data.
Should a model card report fairness or bias?
Yes. Detection accuracy can vary across skin tones, ages, and genders, so a model that is accurate on average may fail a subgroup. A credible card reports performance broken down by demographic group, and the EU AI Act's documentation requirements expect performance testing that surfaces disparate impact. A card with no fairness breakdown leaves a known risk undocumented.
Is a model card required by regulation?
Increasingly, yes, for many systems. The EU AI Act's Annex IV mandates structured technical documentation for high-risk AI, including purpose, training data, performance by demographic group, and oversight, with enforcement in 2026, and standards like ISO 42001 and the NIST AI Risk Management Framework point the same way. Even where not strictly mandated, a model card is fast becoming an expected norm.
What are the red flags in a model card?
A single accuracy figure with no error-rate breakdown, vague training-data descriptions that do not name generators, results only on benchmark datasets rather than unseen models and real-world media, no fairness or bias testing, claims to catch everything with no stated limitations, and no versioning or update policy. Any of these suggests the disclosure, and likely the product, is weaker than it appears.
About the author
