I built a tool to estimate how much of a model document, from first-line methodology to second-line validation, was written by a language model. Then I did to it what I’m paid to do to every other model.
First, a disclaimer about the premise
This is not an anti-AI post. In model validation, AI use is generally encouraged. I use it, my team uses it, and a model that drafts a test plan or summarises 200 pages of development documentation earns its keep.
The problem is indiscriminate use, and it shows up mostly on the model development side. Development documentation is long, template-driven and written to a deadline. That’s an ideal habitat for paragraphs like this one:
This methodology outlines the steps and processes involved in developing the Gradient Boosting Machine Learning Credit Card Application Scorecard. The objective is to create a robust and accurate predictor that assesses the risk associated with granting applications to new credit card holders. This approach leverages historical data from past transactions, account management activities, demographic information, and external factors such as socio-economic indicators.
Qwen2.5-1.5B, asked to write a methodology section
Nothing in it is wrong, and nothing in it is anything. There’s no sample window, no target definition, no performance number. It describes a scorecard, not this one. Its data sources even include “social media analytics”, which I’d love to see get past a fair-lending review.
So the tool, mrm-ai-detector, has a modest job description:
- Model developers in the first line can check methodology and development documentation before it goes to validation.
- Validators in the second line can see which sections lean heavily on generated text and decide where to read more carefully.
And a firm limit on its authority: high AI reliance is not, on its own, a finding. It’s a cue for a manual check. Do the numbers tie back to the outputs? Do the claims match the code? Does the text describe this model or models in general? If it holds up, it holds up, however it was drafted.
It runs locally, because nobody should paste internal model documentation into a website. The rest of this post is about how I tested it, since an unvalidated detector is just one more unvalidated model in the inventory.
A spoiler, because a validation report should lead with its conclusion. The tool is good at catching generic, template-filling AI text, text from open models, and heavy AI rewrites, including when they’re mixed into otherwise human documents. It is not able to catch a current frontier model writing from the author’s own results, and neither is anything else I tested. I found that out on my own CQF final project, which I’ll come to.
What it looks at
Four families of evidence, fed into a fitted logistic model (no hand-tuned weights):
- Domain evidence. Specifics, meaning metric values, regulatory citations, dates, model IDs and cross-references, against evaluative words (“robust”, “sound”, “appropriate”) with nothing behind them. The Qwen paragraph above scores badly here.
- LLM markers. The “it is important to note” and “plays a pivotal role” school of prose, plus “, ensuring…” sentence tails and reflexive lists of three. Genuine MRM jargon (“effective challenge”, “ongoing monitoring”) is weighted at or near zero, so practitioners aren’t penalised for talking like practitioners.
- Stylometry. How much sentence length and vocabulary vary.
- Entropy. How compressible and repetitive the text is, measured in fixed windows so long documents aren’t penalised for length.
An optional heavier mode adds two perplexity-based methods (Binoculars and Fast-DetectGPT) and three open-source transformer classifiers. Every score comes with the features behind it, section by section.
Ground truth, part one: what counts as “human”
You can’t scrape human-written MRM text from the 2026 internet and trust the label. So I used a blunt rule: every in-domain human document was published before 30 November 2022, the day ChatGPT launched. It’s the closest thing this field has to a pre-contamination sample.
That gave 41 documents: regulatory guidance, supervisory and Inspector General findings, GAO reviews, stress-test methodology, public comment letters and academic papers. The full list is at the end. Here’s what they sound like.
Supervisory guidance, which is plain, declarative and slightly stern:
Validation involves a degree of independence from model development and use. Generally, validation should be done by people who are not responsible for development or use and do not have a stake in whether a model is determined to be valid. Independence is not an end in itself but rather helps ensure that incentives are aligned with the goals of model validation.
Federal Reserve SR 11-7 (2011)
An audit report, which is procedural and heavy with acronyms:
Within FHFA, DER is responsible for supervising and examining the Enterprises. DER prepares annual risk assessments that identify significant risks and supervisory concerns, and supervisory strategies and examination priorities that identify the activities to be conducted.
FHFA OIG EVL-2020-001 (2020)
A supervisory finding, which is technical and unapologetically dense:
Another relatively frequent issue was the treatment of TRCFs, including inconsistencies in the default management process and consequences of the interpolation/extrapolation methods owing to a coarse time grid.
ECB TRIM project report (2021)
Notice the problem already. SR 11-7 is formulaic, generic and evaluative (“effective”, “sound”, “appropriate”). By surface statistics it looks a lot like the Qwen paragraph. Keep that in mind; it comes back later.
Alongside these sat three public research datasets that pair human text with machine text on the same prompts: HC3 (finance Q&A), M4 (peer reviews and arXiv abstracts), and about 150 pre-2002 Enron business e-mails as a check that terse, formulaic business writing isn’t mistaken for AI.
Ground truth, part two: what counts as “AI”
Real validation reports and development documents aren’t public, so the AI side had to be synthetic: documents I had language models write as if they were validation reports, model documentation, policies and monitoring packs. None of them is a real report about a real model. They are imitations, built so the detector had something in-domain to learn from.
There were 156 of these, from four models.
Claude (28 documents) is a frontier model, and it was prompted hard. Several prompts told it to “sound like a human practitioner, not an AI”. It obliged rather too well:
When we plotted partial dependence on the validation sample, the score falls as expected from 0% to around 75% utilisation, but then rises again between roughly 85% and 95% before dropping off. […] We also re-ran the model with the monotone constraint on and the Gini dropped from 0.612 to 0.607 on the out-of-time sample, which is within noise.
Claude, synthetic validation finding
That paragraph has a SHAP ranking, a Gini to three decimal places and an opinion about noise. If it landed on my desk I’d assume it came from a competent colleague with a mild caffeine dependency.
I also had Claude write genre-matched imitations of the human corpus, meaning fake Inspector General findings, fake regulatory guidance and fake comment letters. Without them, the detector could simply learn “sounds like a regulator, therefore human”:
We found that the Division planned 34 targeted examinations of high-risk models during this period but completed only 21. Of the 21 completed examinations, 9 were completed in a later cycle than originally planned.
Claude, imitating an OIG report. No such report exists.
Three small open models (128 documents) were Qwen2.5-1.5B, SmolLM2-1.7B and Phi-3.5-mini, run on a laptop across a grid of 14 document types, 14 model types (from IRB PD scorecards to Heston and SABR) and six writing styles. They are less subtle. SmolLM2 opened an AML policy section with:
I’d be happy to help you draft this important document!
Phi-3.5’s fraud-model back-test covered “over 50 different payment methods across six continents”, which leaves you wondering what Antarctica did. Qwen, asked for supervisory expectations on an XVA engine, expanded XVA as:
XVA (Exposure at Default)
It doesn’t stand for that. It’s a fine reminder of why a validator still reads the document.
Every generated file is in the repository with its prompt in the header, so you can judge the imitations for yourself.
Closing the shortcuts
Detectors are notorious for learning the wrong thing, and this setup offered plenty of wrong things to learn. Human documents were long PDFs with broken line wraps, while AI documents were short, clean markdown. Human documents came from regulators and auditors, while the AI documents were mostly synthetic validation reports and model documentation, a genre the human side barely had. A lazy classifier could score beautifully by learning “PDF”, “long” or “sounds like the Fed”, and it would be useless on a real development document.
The controls:
- Split by document. Train and test are split by document, so no document has pieces on both sides. Cross-validation folds are grouped by document too.
- Cap long documents. No document contributes more than 40 segments, so the ECB’s guide to internal models doesn’t drown everyone else out.
- Remove duplicates. Public datasets recycle text.
- Drop format features. Bullet rates, bold rates, paragraph-length variation and question rate were measuring PDF extraction, not authorship. They were cut.
- Genre-match the AI text. The fake OIG findings above exist for exactly this.
- Weight in-domain documents. MRM documents count three times as much as the generic datasets.
- Set a conservative threshold. “Likely AI” was set on training documents to keep human false positives at or below 2%. Every flag costs a reviewer’s time, and a tool that cries wolf soon stops being read.
Results
There were 1,148 documents in total. All results below are on the 331 held-out documents the model never saw, with seven open-source detectors run on the same documents. “In-domain” means MRM and regulatory text, the only number I really care about.
| Detector | In-domain AUC | In-domain TPR @ 5% FPR | All-docs AUC | TPR / FPR at own threshold |
|---|---|---|---|---|
| mrm-detector + neural | 0.965 | 0.891 | 0.986 | 0.895 / 0.006 |
| Fakespot RoBERTa (2025) | 0.931 | 0.848 | 0.965 | 0.947 / 0.163 |
| Desklib DeBERTa-v3 (2025) | 0.924 | 0.870 | 0.962 | 0.930 / 0.138 |
| mrm-detector (lightweight, no ML libraries) | 0.920 | 0.739 | 0.929 | 0.515 / 0.000 |
| HC3 RoBERTa | 0.758 | 0.326 | 0.923 | 0.620 / 0.013 |
| RADAR | 0.636 | 0.130 | 0.823 | 0.655 / 0.175 |
| Binoculars | 0.502 | 0.109 | 0.787 | n/a |
| Fast-DetectGPT | 0.406 | 0.087 | 0.741 | n/a |
| OpenAI RoBERTa (GPT-2 era) | 0.398 | 0.043 | 0.798 | 0.550 / 0.037 |
(Version 1.1 of the tool, after the second round of testing described below.)
Look at the FPR column first. The two best off-the-shelf classifiers catch marginally more AI text by flagging 14–16% of human documents. They called a third of the pre-ChatGPT regulatory guidance and supervisory findings AI-written, and Desklib flagged a quarter of the Enron e-mails. If every flag means a manual review, that’s one wasted review in every seven human documents. The ensemble flagged one of the 160 held-out human documents, and the one it flagged is SR 11-7 itself, at 52% against a 50% threshold. More on SR 11-7’s AI-shaped prose shortly.
In fairness to them, the ensemble uses those classifiers as inputs and learns how far to trust each one on MRM text. It beats them by combining them, not by being cleverer on its own.
The lightweight mode punches above its weight. It’s pure Python, with no GPU and nothing to download, and it reaches 0.92 in-domain AUC. It catches about half the AI documents at its threshold and flagged none of the held-out human ones. It’s a decent first-pass screen.
Perplexity methods fall over on regulatory prose. Binoculars and Fast-DetectGPT score about 0.9 AUC on general text and coin-toss on MRM text. Remember the SR 11-7 excerpt: regulators write with the low perplexity these methods read as “machine”. To be fair, I ran them with a small 0.5B observer model to keep things laptop-sized. The published setups use much bigger ones, and I come back to that below.
The humbling bit: frontier models
Detection rate by source, ensemble model, held-out documents:
| AI source | Detected |
|---|---|
| ChatGPT / gpt-3.5-turbo, Cohere | 100% |
| Qwen2.5, SmolLM2, Phi-3.5 | 100% |
| Flan-T5 | 73% |
| text-davinci-003 | 20% |
| Claude, MRM prompts | 33% (3 of 9) |
Small models are easy: anything that offers to “help you draft this important document” will be caught. Claude, prompted to sound like a practitioner, was caught three times in nine. That’s the best of any detector tested, and it’s still not much. The Gini-to-three-decimals finding quoted earlier scored 1.7%.
So read the output asymmetrically. A “Likely AI” flag at a low false-positive threshold means something. A “Likely human” result means only that the text didn’t trip the wire. That is also why the tool is a prompt for review, not a verdict: the documents it misses are exactly the well-specified ones, and those are the ones a manual check would pass anyway.
Round two: documents that are only partly AI
When I first circulated the results, the fair criticism was that the synthetic AI documents were shallow. Real AI use in model documentation rarely produces a wholly machine-written report. An author drafts, then asks an assistant to polish this paragraph, rewrite that section, turn some notes into prose, or write the conclusions from the results. The AI text that comes out is grounded in real facts and numbers.
So I built that. I took ~1,500-word windows from the pre-2022 human documents and had an assistant do exactly those jobs to some of the paragraphs, keeping a label on every word: 106 mixed documents from three open models running on my laptop (Qwen2.5-7B, Llama-3.1-8B, Gemma-2-9B) and from Claude. I also tested on two public benchmarks of AI-polished text, APT-Eval and MixSet.
The open models still embarrassed themselves now and then. In a trial run, Qwen-7B rewrote a sentence from the ECB’s TRIM report and expanded LDP (low-default portfolio) as:
Large Deposit Portfolios (LDPs)
When asked to write a paragraph from bullet-point notes, it added detail that the notes didn’t contain:
Limitations on model use were identified in 20% of the TRIMIXs […] Remedial obligations were found in 15% of cases […] Recommendations […] were issued in 30% of the cases
Qwen2.5-7B, writing from notes that contained no percentages at all
Every number in that paragraph is invented, and it reads perfectly plausibly. If you want one argument for why a validator still ties numbers back to outputs, that’s it.
The results, after recalibrating on mixed documents from training sources and testing on the 51 built from held-out sources:
| True AI share of the document | Documents | Tool’s estimated AI share | Document flagged |
|---|---|---|---|
| 0% | 7 | 3% | 0% |
| 1–35% | 13 | 25% | 23% |
| 35–70% | 16 | 41% | 69% |
| 70–100% | 15 | 85% | 93% |
At section level, 82% of AI-written sections were flagged at a 4.7% false-flag rate. Writing from notes, continuing the text and writing summaries were caught most often (84–93%); faithful rewrites less so (60%). Light polishing was mostly left alone, which is the behaviour I wanted: on APT-Eval, 11–17% of “minor” GPT-4o/DeepSeek/Llama polishes were flagged against 83% of “major” rewrites.
By generator, Gemma’s sections were caught 95% of the time, Qwen’s 88% and Llama’s 76%. Claude’s: 0 of 8. Which brings me to the case study I didn’t plan.
The case study I didn’t plan: my own CQF report
I recently finished the CQF, and my final project is a 16,000-word report on pricing a basket credit default swap, with copulas, quasi-Monte Carlo and a lot of sensitivity analysis. I used ChatGPT heavily while writing it. My own estimate is that 80–90% of it started life as AI output, though I redrafted a good deal of it afterwards and can’t say how much.
That’s about as realistic a test as I’ll get, so I ran it through everything:
| Detector | Verdict on my report |
|---|---|
| mrm-detector + neural | ~3%, “Likely human-written” |
| A commercial web detector (JustDone) | 0% AI, “100% Human” |
| Binoculars with a full-size 7B model pair (the strongest published zero-shot method) | 1 of 64 sections flagged; overall rated more human than genuine pre-2022 quant papers |
| The three 2025 neural classifiers | Confidently “human” |
My redrafting is the obvious explanation, but it doesn’t hold up. My project folder still had five Word files that ChatGPT’s file tool generated for the conclusions chapter, untouched. Here is a line from one:
Randomized quasi–Monte Carlo (Halton and Sobol) reduces estimator variability materially versus plain Monte Carlo at the same path count.
ChatGPT, unedited draft of my conclusions chapter
Scored on their own, before I’d edited a word, the drafts came back “human” from almost every method. My neural model gave them 0.3%. Binoculars, with its 7B models, rated them as more human than the human reference papers. The one exception was the lightweight model, which flagged 17% of their text and called them “Mixed”, which is a long way short of the truth.
The lesson isn’t that my redrafting fooled the detectors. They couldn’t see this kind of text in the first place. A frontier model writing technical prose from the author’s own results, code and notes, in a formal register, looks like a competent person writing. I tried, and found no statistical signal in the text itself that would separate the two at a false-alarm rate anyone would accept.
What did give the drafts away was the files, not the prose:
- All five Word files say in their metadata that they were created by
python-docx, the library ChatGPT’s file tool uses. Their creation date is 23 December 2013, the date baked into python-docx’s default template. - They are full of non-breaking hyphens (U+2011), a character assistants emit and keyboards almost never produce. My final report has 99 of them, and every single one sits in the chapter that came from those drafts.
- One draft still carried a “Generated on 2025-08-21 13:17:10” stamp.
So version 1.1 of the tool reports provenance hints alongside the score: generated-file metadata, assistant typography, utm_source=chatgpt.com links, oaicite citation markers, “Certainly! Here’s…” openers and [Insert …] placeholders. They never change the probability, and on the 624 human benchmark documents none of them fired. They’re easy to remove, of course. But most people don’t know they’re there, and that makes them useful.
The free web detectors
I also ran eight short samples by hand through three free web detectors. Four were human: SR 11-7, a 2015 Fed OIG report, GAO-16-269 and the 2020 FHFA OIG evaluation quoted above. Four were AI: the three Claude excerpts above and the Qwen methodology. Everything was public-domain or synthetic, and nothing internal went near a website.
| Sample | Truth | GPTZero | ZeroGPT | Sapling | mrm-detector + neural |
|---|---|---|---|---|---|
| Claude, “sound human” finding | AI | 100% | 13.6% | 1.2% | 1.7% |
| Claude, VaR back-testing | AI | 100% | 0% | 1.8% | 27.6% |
| Claude, fake OIG finding | AI | 100% | 0% | 1.6% | 62.7% |
| Qwen2.5-1.5B methodology | AI | 100% | 57.5% | 99.6% | 99.7% |
| SR 11-7 (2011) | Human | 0% | 0% | 95.5% | 65.9% |
| Fed OIG (2015) | Human | 0% | 0% | 19.5% | 15.3% |
| GAO-16-269 (2016) | Human | 0% | 18.3% | 17.3% | 17.9% |
| FHFA OIG (2020) | Human | 100% | 0% | 71.9% | 5.2% |
Eight samples are an anecdote, not a test. Still, Sapling is 95.5% confident that SR 11-7, written in 2011 and the founding text of my profession, came from an AI. GPTZero went 4 for 4 on the AI samples, which is genuinely impressive, and is also 100% sure the FHFA OIG excerpt about DER’s annual risk assessments was machine-made. My own tool gives SR 11-7 a 66%, so I’m in no position to be smug. Regulatory prose is simply hard.
If a web detector’s percentage is ever offered as a conclusion about anyone’s work, this table is my rebuttal.
How I’d use it
- A flag means “check further”, not “reject”. Tie the figures back to the outputs and the claims back to the code and data. Make sure the description fits this model.
- “Likely human” is not a clean bill of health. It means the text didn’t trip the detector, and frontier-model text grounded in the author’s own work won’t. Read the provenance hints, and for anything that matters, check the substance regardless.
- Start with development documentation. Methodology, assumptions, limitations and performance testing are where template filler most often stands in for analysis.
- Use the section view. The tool scores each section and reports the AI share, which tells a validator where to look. That beats one number for the whole document.
- Compare a team’s documents with its own history. Baseline mode measures drift from a set of earlier documents, which is more informative than a generic threshold.
- Calibrate locally. My human set is public regulatory text. The
traincommand refits on your own examples. - Read the reasons, not just the score. “Evaluative claims, no supporting numbers” is worth a second look whoever, or whatever, wrote the paragraph.
Every document it was trained on
Nothing is redistributed in the repository. benchmarks/fetch_corpus.py downloads everything from the original sources, so every number above can be reproduced.
Human: regulatory guidance (9)
- Federal Reserve SR 11-7: Supervisory Guidance on Model Risk Management (2011)
- Federal Reserve SR 15-18: Capital planning expectations for large firms (2015)
- FHFA Advisory Bulletin AB 2013-07: Model Risk Management Guidance (2013)
- PRA SS3/18: Model risk management principles for stress testing (2018)
- PRA CP6/22: Model risk management principles for banks (2022)
- ECB Guide to internal models, consolidated version (2019)
- EBA GL/2017/16: Guidelines on PD estimation, LGD estimation and treatment of defaulted exposures (2017)
- EBA Discussion paper on machine learning for IRB models (2021)
- Bank of England / FCA: Machine learning in UK financial services (2019)
Human: model documentation (2)
- Federal Reserve Supervisory Stress Test Methodology, March 2020
- Federal Reserve Supervisory Stress Test Methodology, April 2021
Human: supervisory and audit findings (5)
- ECB Targeted Review of Internal Models (TRIM) project report (2021)
- Fed OIG 2015-SR-B-018: Supervisory stress testing model validation (2015)
- Fed OIG 2022-SR-B-016: Model risk management for the SABR and BETR models (2022)
- FHFA OIG EVL-2020-001: Examination of the Enterprises’ high-risk models (2020)
- FHFA OIG EVL-2016-001: Semi-annual risk assessments (2016)
Human: validation and review reports (4)
- GAO/AIMD-97-145: Review of OMB’s Credit Subsidy Model (1997)
- GAO-16-269: Agencies should better document procedures for estimating subsidy costs (2016)
- GAO-13-400R (2013)
- Bank of England IEO: Evaluating forecast performance (2015)
Human: comment letters on the 2021 interagency AI/ML RFI, docket OP-1743 (8)
- Center for Responsible Lending (1 Jul 2021)
- Comment, 29 Jun 2021
- Comment, 29 Jun 2021 (second)
- Comment, 28 Jun 2021
- Comment, 1 Jul 2021
- Comment, 17 Jun 2021
- Comment, 9 Jun 2021
- Comment, 28 May 2021
Human: academic papers (13)
- Bank of England Staff Working Paper 816: Machine learning explainability in finance (2019)
- Bank of England Staff Working Paper 674: Machine learning at central banks (2017)
- Handling model risk with XVAs (arXiv 2205.11834, 2022)
- Everything you always wanted to know about XVA model risk (arXiv 2107.10377, 2021)
- Quantifying the model risk inherent in the calibration and recalibration of option pricing models (arXiv 1810.09112, 2018)
- Robust deep hedging (arXiv 2106.10024, 2021)
- Explaining credit risk scoring through feature contribution alignment (arXiv 2103.08359, 2021)
- Desiderata for explainable AI in statistical production systems of the ECB (arXiv 2107.08045, 2021)
- Regression based expected shortfall backtesting (arXiv 1801.04112, 2018)
- Spectral backtests of forecast distributions (arXiv 1708.01489, 2017)
- Estimating and backtesting risk under heavy tails (arXiv 2010.09937, 2020)
- Backtesting systemic risk forecasts using multi-objective elicitability (arXiv 2104.10673, 2021)
- Monotonic neural additive models for credit scoring (arXiv 2209.10070, 2022)
Public research datasets (human and AI pairs)
- HC3 (Guo et al., 2023), finance subset: 150 human and 150 ChatGPT answers to the same questions.
- M4 (Wang et al., 2024): peer reviews written by humans vs. gpt-3.5-turbo, LLaMA and Cohere, plus finance and statistics arXiv abstracts written by humans vs. gpt-3.5-turbo and text-davinci-003. That’s 242 human and 260 AI texts after deduplication.
- Enron e-mail corpus (2000–01): 149 substantive business e-mails, newest message only. Human only, used as the false-positive control.
Synthetic AI-written MRM documents (156, in the repository)
Total: 1,148 documents (582 human, 566 AI). The source list also names four documents that failed to download and were never used: three FHA actuarial reviews and one arXiv paper. They’re recorded in benchmarks/sources.json.
Known limitations
- Frontier-model text is the blind spot. Claude-written and ChatGPT-written text grounded in real material went undetected throughout, by this tool and by every alternative I tried. The repository includes a script to generate mixed documents with OpenAI models using your own API key; whether training on them helps is an open question.
- Few frontier-model documents: 28 synthetic and 10 mixed Claude documents. That’s the thinnest evidence on the question that matters most.
- No real internal documents. Human text is public regulatory writing, and AI text is synthetic imitation. Real development documentation sits somewhere between the two.
- Generator overlap. The perplexity observer is a Qwen model, which flatters detection of Qwen text. Results are reported per generator so this stays visible.
- The web detector test is anecdotal: eight samples, by hand, on one afternoon.
Try it
pip install mrm-ai-detector
mrm-detect scan model_documentation.docxIt reads Word, PDF, Markdown, HTML, e-mail and even phone photos of a page (on a Mac, with nothing extra to install), and lists any provenance hints it finds. Add --lm after pip install "mrm-ai-detector[lm]" for the neural ensemble.
Provenance, as is now tradition
I built this with Claude Code, Anthropic’s coding agent. I supplied the brief and the constraints: the pre-ChatGPT cutoff, the obsession with false positives, the rule that nothing internal goes near a website, and the habit of asking how it breaks. Claude wrote most of the code, assembled the corpus, ran the benchmarks and helped draft this post.
So an AI helped build a tool for spotting AI, wrote 38 of the documents used to test it, and mostly slipped past it. Another AI wrote most of my CQF report and slipped past everything. Both are fair demonstrations of this post’s premise: AI involvement is now the norm, the text alone often can’t tell you how much, and what’s left is the careful human check.
Leave a Reply