Fact retention test
On 9 November 2026, Ink Balanced changed or dropped a checked fact in 5 of 20 eligible documents using the checker from Wang, Zhu and Chen (2026).
Documents with a fact changed or dropped
Lower is better| Tool | Changed or dropped / eligible documents, percentage (Wilson 95% interval) |
|---|---|
| Ink Balanced | 5/20, 25.0% (11.2% to 46.9%) |
| HIX Bypass | 7/20, 35.0% (18.1% to 56.7%) |
| Superhumanizer | 7/20, 35.0% (18.1% to 56.7%) |
| UndetectedGPT | 7/20, 35.0% (18.1% to 56.7%) |
| Smart Humanizer | 8/20, 40.0% (21.9% to 61.3%) |
| GPTinf | 9/20, 45.0% (25.8% to 65.8%) |
| Rehumanize | 9/20, 45.0% (25.8% to 65.8%) |
| Clever AI Humanizer | 10/20, 50.0% (29.9% to 70.1%) |
| StealthGPT | 11/20, 55.0% (34.2% to 74.2%) |
| Undetectable AI | 11/20, 55.0% (34.2% to 74.2%) |
| AIHumanize | 13/20, 65.0% (43.3% to 81.9%) |
| WriteHuman | 13/20, 65.0% (43.3% to 81.9%) |
Facts and meaning
| Tool | Meaning score | Share of documents with a fact changed or dropped |
|---|---|---|
| AIHumanize | 0.9160848484848484 | 0.65 |
| Clever AI Humanizer | 0.913739393939394 | 0.5 |
| GPTinf | 0.9208757575757576 | 0.45 |
| HIX Bypass | 0.9447818181818182 | 0.35 |
| Ink Balanced | 0.9189476606453624 | 0.25 |
| Ink Light | 0.9228950146924388 | 0.15 |
| Ink Strong | 0.8979881797997731 | 0.2 |
| Rehumanize | 0.9217121212121212 | 0.45 |
| Smart Humanizer | 0.9573575757575756 | 0.4 |
| StealthGPT | 0.9192636363636364 | 0.55 |
| Superhumanizer | 0.9379545454545456 | 0.35 |
| Undetectable AI | 0.9323454545454544 | 0.55 |
| UndetectedGPT | 0.943421212121212 | 0.35 |
| WriteHuman | 0.8831030303030303 | 0.65 |
Ink strengths: documents with a fact changed or dropped
Lower is better| Tool | Changed or dropped / eligible documents, percentage (Wilson 95% interval) |
|---|---|
| Ink Light | 3/20, 15.0% (5.2% to 36.0%) |
| Ink Balanced | 5/20, 25.0% (11.2% to 46.9%) |
| Ink Strong | 4/20, 20.0% (8.1% to 41.6%) |
What this measures
The paper’s checker looks for numbers and dates, names of people and organisations, and quotes of three words or more. A document passes only if every checked item survives. Results for the other tools come from the outputs the paper published.
How the check works
Source
Maya Chen said “the doors open tomorrow” before 24 guests arrived.
Rewrite
Before 24 42 guests arrived, Maya Chen said “the doors open tomorrow”.
Underlined: kept. Struck and replaced: changed number.
Limits
- Only 20 documents have checkable facts, so this is not a ranking.
- The checker was validated by AI annotators, not people.
- It does not check meaning or details a rewrite adds.
- We rerun the test when Ink changes.
Sources
- Method and tool counts: Wang, Zhu and Chen, Humanized but Still True? ↗, 30 October 2026. Code ↗ MIT, © 2026 Ziyuan Wang, Genliang Zhu and Xiaobo Chen.
- Documents: AI Humanizer Benchmark, October 2026 ↗, CC BY 4.0. Not WriteHuman’s HumanizerBench.
- Our results: CSV, CC BY 4.0 ↗. Counts only, no text.
Technical details
| Tool | Changed or dropped / eligible documents |
|---|---|
| Ink Balanced | 5/20, 25.0% (11.2% to 46.9%) |
| HIX Bypass | 7/20, 35.0% (18.1% to 56.7%) |
| Superhumanizer | 7/20, 35.0% (18.1% to 56.7%) |
| UndetectedGPT | 7/20, 35.0% (18.1% to 56.7%) |
| Smart Humanizer | 8/20, 40.0% (21.9% to 61.3%) |
| GPTinf | 9/20, 45.0% (25.8% to 65.8%) |
| Rehumanize | 9/20, 45.0% (25.8% to 65.8%) |
| Clever AI Humanizer | 10/20, 50.0% (29.9% to 70.1%) |
| StealthGPT | 11/20, 55.0% (34.2% to 74.2%) |
| Undetectable AI | 11/20, 55.0% (34.2% to 74.2%) |
| AIHumanize | 13/20, 65.0% (43.3% to 81.9%) |
| WriteHuman | 13/20, 65.0% (43.3% to 81.9%) |
Meaning score
How close each rewrite stays to the source wording, by embedding similarity. Higher is closer, which favours light edits. It does not measure accuracy.
| Tool or run | Meaning score |
|---|---|
| Smart Humanizer | 0.957 |
| HIX Bypass | 0.945 |
| UndetectedGPT | 0.943 |
| Superhumanizer | 0.938 |
| Undetectable AI | 0.932 |
| Ink Light | 0.923 |
| Rehumanize | 0.922 |
| GPTinf | 0.921 |
| StealthGPT | 0.919 |
| Ink Balanced | 0.919 |
| AIHumanize | 0.916 |
| Clever AI Humanizer | 0.914 |
| Ink Strong | 0.898 |
| WriteHuman | 0.883 |
Retention and delivery
Delivered: completed, completed_with_warnings and needs_review, including unchanged source sections. Conservative: all source-eligible documents, missing output receives no credit. Unchanged output raises retention without showing rewriting quality. Kept-source markers do not prove every section was rewritten or identify affected units; needs_review always requires review.
| Run | Delivered documents | Conservative documents | Delivered units | Identical outputs | Documents with a fact changed or dropped among changed eligible documents. Lower is better. |
|---|---|---|---|---|---|
| Balanced (API default) | 15/20, 75.0% (53.1% to 88.8%) | 15/20, 75.0% (53.1% to 88.8%) | 79/94, 84.0% (75.3% to 90.1%) | 0/33 overall; 0/20 eligible | 5/20, 25.0% (11.2% to 46.9%) |
| Light | 17/20, 85.0% (64.0% to 94.8%) | 17/20, 85.0% (64.0% to 94.8%) | 88/94, 93.6% (86.8% to 97.0%) | 0/33 overall; 0/20 eligible | 3/20, 15.0% (5.2% to 36.0%) |
| Strong | 16/20, 80.0% (58.4% to 91.9%) | 16/20, 80.0% (58.4% to 91.9%) | 89/94, 94.7% (88.1% to 97.7%) | 2/33 overall; 2/20 eligible | 4/18, 22.2% (9.0% to 45.2%) |
| Run | Delivered | needs_review | Warnings | Kept-source documents / sections | Identical outputs | Missing output |
|---|---|---|---|---|---|---|
| Balanced (API default) | 33/33 | 14 | 12 | 0 / 0 | 0 | 0 |
| Light | 33/33 | 13 | 11 | 0 / 0 | 0 | 0 |
| Strong | 33/33 | 6 | 11 | 5 / 34 | 2 | 0 |
Commercial and strength runs
| Tool | Documents with a fact changed or dropped | Number/date units | Name units | Quoted units |
|---|---|---|---|---|
| AIHumanize | 13/20, 65.0% (43.3% to 81.9%) | 11/56, 19.6% (11.3% to 31.8%) | 2/23, 8.7% (2.4% to 26.8%) | 15/15, 100.0% (79.6% to 100.0%) |
| Clever AI Humanizer | 10/20, 50.0% (29.9% to 70.1%) | 7/56, 12.5% (6.2% to 23.6%) | 5/23, 21.7% (9.7% to 41.9%) | 14/15, 93.3% (70.2% to 98.8%) |
| GPTinf | 9/20, 45.0% (25.8% to 65.8%) | 8/56, 14.3% (7.4% to 25.7%) | 1/23, 4.3% (0.8% to 21.0%) | 15/15, 100.0% (79.6% to 100.0%) |
| HIX Bypass | 7/20, 35.0% (18.1% to 56.7%) | 3/56, 5.4% (1.8% to 14.6%) | 0/23, 0.0% (0.0% to 14.3%) | 11/15, 73.3% (48.0% to 89.1%) |
| Ink (Balanced, API default) | 5/20, 25.0% (11.2% to 46.9%) | 7/56, 12.5% (6.2% to 23.6%) | 1/23, 4.3% (0.8% to 21.0%) | 7/15, 46.7% (24.8% to 69.9%) |
| Ink (Light) | 3/20, 15.0% (5.2% to 36.0%) | 1/56, 1.8% (0.3% to 9.4%) | 0/23, 0.0% (0.0% to 14.3%) | 5/15, 33.3% (15.2% to 58.3%) |
| Ink (Strong) | 4/20, 20.0% (8.1% to 41.6%) | 3/56, 5.4% (1.8% to 14.6%) | 2/23, 8.7% (2.4% to 26.8%) | 0/15, 0.0% (0.0% to 20.4%) |
| Rehumanize | 9/20, 45.0% (25.8% to 65.8%) | 4/56, 7.1% (2.8% to 17.0%) | 2/23, 8.7% (2.4% to 26.8%) | 12/15, 80.0% (54.8% to 93.0%) |
| Smart Humanizer | 8/20, 40.0% (21.9% to 61.3%) | 2/56, 3.6% (1.0% to 12.1%) | 8/23, 34.8% (18.8% to 55.1%) | 13/15, 86.7% (62.1% to 96.3%) |
| StealthGPT | 11/20, 55.0% (34.2% to 74.2%) | 13/56, 23.2% (14.1% to 35.8%) | 6/23, 26.1% (12.5% to 46.5%) | 14/15, 93.3% (70.2% to 98.8%) |
| Superhumanizer | 7/20, 35.0% (18.1% to 56.7%) | 2/56, 3.6% (1.0% to 12.1%) | 2/23, 8.7% (2.4% to 26.8%) | 14/15, 93.3% (70.2% to 98.8%) |
| Undetectable AI | 11/20, 55.0% (34.2% to 74.2%) | 5/56, 8.9% (3.9% to 19.3%) | 1/23, 4.3% (0.8% to 21.0%) | 12/15, 80.0% (54.8% to 93.0%) |
| UndetectedGPT | 7/20, 35.0% (18.1% to 56.7%) | 7/56, 12.5% (6.2% to 23.6%) | 1/23, 4.3% (0.8% to 21.0%) | 13/15, 86.7% (62.1% to 96.3%) |
| WriteHuman | 13/20, 65.0% (43.3% to 81.9%) | 13/56, 23.2% (14.1% to 35.8%) | 8/23, 34.8% (18.8% to 55.1%) | 15/15, 100.0% (79.6% to 100.0%) |
Method and reproduction
Corpus: 56 number/date units, 23 name units, 15 quoted spans; 19 documents with numbers or names, six with quotes. Quoted examples count alongside reported speech; rewording can trigger the checker without making a statement false.
Wilson intervals: z = 1.96; units can depend on each other. Public English corpus: training overlap unknown. Federal corpus: pending, excluded.
Meaning: text-embedding-3-large cosine similarity, clipped to 0 to 1. Reproduction: 12 published rows; maximum absolute difference 0.001089.
Fact reproduction: all eleven commercial tools’ counts matched before live runs. Counts and intervals recomputed from committed text-free records. CSV commercial dates: month precision for October outputs.
Settings and run records
Balanced: standard customer path after staff-only test path disabled; no settings sent. Expected: Balanced, General context, Auto formality, shared profile, rewrite mode. Light and Strong: only strength set. API: no effective-settings attestation or model, deployment or API-version identifier. Unmodified checker: v1; spaCy 3.8.7, en_core_web_sm 3.8.0.
- Balanced (API default): 9 November 2026, 12:17 to 12:25 UTC;
eval/results/facts/2026-11-09-balanced.json - Light: 9 November 2026, 10:32 to 10:37 UTC;
eval/results/facts/2026-11-09-light.json - Strong: 9 November 2026, 10:37 to 10:43 UTC;
eval/results/facts/2026-11-09-strong.json
Balanced (API default): groups and categories
Completed without warnings or observed kept sections: 18 documents; retention 8/8, 100.0% (67.6% to 100.0%) (selected cohort).
- All checked items: documents 15/20, 75.0% (53.1% to 88.8%); units 79/94, 84.0% (75.3% to 90.1%).
- Numbers and dates: documents 10/13, 76.9% (49.7% to 91.8%); units 49/56, 87.5% (76.4% to 93.8%).
- Names: documents 10/11, 90.9% (62.3% to 98.4%); units 22/23, 95.7% (79.0% to 99.2%).
- Quoted spans: documents 3/6, 50.0% (18.8% to 81.2%); units 8/15, 53.3% (30.1% to 75.2%).
- Numbers and names only: documents 15/19, 78.9% (56.7% to 91.5%); units 71/79, 89.9% (81.3% to 94.8%).
- academic essay: documents 1/1, 100.0% (20.7% to 100.0%); units 1/1, 100.0% (20.7% to 100.0%).
- application essay: documents 3/4, 75.0% (30.1% to 95.4%); units 5/8, 62.5% (30.6% to 86.3%).
- blog post: documents 5/6, 83.3% (43.6% to 97.0%); units 37/45, 82.2% (68.7% to 90.7%).
- business email: documents 1/3, 33.3% (6.1% to 79.2%); units 4/7, 57.1% (25.0% to 84.2%).
- discussion board: documents 1/1, 100.0% (20.7% to 100.0%); units 3/3, 100.0% (43.8% to 100.0%).
- marketing copy: documents 3/3, 100.0% (43.8% to 100.0%); units 6/6, 100.0% (61.0% to 100.0%).
- news article: documents 1/2, 50.0% (9.5% to 90.5%); units 23/24, 95.8% (79.8% to 99.3%).
Light: groups and categories
Completed without warnings or observed kept sections: 18 documents; retention 8/8, 100.0% (67.6% to 100.0%) (selected cohort).
- All checked items: documents 17/20, 85.0% (64.0% to 94.8%); units 88/94, 93.6% (86.8% to 97.0%).
- Numbers and dates: documents 12/13, 92.3% (66.7% to 98.6%); units 55/56, 98.2% (90.6% to 99.7%).
- Names: documents 11/11, 100.0% (74.1% to 100.0%); units 23/23, 100.0% (85.7% to 100.0%).
- Quoted spans: documents 4/6, 66.7% (30.0% to 90.3%); units 10/15, 66.7% (41.7% to 84.8%).
- Numbers and names only: documents 18/19, 94.7% (75.4% to 99.1%); units 78/79, 98.7% (93.2% to 99.8%).
- academic essay: documents 1/1, 100.0% (20.7% to 100.0%); units 1/1, 100.0% (20.7% to 100.0%).
- application essay: documents 4/4, 100.0% (51.0% to 100.0%); units 8/8, 100.0% (67.6% to 100.0%).
- blog post: documents 4/6, 66.7% (30.0% to 90.3%); units 40/45, 88.9% (76.5% to 95.2%).
- business email: documents 3/3, 100.0% (43.8% to 100.0%); units 7/7, 100.0% (64.6% to 100.0%).
- discussion board: documents 1/1, 100.0% (20.7% to 100.0%); units 3/3, 100.0% (43.8% to 100.0%).
- marketing copy: documents 3/3, 100.0% (43.8% to 100.0%); units 6/6, 100.0% (61.0% to 100.0%).
- news article: documents 1/2, 50.0% (9.5% to 90.5%); units 23/24, 95.8% (79.8% to 99.3%).
Strong: groups and categories
Completed without warnings or observed kept sections: 22 documents; retention 8/9, 88.9% (56.5% to 98.0%) (selected cohort).
- All checked items: documents 16/20, 80.0% (58.4% to 91.9%); units 89/94, 94.7% (88.1% to 97.7%).
- Numbers and dates: documents 11/13, 84.6% (57.8% to 95.7%); units 53/56, 94.6% (85.4% to 98.2%).
- Names: documents 9/11, 81.8% (52.3% to 94.9%); units 21/23, 91.3% (73.2% to 97.6%).
- Quoted spans: documents 6/6, 100.0% (61.0% to 100.0%); units 15/15, 100.0% (79.6% to 100.0%).
- Numbers and names only: documents 15/19, 78.9% (56.7% to 91.5%); units 74/79, 93.7% (86.0% to 97.3%).
- academic essay: documents 1/1, 100.0% (20.7% to 100.0%); units 1/1, 100.0% (20.7% to 100.0%).
- application essay: documents 4/4, 100.0% (51.0% to 100.0%); units 8/8, 100.0% (67.6% to 100.0%).
- blog post: documents 4/6, 66.7% (30.0% to 90.3%); units 42/45, 93.3% (82.1% to 97.7%).
- business email: documents 3/3, 100.0% (43.8% to 100.0%); units 7/7, 100.0% (64.6% to 100.0%).
- discussion board: documents 1/1, 100.0% (20.7% to 100.0%); units 3/3, 100.0% (43.8% to 100.0%).
- marketing copy: documents 1/3, 33.3% (6.1% to 79.2%); units 4/6, 66.7% (30.0% to 90.3%).
- news article: documents 2/2, 100.0% (34.2% to 100.0%); units 24/24, 100.0% (86.2% to 100.0%).