Claims Adjudication Validators: Fail-Closed Gates Before Payer Copilots Pay
CMS measured $28.83B in Medicare improper payments (FY2025); OIG audits show missing system edits enable silent overpay—fail-closed validators before payer LLM auto-pay.
Executive Summary
Payer operators evaluating large language model (LLM) adjudication copilots face a documented pattern: automation without deterministic gates scales payment errors, while versioned system edits and upstream validation reduce exposure when they work. Public evidence—not vendor offline accuracy decks—defines the stakes. Medicare Fee-for-Service (FFS) improper payments reached $28.83 billion at a 6.55% national rate in FY2025 (CMS CERT) [10]. OIG audits show $22.7 million in DMEPOS overpayments when edits failed—and a sharp drop after CMS fixed them in January 2020 [11]. Change Healthcare, processing roughly half of U.S. medical claims and 15 billion transactions annually, demonstrated concentration risk when its February 2024 outage stalled national payments [1][2]. Optum Real and Humana/Cohere upstream programs show intercept-at-submission architectures payers can mirror—within each source's stated scope [3][4][8][9]. The operator mandate: wrap probabilistic pay/deny suggestions in policy validators, HIPAA minimum-necessary boundaries, and model risk management (MRM) bundle control before auto-pay expand [5][6][7].
Research scope
This case log cites only published government audits, regulator guidance, press releases, and case studies with traceable URLs. It does not report payer-specific LLM adjudication wrongful-payment rates—no primary source publishes them. Validator-stack and human-in-the-loop (HITL) recommendations synthesize MRM and HIPAA requirements [5][6][7]; quantified outcomes below come from CMS, OIG, HHS, Reuters, Optum, Cohere, and KLAS-documented programs only.
The Challenge
Clearinghouse concentration. HHS reported Change Healthcare handles 15 billion health care transactions annually and is involved in one in every three patient records [1]. Reuters documented that Change processed roughly half of U.S. medical claims before the February 2024 ransomware attack; CMS later closed an advance-payment program that had issued billions in accelerated Medicare support to providers blocked from billing [2]. When a single adjudication rail fails, recovered pipelines need replayable validator outcomes—not silent auto-pay out of policy [1][2].
Payment-integrity scale. CMS CERT estimated $28.83 billion in Medicare FFS improper payments (FY2025 reporting period: claims submitted July 2023–June 2024) [10]. High-variance claim types—Part B at 8.4% improper, durable medical equipment (DME) at 24.1%—are where deterministic gates matter most before automation expands [10].
Public fact: Medicare FFS improper payments: 6.55% / $28.83B (FY2025) · Source: CMS CERT · Citation: [10]
Public fact: Change: 15B transactions; 1 in 3 patient records · Source: HHS · Citation: [1]
Public fact: Change: 50% of US medical claims (pre-outage) · Source: Reuters · Citation: [2]
Public fact: DMEPOS inpatient improper pay: $22.7M (2018–2024) · Source: OIG · Citation: [11]
Public fact: Virtual check-in potentially improper: $2.26M; 183,524 lines · Source: OIG · Citation: [12]
Where gates matter first. CMS CERT breaks improper exposure by claim type—not as a single headline rate. DME's 24.1% improper rate sits far above the national 6.55% FFS average, while Hospital IPPS runs 3.2% [10]. Dollar volume tells a complementary story: Part A accounts for $16.9 billion in improper payments despite a lower rate than Part B, because Part A claim volume dominates the program [10]. Payer operators sizing LLM auto-pay pilots should rank workflows by both rate variance and dollar exposure—CERT gives a public benchmark for that prioritization without inventing payer-specific wrongful-payment rates.
Figure 4: Medicare FFS improper payments by claim type — rate and dollar heatmap (CMS CERT FY2025)
Measured: CMS CERT FY2025 supplemental data [10].
The heatmap pairs improper rates with improper dollars by claim type. DME's rate outlier (24.1%) signals high clawback risk per decision even though DME improper dollars ($2.3B) are smaller than Part A ($16.9B) or Part B ($9.6B) [10]. That split is why fail-closed validators should attach to high-variance claim paths before auto-pay expand—not because LLM copilots are inherently unsafe, but because CERT shows where silent auto-approval scales the largest measured Medicare errors.
Figure 1: Medicare FFS improper payment rates by claim type (CMS CERT FY2025)
Measured: CMS CERT FY2025 supplemental data [10]. Part A 5.4%, Part B 8.4%, Hospital IPPS 3.2%, DME 24.1% improper payment rates.
Risk & Privacy Lane
Model risk (MRM). Federal Reserve SR 11-7 and NIST's Artificial Intelligence Risk Management Framework require independent validation and versioned change control before production promote [5][6]. For adjudication copilots, bundle prompt templates, retrieval corpus hash, rules-engine version, and validator suite under one change ticket. MRM sign-off should block promote when production payment-integrity monitors diverge from approved bundles (see Eval Pays Rent for the eval-gate pattern—conceptual cross-link; no quantified adjudication metrics claimed here).
Privacy / HIPAA. The HHS Office for Civil Rights (OCR) Privacy Rule requires minimum necessary use of protected health information (PHI) [7]. Adjudication paths carry member identifiers, dates of service, and coded claim lines; clinical narratives and attachments raise breach and retention risk (compare HIPAA fail-closed validators for clinical summarization paths—this article addresses pay/deny gates).
HIPAA minimum necessary: Allow coded claim lines and member ID/date of service; block full clinical narratives and raw vendor prompt dumps in production logs [7].
PHI field class: Member ID, DOS, CPT/ICD lines · Pre-LLM adjudication copilot: Allow (minimum necessary) · Regulatory basis: HIPAA Privacy Rule [7]
PHI field class: Full clinical note / attachment text · Pre-LLM adjudication copilot: Block · Regulatory basis: Minimum necessary [7]
PHI field class: Adjuster override reason · Pre-LLM adjudication copilot: Allow + immutable log · Regulatory basis: Audit replay (MRM [5])
PHI field class: Vendor prompt/response raw dump · Pre-LLM adjudication copilot: Block in prod logs · Regulatory basis: Minimum necessary + subprocessor risk [7]
Payment integrity (measured). OIG found Medicare improperly paid $22.7 million for DMEPOS furnished during inpatient stays (January 2018–December 2024); $18.2 million occurred before CMS fixed system edits in January 2020 versus $4.5 million after [11]. A separate OIG audit identified 183,524 potentially improper payments totaling $2.26 million for virtual check-in and e-visit services when automated system edits were absent; CMS concurred on implementing corrective edits [12]. Optum press materials cite $20 billion in annual U.S. hospital claim-overturn spend and 84% of first-time denials as avoidable (Optum press release—not independent audit) [9].
The Approach
Public payer programs illustrate deterministic gates before probabilistic steps:
Optum Real surfaces payer contract and coverage rules at claim submission; UnitedHealthcare is the first health plan adopter per Optum and Healthcare Dive reporting [3][9]. Optum's Allina Health pilot cited fewer administrative errors across 5,000+ outpatient visits (radiology/cardiology) in press materials [9].
Humana expanded Cohere Health prior-authorization automation to imaging and sleep after musculoskeletal pilots [4]—upstream clinical-guideline gates, not pay/deny adjudication.
Humana / athenahealth / Availity electronic prior authorization (EPA): KLAS case study reported 70% of requests instantly approved during an evaluation period (prior auth scope only) [8].
Validator stack (synthesis from MRM + HIPAA + OIG edit precedent [5][6][7][11][12]):
Format & plausibility — Valid codes, allowed place of service, duplicate claim keys.
Payer policy engine — LCD/NCD and contract rules; fail-closed on missing authorization or conflicts.
Dollar and confidence thresholds — Route high-dollar or low-confidence LLM suggestions to adjuster queues (HITL).
PHI allowlist — Enforce minimum necessary per table above [7].
MRM promote gate — Bundle version sign-off before auto-pay expand [5][6].
Compliance should answer an OIG-style replay question: show validator outcomes and model bundle hash for claim X on date Y [11].
CERT error categories anchor gate design. CMS attributes gross improper payments to three primary error types: insufficient documentation (3.5% of claims), medical necessity (1.0%), and incorrect coding (0.8%) [10]. Deterministic validators map cleanly to these categories—documentation completeness checks, medical-necessity rule engines, and coding plausibility gates—before an LLM proposes pay/deny/adjust. The categories do not sum to the net national rate because CERT applies adjustments; they still define which gate types public Medicare data says matter most.
Figure 6: Medicare FFS gross error categories (CMS CERT FY2025)
Measured: CMS CERT FY2025 supplemental data [10].
The Results
Medicare improper-payment exposure (CMS CERT, measured). National FFS improper payments: 6.55% / $28.83B (FY2025) [10]. DME's 24.1% rate underscores high-variance paths where auto-adjudication without gates carries disproportionate clawback risk [10].
Figure 2: DMEPOS improper payments before vs after CMS system-edit fix
Measured: OIG audit [11]. ~$18.2M improper Jan 2018–Dec 2019 (broken edits) vs. $4.5M Jan 2020–Dec 2024 (edits fixed); $22.7M audit total.
Service-type split (OIG, measured). The virtual check-in audit is not a single lump-sum finding—it breaks into virtual check-in versus e-visit billing paths. OIG identified $1.96 million potentially improper across 173,287 virtual check-in payment lines and $0.30 million across 10,237 e-visit lines [12]. Both paths lacked automated system edits at audit time; CMS concurred on implementing corrective edits. For payer operators, the lesson is granular: missing gates on new service types accumulate line-level exposure even when headline dollars look small relative to CERT program totals.
Figure 5: OIG-identified improper payments by service type
Measured: OIG audit of virtual check-in and e-visit services [12].
Missing edits enable improper pay (OIG, measured). Virtual check-in and e-visit audits document $2.26 million potentially improper across 183,524 payment lines when system edits were absent [12].
Figure 3: OIG-identified improper virtual check-in and e-visit payments
Measured: OIG audit period January 2019–December 2022; CMS implementing corrective system edits [12].
Audit scope vs improper dollars (OIG, measured). OIG audited $24.15 million in Medicare payments for virtual check-in and e-visit services—$12.48 million virtual check-in and $11.67 million e-visit—against $2.26 million potentially improper [12]. The ratio matters for monitor design: production payment-integrity sampling must cover audited-scope denominators, not vendor golden-set accuracy alone.
Figure 7: OIG audit scope — audited payments vs potentially improper
Measured: OIG audit of virtual check-in and e-visit services [12]. $24.15M audited total; $2.26M potentially improper.
Upstream validation (measured, scope-limited). Humana EPA case: 70% instant approvals in evaluation period—prior auth, not pay/deny adjudication [8]. Optum Real: real-time validation at submission; UHC first adopter [3][9].
Program: CMS CERT FY2025 · Measured public outcome: 6.55% / $28.83B improper (Medicare FFS) · Scope limit: Medicare FFS sample, not commercial payers [10]
Program: OIG DMEPOS audit · Measured public outcome: $22.7M improper; drop post-edit fix · Scope limit: DMEPOS during inpatient stays [11]
Program: OIG virtual check-in · Measured public outcome: $2.26M potentially improper · Scope limit: Virtual check-in / e-visit billing [12]
Program: Humana EPA (KLAS) · Measured public outcome: 70% instant approval (eval period) · Scope limit: Prior auth only [8]
Program: Optum Real (press) · Measured public outcome: UHC first adopter; Allina 5,000+ visit pilot · Scope limit: Press release / trade press [3][9]
What Went Wrong
When system edits fail, improper payments accumulate. OIG's DMEPOS follow-up audit is the receipt: $18.2 million improperly paid while edits were broken, versus $4.5 million after the January 2020 fix [11]. Virtual check-in billing repeated the pattern—$2.26 million potentially improper without automated gates [12].
When national rails stall, payment integrity is a resilience test. Change Healthcare's outage affected claims processing at national scale; HHS and Reuters document payment disruption and CMS's wind-down of emergency advance payments [1][2]. Recovered pipelines must not compensate with unlogged auto-pay.
When a clearinghouse handling fifteen billion transactions a year fails, validators and replay matter as much as uptime [1][2].
What to Do Next
First 30 days: Map adjudication copilot PHI paths to HIPAA minimum necessary [7]; assign a named owner for one workflow_id.
By 60 days: Pilot fail-closed policy validators on highest CERT-variance claim types (e.g., DME, Part B) [10]; shadow-mode production monitors before auto-pay expand.
By 90 days: Bundle prompt + retrieval + rules + validators under one MRM change ticket [5][6]; benchmark exposure against CMS CERT and OIG audit questions—not vendor offline accuracy alone [10][11].
Key Takeaways
Billions measured: CMS CERT: $28.83B Medicare FFS improper payments (FY2025) [10].
Edits work: OIG DMEPOS audit—$22.7M improper; ~80% reduction on audited path after edit fix [11].
No edits, improper pay: OIG virtual check-in—$2.26M / 183,524 lines [12].
Concentration risk: Change—15B transactions, ~50% of US claims [1][2].
Upstream gates (scope-limited): Optum Real [3][9]; Humana/Cohere EPA 70% instant approvals (prior auth only) [4][8].
MRM + HIPAA: Bundle version control [5][6]; minimum necessary field allowlists [7].
References
U.S. Department of Health and Human Services. (2024, March 10). Letter to health care leaders on cyberattack on Change Healthcare. https://www.hhs.gov/about/news/2024/03/10/letter-to-health-care-leaders-on-cyberattack-on-change-healthcare.html
Reuters. (2024, June 17). US to stop advance payments for Medicare providers hit by Change hack. https://www.reuters.com/business/healthcare-pharmaceuticals/us-shut-advance-payments-program-medicare-providers-hit-by-change-hack-2024-06-17/
Healthcare Dive. (2025). Optum launches AI system to speed medical claims. https://www.healthcaredive.com/news/optum-real-ai-speed-claims-review-united-health/803448/
Cohere Health. (2024, April 23). Cohere Health and Humana expand prior authorization partnership. https://www.coherehealth.com/news/cohere-humana-expand-prior-authorization-imaging-sleep
Board of Governors of the Federal Reserve System. (2011). SR 11-7: Guidance on Model Risk Management. https://www.federalreserve.gov/supervisionreg/srletters/sr1107.htm
National Institute of Standards and Technology. (2023). Artificial Intelligence Risk Management Framework (AI RMF 1.0). https://www.nist.gov/itl/ai-risk-management-framework
U.S. Department of Health and Human Services. HIPAA Privacy Rule. https://www.hhs.gov/hipaa/for-professionals/privacy/index.html
Business Wire. (2024, June 4). athenahealth, Availity, Harmony Park Family Medicine, and Humana honored with 2024 KLAS Points of Light Award. https://www.businesswire.com/news/home/20240604470805/en/athenahealth-Availity-Harmony-Park-Family-Medicine-and-Humana-Honored-with-a-2024-KLAS-Points-of-Light-Award-for-Modernizing-Prior-Authorization-Process
Optum. (2025, October 21). Optum reinvents claims & reimbursement process (Optum Real). https://www.optum.com/en/newsroom/health-tech/optum-reinvents-claims-reimbursement-process.html
Centers for Medicare & Medicaid Services. Comprehensive Error Rate Testing (CERT); FY2025 Medicare FFS improper payment data. https://www.cms.gov/data-research/monitoring-programs/improper-payment-measurement-programs/comprehensive-error-rate-testing-cert · Supplemental: https://www.cms.gov/files/document/nov-2025-medicare-ffs-supplemental-improper-payment-data-2025922.pdf
U.S. Department of Health and Human Services, Office of Inspector General. (2025). Medicare improperly paid suppliers $22.7 million over 7 years for DMEPOS provided to enrollees during inpatient stays. https://oig.hhs.gov/reports/all/2025/medicare-improperly-paid-suppliers-227-million-over-7-years-for-durable-medical-equipment-prosthetics-orthotics-and-supplies-provided-to-enrollees-during-inpatient-stays/
U.S. Department of Health and Human Services, Office of Inspector General. (2026). CMS could strengthen Medicare program safeguards to prevent and detect potentially improper payments for virtual check-in and e-visit services. https://oig.hhs.gov/reports/all/2026/cms-could-strengthen-medicare-program-safeguards-to-prevent-and-detect-potentially-improper-payments-for-virtual-check-in-and-e-visit-services/
Monday Morning Checklist
[ ] Map PHI field allowlist to HIPAA minimum necessary [7].
[ ] Bundle prompt + retrieval + rules + validators under one MRM change ticket [5][6].
[ ] Benchmark payment-integrity exposure against CMS CERT rates by claim type [10].
[ ] Design OIG-style replay: validator outcome + bundle hash per claim [11].
[ ] Pilot fail-closed validators on high-variance claim types (Part B, DME per CERT [10]) before auto-pay expand.
[ ] Review clearinghouse concentration in business continuity plan [1][2].
[ ] Scope upstream benchmarks correctly: Humana EPA is prior auth, not pay/deny adjudication [8].
Editorial transparency. Essays at The AI Operator may use AI-assisted research, drafting, and editing tools under staff editorial review. Facts, figures, and recommendations are checked before publication; we correct the record when evidence changes. Questions: hello@theaioperator.net.
Published on [Substack](https://theaioperator2.substack.com/p/claims-adjudication-validator-stack).