Allegheny County DHS
Verified evidenceExceptions onlyContinuous decisioningCreator to judgeAdministrative data can standardize triage only with override and disparity review.
Healthcare screening · Pennsylvania, USA
Collections: Human still decides · Regulated autonomy
Executive brief
Public risk scores require contestable targets and override logs.
AI value · Screening accuracy and disparity
VerifiedImproved identification and modest reduction in racial disparity in case openings
Outcome summary links an independent evaluation; fairness concerns remain contested.
Before
Receives an eligible General Protective Service referral and reviews allegations and available county records. → Decide whether to open an investigation without an algorithmic risk score.
After
Uses administrative data to produce a 1–20 future-placement risk score for eligible referrals. → Combine score with qualitative context and decide screen-in or screen-out.
Human boundary
Humans make final screening decisions; mandatory abuse referrals bypass the model.
Why it matters
Administrative data can standardize triage only with override and disparity review.
Before
How the work ran before the change.
Child-welfare hotline screener
Receives an eligible General Protective Service referral and reviews allegations and available county records.
ControlAbuse and severe-neglect reports are mandatory screen-ins and are outside model discretion.
Screener and supervisor
Decide whether to open an investigation without an algorithmic risk score.
ControlHuman policy and contextual judgment; no baseline cycle time is published.
What changed
Administrative data can standardize triage only with override and disparity review.
Decision rightAI handles the default; humans own exceptions
After
How the same work runs now.
AFST
Uses administrative data to produce a 1–20 future-placement risk score for eligible referrals.
ControlScore is one input; target is placement within two years, not a direct prediction of abuse.
Screener and supervisor
Combine score with qualitative context and decide screen-in or screen-out.
ControlHumans may override; high-score override requires documented explanation.
Exception path
Overrides are allowed and high-score overrides require explanation.
Work removed
Decision authority
Humans make final screening decisions; mandatory abuse referrals bypass the model.
Before
Child-welfare hotline screener
Receives an eligible General Protective Service referral and reviews allegations and available county records.
Control: Abuse and severe-neglect reports are mandatory screen-ins and are outside model discretion.
Screener and supervisor
Decide whether to open an investigation without an algorithmic risk score.
Control: Human policy and contextual judgment; no baseline cycle time is published.
After
AFST
Uses administrative data to produce a 1–20 future-placement risk score for eligible referrals.
Control: Score is one input; target is placement within two years, not a direct prediction of abuse.
Screener and supervisor
Combine score with qualitative context and decide screen-in or screen-out.
Control: Humans may override; high-score override requires documented explanation.
Work that left the path
Human role before
Hotline screeners reviewed allegations and available county records, with a supervisor supporting the screen-in or close-at-intake decision under mandatory-referral rules.
Human role after
Screeners combine score with context; supervisors govern overrides.
AI roleGenerates a 1–20 future-placement risk score for eligible referrals.
Pre-AFST decisions→Improved identification and modest reduction in racial disparity in case openings
Through March 2019 · Eligible GPS hotline referrals
Outcome summary links an independent evaluation; fairness concerns remain contested.
Anti-pattern
Do not call the score an abuse prediction.
Questions
Portability conditions
Reputation risk
high
Evidence and authority
Current · updated
1 independent, 1 primary; publication outcomes are verified.
Bundle 1.0.0 · reviewed 2026-08-23 · stable ID 02134d08a5e8cfdc
Capital Region of Denmark mammography screening programme · Exceptions only
That every screening mammogram must be independently double-read by two radiologists to keep cancer detection and recall rates acceptable.
Waymo · Exceptions only
Within a defined domain, automated driving can control the vehicle while remote assistance handles exceptions.
New York City Fire Department · Exceptions only
Fire companies can prioritize inspections by predicted risk instead of fixed cycles and local knowledge.
Pacific Gas and Electric Company · Exceptions only
That crews should be dispatched mainly after a fault, outage, or visible field report.