Photo checks
Each case is a real recall photo or a copy made from one. The checks keep the refund pool for real owners: a picture that is not the owner’s own is asked to be retaken, kindly.
The recorded runs
| The photo | Recall | Decision | Decided by | Time |
|---|---|---|---|---|
| Brightline's sample charger, cord cutPublic-domain photos of one charger, before and after its cord was cut (Dmitry G, Wikimedia Commons). Sample recall; fictional. | BH-2026-01 | Verified | Right product confirmed by the two reviewers (the photo check scored 0.80, under the pass mark) Vision, Claude Opus 5.5Verified by two agreeing reviewers over soft items (C1 unclear). firm amount 34.99; no linked capture; refund 34.99 USD (firm, the lower); auto cap 50.00 USD. Reviewers: Product reviewer: meets, confidence 0.75; evidence: [right_product] Black charger housing with rounded end caps, opened and lying on its side, matching the shape of the recalled charger body (right side of the photo); [right_product] Black ribbed cord strain relief like the one where the cord meets the charger in official image 2, now pulled out of the housing (left center, held between the fingers); [right_product] Black fixed power cord running from the charger toward a barrel style DC plug (center of the photo, plug near top center); [made_safe] Cord cut at the strain relief with a white insulated wire and two bare stranded wires exposed (center left, sticking out to the right of the strain relief); [made_safe] Yellow insulating sheets removed from inside the housing, one with holes cut in it (top center and top right); [own_photo] Owner's thumb and finger in the foreground, shallow focus with the background blurred, real shadows and a seam line on the white surface (left edge foreground and across the background) | Remedy reviewer: meets, confidence 0.7; evidence: [made_safe] Black power cord end with its molded strain relief collar, with bare stranded metal wires and a white insulated wire sticking out, showing the cord has been severed at the collar where it joined the case (left center, held between the fingers); [made_safe] The separated cord runs loose across the table with no charger body attached to this end (center and bottom of the photo, looping cable); [right_product] Black charger housing, opened, lying on the table near the cut cord (right side of the photo); [right_product] Yellow and black insulating pieces from inside the charger case (top center and top right) | 19.5 s |
| The recall's own picture, heaterThe CPSC's official photo of the DREO space heater. | 27054 | Retake | Your own photo: it matches the recall's official picture No AI callRetake (own_photo). Own-photo check: Matches this recall's official image https://cpsc.gov/s3fs-public/DREO1.png?VersionId=37rR7xQxj_D2T3WYYPVS6bf6pWt6LZqC ("Recalled DREO Space Heater - model DR-HSH015") at 0 bits (upright). dHash 3878686ef8a63c38; nearest official 0 bits, nearest earlier claim none (reused at 10 or less). | 139 ms |
| The recall's own pictureThe CPSC's official photo of the HAIARA power bank, sent as if it were the owner's. | 27056 | Retake | Your own photo: it matches the recall's official picture No AI callRetake (own_photo). Own-photo check: Matches this recall's official image https://cpsc.gov/s3fs-public/HAIARA1.jpg?VersionId=myWygu.qNAqSjgZDm8rx5DlFPHoyO9tW ("HAIARA power bank, model W5 PRO (front)") at 0 bits (upright). dHash e48280e0c0c1c9e0; nearest official 0 bits, nearest earlier claim none (reused at 10 or less). | 59 ms |
| Rotated and recompressed copyThe official power bank photo turned 90 degrees, shrunk and saved again at low quality. | 27056 | Retake | Your own photo: it matches the recall's official picture No AI callRetake (own_photo). Own-photo check: Matches this recall's official image https://cpsc.gov/s3fs-public/HAIARA1.jpg?VersionId=myWygu.qNAqSjgZDm8rx5DlFPHoyO9tW ("HAIARA power bank, model W5 PRO (front)") at 1 bits (turned 270). dHash bcbebe78783a7e38; nearest official 1 bits, nearest earlier claim none (reused at 10 or less). | 55 ms |
| Tilted and cropped copyThe heater's label photo tilted 4 degrees and cropped 6 percent on each side. | 27054 | Retake | Your own photo: it matches the recall's official picture No AI callRetake (own_photo). Own-photo check: Matches this recall's official image https://cpsc.gov/s3fs-public/DREO2.jpg?VersionId=XQ9PGzTKG443Y4cwtuigpQ93h0DOrv6B ("The manufacture date, in YYYY/MM format, and model “DR-HSH015” are printed on a label on the underside of the recalled DREO space heaters.") at 10 bits (upright). dHash d01607070f0d07b4; nearest official 10 bits, nearest earlier claim none (reused at 10 or less). | 81 ms |
| A photo already usedA full-size photo of a power bank, turned and saved again, that an earlier claim on this recall already sent. | 27056 | Firm review | It matches a photo sent with an earlier claim No AI callFirm review: C1 skipped, C2 skipped, C3 fail, C4 skipped. no firm amount; no linked capture; no refund amount; no auto cap. | 200 ms |
| A blurry photoA photo with heavy blur, the way a shaky hand takes it. | 27056 | Retake | Photo quality No AI callRetake (quality). Quality precheck: sharpness 4.9 (min 35), mean luma 124, p95 199, p05 58, size 1280x960 (min side 240). Issues: blurry. | 142 ms |
| Another recall's pictureThe CPSC's made-safe photo from hair dryer recall 26125, sent for a different hair dryer recall, 26754. | 26754 | Retake | Your own photo: it matches the recall's official picture No AI callRetake (own_photo). Own-photo check: Matches recall 26125's official image https://www.cpsc.gov/s3fs-public/RecalledMyOnlyStylerRootBoosterHairDryer.png ("Recalled MyOnlyStyler Root Booster Hair Dryer with the power cord cut in half for disposal") at 0 bits (upright), in the check against every recall's published pictures at 4 bits or under. dHash e0808ed69ce174f8; nearest official 19 bits, nearest earlier claim none (reused at 10 or less). | 98 ms |
| Cord cut in halfThe photo the CPSC published with recall 26125 to show the hair dryer made safe: its power cord cut in half. | 26125 | Retake | Your own photo unclear: Claude saw the CPSC's published picture; right product 0.95; made safe not confirmed Vision, Claude Opus 5.5Retake (made_safe). Retake: cut_cord (not_visible): The cord appears cut through, with a cut end showing exposed wire near the top center and a second loose end near the velcro tie on the upper left. However, this image is identical to the official CPSC recall image, so it may not show the owner's own dryer.. Model tip: Please take your own new photo of your dryer with the cut cord, and turn the handle so the model number on the back shows. | 12.3 s |
| Something else entirelyA photo of a coffee mug (Kurt Kaiser, CC0, Wikimedia Commons) sent for the power bank recall. | 27056 | Retake | Right product, scored 0.02; made safe not confirmed Vision, Claude Opus 5.5Retake (made_safe). Retake: write_recalled (missing): No handwriting found on the product.. Model tip: Please take a photo of the whole black power bank with the word recalled written on it in permanent marker. | 8.0 s |
Brightline's sample charger, cord cut19.5 s
Right product confirmed by the two reviewers (the photo check scored 0.80, under the pass mark)
The recall's own picture, heater139 ms
Your own photo: it matches the recall's official picture
The recall's own picture59 ms
Your own photo: it matches the recall's official picture
Rotated and recompressed copy55 ms
Your own photo: it matches the recall's official picture
Tilted and cropped copy81 ms
Your own photo: it matches the recall's official picture
A photo already used200 ms
It matches a photo sent with an earlier claim
A blurry photo142 ms
Photo quality
Another recall's picture98 ms
Your own photo: it matches the recall's official picture
Cord cut in half12.3 s
Your own photo unclear: Claude saw the CPSC's published picture; right product 0.95; made safe not confirmed
Something else entirely8.0 s
Right product, scored 0.02; made safe not confirmed
What each case shows, and how it was run
- Brightline's sample charger, cord cut: Public-domain photos of one charger, before and after its cord was cut (Dmitry G, Wikimedia Commons). Sample recall; fictional.
- The recall's own picture, heater: The CPSC's official photo of the DREO space heater.
- The recall's own picture: The CPSC's official photo of the HAIARA power bank, sent as if it were the owner's.
- Rotated and recompressed copy: The official power bank photo turned 90 degrees, shrunk and saved again at low quality.
- Tilted and cropped copy: The heater's label photo tilted 4 degrees and cropped 6 percent on each side.
- A photo already used: A full-size photo of a power bank, turned and saved again, that an earlier claim on this recall already sent.
- A blurry photo: A photo with heavy blur, the way a shaky hand takes it.
- Another recall's picture: The CPSC's made-safe photo from hair dryer recall 26125, sent for a different hair dryer recall, 26754.
- Cord cut in half: The photo the CPSC published with recall 26125 to show the hair dryer made safe: its power cord cut in half.
- Something else entirely: A photo of a coffee mug (Kurt Kaiser, CC0, Wikimedia Commons) sent for the power bank recall.
The free checks run before any AI call: the photo’s quality, and a 64-bit perceptual hash compared with the recall’s official CPSC images and every earlier claim photo, in all eight orientations (four turns, each mirrored or not). A match within 10 bits asks for the owner’s own photo, and within 4 bits of a picture published with any other recall or a photo claimed on one, the same. That catches the official picture, a turned and recompressed copy, a tilted and cropped copy, and a photo sent before. The same hash is then compared with the 1,779 published pictures of other recalls that are distinct enough to tell their recall apart (at least 5 bits from every picture of a different recall; 1,825 are stored, making 1,794 distinct hash values), and with every photo already claimed on any recall at any firm, within 4 bits: a re-saved, halved or turned copy of a published picture stays within 2 bits, and none of the test photos that are not a published picture comes within 9. Between firms, only the hash is compared, never the photo.
The vision check reads the photo against the proof rule: the product, each required action, and signs of a screen or printout. The decision, the amount and the payment route are plain code. Unclear cases go to two independent AI reviewers and a referee, and then to a person at the firm.
Brightline's sample charger, cord cut: Brightline Home's sample recall BH-2026-01, refund $34.99, auto-approve cap $50.00. One live Claude Opus 5.5 run recorded 2026-11-08 ($0.1158), replayed here. Your photo checks out.
Cord cut in half: Official-picture match turned off for this run, so the AI check can be seen; reviewers off. Please cut the cord first, then take a new photo.
Something else entirely: Reviewers off. Please write RECALLED on the product first, then take a new photo.
The cord-cut case is the owner flow’s recorded answer for this photo, read with the recall’s official pictures and replayed through the verifier, so it matches the Try page and the home page. Its time is that one live call.
Recorded November 8, 2026 by scripts/public/record-runs.ts. The two vision runs cost $0.02 together. The user’s own photos of products made safe are added as they are taken.
One reviewer against the referee
At the claim path’s setting, 49 recorded decisions: 42 were settled by the six checks before any reviewer ran, and 7 went to the two reviewers and the referee. The sweep kept the referee’s decision for every one of those; it kept what each reviewer said on its own in one row (the recorded preset), so a single reviewer can be set against the referee only there. The sweep script now records both votes for the next run.
| Case | Run | Referee | Final | Known answer | Product reviewer | Remedy reviewer |
|---|---|---|---|---|---|---|
| look-alike-hair-dryer | sweep, trial 1 | A person decides | Firm review | Do not pay | not recorded | not recorded |
| made-safe-hair-dryer-cord-cut | sweep, trial 2 | A person decides | Firm review | Reading only | not recorded | not recorded |
| look-alike-hair-dryer | sweep, trial 2 | A person decides | Firm review | Do not pay | not recorded | not recorded |
| sample-charger-cut | sample sweep, trial 1 | Both agree: pay | Verified | Pay | not recorded | not recorded |
| sample-charger-cut | sample sweep, trial 2 | Both agree: pay | Verified | Pay | not recorded | not recorded |
| sample-charger-cut | sample sweep, trial 3 | A person decides | Firm review | Pay | not recorded | not recorded |
| sample-charger-cut | recorded preset | Both agree: pay | Verified | Pay | meets, 0.75 | meets, 0.70 |
look-alike-hair-dryerFirm review
Each reviewer’s own vote was not recorded in this run.
made-safe-hair-dryer-cord-cutFirm review
Each reviewer’s own vote was not recorded in this run.
look-alike-hair-dryerFirm review
Each reviewer’s own vote was not recorded in this run.
sample-charger-cutVerified
Each reviewer’s own vote was not recorded in this run.
sample-charger-cutVerified
Each reviewer’s own vote was not recorded in this run.
sample-charger-cutFirm review
Each reviewer’s own vote was not recorded in this run.
sample-charger-cutVerified
Product reviewer meets, 0.75; Remedy reviewer meets, 0.70.
- The referee, as designed
- Paid when it should not have been: 0 of 2
- Held when it could have been paid: 1 of 4
- 7 rows
- Product reviewer alone
- Paid when it should not have been: 0 of 0
- Held when it could have been paid: 0 of 1
- 1 row
- Remedy reviewer alone
- Paid when it should not have been: 0 of 0
- Held when it could have been paid: 0 of 1
- 1 row
On the one row with recorded votes, either reviewer alone would have paid what the referee paid; the referee's extra step did not change a payment there. The referee's own record is wider: of the 7 rows it saw, it sent 4 to a person and paid none it should not have. The reason the two reviewers read two different rulebooks, and why a disagreement goes to a person, is in where the design comes from, below.
A reviewer alone pays when its vote says meets and the evidence check counted it. Known answers: Pay and Do not pay from content/verify/fixtures/cases.json; Reading only is a published picture sent with the official-picture match off, outside both money rates. Read by src/components/evidence/reviewer-data.ts from content/verify/sweep-results.json, content/verify/sweep-sample.json, content/presets/recorded/*.json, content/verify/fixtures/cases.json; tests/reviewers-recorded.test.ts counts the files again.
The sample recall’s reference pictures
In all three sweep trials and the recorded preset, the sample charger with its cord cut scored 0.75 on the right-product check, under the 0.85 bar, and was verified only through the two reviewers. The model named what it could not compare: the case is opened and no label is in the owner's photo. The second reference picture it was given was a 440 by 340 crop of the whole-charger picture, mostly the area where the label had been removed, with the cord's strain relief small at its lower edge. A firm's recall notice shows the product, the place the remedy acts on, and its connectors, so the sample recall now publishes three pictures of the same public-domain charger: the whole charger (unchanged), where the cord meets the case (a new crop of the original photo below the label, so no brand mark is in it), and the DC plug (from a second public-domain photo of the same charger). The claim path still sends the first two to Claude.
Brightline 65W laptop charger (sample) 
Where the cord meets the charger 
The charger's DC plug
Right-product score for the sample charger with its cord cut: 0.75 with the earlier pictures, 0.80 to 0.82 with these, in 4 runs each. Both stay under the 0.85 pass mark, so the two reviewers look closer each time. With these pictures, 3 of 4 were verified, and one was sent to a person at the firm. The pass marks and the prompts did not change.
| Run | Before, right product | Before, outcome | After, right product | After, outcome |
|---|---|---|---|---|
| Recorded preset | 0.75 | verified by the two reviewers | 0.80 | verified by the two reviewers |
| Sweep trial 1 | 0.75 | verified by the two reviewers | 0.80 | verified by the two reviewers |
| Sweep trial 2 | 0.75 | verified by the two reviewers | 0.80 | verified by the two reviewers |
| Sweep trial 3 | 0.75 | verified by the two reviewers | 0.82 | sent to a person at the firm |
Changed Nov 8, 2026. Before: content/verify/history/sweep-sample.2026-11-07.json and content/verify/history/sample-charger-cut.2026-11-07.json. After: content/verify/sweep-sample.json and content/presets/recorded/sample-charger-cut.json. Why and which pictures: content/verify/sample-references.json; credits in public/samples/charger/CREDITS.json.
What the model read on the CPSC’s own photos
With the official-picture match off, the claim path’s setting read the CPSC’s own photos of 3 recalled products made safe, 3 trials each: right product 9 of 9, made safe 7 of 9, 16 of 18 readings in all.
- Hair dryer, cord cut (recall 26125): asked for a new photo in 2 of 3, each time by the made-safe check (C2). The model did not confirm the cut, and its own-photo check came back unsure in each: it recognized the CPSC's published picture. On a claim, the hash match answers this picture first.
- Helmet, straps cut (recall 25007): verified in each of 3 trials, both readings right.
- Helmet in the trash (recall 25007): asked for a new photo in 3 of 3, each time by the made-safe check (C2). That is the right reading: the recall asks for two photos, the straps cut and the helmet in the trash, and this one does not show the straps.
Where the two-reviewer design comes from
Two reviewers and a referee is a design with a lineage. The published pattern is one model reviewer that abstains on its own confidence, or a panel of different models. The finding that panels from one model family share errors is why the two reviewers here read two different rulebooks rather than two model names.
- Zheng and others, Judging LLM-as-a-Judge, 2023. A model as the reviewer of another's output, with its position and verbosity biases measured.
- Madras, Pitassi and Zemel, Predict Responsibly, 2018. Learning to defer: a model that hands the cases it should not decide to a person.
- Jung, Brahman and Choi, Trust or Escalate, 2024. A model reviewer that answers only when confident and escalates the rest, with a guaranteed level of agreement with people.
- Badshah, Emami and Sajjad, Judge, Retrieve, or Abstain, 2026. When unsure, fetch evidence and judge again; still unsure, abstain and flag the case for human review.
- Kohli, Nine Judges, Two Effective Votes, 2026. Nine frontier models from seven families made the same mistakes on the same items, so a panel of different model names gave about two independent votes.
The part that is new here. The Product reviewer sees only the recall’s official pictures; the Remedy reviewer sees only the remedy text. Each must cite what it saw, a referee in code counts only votes whose evidence checks out, and a disagreement goes to a person at the firm, in payments: nothing is paid until that person decides. “The agent proposes, a person approves” is a pattern many payment agents share this year; the two rulebooks and the abstention on disagreement are the design choice. The reviewers and the referee: src/lib/verify/reviewers.ts; the decision: src/lib/verify/decide.ts; tests/verify.test.ts. See the two reviewers on a look-alike claim
The two reviewers on one model and on two
The claim path's own setting, run twice: first with both reviewers on Claude Opus 5.5, the default, then with the Remedy reviewer on Claude Sonnet 5.5 (REMEDY_REVIEWER_MODEL_REMEDY) and the Product reviewer unchanged.
Both runs read the 3 photos that reach the two reviewers, 5 trials each: a look-alike hair dryer (known answer: do not pay), the CPSC's own photo of the recall 26125 hair dryer with its cord cut (a reading, in neither money rate) and Brightline's sample charger with its cord cut (known answer: pay). The second run re-read the photo check and the Product reviewer's vote from the first run's replay cache, so the Remedy reviewer's model is the only difference.
| Measure | Both on Opus 5.5 (the default) | Remedy reviewer on Sonnet 5.5 |
|---|---|---|
| Same answer from both reviewers | 4 of 10 | 0 of 13 |
| Paid when it should not have been | 0 of 5 | 0 of 5 |
| Held when it could have been paid | 1 of 5 | 5 of 5 |
| Cost per claim, list price | $0.0562 | $0.0541 |
What it shows. A second model made the pair no safer on these photos: neither run paid a photo it should not have (0 of 5 and 0 of 5), and with the Remedy reviewer on Claude Sonnet 5.5 the sample charger, the one photo here that should be paid, went to a person in every trial (5 of 5), so both reviewers stay on Claude Opus 5.5.
The two reviewers' independence comes from the different rulebook each reads: on Claude Opus 5.5 they gave different answers on the look-alike hair dryer and the CPSC's hair dryer photo in all 6 claims they reviewed, and the same answer on the sample charger in all 4. The sample is small, 3 photos and 5 trials (15 claims a run), so it shows how the two settings behaved on these photos rather than a rate.
On Claude Opus 5.5 the photo that should be paid was held once (1 of 5), in a trial where the reviewers did not finish, and a claim the reviewers do not finish goes to a person. The reviewers did not finish on 3 claims in the first run and on none in the second, each sent to a person, so the first run has 10 reviewed claims and the second 13. Counted by src/components/evidence/reviewer-models.ts from content/verify/reviewer-models.json (scripts/verify/sweep.ts --claim-path, run Nov 9, 2026), against the known answers in content/verify/fixtures/cases.json; cost per claim is the list price of every model call a claim makes, the photo check and any reviewers. The run's model calls cost $1.04 over 51 calls in the usage log ($0.98 by the sweep's own count of its live calls). tests/reviewer-models.test.ts counts the file again.
Which model setting checks the photo, and why
| Setting | Paid when it should not have been | Right, scored the day of the sweep | Re-scored with today’s answers | Would have paid a CPSC picture, match off | Cost per check | Median time |
|---|---|---|---|---|---|---|
| Claude Opus 5.5, medium effort (the claim path) | 0 of 18 | 18 of 24 (75%) | 22 of 24 | 3 of 6 | $0.0297 | 8.7 s |
| Claude Opus 5.5, low effort | 0 of 18 | 20 of 24 (83%) | 23 of 24 | 3 of 6 | $0.0203 | 7 s |
| Claude Sonnet 5.5 | 0 of 18 | 19 of 24 (79%) | 23 of 24 | 5 of 6 | $0.0116 | 7 s |
| Claude Haiku 5.5, medium effort | 0 of 18 | 13 of 24 (54%) | 16 of 24 | 1 of 6 | $0.0009 | 6.7 s |
On the measure that matters for money, the settings tie: none paid a photo it should not have (0 of 18 for each). Scored the day the sweep ran, the claim path’s setting was right on 18 of 24; re-scored after the two revisions listed below, on 22 of 24. Held back can only be measured on an owner’s own photo that should be paid, and only the claim path’s setting has run on one: Brightline’s sample charger, paid in 3 of 4 decisions, the other one sent to a person at the firm. So the claim path stays on Claude Opus 5.5, medium effort. Claude Opus 5.5, low effort scores at least as well on both counts, with no more CPSC pictures paid with the match off, at $0.0203 a check, and is the next to measure on owners' photos. It stays on that setting because every recorded answer on this site, and the verification cache, are keyed to it: switching means a paid re-run of those recordings. The switch itself is one line, effortFor() in src/lib/verify/vision.ts.
Known answers revised after the sweep
Nov 6, 2026, case
made-safe-helmet-in-trash(official/25007-droyd-5-in-trash.jpg): was right product pass, made safe pass, now a new photo asked for; right product pass, made safe fail.Recall 25007 asks for two photos, one of both straps cut off and one of the helmet in a trash can. This is the second, and on its own it does not show the straps cut, so a request for the other photo is the right answer. Every setting in the three-trial sweep read it that way (made safe: fail, 12 of 12).
Trials whose score moved: Opus medium 3 of 3, Opus low 3 of 3, Sonnet 3 of 3, Haiku 0 of 3.
Nov 6, 2026, case
look-alike-hair-dryer(official/26125-hair-dryer-cord-cut.png): was sent to a person at the firm, now anything but paid.What matters for a look-alike is that it is never paid. Sending it to a person and asking for a new photo both keep the refund for the recalled product, so either is a right answer. No setting paid it (0 of 12).
Trials whose score moved: Opus medium 1 of 3, Opus low 0 of 3, Sonnet 1 of 3, Haiku 3 of 3.
Each revision, with the answer before and after, is in content/verify/answer-changes.json. tests/facts.test.ts fails when the day-of-sweep score differs from the pass marks the sweep wrote when it ran, or when a case drifts from its listed answer.
Scored by src/components/evidence/errors.ts from content/verify/sweep-results.json, three trials of eight photos per setting, against the known answers in content/verify/fixtures/cases.json.