COI Verification · implementation

Blank, unreadable, or absent

Every certificate extraction pipeline eventually meets a cell it cannot read. What it returns for that cell decides whether someone chases a subcontractor over coverage that is printed on the page. Most pipelines return null, which is the one answer that cannot tell the two situations apart.

Three answers, because they send you to different places

Collapse absent and unreadable into one null and every smudged fax becomes a coverage finding nobody verified.

First, find out what kind of PDF you have

The same form arrives three ways, and each tells you something different about a blank-looking cell. A fillable form carries its values as form fields, so an empty field really is empty. A flattened PDF has only a text layer — usable, if it is intact. A scan or a photo has no text at all, and whether a cell is readable becomes a judgment.

from pypdf import PdfReader

reader = PdfReader("certificate.pdf")
fields = reader.get_fields() or {}
text = reader.pages[0].extract_text() or ""

if fields:
    kind = "fillable form"                  # an empty field is genuinely blank
elif text.strip():
    kind = "flattened, with a text layer"   # readable as text -- if the text is intact
else:
    kind = "image only"                     # OCR or a vision model; expect unreadable cells

# A text layer can exist and still be wrong. Check for labels every ACORD 25 prints.
EXPECTED = ["EACH OCCURRENCE", "CERTIFICATE HOLDER", "POLICY EXP"]
missing = [label for label in EXPECTED if label not in text.upper()]

print(kind, "| form fields:", len(fields), "| text chars:", len(text.strip()), "| labels missing:", missing)

Run on the specimen ACORD 25 the NYC Department of Buildings publishes, it prints:

flattened, with a text layer | form fields: 0 | text chars: 2973 | labels missing: []

Note the zero: a government-published specimen of the form carries no form fields at all, so a pipeline built only on reading form fields would read nothing from it. And a text layer can be present and still be wrong: some generators map glyphs so that extraction drops spaces or substitutes characters. That is what the label check is for — if a label every ACORD 25 prints is missing from the text, do not trust the values next to it.

A confidence score is not a refusal

The common fix is to attach a confidence to each field. It helps, and it does not solve this. A field returned at 0.62 still hands your code a value, and the code downstream will use it. What you need is a field that can say I am not giving you a value, and here is why.

The rule we use: no value is reported unless it can be quoted from its own cell, and every missing value carries one of the two reasons.

def field_result(value, quote, cell_has_ink):
    """One field, one of three answers -- never two."""
    if value is not None and quote:          # read, and you can point at the characters
        return {"value": value, "status": "present"}
    if cell_has_ink:                         # something is printed there that you could not read
        return {"value": None, "status": "unreadable"}   # ask for a clearer copy
    return {"value": None, "status": "absent"}           # blank on the page: go to the vendor

Deciding cell_has_ink is the hard part, and on a scan it is a judgment. Make the judgment explicit rather than letting a null make it for you.

What it looks like from our API

A synthetic certificate whose policy-expiration cell is smudged beyond reading, beside a limits cell that is simply blank — fixture 4 of the COI Extraction Honesty Test. The real response, as served on 2026-09-11 (the full extraction under parsed omitted here):

{
  "compliant": false,
  "verdict": "cannot_determine",
  "evaluated": [
    "general_liability",
    "min_days_until_expiration"
  ],
  "withheld": [
    "general_liability.policy_expiration_date",
    "auto_liability.any_auto"
  ],
  "deficiencies": [
    {
      "field": "general_liability.personal_adv_injury",
      "required": "Personal & Advertising Injury limit >= $1,000,000",
      "actual": "not found on certificate",
      "status": "absent",
      "reason": "Personal & Advertising Injury limit could not be determined from the certificate."
    },
    {
      "field": "general_liability.policy_expiration_date",
      "required": "Policy expiration date on file, valid at least 30 more day(s)",
      "actual": "present on the certificate but could not be read",
      "status": "unreadable",
      "reason": "General Liability policy expiration date could not be determined. The value was withheld because it could not be quoted from the document -- this is a request for a clearer copy, not a coverage deficiency."
    }
  ]
}

A model does not always declare the cell it could not read. On 2026-09-15, one call in three on this same certificate left the expiration null without declaring it, and the API served it as absent. A blank policy date cell and an illegible one cannot be told apart from a reply that is silent about both, so the API now withholds a policy date missing from any coverage row the certificate shows: status is "unreadable" and actual reads "not established from the certificate". Blank limits and unmarked checkboxes are still absent.

Test your own pipeline

The COI Extraction Honesty Test is four synthetic certificates, each built around one failure — a blank cell between two filled ones, a placeholder date, a document that is not a certificate, and a smudged cell — with the ground truth written down before any tool read them. Free to run against anything, including us.

What this does not prove

One smudged cell is one smudged cell. This is a method and a test set, not an accuracy rate: it does not measure how often any tool reads a real, degraded certificate correctly, and a model that declares a cell unreadable can still be wrong about it. The point is narrower — that the question gets asked, and that the answer is not a silent null.

When a certificate can’t be read · Coverage requirements as data · Certificate holder vs additional insured · Free COI checker · COI Verification API