Databricks Community

suryaprayaga · yesterday

Dear Databricker's

What are some of the most common use cases you have used Document Parsing in Databricks?

How is the success rate measured in the same?

What are some of the lessons learnt?

suryaprayaga

Isaac_18 · 13 hours ago

Good question, and it's a nice change from the usual "how do I" threads. Here are the use cases that come up most often, plus a simple way to put numbers on it.

Common use cases

Invoices, receipts and purchase orders. Parse with ai_parse_document, then pull fields like vendor, invoice number and total with ai_extract. This is probably the most common one.
Contracts and policies. Find specific clauses (renewal terms, liability caps, notice periods) across hundreds of PDFs instead of reading each one.
RAG over manuals, SOPs and internal docs. Parse, chunk, embed into Vector Search, then put an agent or Genie on top.
Forms and onboarding documents (KYC, claims, applications) where the layout is fairly fixed but the files are scans.
Reports with tables and charts. Version 2.0 returns tables as HTML, and chart figures can come back with their data as JSON, which saves a lot of manual retyping.

How to measure success

"Success rate" means different things at different layers, so I'd track three numbers:

Coverage: what share of documents parsed without an error and produced real text. A file can be "processed" and still come back nearly empty, especially low-quality scans.
Extraction quality: for field extraction, label 50 to 100 documents by hand and compare field by field. Exact match for IDs and amounts, a looser match for names and addresses.
Confidence: every element in the output has a confidence score from 0 to 1. Tracking the share of low-confidence elements per document is a cheap way to spot files that need a human look.

Here's a starting point for the first and third:

CREATE OR REPLACE TABLE main.docs.parsed AS
SELECT
  path,
  ai_parse_document(
    content,
    map('version', '2.0', 'descriptionElementTypes', '')
  ) AS parsed
FROM READ_FILES('/Volumes/main/docs/raw/', format => 'binaryFile');

-- One row per element, with its type and confidence
CREATE OR REPLACE TEMP VIEW elements AS
SELECT
  d.path,
  e.value:type::string        AS el_type,
  e.value:confidence::double  AS confidence,
  length(e.value:content::string) AS chars
FROM main.docs.parsed d,
LATERAL variant_explode(d.parsed:document:elements) AS e;

-- Per-document health
SELECT
  path,
  count(*)                                   AS elements,
  sum(chars)                                 AS total_chars,
  round(avg(confidence), 3)                  AS avg_confidence,
  round(avg(CASE WHEN confidence < 0.8 THEN 1 ELSE 0 END), 3) AS low_conf_share
FROM elements
GROUP BY path
ORDER BY low_conf_share DESC;

Documents that don't show up in that last query at all (no elements) are worth checking on their own. The 0.8 cut-off is just a starting point, so adjust it once you've looked at some real output.

Lessons that come up again and again

Test on your messiest documents first, not the clean samples. The Document Parsing UI in the workspace shows the source next to the parsed output, which makes it quick to see what was missed.
Keep the raw parse output in a table. Parsing is the expensive step. If you store the result, you can change chunking or extraction later without parsing everything again.
Pin the output version (map('version', '2.0')) so a future schema change doesn't surprise your downstream code.
Turn off figure descriptions if you don't need them ('descriptionElementTypes' set to an empty string). The docs note this cuts compute and cost for figure-heavy files.
Know the limits: 500 pages and 100 MB per document. For longer files, use the pageRange option. The docs also say the model may be weaker on non-Latin scripts and on documents with digital signatures, so test those separately if you have them.
Cost lands under the AI_FUNCTIONS product in system.billing.usage, so you can track spend per pipeline.

Docs: