Surface per-document extraction results in document indexing status, not just file counts

[suranga]

Problem

When indexing documents from a Unity Catalog volume for retrieval, the status I get back is a count of files processed and a count of errors. In my case that read as every file processed with zero errors, while retrieval returned nothing at all for any question I asked. The count was accurate and it told me nothing, because it measures files enumerated rather than text extracted. Nothing in the status distinguished a document that parsed into usable content from one that parsed into nothing.

That cost me several days. A green status over a broken index looks identical to a green status over a working one, and there is no intermediate state to inspect.

What I would like

1. A per-document extracted-content measure in the indexing status. A character count, a chunk count or a token count. Anything measured on the output side rather than the input side.
2. An explicit count of documents that indexed to zero retrievable content, surfaced as a warning rather than folded into the success total.
3. A way to list those specific documents, so I can go and look at the ones that produced nothing.

Why it helps

It turns a silent failure into a visible one. Any pipeline that indexes files from a volume carries the same exposure: a file can be enumerated, raise no error, and still contribute nothing retrievable, whether it is an image-only scan, an extraction that came back empty, or something about the file itself that the parser could not handle. None of those are currently distinguishable from success. One output-side number makes all of them obvious on the first run instead of after days of elimination.

This applies to anything indexing files in a volume rather than to one specific feature, so the same signal would pay off across several of them.