File formats

Reading a file returns sections: each carries its text, the headings or column names above it, and where it sat. Bytes become text first; pages with no usable text layer are then read from their image, by OCR and — at deeper reading levels — by a vision model.

Key rules

1. A refusal lands at upload, while the dialog is still open, not at the end of an ingest.
2. Vietnamese code pages are detected, and tone marks survive every step.
3. OCR reads every scanned page first; a model only reads at Text + figures or Full rewrite.
4. A model never reads a page with nothing to check itself against.
5. Every passage says how its text was obtained.

Text encoding

The first rule that matches wins.

Evidence	Encoding
Byte order mark	As marked
A zero on every second byte	UTF-16 — checked before UTF-8, because a NUL byte is valid UTF-8
Decodes as UTF-8	UTF-8
A tone mark on a vowel	Vietnamese code page
A capital mid-word	Western European code page; the capital is a misread byte

Choosing a reader

The reader is chosen from the declared type, with the filename settling what the type cannot: a Linux browser reports Markdown and CSV as generic binary, and Windows reports a plain CSV as legacy Excel. A file neither names is decided by its bytes. Characters are read as plain text, so a SQL dump, a YAML playbook or a Terraform module indexes without anyone adding its extension to a list; bytes that are not characters are refused.

An image is one page with no text layer — just like a scanned PDF page — so a photo of a document or a page saved as JPEG or PNG is read the same way, with nothing rendered first: the stored file goes to OCR, and to the model under its own type only when the file asks for Text + figures or Full rewrite.

Refused formats

• Archives, recordings and fonts — by type or extension, which lets a sync skip one before downloading.
• Page markup (HTML) — the words sit among tags and nothing strips them. Viewing such a file on screen is a separate question (see document review view).
• Word, Excel and PowerPoint from before 2007 — a binary format nothing here reads; the refusal names the application that can convert the file.

The upload route already holds the bytes, so it answers the same question the extractor would. A scheduled sync asks twice: from the name before download and from the bytes on arrival, counting an unreadable file as skipped.

Sections per format

Format	Sections	Above each one	Where it sat
Plain text, JSON, any file whose bytes are characters	The whole file	Nothing	Nothing
Markdown	One per heading	The headings above it	Heading path
CSV, TSV	One per row	Header row, and the table’s title if any	Row number
PDF	One per page, minus repeated head and foot; a model-read page splits at its headings	The bookmark covering the page, then the model’s headings	Page number, and the printed one when it differs
Word	Header and footer as one block, then one per heading, plus each table row	Headings, and a table’s own header	Table row
Excel	One per row per visible sheet, skipping hidden rows and columns	Sheet name, and the header row with any band above it	Sheet and row
PowerPoint	One per shown slide, with notes and SmartArt text	Slide title	Slide number

Date and percent formats are rendered — otherwise a delivery date indexes as a five-digit integer and twelve percent as 0.12. Other numbers stay as stored, since bare digits are what an exact search matches.

Left behind	Effect
Multi-column reading order in a PDF with a text layer	Not handled
Word’s automatic heading numbers	A heading numbered 5.2 arrives under its words alone
Hidden sheets, rows and columns; slides marked not shown	Skipped
An image inside a document	Skipped; an image uploaded on its own is read as a page

A Markdown or text file may embed pictures as base64 under an image mark. Those are replaced by the word image before the file is cut into sections; left in, a cut could land inside one, spend an embedding on a passage with no words, and get quoted as evidence for an answer.

Pages the text layer cannot serve

A scanned page has no text layer, and a table of figures flattens into one line of numbers with no columns. Both are read from the page image, at every reading level.

Measured on a 422-page Vietnamese test corpus, with a ruling-line detector marking pages with a printed grid: every wide table (eight columns or more) has a rule-mark share of 4.17% or above. The populations overlap lower down — a page with one small ruled block reaches 12.98% — so the threshold is the largest value that still catches every wide table. Across the same 422 pages the Otsu cutoff ranged from 136 to 252, so no single number would have served; the widest table went from three different column counts across its pages to one.

With the grid attached, the model places marks in the right column. Reading flat text it must count empty cells across the page to place an A, and on a 25-column scan it collapsed them into two; with the grid, marks reach the committee columns at the right-hand edge that the flat reading lost entirely. A multi-level header has no full-width rule inside it, so the model is asked to read headings off the image and give each column one name.

Why both readers are kept

They fail in different shapes. A model reading a weak scan fills gaps from its language prior — on one real page it named a company printed nowhere on it, in fluent prose nothing automatic can catch. Tesseract cannot invent a name; where it fails it emits garbled characters a reader can see. So the model is never shown a page with nothing to check itself against, and is told: OCR is evidence about names, codes, numbers and dates; the image wins where the two disagree on a character; OCR’s spacing and reading order are not to be copied.

The page row keeps OCR’s confidence and which reader produced the transcript. No threshold reads the confidence: whether a model reads a page is the file’s reading level, and nothing else.

How a page reaches the model

• The extractor only decides which pages qualify — reporting each with its bookmark and printed number — and never calls a model, so every call is made where it is priced and billed. Reading a page is the background model’s work.
• Amazon Textract is not used: it reads Latin script only for English, Spanish, German, French, Italian and Portuguese, and drops or misreads Vietnamese accents. A vision model through LiteLLMLiteLLM (model gateway)LiteLLM (cổng mô hình)The gateway every model call passes through. It routes to Bedrock, applies each department's key and budget, and records the cost of every call.Open in glossary on Bedrock reads Vietnamese and stays in your AWS account.
• Each page goes up as an image, because Bedrock answers a PDF document block by extracting its text layer and a scan would arrive empty. The longest edge is the size the model looks at.
• Rasterising is a subprocess, one call per run of consecutive pages. It writes JPEG at a quality where no pixel of a Vietnamese page differs from the PNG by more than five percent, so accents survive the encoding. Pages are read several at a time, bounded by a setting.
• The instruction asks for a transcription: every word in its own language, headings as headings, tables as Markdown tables with each value under its printed column, numbers and codes exactly as printed, no summary, no translation.
• The answer is split at the model’s headings and filed under the page’s bookmark at its locator, with the printed page number at the head of the first non-table part. The running header is removed, as with a text layer. Every transcribed passage is marked, because a transcription can misread a digit.

Empty and cut-off answers. An empty answer has read nothing, and OCR’s text is kept. An answer cut off at the output limit keeps everything above the cut — it is the page’s own text in its own columns — and drops only the half-written last line, since a row missing its closing cells would land under the wrong column names. A failed page is asked once more, then recorded unread; the document shows its unread count.

Wide-table slicing

The provider caps an image at 1,568 visual tokens of 28 × 28 pixels, counting the image rather than the page. An A4- or A3-shaped page reaches the model at about 1318 × 932 pixels whatever is sent, so the larger the paper, the smaller its text: a glyph 20 pixels tall at 300 dpi comes out around 5 on an A3 scan.

Under Full rewrite, a page whose printed rules describe a table is cut into four vertical slices, each sent at the cap, travelling in one call as parts of one page. Cuts land on the rules nearest a quarter, a half and three quarters of the width, so a slice ends where a column does. A page with fewer than eight vertical rules — every page that is not a wide table — is sent whole, as is every page sent for its pictures.

Eight A3 table pages, no OCR attached	Word types recovered (vs Tesseract)	Written but not on the page	Printed column count hit
Whole page	61.2%	20.7%	1 page
Four slices	64.2%	19.9%	3 pages

One page collapsed when read whole, spending its whole output budget on a single row of about 15,930 empty cells, and read correctly in slices. Cost per page did not move: the words the model writes outweigh the image five to ten times over.

Stops before and during a run

A file needing more model pages than the deployment’s cap is refused whole rather than half read, as is a Text + figures file with more pictured pages than the cap. An organisation with no background model assigned fails a file read at either deeper level.

Stop	When
No background model assigned	Before any token is spent
Assigned model cannot read images	Before any token is spent
More pages still need a model after OCR than the cap allows	Before any token is spent; the file is refused whole
A page could not be rendered	During the run
A run died and the sweep restarted it too many times	After the restart ceiling
Every qualifying page came back unreadable	During the run

Under the hood

Related