Chunking and sections

Search scores small passagesPassageĐoạn (passage)A short section of a document, cut along its headings and stored with an embedding. Search returns passages, not whole files.Open in glossary (chunks), but the model reads the whole section around a match. The document screen lets you check both against the original file.

Key rules

1. Structure decides boundaries; size decides where structure runs out.
2. A chunk never spans sections from different places.
3. Every chunk repeats its headings and column names.
4. One result per section, however many of its chunks matched.
5. A citation quotes the chunk, not the whole section.

Chunking

Rule	Behaviour
Packing	Sections that belong together are packed until the window fills; a chunk never spans sections from different places
Headings	Every chunk repeats the headings, sheet name and column names above it, cut back if they would swallow the window
Tables	A table is a section of its own, whether the file stated it or a model transcribed the page. Every chunk cut from one leads with the column names, so a chunk starting at row 37 still says what each number is. Names too long to repeat are cut at a whole column — a header filling the window would leave no room for rows and cut a page into hundreds of almost-all-header chunks
Overlap	Only inside a section that had to be split, starting at a sentence boundary
Locator	The format’s own words: a page or range, a sheet and rows, a slide, or a heading path

Where headings come from

• Markdown states them; Word through its heading styles; a slide through its title; a PDF through its bookmarks.
• A heading is kept as its words only. Bold or link marks around it would otherwise print in the locator and atop every passage from that section.
• A PDF without bookmarks is read by font size: a heading is a short line set noticeably larger than the body, in a size used for only a small share of the document. If no size qualifies, nothing is inferred and the passages carry no section.

Sections

Each run of sections the chunker packed together is kept as a row, and every chunk cut from it points back at it. The vector index scores the chunk, small enough to be about one thing; a search hands the model the whole section around it, so a question landing mid-procedure arrives with the steps either side.

• Citations quote the chunk. A citation quoting a page of text would say nothing about which line matched.
• One result per section. A procedure matching on seventeen of its chunks is one place in the corpus and arrives as one result naming every chunk that matched. Two queries that both reach it answer once between them.
• Context is charged per section. A section is charged once however many chunks matched, and a section longer than 4,000 characters is trimmed. A passage with no section — every memory statement, and every document whose format declared no structure — is charged for its own text.

The embedding model and window

The embedding model is the one your administrator catalogued as emitting vectors, through the route the gateway serves it under. The window is capped by the model’s input limit: Cohere on Bedrock refuses past 2,000 characters; Titan allows far more. A deployment configured beyond the cap is cut down to it, and the embedding client checks the limit before a request leaves. Every embedding call says whether the text is being stored or searched with, since the multilingual model scores better when told which side it is on — the portal, the agent runtime and the developer harness all say it.

Passages on the document screen

The document screen lists each passage’s embedded text with its locator, in chunker order, retired ones left out. Anyone the knowledge base’s visibility rules let search may read them; a refusal says the document is missing.

Re-ingesting retires passages rather than deleting them, and a retired passage stays readable by its id, so the sections it points at stay too. A re-ingest therefore writes a new generation of sections beside the old one; both go when the document is purged. Changing how documents are chunked is one command that re-queues every document in the organisation; stored page transcripts are reused, so the run costs an embedding pass and sends no page to a model.

Document review view

The document screen opens on the file as it was uploaded and, beside it, the words the portal read off it, with the passages on a second tab. The pair is there to be compared: a page a model read can carry a company name the page does not print, fluently enough that nothing automatic catches it — only the page open beside the words does.

• Reader per piece. Each piece says which reader produced it — the file’s text layer, OCR over the page image, or a model reading the image — and carries OCR’s confidence whichever won.
• Markdown renders. Model readings and Markdown uploads render as headings and tables through the chat reply renderer, which builds elements rather than markup and refuses a link whose scheme could carry a script. A Markdown upload reaches the renderer on its own type and filename; nothing is sent to a model to format it.
• Plain text keeps its line breaks. OCR and other text layers stay plain text, because a scan’s line breaks are the page’s rows; Markdown would join them and read a line starting with a stray pipe as a table. The literal characters are one click away.
• Pages without a transcript. A rendered page has its own stored transcript; a page the text layer served stores none, so the screen shows its passages joined back in order.

Jumping from a citation

The reading also answers where one passage sits. Given a passage, it returns the same pieces plus a list of placements: each names a piece by position, a character range for the section the passage was cut from, and — on the one piece holding it — a character range for the passage. A section across two pages gives two placements, only one carrying the passage. A passage the current revision no longer holds gives no placements.

Viewing the file

The bytes travel on the document’s own route, which asks the knowledge base, so a member who may read the base can read the file without having uploaded it. PDFs and images are served under their own type; everything else made of characters is served as plain text, so a SQL dump or YAML file reads in the browser. Rewriting the type is also what makes an uploaded page safe: one origin serves app and API, so markup under its own type would run as the portal. A layout no frame can show — a Word file or a spreadsheet — downloads instead.

Under the hood

Related