Chunking turns a document into retrievable, citable units. Good chunks preserve a coherent idea, enough local context, source hierarchy, permissions, and exact coordinates while fitting the embedding model and downstream context budget. Fixed character windows are a baseline; structure-aware policies usually handle headings, paragraphs, tables, code, and lists more faithfully.
Very small chunks improve specificity but lose context and multiply rows. Very large chunks preserve context but mix topics, increase embedding and prompt cost, and make citations vague. Overlap can preserve boundary context but duplicates evidence and may crowd results with near-identical chunks.
Start with semantic boundaries and target token ranges, then evaluate—not with one universal number.
Tables may need a title and column headers repeated with row groups. Code should preserve function/class boundaries and file path. Never split a security scope away from the content it governs.
Embed a focused child chunk for matching, then fetch a larger authorized parent section for context. This separates retrieval granularity from generation granularity. The parent still needs a token budget and precise citation mapping.
Render sampled chunks in a human review tool. Ask: can this unit answer a plausible question alone, does it retain required definitions, and can a reader locate the source? Compare several policies on the same golden query set. Track retrieval recall, duplicate rate, context tokens, citation precision, and ingestion cost.
Given this representative document [paste excerpt] and query set [paste], propose three chunking policies: baseline fixed window, structure-aware, and parent-child. Specify boundaries, target/maximum tokens, overlap, metadata, stable IDs, and evaluation measures. Do not choose a winner without running the queries.
Verification contract: Inspect at least 30 chunks and compare policies on the same labels, model, metric, filters, and retrieval depth.