IVFFlat Indexes from Training to Tuning

IVFFlat partitions vectors into lists around learned centers and searches only selected lists near the query. It builds faster and uses less memory than HNSW, but needs representative existing data and careful lists and probes tuning. Data distribution changes can reduce quality, so rebuild decisions belong in the operational lifecycle.

What will you be able to do?

Explain IVFFlat clustering and candidate lists.
Choose an initial list count for an experiment.
Tune probes against exact results.
Detect poor training-data or premature-build failures.
Compare IVFFlat with HNSW using evidence.

How do lists and probes work?

lists sets the number of vector partitions at build time. ivfflat.probes sets how many lists a query visits. More lists can narrow each partition; more probes improves recall at extra work. When probes equals lists, search approaches exact behavior and the planner may prefer another plan.

Upstream offers starting heuristics: rows divided by 1,000 up to one million rows, then square root of rows; begin probes around square root of lists. These are experiment seeds, not universal production settings.

sql

CREATE INDEX CONCURRENTLY chunks_embedding_ivf_cos
ON app.chunk_embeddings
USING ivfflat (embedding vector_cosine_ops)
WITH (lists = 1000);

Build after the table has representative data. If the index was created with too little or skewed data, rebuild after the distribution stabilizes.

How do you tune probes?

sql

BEGIN;
SET LOCAL ivfflat.probes = 32;
SELECT chunk_id
FROM app.chunk_embeddings
ORDER BY embedding <=> $1::vector
LIMIT 10;
COMMIT;

Sweep probes using the exact baseline. Include filtered queries because candidate loss interacts with filter selectivity. Record returned row counts as well as overlap.

Why can IVFFlat return poor results?

Too few lists cause large scans and weak speed.
Too many lists relative to data produce sparse or empty partitions.
Too few probes miss nearby clusters.
Training data is not representative of current production data.
The table changed substantially after the build.
Query and index metrics or expressions do not match.

How do you verify it?

Inspect the build phase, index validity, size, and query plan. Compare list/probe combinations against exact recall and latency. Add a newer data slice to determine whether quality differs by ingestion period. Repeat after a large distribution shift.

What breaks in production?

The index is created on an empty table.
Heuristics are treated as final values.
A rebuild has no disk/WAL/time budget.
New tenant or language distributions were absent at build time.
Probes are set globally without workload isolation.

AI pair-work prompt

Propose an IVFFlat experiment for [row count, growth, dimensions, metric, filters]. Generate a list/probe matrix from upstream heuristics, but require exact recall and latency evidence. Include data-representativeness checks, rebuild triggers, disk/WAL capacity, and HNSW comparison.

Verification contract: Reject IVFFlat if representative training data or measurable rebuild operations are unavailable.

Check your understanding

Why should IVFFlat usually be built after loading data?
What does probes control?
Name two distribution changes that could trigger reevaluation.

Official references

IVFFlat Indexes from Training to Tuning

How do lists and probes work?

sql

CREATE INDEX CONCURRENTLY chunks_embedding_ivf_cos
ON app.chunk_embeddings
USING ivfflat (embedding vector_cosine_ops)
WITH (lists = 1000);

Build after the table has representative data. If the index was created with too little or skewed data, rebuild after the distribution stabilizes.

How do you tune probes?

sql

BEGIN;
SET LOCAL ivfflat.probes = 32;
SELECT chunk_id
FROM app.chunk_embeddings
ORDER BY embedding <=> $1::vector
LIMIT 10;
COMMIT;

Sweep probes using the exact baseline. Include filtered queries because candidate loss interacts with filter selectivity. Record returned row counts as well as overlap.

Why can IVFFlat return poor results?

Too few lists cause large scans and weak speed.

Too many lists relative to data produce sparse or empty partitions.

Too few probes miss nearby clusters.

Training data is not representative of current production data.

The table changed substantially after the build.

Query and index metrics or expressions do not match.

AI pair-work prompt

Propose an IVFFlat experiment for [row count, growth, dimensions, metric, filters]. Generate a list/probe matrix from upstream heuristics, but require exact recall and latency evidence. Include data-representativeness checks, rebuild triggers, disk/WAL capacity, and HNSW comparison.

Verification contract: Reject IVFFlat if representative training data or measurable rebuild operations are unavailable.