Skip to content

RFC: Setting search defaults as an index option #235

@jkatz

Description

@jkatz

Use cases

There are cases where a user wants to set a default search value (e.g. ivfflat.probes, hnsw.ef_search) value on a per-index basis, for example:

  1. Data is partitioned across multiple tables (integer-based partition key, not related to vector) with indexes on each table. This avoids setting the value on a per-query basis
  2. User has a single index, but doesn't want to set the search values on a per-query basis

Proposal

The proposal is to add new index options on the index AMs:

  1. ivfflat: default_probes (default: NULL)
  2. hnsw: default_ef_search (default: NULL)

PostgreSQL tracks when a GUC is changed. For this case, we can use the PGC_S_SESSION indicator to determine that GUC is changed in the database sessions and therefore we do not need to consult the index option.

The proposal would then use this heuristic:

if default_probes/default_ef_search is NULL
    use value from `ivfflat.probes`/ `hnsw.ef_search`
else
  if `ivfflat.probes`/`hnsw.ef_search` source is PGC_S_SESSION, use value of `ivfflat.probes`/`hnsw.ef_search`
  else use value from `default_probes`/`default_ef_search`

Alternatives

With additional work, we could set "smarter" automatic defaults for ivfflat.probes/hnsw.ef_search, but folks may still want to set their own index-specific defaults based on the above use cases.

Seeking feedback on this proposal.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions