Thanks for the proposal! I have a few general comments:

1. Why does this proposal implement the quantizer as a separate index access method? It'd be more convenient if quantization were an option for the existing ivfflat index.
2. Generally, I'm not in favor of making specific performance claims (e.g. 2x faster) in documentation, as this can vary based upon workload. With ANN searches, it's also important to discuss the performance / recall tradeoff.

Do you have any tests that you ran against a known data set (e.g. ANN Benchmarks) that show how this implementation compares against ivfflat or others, specifically in:

1. Performance/recall
2. Index build time
3. Index size

I have the same question as @jkatz - why a new index type, why not the "ops" part, so ... vector_l2_ops_uint8) ... which looks like the correct integration point to me

Thanks for the proposal! I have a few general comments:

1. Why does this proposal implement the quantizer as a separate index access method? It'd be more convenient if quantization were an option for the existing ivfflat index.
2. Generally, I'm not in favor of making specific performance claims (e.g. 2x faster) in documentation, as this can vary based upon workload. With ANN searches, it's also important to discuss the performance / recall tradeoff.

Do you have any tests that you ran against a known data set (e.g. ANN Benchmarks) that show how this implementation compares against ivfflat or others, specifically in:

1. Performance/recall
2. Index build time
3. Index size

As for recall - at least for some ANN Benchmarks datasets - there should not be any difference, as for example SIFT 128 dataset has only integer values in range 0 ... 218 in its vectors, though still stored as FP32

Thanks for your comments!
Re 1. Please see my comments above for.
Re 2., point taken. let me update the doc and be more specific about the performance/recall trade-off.

And yes, we ran the ANN benchmark with various datasets, an example gain from Deep1B on Intel(R) Xeon(R) CPU @ 2.00GHz:

• Index build time (single threaded build prior to 0.5.0): 90s (ivfflat) down to 68s (with quantization).
• Index size: 3903MB (ivfflat) down to 1115MB.
• Recall loss varies with datasets but overall we saw 1-1.5% recall loss comparing to ivfflat for the same probe when vectors are not normalized (i.e., l2 distance) and smaller < 1% for normalized cases.

Hi @bohanliu5, thanks for the PR! I really appreciate all the work.

It looks like this introduces a lot of complexity to the code. I think there's some that can be removed (using the existing index type, no cross page storage at first), but I'm also concerned that there's enough that can't (which may not justify the benefits right now).

I'd like to see how the performance and complexity compare to product quantization, which I'm planning to focus on after HNSW, so I think it makes sense to wait on this.

Does scalar quantization only make sense with IVF index, or can it be used with HNSW too?

@hlinnaka SQ is a general technique to reduce the number of bytes required to store a vector, so the short answer is yes.

However, compared to product quantization (PQ), SQ can only reduce the storage so much, as you ultimately can only reduce so many bits before you lose too much information. I agree with Andrew's analysis above (in many ways), but in particular, around focus on PQ first, which should have a more dramatic effect on reducing memory consumption, though we'd have to test for the impact on recall.

@bohanliu5 do you plan to work on this one? Would be awesome to have this integration.