-
Notifications
You must be signed in to change notification settings - Fork 874
Description
Describe the bug
After upgrading to 8.0.9, we encounter a sporadic IndexOutOfRangeException in PgNumeric.Builder.ToDecimal() when synchronously reading a large number of rows containing numeric columns via NpgsqlDataReader.GetValue().
We are aware that #6383 / PR #6385 fixed a similar ArraySegment boundary issue in NumericConverters.ReadAsync (the BigInteger async path) for 8.0.9. However, the synchronous decimal read path through PgNumeric.Builder.ToDecimal() appears to have the same class of bug that was not addressed in 8.0.9.
Environment
- Npgsql version: 8.0.9 (confirmed via
Assembly.GetName().Version= 8.0.9.0) - .NET version: .NET Framework 4.8
- PostgreSQL version: Greenplum 6.x (PostgreSQL 9.4 compatible)
- Operating system: Windows Server 2019
Exception details
System.IndexOutOfRangeException: Index was outside the bounds of the array.
at System.ThrowHelper.ThrowIndexOutOfRangeException()
at Npgsql.Internal.Converters.PgNumeric.Builder.ToDecimal(Int16 scale, Int16 weight, UInt16 sign, Span`1 digits)
at Npgsql.Internal.Converters.PgNumeric.Builder.ToDecimal()
at Npgsql.Internal.Converters.DecimalNumericConverter`1.ConvertTo(Builder& numeric)
at Npgsql.Internal.Converters.DecimalNumericConverter`1.ReadCore(PgReader reader)
at Npgsql.Internal.PgBufferedConverter`1.Read(PgReader reader)
at Npgsql.Internal.PgBufferedConverter`1.ReadAsObject(Boolean async, PgReader reader, CancellationToken cancellationToken)
at Npgsql.Internal.PgConverter.ReadAsObject(PgReader reader)
at Npgsql.NpgsqlDataReader.GetValue(Int32 ordinal)
Key observation: the call goes through PgBufferedConverter<T>.Read (sync), not ReadAsync. The exception source is System.Memory (Span<T> indexer), not Npgsql's own array access.
Steps to reproduce
The issue is sporadic and depends on internal buffer reuse patterns. It occurs more frequently with:
- Large result sets (thousands of rows)
numericcolumns with varying precision- Aggregation functions (
SUM,AVG) that producenumericresults with different digit counts
Minimal reproduction pattern:
using var conn = new NpgsqlConnection(connStr);
conn.Open();
// Read many rows with numeric values - triggers buffer reuse in PgReader
using var cmd = new NpgsqlCommand(
"SELECT 1234567890.123456::numeric AS val FROM generate_series(1, 8000)", conn);
using var reader = cmd.ExecuteReader();
while (reader.Read())
{
// Sporadic IndexOutOfRangeException here
var value = reader.GetValue(0); // sync path → PgNumeric.Builder.ToDecimal()
}Analysis
Comparing v8.0.8...v8.0.9, the fix in NumericConverters.cs (line 219) corrected the ArraySegment boundary in the async BigInteger read path:
// NumericConverters.cs ReadAsync method (line 219)
// Before: for (var i = digits.Offset; i < array.Length; i++)
// After: for (var i = digits.Offset; i < digits.Offset + digits.Count; i++)However, PgNumeric.Builder.ToDecimal(Int16 scale, Int16 weight, UInt16 sign, Span<short> digits) in PgNumeric.cs — which is the sync decimal path — was not modified in 8.0.9. The Span<short> digits parameter appears to receive incorrect bounds when the PgReader's internal buffer is reused across many rows, causing the Span indexer to throw IndexOutOfRangeException from System.Memory.
This is the same root cause as #4313 and #6383 — ArraySegment/Span boundary not properly scoped after buffer reuse — but manifesting in a different code path.
Related issues
IndexOutOfRangeExceptionin theNumericHandler#4313 -IndexOutOfRangeExceptionin theNumericHandler(original report, closed)- NpgsqlBinaryExporter function throws exception when reading BigInteger. #6383 - Reading numerics as
BigIntegerthrowsIndexOutOfRangeException(fixed in 8.0.9, but only asyncBigIntegerpath) - PR Fix reading numerics as BigInteger #6385 - Fix reading numerics as BigInteger (the 8.0.9 fix)
- PR Fixes #6107 missed should buffer in biginteger numeric converter #6117 - Broader fix merged to
main(10.0.0)
Expected behavior
NpgsqlDataReader.GetValue() should return a decimal value without throwing IndexOutOfRangeException, regardless of result set size.
Actual behavior
Sporadic IndexOutOfRangeException in PgNumeric.Builder.ToDecimal() when reading large result sets with numeric columns via the synchronous GetValue() path.
Possible fix
The fix from PR #6117 (merged to main / 10.0.0) likely addresses this. Could the relevant PgNumeric.cs changes be backported to the 8.0.x branch?