FEAT: varbinarymax streaming support in execute()#231
Merged
gargsaumya merged 4 commits intomainfrom Sep 15, 2025
Merged
Conversation
Contributor
There was a problem hiding this comment.
Pull Request Overview
This PR adds support for streaming large binary parameters (bytes and bytearray) to SQL Server using VARBINARY(MAX), removing the previous 8192-byte limitation. The implementation introduces deferred execution (DAE) for binary data larger than 8000 bytes while maintaining direct binding for smaller data.
Key changes:
- Updated parameter type mapping to detect large binary parameters and use VARBINARY(MAX) with streaming
- Modified C++ parameter binding to handle both small (direct) and large (streamed) binary data
- Enhanced SQL execution to stream large binary data in chunks using SQLPutData
Reviewed Changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
| mssql_python/cursor.py | Updated _map_sql_type to conditionally use VARBINARY(MAX) with DAE for binary data >8000 bytes |
| mssql_python/pybind/ddbc_bindings.cpp | Enhanced parameter binding and execution to support streaming large binary data |
| tests/test_004_cursor.py | Split binary data tests into separate small/large test cases |
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
sumitmsft
reviewed
Sep 10, 2025
sumitmsft
reviewed
Sep 10, 2025
sumitmsft
reviewed
Sep 10, 2025
sumitmsft
reviewed
Sep 10, 2025
sumitmsft
approved these changes
Sep 12, 2025
jahnvi480
approved these changes
Sep 15, 2025
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Work Item / Issue Reference
Summary
This pull request adds support for streaming large binary parameters (bytes and bytearray) to SQL Server using VARBINARY(MAX), removing the previous limitation that prevented inserting binary data larger than 8192 bytes. The changes update both the Python and C++ layers to detect large binary parameters, use deferred execution (DAE) for streaming, and add new tests to verify correct handling of both small and large binary data.
Binary parameter streaming support:
_map_sql_typeincursor.pyto detect largebytes/bytearrayparameters (>8000 bytes) and mark them for VARBINARY(MAX) streaming, while still handling small binaries directly.ddbc_bindings.cppto use deferred execution for large binaries, and to allocate/bind buffers correctly for both small and large binary parameters.SQLExecute_wrapinddbc_bindings.cppto stream large binary data usingSQLPutDatain chunks during execution.Testing improvements:
test_004_cursor.pyto separately verify small/medium binary inserts and to add a new test for inserting large binary data (>8000 bytes) using streaming. [1] [2]