fix(django-google-spanner): support JSONField read and write operations by sakthivelmanii · Pull Request #19007 · googleapis/google-cloud-python

sakthivelmanii · 2026-11-06T11:37:00Z

Problem

On write (INSERT/UPDATE), DatabaseOperations did not override adapt_json_value, so BaseDatabaseOperations.adapt_json_value returned a str that spanner_dbapi bound as TypeCode.STRING instead of TypeCode.JSON. Additionally, passing a pre-wrapped array, scalar, or null JsonObject into json.dumps serialized an empty object ({}) because JsonObject subclasses dict.
On read (SELECT), google-cloud-spanner decodes TypeCode.JSON columns into JsonObject instances, and because DatabaseOperations.get_db_converters did not register a converter for JSONField, JSONField.from_db_value passed the JsonObject directly to json.loads, raising TypeError: the JSON object must be str, bytes or bytearray, not JsonObject.

Solution

Implement _unwrap_json_object and DatabaseOperations.adapt_json_value in django_spanner/operations.py so JSONField parameters are serialized with any custom field encoder and wrapped in JsonObject (binding as TypeCode.JSON instead of TypeCode.STRING). Note that Value(None, JSONField()) (JSON null) is stored as SQL NULL because JsonObject.serialize() returns None for null JSON values.
Register DatabaseOperations.convert_jsonfield_value in get_db_converters for JSONField to serialize JsonObject values back into JSON strings (or None) before JSONField.from_db_value runs json.loads.
Add unit tests in tests/unit/django_spanner/test_operations.py and end-to-end mockserver tests in tests/mockserver_tests/test_basics.py.

gemini-code-assist

Code Review

This pull request introduces support for Django's JSONField in the Google Cloud Spanner backend by adding utility functions to adapt and convert JSON values, along with corresponding unit and mock server tests. Feedback is provided to optimize the recursive _unwrap_json_object serialization helper by checking for standard Python types first, which avoids expensive isinstance checks on standard elements in performance-critical paths.

into fix_json_field_spanner

olavloite · 2026-11-07T11:24:34Z

+        Note that JSON null (`Value(None, JSONField())`) maps to a null
+        JsonObject whose `serialize()` returns `None` (bound as SQL NULL by
+        `spanner_v1`).


I think that this is Claude-speak for "Value(None, JSONField()) is equal to SQL NULL", but I am not sure. And the main question is:

How do you write (and read) a JSON NULL in this Django provider? (insert into my_table (col) values (JSON 'NULL'))

How do you write and read a JSON column with a SQL NULL value?

My impression is that the first point is not possible, and that this provider now only supports SQL NULL for JSON columns. Is that intentional?

Updated adapt_json_value to wrap the serialized JSON string in _JsonString(JsonObject) so it is sent directly without re-parsing, and Value(None, JSONField()) now serializes as JSON null rather than SQL NULL.

olavloite · 2026-11-07T11:33:10Z

+from google.cloud.spanner_v1 import JsonObject
+
+
+def _unwrap_json_object(value):


I think that this entire method can be removed if this monkey patching in django-spanner/__inity__.py is removed:

def get_prep_value(self, value): if not isinstance(value, JsonObject) and isinstance(value, dict): return JsonObject(value) return value JSONField.get_prep_value = get_prep_value

Good catch — removed the JSONField.get_prep_value monkey-patch from init.py along with _unwrap_json_object in operations.py.

olavloite · 2026-11-07T11:40:31Z

+        :returns: A serialized JSON string for Django's JSONField.from_db_value,
+                  or None.
+        """
+        if isinstance(value, JsonObject):


The current implementation does a double serialize/parse on both the read and the write path. That is: Whenever a JSON value is read, it is parsed into a JsonObject by the client, and then serialized again into a string by Django, and then parsed back into a JsonObject. Whenever a JSON value is written, it is dumped into a string by Django, then parsed into a JsonObject by the client library, and then dumped back into a string by the client library. I think that we can avoid that double-parsing-serialization if we add an option to the client library to return JSON values as a string.

Updated convert_jsonfield_value to unwrap the parsed JsonObject directly in memory (while still supporting custom decoders and subclasses), and updated SQLCompiler.get_converters to skip the redundant default JSONField.from_db_value pass.

olavloite · 2026-11-07T11:43:27Z

            for db, config in DATABASES.items():
                if config["ENGINE"] == "django_spanner":
                    config.pop("DISABLE_RANDOM_ID_GENERATION", None)
+


This implementation does not support looking up or filtering on JSON keys, e.g. Detail.objects.filter(value__name='Jakob'). Is that intentional?

Added Spanner lookup implementations for KeyTransform, KeyTextTransform, and HasKeyLookup using JSON_QUERY and JSON_VALUE, and added unit, mockserver, and emulator tests covering key filtering.

olavloite · 2026-11-07T11:44:45Z

 # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 # See the License for the specific language governing permissions and
 # limitations under the License.
+import google.cloud.spanner_v1.types.result_set as result_set


There are a number of test gaps:

No custom encoder integration test: Does not test saving a model using JSONField(encoder=DjangoJSONEncoder) with UUID, Decimal, or datetime end-to-end through the ORM.

No test for updates or bulk inserts: No coverage for QuerySet.update() or bulk_create().

No test for JSON filtering: No coverage for Model.objects.filter(value={"name": "Jakob"}).

Emulator tests are still skipped: In tests/system/django_spanner/test_json_field.py, all JSON tests are skipped for the emulator

There are also no tests for reading and writing JSON primitives, and the flag for JSON primitives is still disabled in django_spanner/features.py (the supports_primitives_in_json_field config field is false)

Added mockserver and unit tests for SQL NULL, JSON null, primitives, custom encoders/decoders, bulk_create, update, and key lookups, and enabled TestJsonField on the emulator.

Enabled supports_primitives_in_json_field, has_native_json_field, and supports_json_field in features.py, and added tests for primitive JSON values.

fix(django-google-spanner): support JSONField read and write operations

c1ccb58

sakthivelmanii requested a review from a team as a code owner November 6, 2026 11:37

gemini-code-assist Bot reviewed Nov 6, 2026

View reviewed changes

Comment thread packages/django-google-spanner/django_spanner/operations.py Resolved

sakthivelmanii added 3 commits November 6, 2026 11:44

fix(django-google-spanner): support JSONField read and write operations

6c062a7

fix(django-google-spanner): support JSONField read and write operations

d4f78b0

Merge branch 'main' of https://github.com/googleapis/google-cloud-python

8dc762e

into fix_json_field_spanner

olavloite reviewed Nov 7, 2026

View reviewed changes

sakthivelmanii added 4 commits November 9, 2026 05:52

fix(django-google-spanner): support JSONField read and write operations

9df95ed

fix(django-google-spanner): support JSONField read and write operations

2ec4bf6

fix(django-google-spanner): support JSONField read and write operations

e721bdc

fix(django-google-spanner): support JSONField read and write operations

16ef6a6

		from google.cloud.spanner_v1 import JsonObject


		def _unwrap_json_object(value):

Conversation

sakthivelmanii commented Nov 6, 2026

Problem

Solution

Uh oh!

gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

This branch has not been deployed

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants