Skip to content

Write decimal default values in plain notation - #4420

Open
gedaliakliger wants to merge 1 commit into
apache:mainfrom
gedaliakliger:fix/decimal-default-plain-notation
Open

gedaliakliger wants to merge 1 commit into
apache:mainfrom
gedaliakliger:fix/decimal-default-plain-notation

Conversation

@gedaliakliger

Copy link
Copy Markdown
Contributor

Rationale for this change

to_json for a DecimalType returns str(val), and Python renders a small Decimal in scientific notation. A decimal(20, 8) column with a default of zero gets "initial-default": "0E-8" in the table metadata, and 0.00000010 becomes 1.0E-7.

The spec's JSON single-value serialization uses the number of digits to the right of the decimal point to indicate a positive scale, and keeps scientific notation for negative scales. Java's SingleValueParser.toJson does the same: toPlainString() when scale() >= 0, otherwise toString().

This writes the plain form when the exponent is zero or negative and leaves the other cases as they are.

Are these changes tested?

Three new cases in test_json_single_serialization (tests/test_conversions.py). They fail on main (0E-8, -1.2E-7, 1E-18) and pass with this change.

Ran locally: pytest tests/test_conversions.py tests/test_types.py -m unit, and prek run ruff ruff-format mypy on the two changed files. The full unit and integration suites are left to CI.

Are there any user-facing changes?

Decimal initial-default and write-default values with a small magnitude are written in plain notation. Metadata that already holds the scientific form still reads.

str() renders a Decimal with a small magnitude in scientific notation, so a
decimal(20, 8) default of zero was written to table metadata as 0E-8. The spec
uses the digits after the decimal point to carry the scale, and Java writes
toPlainString() for a non-negative scale.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant