Skip to content

py: Implement PEP 570, positional-only parameters. - #19799

Open
dpgeorge wants to merge 4 commits into
micropython:masterfrom
dpgeorge:py-pep570
Open

dpgeorge wants to merge 4 commits into
micropython:masterfrom
dpgeorge:py-pep570

Conversation

@dpgeorge

@dpgeorge dpgeorge commented Oct 22, 2026 •

Copy link
Copy Markdown
Member

Summary

This PR implements PEP 570: positional-only parameters, which are annotated via a "/" in the function/lambda parameter list; see https://peps.python.org/pep-0570/

The implementation is simple: positional-only parameters are given the name MP_QSTRnull in the prelude, so keyword matching at runtime in mp_setup_code_state_helper() does not match against them. This approach gets the same semantics as CPython while keeping the runtime and .mpy format unchanged.

This PR is an alternative to #8480. The benefit of this one here is that it does not require a change to the .mpy file format. So code compiled with mpy-cross prior to this PR will work on VM's post this PR.

Inspecting generated bytecode is the best way to see how this feature works. For the function definition:

def f(a, b, c):
    pass

The qstr table in the corresponding .mpy file looks like:

qstr_table[6] (* for static qstrs):
    'x.py'
    '<module>' *
    'f'
    'a'
    'b'
    'c'

and the bytecode for the function f is:

1b:0a:02:03:04:05:20:51:63

The 02:03:04:05 bytes are indices into qstr_table for the function name and then the three parameter names.

Changing that function to use positional-only parameters:

def f(a, b, c, /):
    pass

the qstr table is now:

qstr_table[4] (* for static qstrs):
    'x.py'
    '<module>' *
    'f'
    None *

and the bytecode for f is:

1b:0a:02:03:03:03:20:51:63

This is the same number of bytes as before, but the parameter names are the 03:03:03 bytes which index the None entry in qstr_table. The total .mpy file size is eight bytes smaller here because it no longer stores the three parameter names.

The user-facing differences to CPython are minimal:

  • In MicroPython, locals() doesn't know about the positional-only parameter names. But MicroPython already doesn't support local function variables in locals().
  • Error messages are slightly different.

For an example of a different error message, consider executing:

def f(a, /): pass
f(a=1)

CPython gives:

TypeError: f() got some positional-only arguments passed as keyword arguments: 'a'

whereas MicroPython gives:

TypeError: unexpected keyword argument 'a'

Testing

Tests are added to the standard test suite and will be run under CI on all targets (the feature is not optional, so all ports will get it).

Trade-offs and Alternatives

  • This will cost a small amount of code size but should be relatively minimal. It's a standard syntax feature of CPython 3.8+ so it's worth having it enabled unconditionally.
  • RAM usage and size of .mpy data is increased slightly only in rare cases compiling code that includes the "*" string as well as closures (because now the compiler uses MP_QSTR__star_ as well as MP_QSTRnull in the generated mpy data). In the entire micropython-lib library totaling about 600k bytes of .mpy data, the changes here added only +20 bytes to that.
  • Performance is the runtime should be unchanged because there are no changes to the runtime.
  • Error messages here are less informative than CPython (just "invalid syntax") but that follows existing practices in the compiler.
  • This is an alternative to pep 570 positional only parameters. #8480. The PR here is pretty much equivalent to that one in the grammar and compiler. But the emitter is different: here, the bytecode emits MP_QSTRnull for each positional-only parameter, whereas in pep 570 positional only parameters. #8480 the bytecode has a modified prelude which stores the number of positional-only parameters. So the difference is really just the way things are encoded in the mpy file. pep 570 positional only parameters. #8480 is slightly more efficient in use of space for functions with many (4 or more) positional-only parameters, but has overhead in the runtime extracting and using the new number in the prelude. The approach in this PR has no changes to the runtime and no additional overhead there when using positional parameters. It's also backwards compatible with the .mpy format.

Generative AI

I used Claude Opus 5.5 to discuss the problem of implementing PEP 570 and brainstorm some alternative ways of doing it. It came up with the approach here, but I implemented it all by hand. Claude also wrote some of the tests in this PR, and reviewed the commits (finding a few bugs along the way, like the need to use MP_QSTRnull instead of MP_QSTR__star_).

This commit changes "*" to the null qstr for all internal parameter names,
such as closed over variables.

That fixes a subtle bug where passing a dict like `**{"*": 1}` to a closure
would actually match the internal parameter named "*".  Now using the null
qstr means that there's no way to match from a dict.

This has a minor change to the output from `mpy-tool.py`, which is now:

  args: [None]

instead of previously `args: ['*']`.

Signed-off-by: Damien George <damien@micropython.org>
This commit implements PEP 570: positional-only parameters, which are
annotated via a "/" in the function/lambda parameter list; see
https://peps.python.org/pep-0570/

The implementation is simple: positional-only parameters are given the name
MP_QSTRnull in the prelude, so keyword matching at runtime in
`mp_setup_code_state_helper()` does not match against them.  This approach
gets the same semantics as CPython while keeping the runtime and .mpy
format unchanged.

Inspecting generated bytecode is the best way to see how this feature
works.  For the function definition:

    def f(a, b, c):
        pass

The qstr table in the corresponding .mpy file looks like:

    qstr_table[6] (* for static qstrs):
        'x.py'
        '<module>' *
        'f'
        'a'
        'b'
        'c'

and the bytecode for the function `f` is:

    1b:0a:02:03:04:05:20:51:63

The `02:03:04:05` bytes are indices into `qstr_table` for the function name
and then the three parameter names.

Changing that function to use positional-only parameters:

    def f(a, b, c, /):
        pass

the qstr table is now:

    qstr_table[4] (* for static qstrs):
        'x.py'
        '<module>' *
        'f'
        None *

and the bytecode for `f` is:

    1b:0a:02:03:03:03:20:51:63

This is the same number of bytes as before, but the parameter names are the
`03:03:03` bytes which index the `None` entry in `qstr_table`.  The total
.mpy file size is eight bytes smaller here because it no longer stores the
three parameter names.

The user-facing differences to CPython are minimal:
- In MicroPython, `locals()` doesn't know about the positional-only
  parameter names.  But MicroPython already doesn't support local function
  variables in `locals()`.
- Error messages are slightly different.

For an example of a different error message, consider executing:

    def f(a, /): pass
    f(a=1)

CPython gives:

    TypeError: f() got some positional-only arguments passed as keyword
               arguments: 'a'

whereas MicroPython gives:

    TypeError: unexpected keyword argument 'a'

Signed-off-by: Damien George <damien@micropython.org>
@dpgeorge dpgeorge added the py-core Relates to py/ directory in source label Oct 22, 2026
These tests were written by Claude Opus 5.5.

Signed-off-by: Damien George <damien@micropython.org>
Frozen code goes through `mpy-tool.py` and that tool needs to correctly
handle functions that have positional-only parameters, which requires
handling `MP_QSTRnull`.

Signed-off-by: Damien George <damien@micropython.org>
@codecov

codecov Bot commented Oct 22, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 98.59%. Comparing base (b70ae1a) to head (eaa4c30).

Additional details and impacted files
@@           Coverage Diff           @@
##           master   #19799   +/-   ##
=======================================
  Coverage   98.59%   98.59%           
=======================================
  Files         182      182           
  Lines       23346    23356   +10     
  Branches        5        5           
=======================================
+ Hits        23017    23027   +10     
  Misses        328      328           
  Partials        1        1           
Flag Coverage Δ
unix-coverage-32bit 98.59% <100.00%> (+<0.01%) ⬆️
unix-coverage-64bit 98.52% <100.00%> (+<0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@github-actions

Copy link
Copy Markdown

Code size report:

Reference:  stm32/boards/ARDUINO_NICLA_VISION: Lower the QSPI clock to 66MHz. [b70ae1a]
Comparison: tests/ports/unix: Add test for frozen func with positional-only params. [merge of eaa4c30]
  mpy-cross:  +144 +0.038% 
   bare-arm:   +44 +0.078% 
minimal x86:   +60 +0.032% 
   unix x64:   +96 +0.011% standard
      stm32:   +52 +0.013% PYBV10
      esp32:   +60 +0.003% ESP32_GENERIC[incl +16(data)]
     mimxrt:   +48 +0.012% TEENSY40
        rp2:   +48 +0.005% RPI_PICO_W
       samd:   +52 +0.019% ADAFRUIT_ITSYBITSY_M4_EXPRESS
  qemu rv32:   +66 +0.014% VIRT_RV32

@octoprobe-bot

Copy link
Copy Markdown

Octoprobe PR report

Test Tests
passed
Tests
skipped
Tests
xfailed
Tests
failed
format flash 9 8
run-tests.py 23189 3303 21 3
run-tests.py --via-mpy --emit native 26782 4059 17 29
run-tests.py --via-mpy 28970 3999 15 15
run-perfbench.py 394 14
run-natmodtests.py 560 137
run-mpremote-tests.sh 156 9 6
run-tests.py --test-dirs=extmod_hardware 176 296 67
run-tests.py --test-dirs=extmod_hardware --emit-native 173 278 66
Total 80409 12103 186 53
Failures

Group: run-mpremote-tests.sh

Test esp32
0c30-
ESP32_C3_DEVKIT
esp32
5d21-
ESP32_DEVKIT
esp32
472b-
ESP32_S3_DEVKIT
esp32
1830-
LOLIN_C3_MINI
esp8266
7921-
LOLIN_D1_MINI
mimxrt
1133-
TEENSY40
nrf
3c2a-
ARDUINO_NANO_33
rp2
5334-
RPI_PICO2
rp2
5334-
RPI_PICO2-
RISCV
rp2
552b-
RPI_PICO2_W
rp2
6038-
RPI_PICO_W
samd
5f2a-
ADA_ITSYBITSY_M0
stm32
2b35-
NUCLEO_WB55
stm32
7b21-
PYBV11
stm32
7b21-
PYBV11-
DP
stm32
3a21-
PYBV11-
DP_THREAD
stm32
7b21-
PYBV11-
THREAD
test_unicode.sh pass pass pass pass FAIL FAIL FAIL pass skip pass pass pass pass skip pass pass pass pass pass
test_filesystem.sh pass pass pass pass FAIL FAIL FAIL pass skip pass pass pass pass skip pass pass pass pass pass

Group: run-tests.py

Test esp32
0c30-
ESP32_C3_DEVKIT
esp32
5d21-
ESP32_DEVKIT
esp32
472b-
ESP32_S3_DEVKIT
esp32
1830-
LOLIN_C3_MINI
mimxrt
1133-
TEENSY40
nrf
3c2a-
ARDUINO_NANO_33
rp2
5334-
RPI_PICO2
rp2
5334-
RPI_PICO2-
RISCV
rp2
552b-
RPI_PICO2_W
rp2
5f2c-
RPI_PICO_W
samd
5f2a-
ADA_ITSYBITSY_M0
stm32
2b35-
NUCLEO_WB55
stm32
3a21-
PYBV11
stm32
3a21-
PYBV11-
DP
stm32
3a21-
PYBV11-
DP_THREAD
stm32
7b21-
PYBV11-
THREAD
extmod/machine_mem_backup_slice.py pass pass pass pass skip FAIL FAIL FAIL skip skip skip skip skip skip skip skip pass pass pass pass pass pass pass pass pass

Group: run-tests.py --via-mpy --emit native

Test esp32
0c30-
ESP32_C3_DEVKIT
esp32
5d21-
ESP32_DEVKIT
esp32
472b-
ESP32_S3_DEVKIT
esp32
1830-
LOLIN_C3_MINI
mimxrt
1133-
TEENSY40
nrf
3c2a-
ARDUINO_NANO_33
rp2
5334-
RPI_PICO2
rp2
5334-
RPI_PICO2-
RISCV
rp2
552b-
RPI_PICO2_W
rp2
5f2c-
RPI_PICO_W
samd
5f2a-
ADA_ITSYBITSY_M0
stm32
2b35-
NUCLEO_WB55
stm32
7b21-
PYBV11
stm32
7b21-
PYBV11-
DP
stm32
3a21-
PYBV11-
DP_THREAD
stm32
7b21-
PYBV11-
THREAD
basics/array_add.py pass pass pass pass pass pass pass pass pass pass pass pass pass pass pass pass pass pass pass pass pass pass pass pass pass pass pass FAIL pass pass
basics/array_construct.py pass pass pass pass pass pass pass pass pass pass pass pass pass pass pass pass pass pass pass pass pass pass pass pass pass pass pass FAIL pass pass
basics/array_construct2.py pass pass pass pass pass pass pass pass pass pass pass pass pass pass pass pass pass pass pass pass pass pass pass pass pass pass pass FAIL pass pass
basics/array_construct_endian.py pass pass pass pass pass pass pass pass pass pass pass pass pass pass pass pass pass pass pass pass pass pass pass pass pass pass pass FAIL pass pass
basics/array_int_repr.py pass pass pass pass pass pass pass pass pass pass pass pass pass pass pass pass pass pass pass pass skip skip skip pass pass pass pass FAIL pass pass
extmod/machine_mem_backup_slice.py pass pass pass pass skip FAIL FAIL FAIL skip skip skip skip skip skip skip skip skip skip skip skip skip skip skip skip pass pass pass pass pass
extmod/machine_timer.py pass pass pass pass pass skip skip skip pass pass pass pass pass pass pass pass pass pass pass pass FAIL FAIL FAIL pass pass pass pass pass pass
micropython/viper_ptr8_load_boundary.py pass pass pass pass pass pass pass pass pass pass pass pass pass pass pass pass pass pass pass pass FAIL FAIL FAIL pass pass pass pass pass pass
ports/rp2/rp2_lightsleep.py FAIL FAIL FAIL FAIL FAIL FAIL FAIL FAIL FAIL FAIL FAIL FAIL
stress/qstr_limit.py pass pass pass pass pass pass pass pass pass pass pass pass pass pass pass pass pass pass pass pass FAIL FAIL FAIL pass pass pass pass pass pass

Group: run-tests.py --via-mpy

Test esp32
0c30-
ESP32_C3_DEVKIT
esp32
5d21-
ESP32_DEVKIT
esp32
472b-
ESP32_S3_DEVKIT
esp32
1830-
LOLIN_C3_MINI
esp8266
7921-
LOLIN_D1_MINI
mimxrt
1133-
TEENSY40
nrf
3c2a-
ARDUINO_NANO_33
rp2
5334-
RPI_PICO2
rp2
5334-
RPI_PICO2-
RISCV
rp2
552b-
RPI_PICO2_W
rp2
5f2c-
RPI_PICO_W
samd
5f2a-
ADA_ITSYBITSY_M0
stm32
2b35-
NUCLEO_WB55
stm32
7b21-
PYBV11
stm32
7b21-
PYBV11-
DP
stm32
3a21-
PYBV11-
DP_THREAD
stm32
7b21-
PYBV11-
THREAD
extmod/machine_mem_backup_slice.py pass pass pass pass skip skip FAIL FAIL FAIL skip skip skip skip skip skip skip skip skip skip skip skip skip skip skip skip pass pass pass pass pass pass
ports/rp2/rp2_lightsleep.py FAIL FAIL *[FAIL](https://reports.octoprobe.org/github\_selfhosted\_testrun\_787/RUN-TESTS\_STANDARD\_VIA\_MPY@5334-RPI\_PICO2,c/l...*[Comment body truncated]

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

py-core Relates to py/ directory in source

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants