You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
dask pyperformance benchmark hangs on win32 #118715
# Make a venv to install dependencies in
cpython\PCBuild\win32\python.exe -m venv venv
copy cpython\PCBuild\win32\python313.lib cpython
venv\Scripts\activate
python -m pip install pyperformance
python -m pyperformance run -b dask
I'm working on finding a more self contained reproducer, but may not have time for that immediately, so I thought I would report this in the meantime. I suspect it is an integer sizing mismatch bug in that commit, given that it doesn't happen on 64-bit Windows.
@vstinner: I definitely have no experience with these Windows APIs, however, it seems that the change in 1d95451 introduced some stateful interaction between time.time and time.perf_counter, since they both now call py_win_perf_counter_frequency with the same global variable as base. Is this a regression for any code that calls time.time and time.perf_counter interleaved? What are the implications of that?
At a minimum, I think we should improve the "What's new" entry about this so people are aware of the interaction.
the change in 1d95451 introduced some stateful interaction between time.time and time.perf_counter, since they both now call py_win_perf_counter_frequency with the same global variable as base.
time.time() now calls GetSystemTimePreciseAsFileTime()
time.perf_counter() is unchanged
time.monotonic() now calls GetSystemTimePreciseAsFileTime().
Yes, but why did that break dask? Because the assumption about time.time having low resolution no longer holds? I think the concern is for possibly breaking other libraries using these APIs.
Activity
vstinner commentedon May 7, 2024
Do you mean that it uses all the CPU, or that the process hangs and do nothing?
Can you check which lines are executed when it "hangs"?
vstinner commentedon May 7, 2024
I reproduced the issue. The problem comes from
_WindowsTimeofdistributed/metrics.py.https://github.com/dask/distributed/blob/d5edb4e536c9acc4998f803aacc0dd0188af21ce/distributed/metrics.py#L45-L98
vstinner commentedon May 7, 2024
I reported the issue to the dask/distributed project and I proposed a fix:
vstinner commentedon May 7, 2024
@mdboom: I suggest to close this issue since it's not a bug in Python, and it's now tracked in dask/distributed.
mdboom commentedon May 9, 2024
I want to bring a discussion elsewhere back here.
@vstinner: I definitely have no experience with these Windows APIs, however, it seems that the change in 1d95451 introduced some stateful interaction between
time.timeandtime.perf_counter, since they both now callpy_win_perf_counter_frequencywith the same global variable as base. Is this a regression for any code that callstime.timeandtime.perf_counterinterleaved? What are the implications of that?At a minimum, I think we should improve the "What's new" entry about this so people are aware of the interaction.
vstinner commentedon May 10, 2024
It's documented at: https://docs.python.org/dev/whatsnew/3.13.html#time
mdboom commentedon May 10, 2024
Yes, but why did that break dask? Because the assumption abouttime.timehaving low resolution no longer holds? I think the concern is for possibly breaking other libraries using these APIs.Sorry, I see you answered my question elsewhere.