Skip to content

Conversation

@dscho
Copy link
Member

@dscho dscho commented Dec 18, 2025

In 3e1d800 (potential fix for windows static linking with thread creation in dll's, 2022-11-07), mimalloc was changed to ensure that certain code paths use "weak" random seeds when initializing mimalloc.

This was a crucial fix for issues that were also discovered (and fixed independently) in Microsoft Git, where initializing with "strong" random seeds could cause Access Violations in bcryptPrimitves.dll. The symptom would look like this:

ntdll!ZwWaitForMultipleObjects+0x140 ...
KERNELBASE!WaitForMultipleObjectsEx+0x123 ...
KERNELBASE!WaitForMultipleObjects+0x11 ...
kernel32!WerpReportFaultInternal+0x62c ...
kernel32!WerpReportFault+0xc5 ...
KERNELBASE!UnhandledExceptionFilter+0x34c ...
ntdll!RtlpThreadExceptionFilter+0x2e ...
ntdll!RtlUserThreadStart$filt$0+0x3f0 ...
ntdll!_C_specific_handler+0x93 ...
ntdll!RtlpExecuteHandlerForException+0xf ...
ntdll!RtlDispatchException+0x437 ...
ntdll!KiUserExceptionDispatch+0x2e() ...
bcryptPrimitives!AesRNGState_generate+0x79 ...
bcryptPrimitives!GenRandomAes+0xf70 ...
bcryptPrimitives!MSCryptGenRandom+0x15f ...
bcrypt!BCryptGenRandom+0x7a ...
bcrypt!BCryptGenSysternPreferredRandom+0x340 ....
bcrypt!BCryptGenRandom+0x119 ...
git!_mi_prim_random_buf+0x23() [compat\mimalloc\prim\windows\prim.c @ 648]
git!mi_random_init_ex+0x61() [compat\mimalloc\random.c @ 179]
git!_mi_heap_init+0xe8() [compat\mimalloc\heap.c @ 226]
git!mi_thread_init+0x21b() [compat\mimalloc\init.c @ 396]
git!mi_heap_get_default+0x9() [compat\mimalloc\mimalloc\prim.h @ 415]
git!_mi_malloc_generic+0x9d() [compat\mimalloc\page.c @ 993]
git!mingw_getenv+0x4f() [compat\mingw.c @ 2446]
git!run_post_command_hook+0x84() [git.c @ 515]

This symptom resurfaced in Microsoft Git, and the root cause for this regression is that 710d613 (refactor thread meta-data initilazation, upstream of python/cpython#113263, 2024-05-18) renamed the original _mi_heap_init() function (which did contain that weak random seed special-case)to _mi_thread_heap_init(), and implemented a new _mi_heap_init() that does not special-case the weak random seed initialization.

The good news is that we can now characterize much better under what circumstances this crash occurs: The code path in which this Access Violation occurs is in an atexit() handler. That is important because mimalloc's own atexit() handler has deinitialized mimalloc just before that handler wants to allocate something using mimalloc, which now faithfully tries to reinitialize mimalloc. And that's apparently the crux: something in bcryptPrimitives.dll is totally unhappy to initialize things when already running in the atexit() phase.

This patch lets _mi_heap_init() once again use the week random seed when linking statically.

An alternative approach to address this might be to figure out what exactly is bothering bcryptPrimitives.dll when calling BCryptGenRandom() in an atexit() handler, and then trying to accommodate BCrypt's needs, but that looks like a tough nugget to crack.

In 3e1d800 (potential fix for windows static linking with thread
creation in dll's, 2022-11-07), mimalloc was changed to ensure that
certain code paths use "weak" random seeds when initializing mimalloc.

This was a crucial fix for issues that were also discovered (and fixed
independently) in Microsoft Git, where initializing with "strong"
random seeds could cause Access Violations in `bcryptPrimitves.dll`.
The symptom would look like this:

 ntdll!ZwWaitForMultipleObjects+0x140 ...
 KERNELBASE!WaitForMultipleObjectsEx+0x123 ...
 KERNELBASE!WaitForMultipleObjects+0x11 ...
 kernel32!WerpReportFaultInternal+0x62c ...
 kernel32!WerpReportFault+0xc5 ...
 KERNELBASE!UnhandledExceptionFilter+0x34c ...
 ntdll!RtlpThreadExceptionFilter+0x2e ...
 ntdll!RtlUserThreadStart$filt$0+0x3f0 ...
 ntdll !_ C_specific_handler+0x93 ...
 ntdll!RtlpExecuteHandlerForException+0xf ...
 ntdll!RtlDispatchException+0x437 ...
 ntdll!KiUserExceptionDispatch+0x2e() ...
 bcryptPrimitives!AesRNGState_generate+0x79 ...
 bcryptPrimitives!GenRandomAes+0xf70 ...
 bcryptPrimitives!MSCryptGenRandom+0x15f ...
 bcrypt!BCryptGenRandom+0x7a ...
 bcrypt!BCryptGenSysternPreferredRandom+0x340 ....
 bcrypt!BCryptGenRandom+0x119 ...
 git!_ mi_prim_random_buf+0x23() [compat\mimalloc\prim\windows\prim.c @ 648]
 git!mi_random_init_ex+0x61() [compat\mimalloc\random.c @ 179]
 git!_ mi_heap_init+0xe8() [compat\mimalloc\heap.c @ 226]
 git!mi_thread_init+0x21b() [compat\mimalloc\init.c @ 396]
 git!mi_heap_get_default+0x9() [compat\mimalloc\mimalloc\prim.h @ 415]
 git!_ mi_malloc_generic+0x9d() [compat\mimalloc\page.c @ 993]
 git!mingw_getenv+0x4f() [compat\mingw.c @ 2446]
 git!run_post_command_hook+0x84() [git.c @ 515]

This symptom resurfaced in Microsoft Git, and the root cause for this
regression is that 710d613 (refactor thread meta-data initilazation,
upstream of python/cpython#113263, 2024-05-18) renamed the original
`_mi_heap_init()` function (which did contain that weak random seed
special-case)to `_mi_thread_heap_init()`, and implemented a new
`_mi_heap_init()` that does not special-case the weak random seed
initialization.

The good news is that we can now characterize much better under what
circumstances this crash occurs: The code path in which this Access
Violation occurs is in an `atexit()` handler. That is important because
mimalloc's own `atexit()` handler has deinitialized mimalloc _just_
before that handler wants to allocate something using mimalloc, which
now faithfully tries to reinitialize mimalloc. And that's apparently
the crux: _something_ in `bcryptPrimitives.dll` is totally unhappy to
initialize things when already running in the `atexit()` phase.

This patch lets `_mi_heap_init()` once again use the week random
seed when linking statically.

An alternative approach to address this might be to figure out
what exactly is bothering `bcryptPrimitives.dll` when calling
`BCryptGenRandom()` in an `atexit()` handler, and then trying to
accommodate BCrypt's needs, but that looks like a tough nugget to
crack.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
@dscho
Copy link
Member Author

dscho commented Dec 18, 2025

Please note that this patch applies cleanly to dev, and with relatively uncomplicated merge conflicts also to dev3.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant