Skip to content

Commit 97aafe2

Browse files
ShogunPandaaduh95
ShogunPanda
authored and
aduh95
committed
ffi: add fast support for almost all other platforms
Signed-off-by: Paolo Insogna <paolo@cowtech.it> PR-URL: #63941 Reviewed-By: Matteo Collina <matteo.collina@gmail.com> Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
1 parent 54948c7 commit 97aafe2

12 files changed

Lines changed: 1209 additions & 67 deletions

File tree

‎doc/contributing/ffi-fast-api-internals.md‎

Lines changed: 71 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -41,10 +41,9 @@ The implementation is split across these files:
4141
* `src/ffi/types.{h,cc}` parses public FFI signatures and implements
4242
`IsFastCallEligible()`, which rejects signatures that the current Fast API
4343
trampolines cannot represent.
44-
* `src/ffi/platforms/arm64.cc` and `src/ffi/platforms/x64.cc` contain the
45-
platform trampoline generators. These files follow the contract exposed by
46-
`node_ffi_create_fast_trampoline()` and release code with
47-
`node_ffi_free_fast_trampoline()`.
44+
* `src/ffi/platforms/*.cc` contain the platform trampoline generators. These
45+
files follow the contract exposed by `node_ffi_create_fast_trampoline()` and
46+
release code with `node_ffi_free_fast_trampoline()`.
4847
* `src/node_ffi.cc` decides whether a function gets a Fast API callable,
4948
SharedBuffer callable, or generic callable, and attaches hidden metadata used
5049
by JavaScript wrappers.
@@ -88,8 +87,7 @@ true only on supported architectures when `IsJitMemorySupported()` succeeds.
8887
`IsJitMemorySupported()` runs a one-time self-test:
8988

9089
* Map one writable anonymous page.
91-
* Write a minimal return instruction (`0xD65F03C0` on AArch64, `0xC3` on
92-
x86\_64).
90+
* Write a minimal return instruction for the current architecture.
9391
* Flush the instruction cache where required.
9492
* Try to transition the page to read/execute with `mprotect(PROT_READ |
9593
PROT_EXEC)`.
@@ -99,8 +97,8 @@ The probe deliberately does not execute the generated instruction. Executing a
9997
freshly written capability probe could terminate the process on systems that
10098
block generated code. The real trampoline emitter performs the same writable to
10199
executable transition when creating a callable trampoline and falls back when it
102-
is rejected. Windows currently returns false because the branch does not yet
103-
have a Win64 trampoline emitter or `VirtualAlloc`-based JIT memory support.
100+
is rejected. Windows uses `VirtualAlloc`, `VirtualProtect`, and
101+
`FlushInstructionCache` for the same probe.
104102

105103
## Signature Eligibility
106104

@@ -110,8 +108,8 @@ keeps unsupported cases out of the trampoline emitters and lets
110108

111109
Eligibility requires:
112110

113-
* A supported platform emitter: AArch64 or x86\_64 SysV. Win64 is currently
114-
ineligible.
111+
* A supported platform emitter: AArch64, x86\_64 SysV, Win64 x64, PPC64LE
112+
ELFv2, LoongArch64, RISC-V 64, or s390x.
115113
* A return type that is numeric, pointer, or `void`.
116114
* Argument types that are numeric or pointer. `void` cannot be an argument.
117115
* No `function` typed argument or return value.
@@ -141,6 +139,56 @@ x86\_64 SysV eligibility mirrors `src/ffi/platforms/x64.cc`:
141139
incoming GP count is capped at 5 and buffer-shaped arguments cannot coexist
142140
with FP arguments.
143141

142+
Win64 x64 eligibility mirrors the conservative Windows emitter in
143+
`src/ffi/platforms/x64.cc`:
144+
145+
* The JavaScript receiver occupies the first positional register slot.
146+
* Public arguments are shifted from positions 1..3 into positions 0..2.
147+
* Integer and FP arguments are handled according to their positional Win64
148+
register slots.
149+
* Only scalar register-only signatures with at most three public arguments are
150+
currently eligible.
151+
* Buffer-shaped arguments and stack-passed arguments fall back.
152+
153+
PPC64LE eligibility mirrors `src/ffi/platforms/ppc64.cc`:
154+
155+
* `r3` is occupied by V8's receiver, so user GP arguments arrive in `r4..r10`.
156+
* FP arguments use FPRs and are not shifted by the receiver slot.
157+
* The generated trampoline shifts only GP registers and tail-branches to the
158+
target through `ctr`, with the target address in `r12` for ELFv2 global entry.
159+
* Only scalar register-only signatures are currently eligible.
160+
* Buffer-shaped arguments, stack-passed arguments, narrow returns, and PPC64BE
161+
platforms fall back. AIX/PPC64BE is intentionally a non-target for the current
162+
Fast FFI trampoline work because its ABI/linkage shape needs separate design.
163+
164+
LoongArch64 eligibility mirrors `src/ffi/platforms/loong64.cc`:
165+
166+
* `a0` is occupied by V8's receiver, so user GP arguments arrive in `a1..a7`.
167+
* FP arguments use `fa0..fa7` and are not shifted by the receiver slot.
168+
* The generated trampoline shifts only GP registers and tail-branches to the
169+
target through `jirl`.
170+
* Only scalar register-only signatures are currently eligible.
171+
* Buffer-shaped arguments, stack-passed arguments, and narrow returns fall back.
172+
173+
RISC-V 64 eligibility mirrors `src/ffi/platforms/riscv64.cc`:
174+
175+
* `a0` is occupied by V8's receiver, so user GP arguments arrive in `a1..a7`.
176+
* FP arguments use `fa0..fa7` and are not shifted by the receiver slot.
177+
* The generated trampoline shifts only GP registers and tail-branches to the
178+
target through `jalr`.
179+
* Only scalar register-only signatures are currently eligible.
180+
* Buffer-shaped arguments, stack-passed arguments, and narrow returns fall back.
181+
182+
s390x eligibility mirrors `src/ffi/platforms/s390x.cc`:
183+
184+
* `r2` is occupied by V8's receiver, so user GP arguments arrive in `r3..r6`.
185+
* FP arguments use `f0`, `f2`, `f4`, and `f6` and are not shifted by the receiver
186+
slot.
187+
* The generated trampoline shifts only GP registers and tail-branches to the
188+
target through `br`.
189+
* Only scalar register-only signatures are currently eligible.
190+
* Buffer-shaped arguments, stack-passed arguments, and narrow returns fall back.
191+
144192
The native trampoline generator still repeats its own register checks. The
145193
eligibility function is the early, centralized rejection point; the generator
146194
checks are a defense against direct or future callers.
@@ -395,9 +443,22 @@ Important limits are:
395443
* No stack arguments in the current AArch64 trampoline.
396444
* At most one stack-loaded scalar GP argument in the current x86\_64 SysV
397445
trampoline.
446+
* No stack arguments or buffer-shaped arguments in the current Win64 x64
447+
trampoline.
448+
* No stack arguments, buffer-shaped arguments, or narrow returns in the current
449+
PPC64LE trampoline.
450+
* No stack arguments, buffer-shaped arguments, or narrow returns in the current
451+
LoongArch64, RISC-V 64, and s390x trampolines.
398452
* No mixed buffer-shaped and FP arguments.
399453
* No `function` argument or return type in the Fast API path.
400454

455+
Linux x86 and armv7 are experimental Node.js platforms, but the current Fast FFI
456+
trampoline model remains 64-bit only. They continue to use SharedBuffer or
457+
generic libffi fallback paths. Linux s390x is a Tier 2 Node.js platform, but
458+
bundled FFI is not currently enabled for that target; if built with
459+
`--shared-ffi`, scalar register-only Fast API FFI can use the s390x emitter. AIX
460+
PPC64BE is intentionally not covered by this implementation.
461+
401462
These are optimization boundaries, not public FFI signature boundaries. User
402463
code can still call supported public FFI signatures through fallback paths.
403464

‎node.gyp‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -475,6 +475,10 @@
475475
'src/node_ffi.cc',
476476
'src/node_ffi.h',
477477
'src/ffi/platforms/arm64.cc',
478+
'src/ffi/platforms/loong64.cc',
479+
'src/ffi/platforms/ppc64.cc',
480+
'src/ffi/platforms/riscv64.cc',
481+
'src/ffi/platforms/s390x.cc',
478482
'src/ffi/platforms/x64.cc',
479483
'src/ffi/data.cc',
480484
'src/ffi/data.h',

‎src/ffi/fast.cc‎

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -222,7 +222,10 @@ FastFFIMetadata::~FastFFIMetadata() {
222222

223223
bool IsFastCallSupported() {
224224
// Fast call requires both a platform stub emitter and working JIT memory.
225-
#if defined(__aarch64__) || defined(_M_ARM64) || defined(__x86_64__)
225+
#if defined(__aarch64__) || defined(_M_ARM64) || defined(__x86_64__) || \
226+
defined(_M_X64) || defined(__powerpc64__) || defined(__ppc64__) || \
227+
defined(__PPC64__) || defined(__loongarch64) || \
228+
(defined(__riscv) && __riscv_xlen == 64) || defined(__s390x__)
226229
return IsJitMemorySupported();
227230
#else
228231
return false;

‎src/ffi/jit_memory.cc‎

Lines changed: 56 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -2,42 +2,85 @@
22

33
#include "ffi/jit_memory.h"
44

5-
#if !defined(_WIN32)
6-
7-
#include <sys/mman.h>
8-
#include <unistd.h>
9-
105
#include <cstdint>
116
#include <cstring>
127
#include <mutex>
138

9+
#if defined(_WIN32)
10+
#include <windows.h>
11+
#else
12+
#include <sys/mman.h>
13+
#include <unistd.h>
14+
1415
#if defined(__APPLE__)
1516
#include <libkern/OSCacheControl.h>
1617
#endif
1718

18-
#endif // !defined(_WIN32)
19+
#endif // defined(_WIN32)
1920

2021
namespace node::ffi {
2122

2223
namespace {
2324

24-
#if !defined(_WIN32)
25-
2625
bool SelfTest() {
27-
#if !defined(__aarch64__) && !defined(_M_ARM64) && !defined(__x86_64__)
26+
#if !defined(__aarch64__) && !defined(_M_ARM64) && !defined(__x86_64__) && \
27+
!defined(_M_X64) && !defined(__powerpc64__) && !defined(__ppc64__) && \
28+
!defined(__PPC64__) && !defined(__loongarch64) && \
29+
!(defined(__riscv) && __riscv_xlen == 64) && !defined(__s390x__)
2830
// No stub emitter for this platform; nothing to test.
2931
return false;
3032
#else
3133
#if defined(__aarch64__) || defined(_M_ARM64)
3234
// AArch64 BR LR: 0xD65F03C0
3335
constexpr uint32_t kInstruction = 0xD65F03C0;
3436
constexpr size_t kInstructionSize = sizeof(uint32_t);
37+
#elif defined(__powerpc64__) || defined(__ppc64__) || defined(__PPC64__)
38+
// PPC64 BLR: 0x4E800020
39+
constexpr uint32_t kInstruction = 0x4E800020;
40+
constexpr size_t kInstructionSize = sizeof(uint32_t);
41+
#elif defined(__loongarch64)
42+
// LoongArch64 JIRL zero, ra, 0
43+
constexpr uint32_t kInstruction = 0x4C000020;
44+
constexpr size_t kInstructionSize = sizeof(uint32_t);
45+
#elif defined(__riscv) && __riscv_xlen == 64
46+
// RISC-V JALR zero, ra, 0
47+
constexpr uint32_t kInstruction = 0x00008067;
48+
constexpr size_t kInstructionSize = sizeof(uint32_t);
49+
#elif defined(__s390x__)
50+
// s390x BR r14
51+
constexpr uint16_t kInstruction = 0x07fe;
52+
constexpr size_t kInstructionSize = sizeof(uint16_t);
3553
#else
3654
// x86_64 RET: 0xC3
3755
constexpr uint8_t kInstruction = 0xC3;
3856
constexpr size_t kInstructionSize = sizeof(uint8_t);
3957
#endif
4058

59+
#if defined(_WIN32)
60+
void* page = VirtualAlloc(
61+
nullptr, kInstructionSize, MEM_COMMIT | MEM_RESERVE, PAGE_READWRITE);
62+
if (page == nullptr) {
63+
return false;
64+
}
65+
66+
uint8_t* code = static_cast<uint8_t*>(page);
67+
#if defined(__aarch64__) || defined(_M_ARM64) || defined(__powerpc64__) || \
68+
defined(__ppc64__) || defined(__PPC64__) || defined(__loongarch64) || \
69+
(defined(__riscv) && __riscv_xlen == 64) || defined(__s390x__)
70+
std::memcpy(code, &kInstruction, kInstructionSize);
71+
#else
72+
code[0] = kInstruction;
73+
#endif
74+
75+
FlushInstructionCache(GetCurrentProcess(), page, kInstructionSize);
76+
77+
DWORD old_protect;
78+
const bool ok =
79+
VirtualProtect(page, kInstructionSize, PAGE_EXECUTE_READ, &old_protect) !=
80+
0;
81+
VirtualFree(page, 0, MEM_RELEASE);
82+
return ok;
83+
#else
4184
const size_t page_size = static_cast<size_t>(getpagesize());
4285
void* page = mmap(nullptr,
4386
page_size,
@@ -50,7 +93,9 @@ bool SelfTest() {
5093
}
5194

5295
uint8_t* code = static_cast<uint8_t*>(page);
53-
#if defined(__aarch64__) || defined(_M_ARM64)
96+
#if defined(__aarch64__) || defined(_M_ARM64) || defined(__powerpc64__) || \
97+
defined(__ppc64__) || defined(__PPC64__) || defined(__loongarch64) || \
98+
(defined(__riscv) && __riscv_xlen == 64) || defined(__s390x__)
5499
std::memcpy(code, &kInstruction, kInstructionSize);
55100
#elif defined(__x86_64__)
56101
code[0] = kInstruction;
@@ -84,25 +129,18 @@ bool SelfTest() {
84129
munmap(page, page_size);
85130
return ok;
86131
#endif
132+
#endif
87133
}
88134

89-
#endif // !defined(_WIN32)
90-
91135
} // namespace
92136

93137
bool IsJitMemorySupported() {
94-
#if defined(_WIN32)
95-
// Windows stub emitter and VirtualAlloc-based JIT memory support not yet
96-
// implemented. Return false so the fast-call path falls back to libffi.
97-
return false;
98-
#else
99138
// Run the self-test exactly once and publish only the final result, so
100139
// concurrent callers never observe a provisional value.
101140
static std::once_flag once;
102141
static bool supported = false;
103142
std::call_once(once, [] { supported = SelfTest(); });
104143
return supported;
105-
#endif
106144
}
107145

108146
} // namespace node::ffi

0 commit comments

Comments
 (0)