How RDMA map remote memory into local virtual memory?

Question

I am new to RDMA and have just started looking into OPENSHMEM and UCP. I saw that both of them allow mapping remote memory region into local virtual memory space and access it using regular load and write. But in my understanding, RDMA operations needs to be invoked explicitly by the programmer?

As an example, in SHMEM's documentation, we have the following

#include <shmem.h>
#include <stdio.h>
int main(void) {
static int dest[4];
shmem_init();
int mype = shmem_my_pe();
if (mype == 0) { /* initialize PE 1’s dest array */
int *ptr = shmem_ptr(dest, 1);
if (ptr == NULL)
printf("can’t use pointer to directly access PE 1’s dest array\n");
else
for (int i = 0; i < 4; i++)
*ptr++ = i + 1;
}
shmem_barrier_all();
if (mype == 1)
printf("PE 1 dest: %d, %d, %d, %d\n", dest[0], dest[1], dest[2], dest[3]);
shmem_finalize();

In the above example, ptr id dereferenced directly and modified. I wonder how this is implemented? Is it true that RDMA is capable of mapping memory region into local memory space. This region is always protected and whenever accessed, RDMA operations will be invoked by the kernel?

okaeiz · Accepted Answer · 2024-09-18 06:24:01Z

-1

As far as I know, RDMA doesn't natively map remote memory into local address space, programming models like OpenSHMEM provide abstractions that make it appear as if you're accessing local memory. The actual implementation involves sophisticated runtime systems that translate these seemingly local accesses into appropriate RDMA operations, often without kernel involvement for each access.

So yeah, this approach allows for easier programming while still leveraging the performance benefits of RDMA hardware.

answered Sep 18, 2024 at 6:24

okaeiz

4285 silver badges17 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

How RDMA map remote memory into local virtual memory?

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related