3,205 questions
1
vote
0
answers
104
views
How would I create a shared memory buffer to display animations using the Kitty Graphics Protocol in Rust?
I'm using Arch Linux with Wayland. My terminal is Kitty.
I'm trying to create an animation using the Kitty Graphics Protocol. However, I want this animation to cover the whole screen and run at 60 fps....
- 42
Tooling
1
vote
3
replies
72
views
using persistent-memory gawk how variables can created to be local and issolated from other execution instances?
The idea of Persistent-Memory gawk is fabulous because it improves the performance, size, and clarity of many scripts on static and reference data.
However, I have a significant problem in adopting ...
3
votes
1
answer
77
views
Accessing any object type from multiprocessing shared_memory?
Suppose I create a shared memory object:
from multiprocessing import shared_memory
shm_a = shared_memory.SharedMemory(create=True, size=1024)
buffer = shm_a.buf
and put a generic object of a generic ...
- 5,844
1
vote
1
answer
92
views
php Sharing data between Cli and WEB ownership problems
I have a service that execute some bash scripts based on external events, in this scripts I call php cli to execute some php code running with account "A".
I have a web site that needs some ...
- 11
2
votes
2
answers
89
views
Is it possible to create numpy array in shared_memory using custom dtypes?
I can succesfully create numpy arrays with simple dtypes (int32 etc).
But when i try to use something like :
b_shared_memory = shared_memory.SharedMemory(create=True, name = "test235", size =...
- 21
0
votes
1
answer
201
views
How to properly use VK_KHR_external_memory for sharing memory between two processes using Vulkan API
I am trying to share memory between two Vulkan processes (using the same NVidia gpu device) using the VK_KHR_external memory extension on Linux (Ubuntu 22). I create a buffer/device memory (and ...
- 343
0
votes
1
answer
196
views
Shared memory leaks in Python 3
I experience problems with SharedMemory() in Python 3.12.0, it is not properly released. I use below context manager to handle share memory segments:
@contextmanager
def managed_shm(name=None, size=0, ...
- 563
2
votes
1
answer
107
views
shared red black tree among processes
I want to create a red-black tree that it is shared among different forked processes.
Linking to an old question of mine old_question there cannot be resizable data structures in the shared memory (...
0
votes
0
answers
76
views
Does the mirai package use shared memory parallelism?
Does the mirai R package use shared memory parallelism?
The parallel package uses shared memory parallelism with the mc* (e.g. mclapply) functions and fork clusters, in the sense that the input data ...
- 1,150
0
votes
0
answers
23
views
lockf for shm_open fd
Using shm_open to communicate between client and server. How to guarantee client and server will not double initialize.
// server.c
int main() {
int shm_fd = shm_open("/my_shm", O_CREAT |...
- 21
1
vote
1
answer
90
views
Can pthread_barrier_t be used across multiple processes with shared memory?
I'm working on a C++ application that launches multiple processes. All of them load the same executable and share a memory region using shm_open + mmap.
In that shared memory, we store a ...
- 111
2
votes
0
answers
117
views
Shared memory between RISCV vm (using Qemu) and host
I can't set correctly shared memory for Qemu RISCV emulation.
My intention is to create a portion of shared memory between Qemu RISCV emulator and the host server. I need this because my intention ...
- 21
4
votes
1
answer
141
views
Shared memory disappears after writer process finishes, even though shmctl is not used to remove it
I am using System V shared memory in my program. The writer process successfully creates the shared memory using shmget(). When the writer process ends, I notice that the shared memory segment ...
- 235
7
votes
2
answers
617
views
Best Approach for Managing Shared Memory Cleanup on Linux?
I am trying to create a Rust library to simplify working with shared memory on Linux. Processes would only be able to request shared memory through the library, but I cannot guarantee that they won't ...
- 95
2
votes
0
answers
83
views
Memory Lock Without Atomic RMW Operations
I'm working with a fiber-optic reflective memory card (RFM2g model), which lacks atomic read-modify-write operations. My goal is to implement a spinlock across multiple processes (potentially running ...
- 45
0
votes
1
answer
89
views
shared list among processes in C modifications do not reflect in the list again
I am trying a simple programm that creates multiple processes, each process takes a pair of an int and a bool from a list changes the boolean value and reinserts it if it is found false otherwise just ...
2
votes
3
answers
99
views
Splitting a MemoryMappedFile into 2 chunks of data in C#
I'm trying to create a MemoryMappedFile with 2 view accessors, each accessing different regions of this shared memory.
This is a simplified version of what I'm doing:
const int block1Size = 20;
const ...
- 267
3
votes
1
answer
170
views
Why is my shared memory reading zeroes on MacOS?
I am writing an interface to allow communication between a main program written in C and extension scripts written in python and run in a separate python interpreter process. The interface uses a UNIX ...
- 123
0
votes
0
answers
36
views
Network and shared storage issues when setting up a PolarDB cluster
I am encountering some issues while setting up a PolarDB cluster using the polardb_pg_devel:ubuntu22.04 image. After configuring shared storage and network, I am seeing persistent warning messages ...
- 11
2
votes
2
answers
166
views
C++ shared memory among forked processes (what do i do wrong)
I am making a basic program when the main process creates processes with fork that take the first element from a list and print it. I used boost library to manage shared memory, but this is my first ...
2
votes
1
answer
73
views
Shared memory among forked processes c++
I am writing a c++ program and I want a program where there are two processes (running with fork()) and they both share a common list and an interprocess mutex is there to guard the shared list ...
1
vote
0
answers
228
views
Using Transparent Huge Pages (THP) with shm_open and mmap on a Read-Only File in /dev/shm
I have a read-only file located in /dev/shm with an arbitrary size (not necessarily a multiple of 2MB or maybe smaller 120KB). I want to use shm_open and mmap to map this file into memory and then use ...
- 29
1
vote
1
answer
120
views
Can child processes create shared memory and share it with parent processes?
Can a child process create a SharedMemory object and share it with a parent process? Currently I am getting errors when I try. I need this because creating and copying memory is the major performance ...
- 403
2
votes
0
answers
76
views
linux shared memory doesn't showed as expect
I use mmap create a shared memory, two process shared the memory. I use the 'top' command to watch the shared memory output, it looks like:
1825191 root 20 0 10972 88 0 S 0.0 0.0 ...
- 23
1
vote
2
answers
171
views
conflict when using multiprocessing's share memory
I am using multiprocessing's shared memory to share a numpy array between tasks. While each task should originally just read the array, I was curious if writing was also possible. I wrote the ...
- 1,253
0
votes
0
answers
103
views
How to initialize values in shared memory
I am cross-compiling on a Ubuntu 24.04 host for a Raspberry Pi Zero2 W. I want to be able to access data for a device on the i2c bus not only from multiple instances in the same program but also from ...
- 39
0
votes
0
answers
102
views
How to Parallelize Nested Loops in Fortran on a Shared Memory System with OpenMP, Assigning Threads Explicitly to Inner and Outer Loops?
I am working on a Fortran program with nested loops that I want to parallelize using OpenMP. The program runs on a shared memory multi-processor system, and I want the following:
The inner loop to run ...
0
votes
1
answer
241
views
python3 multiprocessing SharedMemory can not solve connecting to existing memory
What I want to accompish is to first run testb.py and create shared memory, then run testc.py and connect to it. Then shutdown testc.py and rerun it and connect to shared memory again. But I stuck at ...
1
vote
0
answers
95
views
How to share a "global variable" from a static library across DLL's
We are working on breaking apart a large application that has a main exe plus many DLL's/so's. We want some DLL's to build independently. As part of doing this, we want to move some code from DLL's ...
- 2,540
2
votes
1
answer
310
views
How do I correctly determine NAME_MAX for shm_open?
In the manual page for shm_open, we see that the argument for the name parameter should be up to NAME_MAX characters long for portable use.
However, in The GNU C Library Reference Manual's section ...
- 33
1
vote
0
answers
136
views
multiprocessing and shared memory
I am trying to get the basics of multiprocessing in python. I have a quite complex routine that takes a large array (c.a 1Gb) and a double as inputs and returns a double. The large array is not going ...
- 31
0
votes
2
answers
131
views
Utilize multiprocessing in Python with functions that do not return any value
Consider the following codes in Python:
from multiprocessing import Pool
data = [4] * 10
def update(j):
data[j] += j ** 2
def solver():
with Pool(8) as po:
po.map(update, range(10))
...
- 19
0
votes
1
answer
38
views
Shared memory sizing for a numpy array
In the example seen on superfastpython.com, the size of a shared memory segment to be used to support a 1-dimensional numpy array is calculated as the number of elements multiplied by the data type ...
- 29.1k
1
vote
1
answer
125
views
SharedMemory numpy array multiprocessing probelm
The problem
I have a lot of data that I first load in RAM with SharedMemory and then I read with many child-processes with multiprocessing.Pool.map.
The code
This is a simplified version (not really ...
- 69
0
votes
1
answer
131
views
NumPy ndarray in Shared Memory not attaching to Pandas DataFrame
I am trying to create a Pandas DataFrame that is attached to an array stored in shared memory. I found a useful example from a response to another SO question. However, I have not been able to use ...
- 17
1
vote
0
answers
82
views
How to know how many processes linked with this shared memory?
I am using System V shared memory api in linux.
My model is 1 writer vs N reader.
I want to avoid N writer, write together, so I need to check if there is other process is writing.
How Can I know this?...
- 143
1
vote
0
answers
116
views
How to map shared memory to fixed address on older linux (version < 4.17)
I'm using shared memory to share data between multiple processes that run my code (written in c).
The data I need to share is many interconnected structs that use pointers.
Since it will be tough to ...
- 115
0
votes
0
answers
46
views
NodeJS - read-only large JS Object shared amongst processes
I've been reading a lot about this issue but I can't come with a definite reliable solution. I know NodeJS spawns different instances as different CPU processes, hence sharing data or memory amongst ...
- 16.5k
1
vote
0
answers
105
views
linux sys/shm.h latency can be 20us?
I want to exchange data between processes.
I choose shared memory, the reason is i think it's fastest IPC method.
But the following code tells different story.
string_worker.hpp: this is a base class, ...
- 143
0
votes
2
answers
146
views
Why race condition occurs when hardware has ensured coherency
#include <iostream>
#include <thread>
#include <vector>
#include <chrono>
#include <mutex>
using namespace std::chrono;
const int nthreads = 4;
const int64_t ndata = ...
2
votes
1
answer
89
views
Why does one process not reach `exit(0)` when forking?
I have the following program, just to test if I'm understanding forking and shared memory. In summary it makes shared memory, forks, forks, forks, and in each fork, writes data and exits.
#include &...
- 4,013
0
votes
0
answers
125
views
How to use shm_open() with a Unique Filename (or alternatives)
I am writing a set of programs on Linux that use shared memory. Currently I call shm_open() with a predetermined name for the shared memory. This works fine as I currently only have one set of ...
1
vote
1
answer
108
views
How can decrease time of creating and insert in boost::interprocess::map?
In the below code I unpack a msgpack file to std::map. The duration of this process is almost 85 seconds.
#include <map>
#include <vector>
#include <string>
#include <iostream>
...
- 190
1
vote
1
answer
249
views
--shm_size change for Docker in Nextflow pipeline: containerOptions not seeming to work
I am trying to increase the shared memory size for the Docker image being used to run a Nextflow pipeline. Per some indications online, I have added the following to my config file:
docker {
...
- 419
0
votes
1
answer
116
views
How should I initialize pthread mutexes in shared memory, since at program start they can be already initialized?
I'm using shared, robust pthread mutexes to protect some shared memory areas I have in my application. Since my program is managed by a systemd service, it could potentially be restarted, in which ...
2
votes
1
answer
184
views
Multiprocessing shared memory to pass large arrays between processes
Context: I need to analyse some weather data for every hour of the year. For each hour, I need to read in some inputs for each hour before performing some calculation. One of these inputs is a very ...
- 23
0
votes
1
answer
307
views
How does mmap'ed eBPF map shared between processes synchronizes operations?
It's possible to mmap eBPF map of type BPF_MAP_TYPE_ARRAY. I have a configuration where multiple processes access this mmaped map that contains entries of type "some cacheline-sized struct" ...
- 531
0
votes
0
answers
95
views
Is there a way to write an image to memory with Python and read it from another application written in C++?
I would like to create and write an image to the memory in such a way that other processes are able to read it. The goal is to have a Python script create some graph using matplotlib.pyplot, write it ...
- 17
1
vote
2
answers
118
views
Error with operations on the elements in shared memory C
I made this code that inserts a structure into shared memory. I made debug prints to try to figure out where the processes were stopping and why there was an error. Apparently by doing operations on ...
- 203
1
vote
0
answers
150
views
Problem with malloc and shared memory in C
I am trying to put shared_pid_data in shared memory. I need to have a dynamic array, so I use malloc, I also tried using calloc getting the same result. The problem I am encountering is that the data ...
- 203