36,461 questions
Advice
1
vote
8
replies
196
views
Efficient memory usage in c programming
If we have dynamic memory allocation, which is best for using the memory efficiently, then why do we use static memory allocation every time? Can't we use dynamic memory everywhere??
Advice
0
votes
7
replies
171
views
Usage of pointers
As a beginner I don't understand why pointers are being used everywhere . When I got to know about pointers, I had lots of questions: Why can't we simply use another variable to store the value of ...
-4
votes
0
answers
45
views
How should I reduce the memory usage on the web page while rendering data from streams which are coming through websockets [closed]
I am using fastapi as backend and react for frontend.
Currently, I'm rendering data (as chunks i.e., 500 records per chunk) which is a stream coming from websockets. And then I'm using visualization ...
-5
votes
0
answers
80
views
How should I reduce the memory usage on the web page while rendering streaming data through websockets? [closed]
I am using fastapi as backend and react for frontend.
Currently, I'm rendering data (as chunks i.e., 500 records per chunk) which is a stream coming from websockets. And then I'm using visualization ...
Advice
0
votes
0
replies
77
views
Cross-Platform Desktop Wars: Electron vs Tauri: How do you explain the tradeoffs to users (without sounding defensive)?
I am writing cause I wanted to get some opinions from folks here that have actually built and shipped with Electron.
Background: Building an API IDE on Electron. Not really “just an API client”, and ...
Advice
0
votes
10
replies
168
views
How malloc() assigns the value to the allocated memory?
I am trying to learn malloc() in C. My question is regarding how the values are stored once I malloc the memory.
To understand this, I wrote the following program:
size_t n_bytes = 4;
char *mmry_ptr = ...
2
votes
0
answers
124
views
Visualizing memory as binary in natvis [closed]
I have a C++ hash map class that I'm writing natvis code for in order to easily inspect its members while debugging. One member of the class is char* myControlBytes which is an array of chars/bytes ...
- 99
0
votes
1
answer
126
views
Corrupt object variables in C++ for ESP32 [duplicate]
I am creating a project for an ESP32. I'm quite new to ESP and C++ coding. I use PlatformIO inside VSCode.
It's a robot that will be driving around. It has Ultrasonic sensors to detect any obstacles. ...
- 13
Advice
0
votes
7
replies
125
views
Number of bits used for representing different types vs what the CPU uses
I was thinking about floats and doubles and was wondering if the fact that they use 32 bits and 64 bits respectively means there is a performance increase either in terms of memory used or time for ...
- 1
Best practices
0
votes
1
replies
115
views
How to reduce GPU memory usage when fine-tuning a large transformer model?
“I’m fine-tuning a transformer with batch size 8 and getting CUDA out-of-memory errors. Would gradient checkpointing or mixed precision help?”
- 31
Best practices
0
votes
12
replies
174
views
Using a C++ union for uninitialized memory
I am needing to manage memory for object allocation manually. I have been using a alignas(T) char storage[] array with dynamic casts. But what I really want is an array of uninitialized values.
Can I ...
- 433
4
votes
1
answer
195
views
Memory leak happening with Python when creating numpy diagonal array and doing matrix math
P = y.T @ (np.diag(Wc) @ y) This line is creating increased memory usage for me in the below code on my Windows 11 machine. As soon as I run the code, I see the RAM usage increasing and skyrocketing. ...
- 115
4
votes
1
answer
157
views
Reducing memory usage when writing chunks in order with Go parallel workers
I have a Go function for encrypting data in parallel using AES-GCM:
func encryptParallel(ctx context.Context, aead cipher.AEAD, w io.Writer, r io.Reader, workers uint64) error
The design is:
One ...
- 83
-1
votes
1
answer
307
views
What is the size of a std::function?
What is the size of a std::function? If the size is dependent on something, is there a theoretical maximum?
The size of closure (lambda function) not same as std::function or pointer explores the ...
- 371
Best practices
2
votes
3
replies
106
views
Is std::launder needed in this scenario?
I am currently trying to implement a slot map and I have a few issues with memory management, if I have an std::array<Storage, Size> where Storage is the following struct:
struct Storage final
{
...
- 384
Advice
0
votes
5
replies
126
views
C# external http request response as streaming API-response
Our ASP.NET Core Web API needs to request data from an external web service (and no, the external web service cannot be called directly from the client) and return it without modification to the ...
- 1,145
6
votes
1
answer
245
views
How does provenance work in a kernel in Rust?
In my kernel, I receive a physical pointer to a device tree through the C ABI, should I make the type of this parameter a pointer or a usize? If I use a pointer, does Rust assume I have provenance ...
- 417
3
votes
2
answers
283
views
C Struct Passed By Value Memory Addresses in Called Function
I've seem to run into something I can't really explain. I'm testing out passing a struct (typedef with an alias) by value into a function and printing out the addresses of each. I do realize that ...
- 79
Best practices
0
votes
3
replies
157
views
Can Linux kernel software mechanisms track the sequence of accessed memory pages, or is hardware support required?
I want to generate a memory access trace in Linux where every fixed number of memory accesses (e.g., every 32 accesses) are grouped and recorded as one batch. Is it possible to implement this kind of “...
- 19
0
votes
0
answers
57
views
How can a segment register with 16bit point to a segment descriptor in 32 protected mode
My understanding is i386 has segment registers with 16bit like CS it can point to code_descriptor in the GDT that should be in the RAM, to be able to reach to it we need 32bit address : How can CS ...
- 129
Best practices
0
votes
2
replies
154
views
What are some simple / practical ways to reduce memory usage in a React Native app?
I’m looking for simple, commonly used techniques to reduce memory usage in a React Native app.
One example I’m already using is avoiding unnecessary screen instances in React Navigation
(e.g. using ...
- 314
Advice
0
votes
2
replies
28
views
Memory management of data embedded into executables
Most compiled languages provide macros for embedding data into executables (#embed in C, include_bytes! in Rust etc.) however, one thing that bothers me about them is that they put the data into ...
- 1
0
votes
0
answers
106
views
Why large string log data cause the java progress memory high?
problem--why the large string log data cause the jvm memory go high?
problem: the java process is using more and more memory over time.
jvm argument:java -Xmx5120M -Xms5120M -XX:NewRatio=1 -XX:...
- 79
3
votes
2
answers
249
views
Force the C compiler to read the object's memory
I duplicated the topic because (possibly) I made a mistake with the section of the site and initially posted in the wrong place: Ways to reliably read an object in C
I have a regular byte buffer ...
- 509
2
votes
1
answer
102
views
Freeing memory to OS after heavy function without forcing GC
I have a Python function that consumes a large amount of memory. When the function finishes, I want to release the memory and return the freed pages to the operating system by calling malloc_trim(0).
...
- 21
1
vote
1
answer
163
views
Cause of AlignmentMismatch error while using bytemuck?
I'm facing an AlignmentMismatch error while trying to cast a Vec<u8> to Vec<T>, where T is a struct that uses repr(C) and Pod + Zeroable.
I've also tried the slice road &[u8] -> &...
- 984
0
votes
1
answer
115
views
Taking advantage of memory contiguousness in HLSL
This is a bit of a slog so bare with me.
I'm currently writing a 3D S(moothed) P(article) H(ydrodynamics) simulation in Unity with a parallel HLSL backend. It's a Lagrangian method of fluid simulation,...
- 13
0
votes
1
answer
99
views
Unable to find required GLIBC private symbols
I am trying to analyze a memory dump obtained with dotnet-dump from a Ubuntu container in WinDbg to find an unmanaged memory leak.
I have been able to load the dump, but whenever I try to run !heap -s,...
- 9,635
2
votes
0
answers
127
views
Why do I observe so high memory usage for copying a Parquet file with streaming using Polars with Rust?
Goal
I want to write a function in Rust using the Polars crate that does the following:
Copy a Parquet file from one location to another
Handle files larger than RAM
Not load the entire file into ...
- 477
Best practices
2
votes
6
replies
177
views
Memory accounting and quotas for subtasks in golang programs
If I have a server written in golang that processes requests containing user queries, is there any viable method to track and limit the memory used by any given user request?
The server process does ...
- 330k
2
votes
1
answer
161
views
Does `jax` compilation save runtime memory by recognizing array elements that are duplicated by indexing
Consider the example code:
from functools import partial
from jax import jit
import jax.numpy as jnp
@partial(jit, static_argnums=(0,))
def my_function(n):
idx = jnp.tile(jnp.arange(n, dtype=int)...
- 613
0
votes
0
answers
64
views
Confused by Valgrind Massif output - appears memory allocated under "free"
I'm attempting to use Massif to profile heap usage of an application. I'm running with the --pages-as-heap=true option and am confused by some output I'm seeing. Specifically I'm seeing allocation by ...
- 549
3
votes
3
answers
363
views
Trying to create Buffer overflow for an example for school in C++
I'm trying to do the very basics of a buffer overflow, nothing illegal at all. My understanding of the code below is that the overflowing buffer doesnt hold enough space for all the data being written,...
- 37
0
votes
1
answer
103
views
Handling cleanup for tasks which might be OOMKilled
I have some Python code running in k8s which in some cases is being OOMKilled and not leaving me time to cleanup which is causing bad behavior.
I've tried multiple approaches but nothing seems quite ...
- 165
3
votes
2
answers
171
views
.NET 8 console application slower after publish
I am confused why my application is somehow slower when I run it by executing the .exe file from the publish folder, compared to running it from Visual Studio debugger (F5).
I use .NET 8.0 and this is ...
- 586
0
votes
0
answers
54
views
Is there a performance difference between writing bytes to RAM instead of longs?
My single threaded program allocates and initializes a volatile block of memory on an unspecified hardware architecture.
It then writes into this block in a loop using a stride equal to the cache line ...
- 33.1k
0
votes
0
answers
69
views
H2O-3 Calling a lot of GC calls in java when trying to load CSV file with 16M rows
I am using h2o-3 java repo to load this frame but have been running into memory issues with constant GC pressure.
The actual frame size is 3.31 GB as per h2o logs, but the peak JVM usage comes to be ...
0
votes
1
answer
215
views
Duckdb Wasm limitation
I don't know how to check or increase the memory limitation of duckdb wasm.
I'm using chrome and I import some parquet into the browser, one of them has 234Mb of data
I did my research and the limit ...
- 1
-2
votes
1
answer
135
views
Is this a valid JavaScript approach for sorting large arrays of unique integers efficiently? [closed]
I’m experimenting with sorting large arrays of unique integers in JavaScript and came up with a simple approach. I’m curious if it’s a recognized pattern or if there are potential pitfalls I might be ...
- 1
0
votes
1
answer
91
views
Python moviepy V2 memory management
I am trying to combine a background video with captions generated word by word from a tts file. I am using nvenc for ffmpeg but my gpu's cuda version is too old to use it for whisper. The script works ...
- 23
1
vote
1
answer
150
views
Is address sanitizer supposed to catch this error?
I tried it with -fsanitize=address and it didn't show any error. Is this a problem with my address sanitizer or system, or is this normal? Also, how to catch these situations?
#include <iostream>...
- 16.9k
0
votes
1
answer
550
views
Nuxt 3 to 4 migration : will it fix memory issues?
I have a Nuxt 3 production app hosted on Scalingo (it's like Heroku but is a french company)
I have so many container crashes because of memory issues when a lot of people are browsing the website.
...
- 814
0
votes
1
answer
138
views
Why can't successfully kmalloc pages in this case?
See the below logs, A critical kernel driver cannot apply for a large block of continuous memory and then panic:
[29001.362425] kworker/u12:4: page allocation failure: order:4, mode:0x40dc0(GFP_KERNEL|...
1
vote
1
answer
125
views
Aws::ShutdownAPI crashes when called by shared_ptr
I am designing a library where I need AWS SDK for S3 and Kinesis as wrapped sub-components. In an executable, Aws::InitAPI() and Aws::ShutdownAPI() can be called once, so I am trying to design a ...
- 319
0
votes
0
answers
141
views
Does accessing the contents of the string after calling reserve causes UB? [duplicate]
From another thread I found that
indeed allocates enough storage to hold at least n elements, but it doesn't actually fill the container with any elements
If elements are already allocated why ...
- 87
3
votes
0
answers
144
views
What is the best/correct way to handle shared memory objects in a heterogeneous multi-CPU embedded microprocessor context [closed]
I'm working on several projects that are targeting bare-metal multi-core microprocessors, with hardware provisions for shared memory.
In other words, the device has several separate CPU cores (of ...
- 5,953
1
vote
1
answer
134
views
Which resources of a modern x86 CPU core are occupied by memory transactions in flight?
I want to clarify how modern x86 architectures handle the latency of memory transactions that go all the way to DRAM. Specifically, which resources (which queues) get occupied waiting for the memory ...
- 4,838
0
votes
0
answers
26
views
Does alloc_calls of slab_info will decrease if the memory is freed?
I found the count of an API in alloc_calls is not increased over time and decreased if I free allocated memory. Is this true?
So if I unload the driver, the API should disappear in the alloc_calls ...
- 1
1
vote
1
answer
127
views
Reading another process' memory in C not returning results after multiple queries
I'm trying to read a variable from another process' memory in C and upon first query, for which I'm using VirtualQueryEx function, some potentially matching values are returned. However, when I change ...
- 63
2
votes
2
answers
212
views
How CPU read memory in alignment?
I have read a post about Why we need memory alignment.
I mostly focus the effects on the performance when CPU read memory.
The CPU always reads at its word size (4 bytes on a 32-bit processor), so ...
- 33