186 questions
Advice
1
vote
2
replies
71
views
Optimizing Node.js worker threads memory pooling for heavy Monte Carlo slot simulations (RTP verification)
I'm developing the math engine for a high-volatility cascading slot game (similar to Megaways mechanics). As part of our global compliance and certification process, we need to run Monte Carlo ...
0
votes
2
answers
202
views
How to wait for Worker-Threads execution with Promises
I have a Node.js script where I need to parse a bunch of big files. My first attempt was using Promises, in general it is working. However, the script is as slow as reading the files one-by-on in a ...
- 60.7k
0
votes
1
answer
115
views
How do I import js files in a worker_thread when using Webpack and Electron Forge?
Electron Forge and Webpack cannot do worker_threads if the worker imports something.
We'd want to do this to do heavy computation without blocking the main thread, allowing the UI to remain responsive....
2
votes
1
answer
596
views
How to use NodeJS worker_threads with typescript?
I need to use worker_threads in Nodejs (server side question only) + Typescript.
I tried the following without success:
scripts/worker.ts
import { parentPort, workerData } from 'worker_threads';
if (...
- 1,991
0
votes
2
answers
533
views
How to test a worker thread in NodeJS?
I have a node script that uses a worker thread. I want to test it with Jest. The file is called, but the code is never executed.
Here is the function that calls the worker:
// index.ts
function foo(){...
- 6,457
1
vote
0
answers
259
views
nodejs write to same file with multiple worker threads concurrently
I'm wondering whether we can write to the same file in multiple worker threads concurrently .
I've goolged around, and many people say that this operation may cause file write collisions, so I write ...
- 660
0
votes
0
answers
144
views
Using server-side streaming RPC in tonic, is it safe to do heavy compute without spawn_blocking?
When doing server-side streaming in tonic, the generated async function is expected to return something which implements Stream. If I construct my stream by taking a receiver and then calling .map(...
- 5,604
-1
votes
1
answer
251
views
How does completablefuture works? [closed]
Need to know the insights of the complete future task without an executor.
Here is my sample code
CompletableFuture.runAsync(() -> {
dosomething();
});
How is this task assigned to the thread? ...
- 1
1
vote
0
answers
227
views
How many threads Node creates per core when using worker threads?
I'm running node js worker threads on MacBook Air M1. There are 8 core. And using PISCINA to create threads. I'm getting 12 threads. Are those for one core or on all 8 core. How do I know if main ...
1
vote
1
answer
2k
views
How to use ES6 import in worker thread in Node.js?
How to use ES6 import in worker thread?
I'm trying to utilize ES6 import syntax in a worker thread in a Node.js environment. However, I'm encountering issues with the import statement. Here's a ...
- 21
1
vote
0
answers
325
views
How to use mongo connection pool shared with worker threads
I am implementing a nodejs app where I am using worker threads for parallel data processing which requires mongo connection. I am using mongoose here for mongo operation but if I create connection in ...
- 626
1
vote
1
answer
392
views
Singleton worker thread with multiple functions
I am trying to implement a singleton worker thread which has multiple functions which can be requested by the user. I have seen implementations for worker threads using always the same function but I ...
- 53
0
votes
1
answer
362
views
Worker threads in Node js
Unable to do axios calls inside worker threads
Hi,
I have a node js application and I am trying to use worker threads for one of my use case.Inside my worker thread I need to do axios call.I was not ...
2
votes
1
answer
1k
views
Worker Thread, other file than self, cannot find module
I want to have my worker thread as an external file from the runtime file.
My current folder structure is
src/
> service.ts // my 'main'
> thread/
>> test.js
inside my service.ts i have ...
0
votes
1
answer
264
views
worker_threads with node.js in nwjs
worker_threads with node.js in nwjs (v0.77.0) crashes without error message
Here a list which versions work and where you can find the crash. It started with version v0.69.1
0.77.0 ⛔
0.76.1 ⛔
...
0....
- 103
0
votes
1
answer
252
views
only one worker_threads running when i spawned two in nodejs
I have this file analyze-caption-job.ts
async function analyzeCaptionJob(): Promise<void> {
while (true) {
console.log("analyzing!");
await new Promise(f => setTimeout(f, ...
- 11
0
votes
1
answer
129
views
I have a question about C#'s worker threads
In this code, when you use await Task.Delay(5000);, the line Console.WriteLine("START TestSemaPhoreSlims:ID[" + ids + "]... is executed without any delay.
However, when you use Thread....
0
votes
1
answer
349
views
Does workerpool declared in multiple routes can still maintain its cpu usage without caring about threshold
I hope to figure out a node.js system with workerpool to handle cpu intensive tasks, but
there is something confused me about the cpu usage in multiple routes.
A scenario is like this:
route1.js:
...
- 469
0
votes
2
answers
946
views
How to specify a local file path in Firebase Functions using worker_threads?
I have a firebase function that uses worker_threads to off load some cpu intensive tasks. To initialize the workerThread I give it a local file path to the worker file. But when the firebase function ...
- 24
1
vote
0
answers
323
views
Would it be possible to iterate over typescript files in a folder and start a worker thread for each using `ts-node`?
Overview
How would one start a worker thread with ts-node? Every attempt that I've made leads to a MODULE_NOT_FOUND error.
Various Attempts
Currently, I'm attempting to spawn a new worker thread using ...
- 36
0
votes
1
answer
522
views
Error trying to terminate worker thread using setTimeout()
When trying to terminate a worker thread using setTimeout() I get the following error:
node:internal/worker:361
this[kHandle].stopThread();
^
TypeError: Cannot read properties ...
0
votes
0
answers
1k
views
NodeJS Worker Threads - message sending before function finished?
I am using a pool of workers to complete some CPU intensive tasks in Node.
However, I have a problem in my code below. When a task is first run, everything goes as expected, the pool is created, then ...
- 76
0
votes
0
answers
334
views
Parallel processing in Node.js
I'm trying to write a node code that will read multiple json files from directory and insert mapped data into mongodb collection. In order to speed up the process, I have follwed following steps:
Put ...
- 833
1
vote
0
answers
442
views
Cannot load data url in Worker Thread
My current worker thread which works fine with file path that tries to console log in that JS file:
console.log("hit here");
Now I get the base64 of that file and use Data URL to replace JS ...
- 450
0
votes
0
answers
184
views
Node Worker thread is not working inside api call
I am using node version 16 and following all documented steps. Still worker thread is not working.
`
const addPaymentApi = async (req) => {
const { payload, adminId } = req.body;
logInfo(`...
- 39
2
votes
1
answer
2k
views
Node.js- share a heavy variable from parent to worker threads
I want to share a variable between parent and worker threads. The variable is heavy (100+ MB - arrays of JSON data) so if I pass it as an argument or re-read from data source, it drastically increases ...
- 363
2
votes
1
answer
2k
views
Node.js thread pool vs worker_thread
thread pool size can be determined with UV_THREADPOOL_SIZE.
Worker_thread can create threads dynamically in runtime.
Is this a only difference between thread pool and worker thread?
Recently, I'm so ...
2
votes
1
answer
154
views
Exiting the stream NodeJS
Only 'text' is output to the console, but 'text2' and 'text3' are not output, because exit from the stream is faster. This is the most simplified code of the real project structure. I can't figure out ...
- 161
3
votes
0
answers
743
views
how to work with response object in nodejs stream, exceljs and worker thread
I am using worker thread and stream at same time in node JS project. At initial I was not able to pass res object through main process to worker thread. I saw many stackoverflow question and solution ...
2
votes
0
answers
98
views
Using wokerpool on multi node.js processes running on the same server
I have a server with 6 CPU logical cores, there is 6 node.js (v 16) processes running express.js app within the server and managed by pm2 process manager, and I'm using Nginx proxy to load balance the ...
1
vote
0
answers
659
views
Multiple node processes vs multiple worker threads
I have a project which I have to convert about 300+ different small application to node. The plan is to make one linux service per application, something like:
Service 1
node /le-path/le-service1.js
...
- 1,404
1
vote
0
answers
1k
views
Pass data between worker threads in node js
I have a nodejs application where I need to use a worker thread.
The worker thread gets a message from the main thread. The message is sent inside a function in a class.
I need to pass the instance of ...
- 11
0
votes
0
answers
425
views
Using Node JS worker threads for I/O intensive operation and make use of multiple core CPU
How effective is using worker threads for an I/O intensive application?
Context- I am developing an application which sends out multiple API requests in parallel. There is a limit of how many requests ...
- 41
0
votes
0
answers
2k
views
How to run function in different thread with node.js
I have a REST API coded with Node.js and there is one route that applies NTFS Rights to folders. The request can take 1 second but can also last several minutes (it depends on the size of the folder).
...
- 45
1
vote
1
answer
1k
views
create worker file (worker threads nodejs)
I want to create a worker to send back an array as response to my main file, but I can't understand why the worker is not sending postMessage to main. Here is some dummy code to explain
My main file:
...
1
vote
1
answer
2k
views
Cannot import modules in worker script
I'm new to using worker threads, thanks in advance for your help and time.
Using node v12.22.4, mongodb v4.2.19
What I'm trying to achieve is: on client-side, user asks for a big export/download ...
- 11
0
votes
0
answers
2k
views
Importing/exporting Typescript type definitions in Node.js worker threads
I am learning Typescript and ran into the following problem with Node.js worker threads:
In a tutorial I found this JS code to be implemented by the worker:
//worker.js
const {parentPort} = require(&...
- 299
2
votes
0
answers
526
views
Unable to import modules in node-worker-thread-pool
I am using node-worker-thread-pool package.I am unable to import certain node modules into my worker.ts, giving an error SyntaxError: Cannot use import statement outside a module. How to resolve this. ...
- 21
4
votes
1
answer
506
views
Why is all of my output appearing before each brute-force attempt when attempting to crack an ethereum keystore wallet file in node.js
I found a post describing how to recover an Ethereum wallet keystore by guessing a single password, however, it uses node synchronous code, and I'm trying to convert it into asynchronous code so that ...
- 65
0
votes
0
answers
605
views
how to use threads and sleep() functionality with node.js
I am trying to learn node.js after python and trying to thread an infinitely running process in the background with a 2 minute sleep().
main.js
const TeleBot = require("telebot");
const fun ...
- 246
0
votes
1
answer
460
views
Node.js: How to process objects of a big JSON file one by one to avoid heap limit errors
im trying to process a few hundreds of json.gz files using worker threads.
at some point im getting js heap limit error due to 3 big files(about 3gb each unzipped).
i tried to find a way to process ...
- 58
0
votes
1
answer
728
views
How to properly handle worker_threads termination, when there are some node instances? (Like docker-swarm or Kubernetes cluster)
I'm creating worker_threads in node and I'm collecting them to custom WorkerPool. All workers are unique, because they have unique worker.threadId. My app have ability to terminate specific worker -- ...
1
vote
0
answers
189
views
Can a SharedArrayBuffer be picked up by garbage collection in Node?
I'm trying to build a Node application using worker threads, divided into three parts.
The primary thread that delegates tasks
A dedicated worker thread that updates shared data
A pool of worker ...
- 229
0
votes
1
answer
228
views
Undefined object properties inside callback function in Worker Thread's parentPort listener
I have a worker object something like this:
const WorkerObject = {
mapProp: new Map(),
funA() {
// some task
},
funB() {
// some task
},
workerListener() {
parentPort.on('...
- 931
1
vote
3
answers
3k
views
What is the importance of using WorkerThread annotation for a method that run in coroutine
I have noticed some methods in a project that I started to work recently as mentioned below. In this method functionality is executed in a coroutine. I wanted to know the purpose of the WorkerThread ...
- 524
1
vote
0
answers
505
views
What happens if @WorkerThread annonated function calls another function in android?
Which thread the subsequently called functions are executed?
You may consider a function like the getNewsFeed() function inside the repository class given below -
@Singleton
class NewsFeedRepository @...
2
votes
1
answer
3k
views
How to bundle worker_thread dependencies with webpack
I'm writing an electron application that uses electron forge for setup and configuration.
The project uses Webpack 5 as a bundler and typescript as the development language. I'm having trouble running ...
- 487
0
votes
0
answers
739
views
Worker Thread Implementation to connect SQL Server in Node JS
I need to implement Worker thread for an API to execute in concurrent way,
API work is pull the data from SQL server,
How to implement worker thread concept to connect SQL server and get the data from ...
0
votes
0
answers
690
views
Using worker_threads to loop through array of 300k items and run tests with jest
I have never used Workers with node before, but I have to run a series of tests with Jest with a large array of objects with size 300k+ items.
I'd like to run workers with subportions of this array ...
- 100
6
votes
1
answer
4k
views
How to share context between worker threads
I would like to create a worker thread in a node.js app and pass the current context to the new thread, so I would be able to access my variables and functions within the new thread, Is there is a ...
- 168