1,309 questions
Best practices
0
votes
0
replies
66
views
Multi-provider RPC failover in Rust - circuit breaker vs retry-only for blockchain nodes?
I'm building a production RPC transport layer in Rust that connects to
multiple blockchain RPC providers (Alchemy, Infura, QuickNode, public nodes)
with automatic failover.
The challenge: not all ...
Best practices
0
votes
2
replies
86
views
Rebalancing Traffic In Leaderless Distributed Architecture
Using Go I am creating an in-memory distributed store similar to Cassandra.
I have concept of storage_node with get_by_key and put_key_value. When a new node starts it starts to gossip with a seed ...
- 1
-1
votes
1
answer
135
views
Should Post and PostLike be separate microservices, and how to validate entity existence across services?
I'm designing a microservices architecture for a social media–like system and need guidance on service boundaries and cross-service validation.
Current Design
I have two separate microservices:
Post ...
Best practices
0
votes
0
replies
51
views
Architectural pattern for binding user authorization to demonstrated comprehension before system execution?
In many distributed systems, user authorization is treated as sufficient proof of intent.
Examples:
Clicking “Approve”
Signing digitally
Submitting a confirmation request
Triggering a smart contract ...
0
votes
0
answers
29
views
Handling log inconsistencies in Raft: how should a follower reconcile conflicting entries?
I'm trying to learn Raft, but some edge cases I find hard to understand.
Here’s an example scenario:
The leader has the following log:
1: A, 2: B, 3: C
A new entry arrives: 4: D. The leader will ...
Advice
0
votes
1
replies
56
views
What is the chance of losing a redis pub/sub message?
I understand that message loss in Redis Pub/Sub is possible and is considered normal behavior. The probability is low, but not zero.
What I’m trying to understand is how often this can realistically ...
- 1
Advice
0
votes
2
replies
87
views
How to make devices discover each other using WIFI
What's the best way to allow programs to discover each other on the network?
Let's say we are writing a system that tracks the usage of computers over the network.
We have an agent program that sends ...
- 13
1
vote
1
answer
61
views
How do I redesign a broken multi-service system where the entry point and child services are out of sync?
I recently joined a startup that has a pretty messy backend setup, and I’ve been assigned to sort it out.
Here’s the situation:
There’s one main entry point (a federation/onboarding service) that’s ...
1
vote
1
answer
212
views
Should I store Stripe Subscription details in local DB or query Stripes API to determine if a user is subscribed?
I have the following requirement.
A website (lets call it Website A) where I sell subscription plans for my SaaS
Payments are handled with Stripe
I am using an authentication service (Auth0) so users ...
- 31
0
votes
0
answers
77
views
ADKG-based threshold ECDSA signature recovers different address per transaction—how to compute aggregate `r` and signature parameters?
Background
I’m implementing Asynchronous Distributed Key Generation (ADKG) over secp256k1 so that N nodes collectively hold a threshold private key. After DKG each node has a secret share. To sign an ...
- 121
0
votes
0
answers
43
views
gRPC: HTTP-level resilience handler (retries)?
Should gRPC clients implement an HTTP-level resilience handler? Or only rely on the gRPC-level RetryPolicy? Why/why not?
For example, if the server responds with a 5xx status code (unexpected but ...
- 121
0
votes
0
answers
48
views
Is there a problem having min.insync.replicas < half the RF?
Many examples of Kafka topic configuration have RF = 3, min.insync.replicas = 2.
In the case of a cluster of 5 brokers, if we use RF = 5, should min.insync.replicas = 3? That seems "natural" ...
- 130
0
votes
1
answer
140
views
How do I limit the count of rows in a result set in Postgres without unnecessary locking?
This question is inspired by a 'general admission' variant of the common 'event ticketing' System Design interview question. Critically in this version, the user does not select a seat - they only say ...
- 33
0
votes
0
answers
35
views
How can I ensure this UDF works reliably across nodes if OceanBase scales horizontally?
OceanBase Version:
V4.2
I’m using OceanBase in MySQL mode, and I noticed that functions like NOW() and CURRENT_TIMESTAMP only provide microsecond (6-digit) precision. So I’m trying to create a custom ...
user30195804
0
votes
0
answers
46
views
Proving the Log Matching property in Raft consensus protocol
I have been trying to understand the Raft protocol for quite some time now. One thing that has always stumped me is the proof of the Log Matching property. One of my concerns is that the proof in the ...
- 118
0
votes
0
answers
47
views
How to use socketio and ipc together in python
I am trying to building an server that communicates with the client using socketio protocol, Now the server starts multiple application as a sub processes, the server communicates with this ...
- 70
0
votes
1
answer
71
views
is last write wins redundant for immutable keys?
In the book Designing Data-Intensive Applications > chapter-5 > Leaderless replication > Detecting Concurrent Writes, below is what Author says while talking about Last write wins (LWW)
The ...
- 1,394
0
votes
1
answer
91
views
Apache JMeter Master Slave Setup on Azure VM
I am new to JMeter Distributed Environment setup. I dont have any knowledge on how to setup the master slave configuration. I just have the information that we have a Master VM and we can spin up ...
1
vote
0
answers
38
views
In which cases Multi-Paxos couldn't support Primary Order required in Primary-Backup replication system?
In paper Zab: High-performance broadcast for primary-backup systems, the figure 1 shows that Paxos could violate primary order of requests.
I understand the result will be like that if each proposer ...
- 313
2
votes
1
answer
61
views
How can I sync JWT blacklists in PHP microservices without a bottleneck or SPOF?
I’m working on a microservices-based project where each service is a separate PHP application. They all rely on JWT for authentication and authorization. The tricky part is revoking (or blacklisting) ...
- 690
0
votes
1
answer
43
views
is Lost update possible with RAFT?
Lets say We have a cluster of 5 nodes and A is the leader. Following sequence of events take place:
A sends the replicate change request in parallel to all the followers.
Only B could receive the ...
- 3,185
-1
votes
1
answer
193
views
Does Raft follower store leader ID? If not, how does it redirect requests to leader?
Referring to this table depicted in the Raft paper, I did not find where do followers memorize the leader in any form such as identifier, physical addr, etc. Instead, I only find the leader ID in ...
- 2,311
1
vote
1
answer
228
views
Kafka and hotspots in a partition
I am new to Kafka and I understand that there is only guarantee of message order within one partition and not across partitions.
What I am not sure is if this can create scalability issues e.g. in ...
- 311
-3
votes
1
answer
45
views
Apache Kafka Problems with offset
I'm using confluent kafka library to create a distributed system, but I'm failing to understand some principles of Kafka itself.
Lets say right now I'm working with a Central, that has to listen to ...
- 1
0
votes
1
answer
40
views
NServicebus command handlers and multi region
We have an NServiceBus application running in two Azure regions: North Europe and West Europe. We are using SQL transport, and both applications in these regions connect to a shared database. ...
- 145
0
votes
1
answer
48
views
How to maintain synchronization between distributed python processes?
I have a number of workstations that run long processes containing sequences like this:
x = wait_while_current_is_set
y = read_voltage
z = z + y
The workstations must maintain synchronization with a ...
- 2,696
0
votes
0
answers
90
views
Is MassTransit with Azure Service Bus a good fit for tenant-isolated on-prem agents?
We are using MassTransit with Azure Service Bus in our backend system for messaging. Now, we need to extend our solution to communicate with on-premises agents that will be installed for each of our ...
- 595
2
votes
1
answer
172
views
What data is stored in the log compaction snapshot of a Raft-based distributed file system?
I'm working on a Raft implementation as part of my distributed file system and I've run into a problem with the log compaction process.
Accurding to the official Raft paper, when a log reaches a ...
- 21
0
votes
1
answer
229
views
Microservices inbox-outbox pattern
I have a question I'm curious about. Let's say we are developing a microservice social media application (I chose this topic for practical purposes :)). I'm using the inbox-outbox pattern to ensure ...
- 11
1
vote
4
answers
203
views
How does client handle failures in RAFT-replicated datastores? [closed]
Consider a database like cockroachDB that uses RAFT protocol for replicating data to a replica group owning a partition of the data. How does a client handle a request that fails in such DBs? Because, ...
- 129
2
votes
1
answer
205
views
Handling Correlation ID Changes in Event Sourcing When an Entity Switches Context
I'm working on an event-sourced application that crawls sports betting games from different bookmakers. I have two primary aggregates in my system:
Game: Represents a sports betting event for a ...
- 12.8k
0
votes
1
answer
537
views
Multiple DbContext in .NET Aspire
I am trying to add multiple DbContext instances in app, which is launched with .NET Aspire. I also want those separate contexts to have configuration available (in this case to have migration history ...
0
votes
1
answer
368
views
Log viewer for binary logs
I have a clustered real time system that produces a very large amount of binary logs.
I get a bunch of binary logs from each node in the system and I want to view the logs in a convinent way. Mostly, ...
- 115
0
votes
1
answer
78
views
Fault-tolerant queue-worker architecture in Kafka?
I am new to using queue-worker architectures and I'm interested in how to make it resilient to a worker failing. For example
We have a pool of workers Alpha that put entries onto queue A
Then the ...
- 178
1
vote
1
answer
263
views
Redis backed rate limiter -- Inconsistent?
I was following this blog on implementing Rate Limiter using Redis.
Link to the blog
Here they have used MULTI to pack all the atomic commands. This ensures that we're not concurrently writing wrongly ...
- 142
2
votes
0
answers
76
views
Tomcat Web Application Not Loading Correctly in Docker Container: HTTP Status 404
I am running a TomEE server inside a Docker container, but my web application is not loading as expected. Here is the setup I'm using:
Docker Image and Container:
Image: interesting_picture:latest (...
- 23
1
vote
1
answer
777
views
Is my understanding of a Distributed Lock correct?
I'm having some trouble understanding the need of a distributed lock. I did think of an example where it may be required but I'm not completely sure. I would appreciate some comments if I'm thinking ...
1
vote
0
answers
59
views
How to Aggregate Asynchronous Messagges from a Queue into a Final Result at Scale?
I'm working on a system where:
A producer sends approximately 100 million messages daily to a message queue.
The consumer processes each message from the queue and produces multiple parts as output.
...
- 222
1
vote
1
answer
126
views
Opentelemetry traceid for Couchbase Database Change Protocol
I wanted to address an important aspect of our microservices architecture, specifically regarding our tracing implementation with OpenTelemetry. We have multiple microservices operating seamlessly ...
- 347
0
votes
0
answers
61
views
An interesting scheduling problem: how to serve multi-stage microservices chain that share resources
Recently, I encountered a scheduling problem in a distributed system and I hope to get some help: for a multi-stage microservice that has two stages calling the same instance, such as A-->B-->A, ...
0
votes
1
answer
241
views
Replicated & Distributed Clickhouse - Keeper Replication
I need to create the replication in clickhouse; on two different machines that are under the same network. I have tried to configure it but I have the following error:
SQL Error [999] [07000]: Code: ...
3
votes
2
answers
403
views
Why is read repair not sufficient in making dynamo-style database linearizable?
I am reading DDIA. It says "possible to make Dynamo-style quorums linearizable at the cost of reduced performance: a reader must perform read repair (see “Read repair and antientropy” on page 178)...
- 372
0
votes
1
answer
81
views
Table primary key uniqueness across different / multi-region Amazon RDS postgres
I have an application that uses postgres database on one region (US West) containing several tables, one of which contains several hundred thousand records (let's call it "events" table with ...
- 1
0
votes
0
answers
31
views
How can a distributed system satisfy CP in CAP theorem?
If a system is partition tolerant, it's impossible for it to be consistent since there's no way for one node to update another. How can you be both consistent while partition intolerant be possible?
- 2,453
0
votes
0
answers
63
views
Deduplication , Grouping for events table at scale
I'm working with an events table where different source tables trigger writes into this table with columns: entity_id and payload. These events are then published to a Kafka topic using a message ...
- 518
1
vote
1
answer
724
views
Message ordering in event driven architecture
Consider there are 3 microservices - s1, s2 and s3.
s1 sends message m1.
s2 consumes message m1, applies some business logic and then sends message m2.
The problem is that s3 receives message m2 ...
- 27
0
votes
1
answer
184
views
Citus Colocation Behavior
I am using Citus as a managed service in the cloud with Azure Cosmos DB for PostgreSQL.
I have 1 coordinator and 2 worker nodes setup.
There are distributed tables and reference tables created.
...
- 1
2
votes
0
answers
81
views
(Re)attaching to an App Insights Operation from another machine/process (not using HTTP)
I have a .NET 8 distributed system in AKS where work is divided among workers using a Manager/Worker pattern. With work shared out on a Redis List. I'm aiming to get unified logging via Application ...
- 3,184
1
vote
0
answers
480
views
How to efficiently pagination and sort data from multiple services?
I need to fetch and filter data from three different services: ProductService, PriceService, and StockService. My goal is to get products that belong to Category = 54, have stock available, and are ...
- 11
2
votes
2
answers
174
views
Handling Race Condition in distributed system
Hi I have order creation functionality in my project and I am giving a order_id to client which is a auto-increment ID
order = serializer.save(user=user,created_by=user,platform=platform)
Now how ...
