1

I'm working on an event-driven system using AWS SQS and facing a design decision when sending commands between services. The command triggers an operation in the receiving service, and that operation requires multiple pieces of data — potentially 15+ parameters.

The dilemma:

  • Should the SQS message contain all the necessary information for the command to be processed (i.e. fully self-contained payload)? OR
  • Only include a reference ID, so that the receiving service can fetch the data it needs (e.g. from a shared database or API)?

I’m trying to keep the architecture clean, scalable, and maintainable

My gut: It feels messy to cram all parameters into a single message, but I'm aware of introducing latency and tight coupling by forcing the consumer to retrieve external data.

What’s the most robust and clean approach in your experience? Any best practices for hybrid strategies?

Improve this question

2 Answers 2

Reset to default
0

I'd always prefer including the necessary data in the message and avoid the "callback". Only sending a notification with a reference, and have the receiver get the actual data via query

  • introduces an unnecessary runtime coupling

  • makes the receiver more complex, it now has to handle more failure cases - e.g. if the service providing the data is unavailable, the receiver needs to stash the command for later retry.

  • can cause load peaks on the sender / data provider

  • decouples the command from the data - maybe the parameter values change between the time the command has been issued and the callback, leading to possible confusion.

I'd only use a notification with a reference if the payload is too big for the message channel and if the referenced data is immutable. E.g. if your service generates a PDF document and wants to notify others about it.

P.S.: You should be mindful of the terminology. Your saying you're developing an event- driven system, but then you talk about sending commands.

Events and commands are very different things!

  • Events describe something that happened in the past, a fact, and are fire-and-forget. An event can't "fail".

  • Commands are a request to perform an operation or change a state, and are request-reply, they yield the response if the command was executed successfully or not. Commands can fail.

(In almost all cases... for a more differentiated view see https://www.reactivesystems.eu/2024/11/30/not-all-commands-are-equal.html)

If you're sending commands, you're implementing a message-driven system, but it's not an event-driven system.
In fact, if you want to keep it scalable and maintainable, you should think about making it actually event-driven, and publish events, not send commands.

Improve this answer
Sign up to request clarification or add additional context in comments.

1 Comment

Thank you for your response. I’ll go with that approach! And yes, thank you for pointing out my error — I sometimes mix up the terminology. The case I presented is indeed not part of an "event-driven system." We are using the concepts of an event-driven system elsewhere in our application, where different systems listen for events.
0

First of all - as for any other architecture question - actual answer is "it depends" and "everything is a trade-off".

As a rule of thumb in event-driven systems the owner of the event/domain determines the event (message) data based on the domain and should add all the necessary information to it (which has an additional benefit of "snapshoting" the data).

But there are nuances here to consider:

  1. Ownership of the data - if the event producer owns some data it usually should add it to the message but if not - it hugely depends on the case (for example for some pipeline processing it can be totally fine to pass through incoming data, but if you produce change event for some domain you should think before enriching it with data from external dependency even if you have it at hand at the moment).
  2. Data size - according to the docs maximum SQS message size is 1Mb (though some sources state that it is 256Kb), for larger messages SQS already implements the reference approach (i.e. using S3 and reference to it in the message via Amazon SQS Extended Client). But limits are not only consideration here, for event-heavy systems you can observe performance hit on smaller message sizes.

So to sum up:

Should the SQS message contain all the necessary information for the command to be processed (i.e. fully self-contained payload)

As a rule of thumb - yes, this should be the go to approach.

potentially 15+ parameters.

In systems I worked with it is not a rare case to have a message with 15+ fields.

but I'm aware of introducing latency

As said before - latency can be a lesser trade-off than big message size (especially when there are multiple consumers to the event stream)

Improve this answer

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.