-
|
What would be the best practice to deal with duplicate messages like this? We have a few Or an API endpoint may update objects irregularly and each update will add the object to the queue for further processing, how do we prevent the same object from being added multiple times? |
Beta Was this translation helpful? Give feedback.
Replies: 1 comment 2 replies
-
|
I don't think Symfony Messenger has a built-in solution for this problem. You will have to implement a solution yourself. I would write a middleware that drops the second and every next message for the configured message types. If you run you PHP app in a fire and forget way (e.g. PHP-FPM) then you can just track the FQCN of the messages in your middleware and drop message if it is a duplicate. However if you run PHP app as long running process (CLI based solutions like PHP-PM or RoadRunner) then you can't just track already sent FQCNs because 2 messages of the same type might be coming from 2 different requests. In this case you will have do some request differentiating. If your problem applies also to your cron jobs (console commands) then remember that you don't have request context in this case (but you can still implement some kind of tracking, but in this case not with a use of request). The same with async messenger workers (which are also a console commands) - the message that you don't want to duplicate may be sent multiple times in an async message handler that handles completely different message. |
Beta Was this translation helpful? Give feedback.
-
Do you have any idea where to start with this type of tracking? Is the Header fields in the DB table the right place? |
Beta Was this translation helpful? Give feedback.
-
|
I would start with creating a service class that creates and holds a unique id for the request (e.g. inside event listener on kernel.request) and for the currently processing message in case of messenger consumers (inside listener on WorkerMessageReceivedEvent). Then inside a middleware you inject this service and maintain an internal array that groups handled message FQCNs by the unique id provided by this service. However I'm not sure if it works for requests in PHP as long running process (like RoadRunner). Although you receive a new instance of Request object every new request but you may still receive the same instance of TrackingService for two different requests. If so then this service cannot create a new id because another request might not be finished yet and we will be replacing the id it created and is relaying on. But I repeat, I didn't have time to go deep inside such solutions internals and I'm just saying what I read on the Internet (I could misunderstood something). |
Beta Was this translation helpful? Give feedback.
I don't think Symfony Messenger has a built-in solution for this problem. You will have to implement a solution yourself. I would write a middleware that drops the second and every next message for the configured message types. If you run you PHP app in a fire and forget way (e.g. PHP-FPM) then you can just track the FQCN of the messages in your middleware and drop message if it is a duplicate. However if you run PHP app as long running process (CLI based solutions like PHP-PM or RoadRunner) then you can't just track already sent FQCNs because 2 messages of the same type might be coming from 2 different requests. In this case you will have do some request differentiating.
If your problem …