Add message polling mechanism for Lambda-SQS integration #2145
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
whummer
force-pushed
the
lambda-sqs-poll
branch
from
4e278fc to
53be5d9
Compare
whummer
force-pushed
the
lambda-sqs-poll
branch
from
53be5d9 to
629b9da
Compare
whummer
force-pushed
the
lambda-sqs-poll
branch
from
57c97cf to
79ff742
Compare
jgbmattos
pushed a commit
to jgbmattos/localstack
that referenced
this pull request
Mar 11, 2020
…ocalstack#2143) Add message polling mechanism for Lambda-SQS integration (localstack#2145)
jgbmattos
pushed a commit
to jgbmattos/localstack
that referenced
this pull request
Mar 11, 2020
…ocalstack#2143) Add message polling mechanism for Lambda-SQS integration (localstack#2145)
jgbmattos
pushed a commit
to jgbmattos/localstack
that referenced
this pull request
Mar 11, 2020
…2145) Added support to redrive policy on SNS product. It was necessary to change the lambda_api.process_sns_notification raise exception to the function caller and then handle the error for redrive policy work properly. add test to assert presence of message Subject in SNS Lambda events (localstack#2143) Add message polling mechanism for Lambda-SQS integration (localstack#2145)
jgbmattos
pushed a commit
to jgbmattos/localstack
that referenced
this pull request
Mar 11, 2020
…2145) Added support to redrive policy on SNS product. It was necessary to change the lambda_api.process_sns_notification raise exception to the function caller and then handle the error for redrive policy work properly. add test to assert presence of message Subject in SNS Lambda events (localstack#2143) Add message polling mechanism for Lambda-SQS integration (localstack#2145)
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Add message polling mechanism for Lambda-SQS integration.
This is a follow-up PR for the change in #2139 . Prior to this change, we used to intercept incoming SQS messages, check if they are linked to a Lambda function (via an event source mapping), and in that case call the Lambda directly with the event message, without forwarding the message to the actual SQS queue. This approach is fundamentally flawed, because Lambdas need to have access to the actual message
receiptHandlein order to manipulate messages in the queue.With this PR, we now implement an approach that is more in line with what's happening in real AWS. We're forwarding all SQS messages to the queues in all cases, and use a polling thread in the Lambda API to poll SQS queues for new messages. We're using a short-polling approach (
receive_messagewith zero wait time), as we want to monitor several queues in parallel, in regular intervals (currently configured to poll every second).Should fix #2039.