14,324 questions
-1
votes
0
answers
73
views
Migrate aws dynamo db from one region to another [closed]
I have a service running on aws fargate. This service is using dynamo db hosted in x region. I want to migrate my dynamo db from one aws region (x) to another (y).
If our services can accomodate ...
- 11
0
votes
0
answers
38
views
Using Kotlin Data Classes for DynamoDB
I have the following class that I am using as a dynamodb item in Kotlin
@DynamoDbBean
data class Transaction(
@get:DynamoDbPartitionKey
val userId: Int,
@get:DynamoDbSortKey
val ...
- 2,375
0
votes
1
answer
72
views
Invalid UpdateExpression trying to upgrade DynamoDB SDK from v1 to v2
I'm trying to upgrade my DynamoDB client from SDK 1 to SDK 2 (full versions at bottom), and I'm running into a strange issue. Server is unchanged (literally hitting the same tables).
We're trying to ...
- 421
1
vote
0
answers
58
views
Batch unprocessed items are actually processed
I'm implementing a solution with high throughput, 500 - 700 items per second per instance, and I'm using DynamoDB for persistence. For every item I performat least 2 writes and one read, some like:
...
- 501
1
vote
1
answer
86
views
Lock a DynamoDB table on reading/writing operations
I have a DynamoDB table with entries that can have a status (waiting, error, running...). Only up to 25 entries can have running status.
My objective is to have an AWS Lambda function that checks if ...
- 3,082
0
votes
0
answers
58
views
Can't get a row from dynamoDB by id
I’m seeing a weird intermittent read issue with DynamoDB using the AWS SDK for JavaScript v3. For some (but not all) item IDs, the first call to GetItem returns no item, and the second call ...
- 603
3
votes
1
answer
126
views
Amazon DynamoDB changes number format with Boto3
I have this data to entry into AWS DynamoDB:
item = {
"PK": {
"S": "DEBUG"
},
"SK": {
"S": "#ID_LOG#2025-10-09T07:48:...
- 2,626
0
votes
1
answer
46
views
Effective pagination for descending order queries in DynamoDB
I have an use-case of building a news feed for which I store the articles along with the timestamps in a DynamoDB table. I want to build infinite scrolling capability, and retrieve the article with ...
- 187
0
votes
1
answer
98
views
How DynamoDb console maintains pagination along with filtering
Suppose I have 110 records and I filter upon status for the first page two calls are made the first calls scans 25 items and after filter it receives 20 matching items , it uses the id of the 25th ...
-1
votes
1
answer
62
views
How should I be storing types before sending custom Json data to DynamoDB
Working on an API endpoint where I need to send custom-formatted Json data to a DynamoDB table, store the original types of each field, and then convert the DynamoDb Json export back into my custom ...
- 1
0
votes
1
answer
52
views
How to model a many-to-many relationship in DynamoDB with composite keys and enforce unique (id, ruleId) pairs?
I have a DynamoDB table defined as follows:
Partition key: id (string)
Sort key: ruleId (string)
Example item:
{
"id": "123",
"ruleId": "abc"
}
What I want is ...
0
votes
0
answers
79
views
How to properly use "limit" in AWS Amplify Gen2 list-function?
Please help me to understand how to properly and correctly use the list-function of AWS Amplify.
After running into many issues of existing data not being found/retrieved, I found out that if no "...
- 45
0
votes
1
answer
74
views
DynamoDB async query calling subscribed consumer after timeout
TLDR: DynamoDB async query on GSI is returning duplicates when forcibly timing out the query request.
I'm running performance tests on a solution built with CompletionStage and the non-blocking ...
- 501
0
votes
1
answer
76
views
How to query multiple DynamoDB objects at the time
I have a table Tours with a Partition Key (PK) id, which is unique and a Global Secondary Index (GSI) geohash, which has repeated values.
I need to get all the geohashes in the array geohashNeighbors, ...
- 1,915
0
votes
1
answer
51
views
Error creating bean with name 'dynamoDbEnhancedClientLocal' defined in class path resource [com/sgeidatareconciliation/config/DynamoDBConfig.class]
I am doing dynamoDb connection for the first time. My goal is to save some data in dynamodb and I am using AWS SDK2 to doing so below is the error I am getting Few things I have tried
Defined a @...
1
vote
1
answer
68
views
ECS Fargate Task performance worsened after re-deploying same task definition
We have an ecs service that uses Fargate tasks to connect to dynamoDB.
The application has an optimized fetch time under 100ms when querying dynamoDB tables in our testing environment.
For some R&...
- 17
0
votes
1
answer
118
views
DynamoDB table: How to set the Epoch timestamp from Date in Java using Spring JPA? [closed]
I am using Java 17, Spring Boot 2.6.2, Lombok, & aws-java-sdk-1.12.136 to create a RESTful API CRUD service.
I need to create a DynamoDB table which should be able to sort books in chronological ...
- 5,494
0
votes
1
answer
98
views
Changing DynamoDB from Provisioned to On Demand via CDK process?
I'm trying to change my DynamoDB table from Provisioned to On Demand via CDK.
The commented out code is what my table was provisioned with before, for autoscaling.
table = new TableV2(this, props....
1
vote
0
answers
57
views
DynamoDB to Athena - dynamic schema detection for evolving "map" field
In AWS, I have a DynamoDB table with one of the fields as a map (my_map). This table is exposed to Athena, so data can be queried there (and eventually in Grafana).
The Athena table shows a schema ...
- 336
0
votes
1
answer
62
views
Flink DynamoDB Sink: "The provided key element does not match the schema"
I'm trying to write data from a Flink SQL job to a DynamoDB table, but I'm consistently getting the following error:
Caused by: software.amazon.awssdk.services.dynamodb.model.DynamoDbException: The ...
1
vote
1
answer
93
views
How to avoid full table scan in Glue's create_dynamic_frame.from_options for dynamodb
My Dynamodb table has both PK and SK. it has huge data set(500 GB).
I'm using below syntax for querying data based on PK in Glue, But it does a full table scan leading to the glue timeout. Have ...
- 541
0
votes
0
answers
54
views
Amazon DynamoDBLockClient not updating heartbeat
I'm using the AmazonDynamoDBLockClient for some synchronizations and something strange is happening when breakpoints are enabled.
Whenever I'm debugging something or the thread is paused on a ...
- 501
1
vote
0
answers
71
views
DynamoDB returning empty between two dates
I have this table that has this line on it as you can see below:
id,"data_dia","nome_completo","nome_produto","nome_tuna","numero_socio","...
- 21
0
votes
0
answers
157
views
What is the default HTTP client in AWS Java SDK v2: UrlConnectionHttpClient or ApacheHttpClient?
I found conflicting information w.r.t AWS docs, as the AWS developer guide says: “Synchronous service clients in the AWS SDK for Java 2.x use an Apache-based HTTP client, ApacheHttpClient by ...
- 53
2
votes
1
answer
53
views
Transactional delete of parent item only if specific child items don't exist
I have a DynamoDB table with the following item structure:
PK: PARENT#<ID>, SK: PARENT#<ID>
PK: PARENT#<ID>, SK: CHILD_A#<ID>
PK: PARENT#<ID>, SK: CHILD_B#<ID>
PK: ...
- 773
1
vote
1
answer
53
views
How should I model a DynamoDB entity with partially unique external IDs
We're using DynamoDB with the Single Table Design pattern for our system, and I need help modeling the Response entity.
Context
Our platform allows users to create Panels for recruiting participants ...
- 137
0
votes
0
answers
78
views
Migrating to AWS SDK V3 and IRSA, but session is not refreshed correctly
I am migrating some services to aws SDK V3, and I need to use web identity token for the credentials, but I am facing a strange problem where it works for an hour and the first request after this hour ...
- 49
0
votes
1
answer
37
views
Return all calls between 2 date in dynamoDB
I want to select all calls happened or at least started or ended between 2 date range in dynamodb. Partition key is some id and sort key is leftRoom. Is it correct approach? left is end time, started ...
0
votes
1
answer
41
views
Document.FromAttributeMap(item) all properties empty objects
I'm using IAmazonDynamoDB.QueryAsync(request) to query and filter on an index.
Now, in trying to cast the result to my model type, I'm doing the following:
var item = response.Items.First(); // I have ...
- 738
1
vote
1
answer
73
views
How to delete and add an element from set in one dynamodb operation
I am getting an error of:
ValidationException: Invalid UpdateExpression: Two document paths overlap with each other; must remove or rewrite one of these paths; path one: [photos], path two: [photos]
...
- 478
-4
votes
1
answer
55
views
Filter by sort key is substring of the given input [closed]
I have an interesting use-case where I want to check if in my dynamo table there exists a key value pair whose sort key is a substring of the given input string. I dont want to fetch all the data in ...
- 13
0
votes
0
answers
192
views
Auto populate timestamp Issue AWS SDK v2 in DynamoDB Java
In aws sdk v1 :
@DynamoDBAutoGeneratedTimestamp(strategy = DynamoDBAutoGenerateStrategy.CREATE)
Above annotation was there to create auto generated time field.
In aws sdk v2 replaced with below code:
...
- 142
0
votes
1
answer
68
views
AWS Atomic update with put item wrapped in a TransactWriteItems
I want to perform an atomic write in DynamoDB using TransactWriteItems that includes:
An atomic update (ADD) on a wallet balance.
A PutItem to write a corresponding transaction record.
Here is the ...
- 73
0
votes
1
answer
60
views
Unable to fetch an item in AWS DynamoDB
Environment:
NodeJS used with "aws-sdk": "^2.1692.0" to test DynamoDB operations
Goal:
To fetch an item in an exsiting table called CUSTOMER_LIST which contains a list of items.
...
- 435
0
votes
1
answer
43
views
How to send a complex object in dynamo attribute using sdk?
I am trying to insert a record in dynamodb and then get it processed using a lambda via SQS. This record contains a complex object. When I am inserting it using the sdk, i am able to see the record in ...
- 1,541
0
votes
1
answer
89
views
How to use CloudWatch Contributor Insights in DynamoDB if my organization only allows for it to be enabled using Machine Roles
Our team has been running into challenges with hot partitions and we have determined that Contributor Insights would be a great tool to use for future hot partitioning issues.
The problem is we do no ...
- 255
1
vote
1
answer
89
views
How does DynamoDB's LIMIT parameter work exactly?
Extending from DynamoDB Limit on query.
I have a use case where the need is to fetch the latest available exchange rate from table. I've considered the following setup:
partition key as currency_pair ...
- 15
0
votes
1
answer
104
views
Facing issues with DynamoDB Dax - Incorrect client builder
Working with Java and DynamoDB Dax, unable to find any clear examples for Dax. There are 3 packages
https://mvnrepository.com/artifact/software.amazon.dax/amazon-dax-client
https://mvnrepository.com/...
0
votes
0
answers
33
views
Process Dynamodb CDC stream to convert it to a pandas dataframe
I’m using a DynamoDB stream to read data. The data contains several columns, and a few of them are expected to be of SUPER type in Redshift, as they include lists, dictionaries, or string values.
I'm ...
- 11
0
votes
0
answers
62
views
Why does table.deleteItem(primaryKey) behave differently when using with DynamoDBLocal vs aws-sdk (actual dynamoDb table)?
I have below code:
DeleteItemOutcome outcome = table.deleteItem(primaryKey);
int status = outcome.getDeleteItemResult().getSdkHttpMetadata().getHttpStatusCode();
This is working fine when using with ...
- 394
0
votes
0
answers
39
views
How should one prevent a DDB item being deleted unless there are no items left which reference it?
Let's say I have 2 types of item in my DDB table: Trains and Passengers. They look like this:
{
"type": "train",
"id": "OrientExpress" // partition key
}
{
...
- 1,017
0
votes
0
answers
22
views
Retrieve paginated results using DynamoDbEnhancedAsyncClient
I have a use-case when I need to get all items in a dynamodb table by their partition key. I am using the DynamoDbEnhancedAsyncClient and I want my items to be paginated so that when I do the get ...
- 1,449
0
votes
1
answer
66
views
dynamoDB table access : how to authorize guest user to read and grant more rights to logged users on a group-basis
# context : AWS Amplify gen 2 (angular)
According to Amplify doc https://docs.amplify.aws/angular/build-a-backend/data/customize-authz/ ,the available authorization strategies do not consider my case,...
0
votes
1
answer
105
views
Running AWS DynamoDB inside a Dev Container
I'm trying to setup a devcontainer for rust project which uses dynamodb as its database.
This is the compose file (referenced AWS docs)
services:
api:
build:
context: .
...
- 275
0
votes
1
answer
121
views
DynamoDB item with NULL
I have this data model
public class ChallengePart_dynamoDB
{
[DynamoDBProperty("numPlayers")]
public string NumPlayers { get; set; }
[DynamoDBProperty("header")]
...
- 475
0
votes
0
answers
132
views
Polars read_ndjson DynamoDB file high memory usage issue
I'm trying to flatten a DynamoDB JSON file with Polars.
My JSON file is 1GB, 100mo after compressing to gz but in memory when I read it, it uses 16GB.
Because the data comes from DynamoDB, my columns ...
- 319
1
vote
1
answer
63
views
Mass batchGetItems from Dynamo
So I currently have tens of thousands of items I need to get from dynamo on a call. In my testing, with just 16K+ items, it takes around 750-1000ms for dynamo to return all the data. Not terrible, ...
- 2,796
0
votes
1
answer
75
views
AWS Java SDK v2.0: How to handle providing indexNames at runtime rather than using annotation
I have a project using DynamoDB which I'm upgrading to Java SDK v2.0
Using the new enhanced client and type mapping.
I have an index name which I need to pass to @DynamoDbSecondaryPartitionKey,
...
- 184
0
votes
0
answers
41
views
DynamoDB query with FilterExpression returns no results even when records exist
I'm querying a DynamoDB GSI (parentUploadId-index) to get the first incomplete record (where status <> "COMPLETED") for a given parentUploadId. Here's my code:
public async Task<(...
- 107
0
votes
2
answers
74
views
Testing DLQ in Opensearch ingestion pipeline
I am using the AWS OpenSearch ingestion pipeline for linking data from DDB to OpenSearch. I have set up the DLQ configuration.
But I am stuck on forcing a pipeline to send messages to DLQ.
Any easy ...
- 1,750