From the dynamodb tables, I'm trying to get max sized item (top 3 with their ID) from the table. Is there a way that we can get the max sized item using scan or query. The table doesn't have any item size stored in a separate column
1 Answer
You can Scan the table, and run each item through an item calculator and keep a track of the largest items. If you use JS you can install this npm package:
https://www.npmjs.com/package/ddb-calc
You can easily build something in a different language if you do not use JS.
Another solution is to track the consumed WCU each time you insert or update an item, the larger the WCU consumed indicates the size of the item.
The same approach can be used for GetItem operations.
4 Comments
Harry
Thanks Leeroy, the problem with scan is that its money for each request.. so can't think how much is it going to cost for say billion records. I'm trying to see if I can get on to something that saves money...
Leeroy Hannigan
Monitoring the item size as you read/write is free. Is this for w one time check or something you'll want to run frequently?
Harry
This is one timer but for multiple tables
Leeroy Hannigan
If the tables are too large to Scan, then you can export the data to S3 and run your analysis there instead. But if they're less than a couple of 100GB then Scan is the best option.