Boost the performance of the program where the file has more than 1 Million data and is stored in a database by batch processing and exporting the data into separate files (100K each)
Step 1: Create a database (mysql) and replace the credentials (databaseName, username, password) application.properties files
Now will see the application is running on port http://localhost:8080 and database tables will be created
URL Endpoint : http://localhost:8080/file/upload
Request Type : POST
Request Param: file
Download sample file from here https://drive.google.com/file/d/1ODHlgplC5lhZorncmswz8HFsKBwLuTmf/view
URL Endpoint : http://localhost:8080/file/export
Request Type : GET
Result: Two valid customers text file will be created root directory with prefix "valid_customers_"
Upload Api:
- Used synchronized multithreading for process every line from content
- Jpa batch process for bulk insert
Export Api:
- Used multithreading for exported data to write into file
- Total Execution Time: 14.62 second (1.2 Million data)