Microservice for user management exemplifying part of the ML development architecture, implemented with Systems Manager Parameter Store, Api-Gateway, Serverless-Framework, Lambda, NodeJs, Sequelize, Mysql, Docker, Docker-compose, Amazon RDS, Unit Test with Jest, among others. AWS services are tested locally. The project code and its documentation (less technical doc) have been developed in English.
View
Latest update: 16/11/25
- 1.0) Project Description.
- 1.1) Project Execution.
- 1.2) Docker Setup and Database Migration
- 1.3) Technologies.
1.0) Project Description 🔝
See
- The Microservice is designed under the MVC architecture. This architecture consists of and is divided into the model layer (definition of the user table), the service layer (the connection and transactions to the db with sequelize) and the controller layer (the implemented lambdas).
- Each lambda performs the token authentication check, those that wait for a body type event check these fields and all the logic to be performed is abstracted from it to decouple functionalities together with low coupling.
- Endpoints that allow the return of more than one object according to the applied search logic are handled with pagination if required. Default pagination is applied.
- The image of the AWS architecture used describes the operating flow of the microservice in a general way. Any request to the microservice starts from a client (Postman, server, etc.).
Step 0: This request is received by the api-gateway and will only be validated if the correct x-api-key is found within the headers of said request.Steps 1A, 1B, etc: All these steps correspond to an endpoint with its specific resource. For ex. for getAllUsers (1A) it is http://localhost:4000/dev/users/list ....check those endpoints in endpoints section. Each lambda performs x-api-key and token checking.Steps 2: The lambdas perform the validations of the corresponding ssm with the System Manager Paramater Store... they validate token, connection values with the db, etc.Steps 3: The lambdas perform the necessary transactions and operations with the db (Mysql).Clarifications: This operation is emulated within the same network and in a local environment with the corresponding serverless plugins.
1.1) Project Execution 🔝
See
- Once a work environment has been created through some IDE, we clone the project
git clone https://github.com/andresWeitzel/Microservice_Mercadolibre_Users_AWS
- We position ourselves on the project
cd 'projectName'
- We install the LTS version of Nodejs(v18)
- We install the Serverless Framework globally if we haven't already. I recommend version 3 so we don't need credentials. You can use the latest version 4 without any problems (it's paid).
npm install -g serverless@3
- We verify the version of Serverless installed
sls -v
- We install all the necessary packages
npm i
-
Important: Make sure Docker are installed on your system (for Windows, use Docker Desktop) -
Start and build the MySQL database container:
docker-compose up -d- Verify the container is running:
docker ps- If you need to reset the database (optional) :
docker-compose down -v
docker-compose up -d- To view database logs (optional):
docker-compose logs mysql- To access MySQL directly (optional):
docker exec -it mercadolibre_users_mysql mysql -u mercadolibre_user -p-
The ssm and env variables used in the project are maintained to simplify the project configuration process. It is recommended to add the corresponding files (serverless_ssm.yml and .env) to the .gitignore.
-
The start script configured in the project's package.json is responsible for launching
-
The serverless-offline plugin
-
The remark-lint plugin for .md files (only --output is applied for check and autoformat without terminating the process and being able to execute the serverless script)
-
We run the app from terminal.
npm start
- If a message appears indicating that port 4000 is already in use, we can terminate all dependent processes and run the app again (optional) :
npx kill-port 4000
npm start
1.2) Docker Setup and Database Migration 🔝
See
-
Docker Compose Configuration The following configuration sets up a MySQL 8.0 container with persistent storage and automatic initialization:
version: '3.8' services: mysql: image: mysql:8.0 container_name: mercadolibre_users_mysql environment: MYSQL_ROOT_PASSWORD: root # Root password for MySQL MYSQL_DATABASE: microdb_mercadolibre # Database name MYSQL_USER: mercadolibre_user # Application user MYSQL_PASSWORD: mercadolibre_pass # Application user password ports: - "3306:3306" # Maps container port to host port volumes: - mysql_data:/var/lib/mysql # Persistent data storage - ./init:/docker-entrypoint-initdb.d # Initialization scripts command: --default-authentication-plugin=mysql_native_password # Authentication method healthcheck: test: ["CMD", "mysqladmin", "ping", "-h", "localhost"] # Health check command interval: 10s # Check every 10 seconds timeout: 5s # Timeout after 5 seconds retries: 5 # Retry 5 times before marking unhealthy volumes: mysql_data: # Named volume for data persistence
-
Essential Docker Commands These commands are essential for managing your Docker environment:
# Start container in detached mode (runs in background) docker-compose up -d # Check container status and health docker ps # Reset database (removes all data and recreates container) docker-compose down -v docker-compose up -d # View database logs for troubleshooting docker-compose logs mysql # Access MySQL command line interface docker exec -it mercadolibre_users_mysql mysql -u mercadolibre_user -p
-
Sample Data Here are some example queries to populate your database:
-- Sample user insertion with all required fields INSERT INTO users ( nickname, first_name, last_name, email, identification_type, identification_number, country_id ) VALUES ( 'USER123', 'John', 'Doe', 'john@example.com', 'DNI', '12345678', 'AR' ); -- Sample product insertion INSERT INTO products ( title, price, currency_id, available_quantity, condition ) VALUES ( 'iPhone 12', 999.99, 'USD', 10, 'new' );
-
Database Initialization The database setup process follows these steps:
- When the container starts, it automatically creates the database specified in MYSQL_DATABASE
- The initialization scripts in the
./initdirectory are executed in alphabetical order - Data persists between container restarts thanks to the Docker volume
mysql_data - The first script (01_*) typically contains table definitions
- The second script (02_*) typically contains initial data
-
File Structure The initialization process uses this file structure:
init/ ├── 01_microdb_mercadolibre_DDL.sql # Database schema and table definitions └── 02_microdb_mercadolibre_DML_INSERTS.sql # Initial data and seed records -
Considerations Important points to remember:
- The
mysql_datavolume ensures your data persists even if the container is removed - To completely reset the database, you need to remove the volume using
docker-compose down -v - Database credentials are defined in the
docker-compose.ymlfile - The container uses MySQL 8.0 with native password authentication
- The database is accessible on port 3306 of your host machine
- The
-
Container Management Advanced container management commands:
# Stop all containers gracefully docker-compose down # Remove all containers, networks, and volumes docker-compose down -v # Rebuild containers with latest changes docker-compose build # View container logs in real-time (follow mode) docker-compose logs -f mysql # Execute interactive shell in container docker exec -it mercadolibre_users_mysql bash
-
Database Backup and Restore Commands for database maintenance:
# Create a full database backup docker exec mercadolibre_users_mysql mysqldump -u mercadolibre_user -p microdb_mercadolibre > backup.sql # Restore database from backup docker exec -i mercadolibre_users_mysql mysql -u mercadolibre_user -p microdb_mercadolibre < backup.sql
-
Troubleshooting Common troubleshooting commands:
# Check container status and details docker ps -a # Inspect container configuration docker inspect mercadolibre_users_mysql # View container logs docker logs mercadolibre_users_mysql # Monitor container resource usage docker stats mercadolibre_users_mysql
-
Additional Sample Queries Useful SQL queries for common operations:
-- Create new user with all fields INSERT INTO users ( nickname, first_name, last_name, email, identification_type, identification_number, country_id ) VALUES ( 'MARIA123', 'Maria', 'Garcia', 'maria.garcia@example.com', 'PASSPORT', 'AB123456', 'ES' ); -- Update user information UPDATE users SET email = 'new.email@example.com', update_date = CURRENT_TIMESTAMP WHERE id = 1; -- Delete user DELETE FROM users WHERE id = 1; -- Search users by country with pagination SELECT * FROM users WHERE country_id = 'AR' ORDER BY creation_date DESC LIMIT 10 OFFSET 0; -- Count users by country SELECT country_id, COUNT(*) as user_count FROM users GROUP BY country_id;
-
Common Issues and Solutions Solutions for frequent problems:
-
Port Conflict: If port 3306 is already in use
# Find process using port netstat -ano | findstr :3306 # Kill process taskkill /PID <process_id> /F
-
Container Won't Start: Check logs for errors
# View detailed logs docker-compose logs mysql # Check container status docker ps -a
-
Database Connection Issues: Verify credentials and network
# Test connection docker exec -it mercadolibre_users_mysql mysql -u mercadolibre_user -p # Check network docker network ls docker network inspect <network_name>
-
-
Performance Optimization Tips for optimizing database performance:
-
Adjust MySQL configuration in
my.cnf:[mysqld] innodb_buffer_pool_size = 256M # Buffer pool size for InnoDB max_connections = 100 # Maximum concurrent connections query_cache_size = 32M # Query cache size
-
Monitor performance:
-- Check slow queries SHOW VARIABLES LIKE '%slow%'; -- Check connection status SHOW STATUS LIKE '%onn%'; -- Check table status SHOW TABLE STATUS; -- Check process list SHOW PROCESSLIST;
-
1.3) Technologies 🔝
See
| Technologies | Version | Purpose |
|---|---|---|
| SDK | 4.3.2 | Automatic Module Injection for Lambdas |
| Serverless Framework Core v3 | 3.23.0 | AWS Services Core |
| Systems Manager Parameter Store (SSM) | 3.0 | Environment Variables Management |
| Jest | 29.7 | Framework for unit testing, integration, etc. |
| Amazon Api Gateway | 2.0 | API Manager, Authentication, Control and Processing |
| NodeJS | 14.18.1 | JS Library |
| Sequelize | ^6.11.0 | ORM |
| Mysql | 10.1 | DBMS |
| XAMPP | 3.2.2 | Server Package |
| VSC | 1.72.2 | IDE |
| Postman | 10.11 | Http Client |
| CMD | 10 | Command Line Interface |
| Git | 2.29.1 | Version Control |
| Others | Others | Others |
| Plugin |
|---|
| Serverless Plugin |
| serverless-offline |
| serverless-offline-ssm |
| Extension |
|---|
| Prettier - Code formatter |
| YAML - Autoformatter .yml |
| Error Lens - Error identifier |
| Tabnine - AI Code |
| Others - Others |
2.0) Endpoints and resources 🔝
See
-
http://localhost:4000/dev/v1/users/identification-number/{ident-number}
-
http://localhost:4000/dev/v1/users/identification-type/{ident-type}
-
http://localhost:4000/dev/v1/users/creation-date/{creation-date}
-
http://localhost:4000/dev/v1/users/update-date/{update-date}
-
All endpoints are optional paginated except /test, /db-connection and /id/{user-id}}
- {required-value}
- Default pagination: ?page=0&limit=5
- Optional pagination: ?page={nro}&limit={nro}
2.1) Examples 🔝
View
| Variable | Initial Value | Current Value |
|---|---|---|
| base_url | http://localhost:4000/dev/ | http://localhost:4000/dev/ |
| x-api-key | f98d8cd98h73s204e3456998ecl9427j | f98d8cd98h73s204e3456998ecl9427j |
| bearer_token | Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiIxMjM0NTY3ODkwIiwibmFtZSI6IkpvaG4gRG9lIiwiaWF0IjoxNTE2MjM5MDIyfQ.SflKxwRJSMeKKF2QT4fwpMeJf36POk6yJV_adQssw5c | Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiIxMjM0NTY3ODkwIiwibmFtZSI6IkpvaG4gRG9lIiwiaWF0IjoxNTE2MjM5MDIyfQ.SflKxwRJSMeKKF2QT4fwpMeJf36POk6yJV_adQssw5c |
curl --location 'http://localhost:4000/dev/v1/users/list?page=0&limit=2&orderBy=id&orderAt=asc' \
--header 'Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiIxMjM0NTY3ODkwIiwibmFtZSI6IkpvaG4gRG9lIiwiaWF0IjoxNTE2MjM5MDIyfQ.SflKxwRJSMeKKF2QT4fwpMeJf36POk6yJV_adQssw5c' \
--header 'Content-Type: application/json' \
--header 'x-api-key: f98d8cd98h73s204e3456998ecl9427j' \
--data ''{
"message": [
{
"id": 3,
"nickname": "HECTOR SS G",
"first_name": "Hector",
"last_name": "Gomez",
"email": "hectorGomez78@gmail.com",
"identification_type": "DNI",
"identification_number": "2172265827",
"country_id": "AR",
"creation_date": "2023-03-20 21:02:33",
"update_date": "2023-03-20 21:02:33"
},
{
"id": 4,
"nickname": "GABRIELA JIMENEZ",
"first_name": "Gabriela",
"last_name": "Jimenez",
"email": "gabriela.consultas@hotmail.com",
"identification_type": "DNI",
"identification_number": "410871223",
"country_id": "AR",
"creation_date": "2023-03-20 21:02:33",
"update_date": "2023-03-20 21:02:33"
}
]
}{
"message": "Bad request, check missing or malformed headers"
}{
"message": "Bad request, could not get the paginated list of users."
}{
"message": "Not authenticated, check x_api_key and Authorization"
}{
"message": "ECONNREFUSED. An error has occurred with the connection or query to the database. Verify that it is active or available"
}curl --location 'http://localhost:4000/dev/v1/users/list-without-dates?page=0&limit=2&orderBy=id&orderAt=asc' \
--header 'Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiIxMjM0NTY3ODkwIiwibmFtZSI6IkpvaG4gRG9lIiwiaWF0IjoxNTE2MjM5MDIyfQ.SflKxwRJSMeKKF2QT4fwpMeJf36POk6yJV_adQssw5c' \
--header 'Content-Type: application/json' \
--header 'x-api-key: f98d8cd98h73s204e3456998ecl9427j' \
--data ''{
"message": [
{
"id": 3,
"nickname": "HECTOR SS G",
"first_name": "Hector",
"last_name": "Gomez",
"email": "hectorGomez78@gmail.com",
"identification_type": "DNI",
"identification_number": "2172265827",
"country_id": "AR"
},
{
"id": 4,
"nickname": "GABRIELA JIMENEZ",
"first_name": "Gabriela",
"last_name": "Jimenez",
"email": "gabriela.consultas@hotmail.com",
"identification_type": "DNI",
"identification_number": "410871223",
"country_id": "AR"
}
]
}{
"message": "Bad request, check missing or malformed headers"
}{
"message": "Bad request, could not get the paginated list of users."
}{
"message": "Not authenticated, check x_api_key and Authorization"
}{
"message": "ECONNREFUSED. An error has occurred with the connection or query to the database. Verify that it is active or available"
}curl --location 'http://localhost:4000/dev/v1/users/id/4' \
--header 'Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiIxMjM0NTY3ODkwIiwibmFtZSI6IkpvaG4gRG9lIiwiaWF0IjoxNTE2MjM5MDIyfQ.SflKxwRJSMeKKF2QT4fwpMeJf36POk6yJV_adQssw5c' \
--header 'Content-Type: application/json' \
--header 'x-api-key: f98d8cd98h73s204e3456998ecl9427j'{
"message": {
"id": 4,
"nickname": "GABRIELA JIMENEZ",
"first_name": "Gabriela",
"last_name": "Jimenez",
"email": "gabriela.consultas@hotmail.com",
"identification_type": "DNI",
"identification_number": "410871223",
"country_id": "AR",
"creation_date": "2023-03-20 21:02:33",
"update_date": "2023-03-20 21:02:33"
}
}{
"message": "Bad request, check missing or malformed headers"
}{
"message": "Bad request, could not fetch user based on id."
}{
"message": "Bad request, the id passed as a parameter is not valid."
}{
"message": "Not authenticated, check x_api_key and Authorization"
}{
"message": "ECONNREFUSED. An error has occurred with the connection or query to the database. Verify that it is active or available"
}curl --location 'http://localhost:4000/dev/v1/users/country-id/AR?page=0&limit=3' \
--header 'Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiIxMjM0NTY3ODkwIiwibmFtZSI6IkpvaG4gRG9lIiwiaWF0IjoxNTE2MjM5MDIyfQ.SflKxwRJSMeKKF2QT4fwpMeJf36POk6yJV_adQssw5c' \
--header 'Content-Type: application/json' \
--header 'x-api-key: f98d8cd98h73s204e3456998ecl9427j' \
--data ''{
"message": [
{
"id": 3,
"nickname": "HECTOR SS G",
"first_name": "Hector",
"last_name": "Gomez",
"email": "hectorGomez78@gmail.com",
"identification_type": "DNI",
"identification_number": "2172265827",
"country_id": "AR",
"creation_date": "2023-03-20 21:02:33",
"update_date": "2023-03-20 21:02:33"
},
{
"id": 4,
"nickname": "GABRIELA JIMENEZ",
"first_name": "Gabriela",
"last_name": "Jimenez",
"email": "gabriela.consultas@hotmail.com",
"identification_type": "DNI",
"identification_number": "410871223",
"country_id": "AR",
"creation_date": "2023-03-20 21:02:33",
"update_date": "2023-03-20 21:02:33"
},
{
"id": 5,
"nickname": "GUSTA G K",
"first_name": "Gustavo",
"last_name": "Gomez",
"email": "gustavo_andaluz@gmail.com",
"identification_type": "PASAPORTE",
"identification_number": "748000221",
"country_id": "AR",
"creation_date": "2023-03-20 21:02:33",
"update_date": "2023-03-20 21:02:33"
}
]
}{
"message": "Bad request, check missing or malformed headers"
}{
"message": "Bad request, could not get paginated list of users according to country id. Try again."
}{
"message": "Bad request, the country id passed as a parameter is not valid."
}{
"message": "Not authenticated, check x_api_key and Authorization"
}{
"message": "ECONNREFUSED. An error has occurred with the connection or query to the database. Verify that it is active or available"
}curl --location 'http://localhost:4000/dev/v1/users/add-user/' \
--header 'Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiIxMjM0NTY3ODkwIiwibmFtZSI6IkpvaG4gRG9lIiwiaWF0IjoxNTE2MjM5MDIyfQ.SflKxwRJSMeKKF2QT4fwpMeJf36POk6yJV_adQssw5c' \
--header 'Content-Type: application/json' \
--header 'x-api-key: f98d8cd98h73s204e3456998ecl9427j' \
--data-raw '{
"nickname": "MARTIN-SUAREZ",
"first_name": "Martin",
"last_name": "Suarez",
"email": "martin_electro_todo@gmail.com",
"identification_type": "DNI",
"identification_number": "4459388222",
"country_id": "AR12"
}'{
"message": {
"id": null,
"nickname": "MARTIN-SUAREZ",
"first_name": "Martin",
"last_name": "Suarez",
"email": "martin_electro_todo@gmail.com",
"identification_type": "DNI",
"identification_number": "4459388222",
"country_id": "AR12",
"creation_date": "2023-06-28T16:46:31.000Z",
"update_date": "2023-06-28T16:46:31.000Z"
}
}{
"message": "Bad request, check missing or malformed headers"
}{
"message": "Bad request, check request attributes. Missing or incorrect. CHECK: nickname, first_name and last_name (required|string|minLength:4|maxLength:50), email (required|string|minLength:10|maxLength:100), identification_type and identification_number (required|string|minLength:6|maxLength:20), country_id (required|string|minLength:2|maxLength:5)"
}{
"message": "Bad request, could not add user.CHECK: The first_name next together the last_name should be uniques. The identification_type next together the identification_number should be uniques."
}{
"message": "Not authenticated, check x_api_key and Authorization"
}{
"message": "ECONNREFUSED. An error has occurred with the connection or query to the database. CHECK: The first_name next together the last_name should be uniques. The identification_type next together the identification_number should be uniques."
}curl --location --request PUT 'http://localhost:4000/dev/v1/users/update-user/32' \
--header 'Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiIxMjM0NTY3ODkwIiwibmFtZSI6IkpvaG4gRG9lIiwiaWF0IjoxNTE2MjM5MDIyfQ.SflKxwRJSMeKKF2QT4fwpMeJf36POk6yJV_adQssw5c' \
--header 'Content-Type: application/json' \
--header 'x-api-key: f98d8cd98h73s204e3456998ecl9427j' \
--data-raw '{
"nickname": "MARTIN-SUAREZ2221",
"first_name": "Martin2221",
"last_name": "Suarez2221",
"email": "martin_electro_todo@gmail.com",
"identification_type": "DNI",
"identification_number": "445938812313222",
"country_id": "AR12",
"creation_date": "2023-10-11 21:18:29",
"update_date": "2023-10-11 21:18:29"
}'{
"message": {
"id": 32,
"nickname": "MARTIN-SUAREZ2221",
"first_name": "Martin2221",
"last_name": "Suarez2221",
"email": "martin_electro_todo@gmail.com",
"identification_type": "DNI",
"identification_number": "445938812313222",
"country_id": "AR12",
"creation_date": "2023-10-11 21:18:29",
"update_date": "2023-10-11 21:18:29"
}
}{
"message": "Bad request, check missing or malformed headers"
}{
"message": "Bad request, check request attributes and object to update"
}{
"message": "Bad request, could not add user.CHECK: The first_name next together the last_name should be uniques. The identification_type next together the identification_number should be uniques."
}{
"message": "Not authenticated, check x_api_key and Authorization"
}{
"message": "ECONNREFUSED. An error has occurred with the connection or query to the database. CHECK: The first_name next together the last_name should be uniques. The identification_type next together the identification_number should be uniques."
}curl --location --request DELETE 'http://localhost:4000/dev/v1/users/delete-user/18' \
--header 'Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiIxMjM0NTY3ODkwIiwibmFtZSI6IkpvaG4gRG9lIiwiaWF0IjoxNTE2MjM5MDIyfQ.SflKxwRJSMeKKF2QT4fwpMeJf36POk6yJV_adQssw5c' \
--header 'Content-Type: application/json' \
--header 'x-api-key: f98d8cd98h73s204e3456998ecl9427j' \
--data ''{
"message": "User has been deleted successfully."
}{
"message": "Bad request, check missing or malformed headers"
}{
"message": "Bad request, a non-existent user cannot be deleted. Operation not allowed"
}{
"message": "Not authenticated, check x_api_key and Authorization"
}{
"message": "ECONNREFUSED. An error has occurred with the connection or query to the database. CHECK: The first_name next together the last_name should be uniques. The identification_type next together the identification_number should be uniques."
}3.0) Functionality Test 🔝
3.1) References 🔝
See
- AWS Lambda Documentation
- API Gateway Best Practices
- Systems Manager Parameter Store
- Amazon RDS Documentation
- AWS CloudWatch Logs
- AWS IAM Best Practices
- Serverless Framework Documentation
- Serverless Framework Plugins
- Serverless Offline Plugin
- Serverless SSM Plugin
- Serverless OpenAPI Documentation
- Serverless Auto Swagger
- Sequelize Documentation
- MySQL Documentation
- Docker MySQL Image
- Docker Compose Documentation
- Sequelize Migrations
- Sequelize Associations
- Jest Documentation
- Node.js Documentation
- Postman Documentation
- VS Code Documentation
- Git Documentation
- Docker Desktop Documentation
- REST API Best Practices
- API Security Best Practices
- OpenAPI Specification
- API Documentation Best Practices
- HTTP Status Codes
- Mercadolibre API Documentation
- Mercadolibre Users API
- Mercadolibre Products API
- Mercadolibre Authentication
- AWS Design Tool (draw.io)
- Postman Collection Examples
- VS Code Extensions for AWS
- Docker Hub
- GitHub Actions
- Node.js Best Practices
- AWS Community Builders
- Serverless Framework Forum
- Stack Overflow
- AWS YouTube Channel
- Serverless Framework YouTube
- Mercadolibre Developers Blog
