This is a sample CDK application to illustrate a cross account setup of OpenSearch Serverless with private access through a VPC endpoint. It allows you to ingest data from one AWS account to an OpenSearch Serverless collection in a different AWS account.
The setup is split in two accounts:
- ingestion account: a Lambda Function that ingests data from certain sources (DynamoDB, S3, ...) to an OpenSearch Serverless collection in a different account. The Lambda Function is placed in a VPC and assumes an IAM role in the search account to write data to the OpenSearch Serverless collection by leveraging a VPC endpoint.
- search account: an OpenSearch Serverless collection that allows network access from a VPC endpoint and data access for an IAM role.
💡 You can use the same setup to read from an OpenSearch Serverless collection, i.e. query data. Instead of having an ingestion account, just replace it with a read account and let your Lambda Function perform queries against OpenSearch (but consider the supported API operations).
The setup unfortunately requires step-wise deployments because the resources have cyclic dependencies. Look at bin/main.ts which contains comments explaining which fields you need to set. Here's a general overview:
- Deploy the
IngestionStackto create the Lambda Function and a VPC endpoint - Note the VPC endpoint id of the stack output because it's required for the
SearchStack. - Deploy the
SearchStackto create the OpenSearch Serverless collection -> it requires a VPC endpoint id. - Note the collection id and IAM role ARN of the stack outputs because they are required for the Lambda Function of the ingestion stack.
- Deploy the
IngestionStackagain with the updated parameters.
Now you can run the Lambda Function to test the connection and ingestion of data.
⚠️ This is a sample application. You might want to further restrict permissions of the VPC endpoints, Lambda Function or OpenSearch Serverless collection.
npm run buildcompile typescript to jsnpm run watchwatch for changes and compilenpm run testperform the jest unit testsnpx cdk deploydeploy this stack to your default AWS account/regionnpx cdk diffcompare deployed stack with current statenpx cdk synthemits the synthesized CloudFormation template
I'm a cloud consultant focusing on AWS and happy to help & learn. Just create an issue or send me a message.