Skip to content

Include CFn tests in test selection#12518

Draft
simonrw wants to merge 14 commits intomainfrom
test-selection/include-cfn-tests-in-service-tests
Draft

Include CFn tests in test selection#12518
simonrw wants to merge 14 commits intomainfrom
test-selection/include-cfn-tests-in-service-tests

Conversation

@simonrw
Copy link
Contributor

@simonrw simonrw commented Apr 11, 2025

  • Ensure CloudFormation tests are executed on service changes

Motivation

TBD

Changes

  • Move test files
  • Validate api gateway tests
  • Revalidate tests

@simonrw simonrw added the semver: patch Non-breaking changes which can be included in patch releases label Apr 11, 2025
@simonrw simonrw self-assigned this Apr 11, 2025
@simonrw simonrw force-pushed the test-selection/include-cfn-tests-in-service-tests branch from 2164675 to a7b9f5a Compare April 11, 2025 15:59
@github-actions
Copy link

github-actions bot commented Apr 11, 2025 •
edited
Loading

LocalStack Community integration with Pro

    2 files  ±0      2 suites  ±0   1h 53m 40s ⏱️ - 2m 43s
4 354 tests  - 3  4 008 ✅  - 3  346 💤 ±0  0 ❌ ±0 
4 360 runs  +1  4 012 ✅ +1  348 💤 ±0  0 ❌ ±0 

Results for commit 0a67334. ± Comparison against base commit 5ad7f8e.

This pull request removes 136 and adds 133 tests. Note that renamed tests count towards both. 
tests.aws.services.cloudformation.resources.test_acm ‑ test_cfn_acm_certificate
tests.aws.services.cloudformation.resources.test_apigateway ‑ test_account
tests.aws.services.cloudformation.resources.test_apigateway ‑ test_api_gateway_with_policy_as_dict
tests.aws.services.cloudformation.resources.test_apigateway ‑ test_cfn_apigateway_aws_integration
tests.aws.services.cloudformation.resources.test_apigateway ‑ test_cfn_apigateway_rest_api
tests.aws.services.cloudformation.resources.test_apigateway ‑ test_cfn_apigateway_swagger_import
tests.aws.services.cloudformation.resources.test_apigateway ‑ test_cfn_deploy_apigateway_from_s3_swagger
tests.aws.services.cloudformation.resources.test_apigateway ‑ test_cfn_deploy_apigateway_integration
tests.aws.services.cloudformation.resources.test_apigateway ‑ test_cfn_deploy_apigateway_models
tests.aws.services.cloudformation.resources.test_apigateway ‑ test_cfn_with_apigateway_resources
…

tests.aws.services.acm.resources.test_acm ‑ test_cfn_acm_certificate
tests.aws.services.apigateway.resources.test_apigateway ‑ test_account
tests.aws.services.apigateway.resources.test_apigateway ‑ test_api_gateway_with_policy_as_dict
tests.aws.services.apigateway.resources.test_apigateway ‑ test_cfn_apigateway_aws_integration
tests.aws.services.apigateway.resources.test_apigateway ‑ test_cfn_apigateway_rest_api
tests.aws.services.apigateway.resources.test_apigateway ‑ test_cfn_apigateway_swagger_import
tests.aws.services.apigateway.resources.test_apigateway ‑ test_cfn_deploy_apigateway_from_s3_swagger
tests.aws.services.apigateway.resources.test_apigateway ‑ test_cfn_deploy_apigateway_integration
tests.aws.services.apigateway.resources.test_apigateway ‑ test_cfn_deploy_apigateway_models
tests.aws.services.apigateway.resources.test_apigateway ‑ test_cfn_with_apigateway_resources
…
This pull request removes 5 skipped tests and adds 5 skipped tests. Note that renamed tests count towards both. 
tests.aws.services.cloudformation.resources.test_ec2 ‑ test_cfn_update_ec2_instance_type
tests.aws.services.cloudformation.resources.test_lambda ‑ test_lambda_vpc
tests.aws.services.cloudformation.resources.test_opensearch ‑ test_domain
tests.aws.services.cloudformation.resources.test_redshift ‑ test_redshift_cluster
tests.aws.services.cloudformation.resources.test_stepfunctions ‑ test_retry_and_catch

tests.aws.services.ec2.resources.test_ec2 ‑ test_cfn_update_ec2_instance_type
tests.aws.services.lambda_.resources.test_lambda ‑ test_lambda_vpc
tests.aws.services.opensearch.resources.test_opensearch ‑ test_domain
tests.aws.services.redshift.resources.test_redshift ‑ test_redshift_cluster
tests.aws.services.stepfunctions.resources.test_stepfunctions ‑ test_retry_and_catch

♻️ This comment has been updated with latest results.

@simonrw simonrw force-pushed the test-selection/include-cfn-tests-in-service-tests branch from 3ee0658 to 10956fa Compare April 11, 2025 21:08
@github-actions
Copy link

github-actions bot commented Apr 11, 2025 •
edited
Loading

S3 Image Test Results (AMD64 / ARM64)

    2 files  ±0    2 suites  ±0   8m 0s ⏱️ -4s
  517 tests ±0  467 ✅ ±0   50 💤 ±0  0 ❌ ±0 
1 034 runs  ±0  934 ✅ ±0  100 💤 ±0  0 ❌ ±0 

Results for commit de821d7. ± Comparison against base commit 5bb9747.

♻️ This comment has been updated with latest results.

@simonrw simonrw force-pushed the test-selection/include-cfn-tests-in-service-tests branch 3 times, most recently from 0a67334 to 5432c3f Compare April 16, 2025 15:30
@simonrw simonrw added this to the Playground milestone Apr 25, 2025
@simonrw simonrw force-pushed the test-selection/include-cfn-tests-in-service-tests branch from 5432c3f to 01a3f3f Compare July 30, 2025 21:19
@github-actions
Copy link

github-actions bot commented Jul 30, 2025 •
edited
Loading

Test Results - Preflight, Unit

22 159 tests  +1   20 422 ✅ +1   6m 23s ⏱️ -2s
     1 suites ±0    1 737 💤 ±0 
     1 files   ±0        0 ❌ ±0 

Results for commit de821d7. ± Comparison against base commit 5bb9747.

♻️ This comment has been updated with latest results.

@github-actions
Copy link

github-actions bot commented Jul 30, 2025 •
edited
Loading

Test Results (amd64) - Acceptance

7 tests  ±0   5 ✅ ±0   3m 12s ⏱️ +3s
1 suites ±0   2 💤 ±0 
1 files   ±0   0 ❌ ±0 

Results for commit de821d7. ± Comparison against base commit 5bb9747.

♻️ This comment has been updated with latest results.

@github-actions
Copy link

github-actions bot commented Jul 30, 2025 •
edited
Loading

Test Results (amd64) - Integration, Bootstrap

    5 files      5 suites   2h 23m 20s ⏱️
5 003 tests 4 402 ✅ 601 💤 0 ❌
5 009 runs  4 402 ✅ 607 💤 0 ❌

Results for commit de821d7.

♻️ This comment has been updated with latest results.

@simonrw simonrw force-pushed the test-selection/include-cfn-tests-in-service-tests branch from 4aeada4 to 569f66d Compare August 28, 2025 16:12
@simonrw simonrw added the docs: skip Pull request does not require documentation changes label Aug 28, 2025
@github-actions
Copy link

github-actions bot commented Aug 28, 2025 •
edited
Loading

Test Results - Alternative Providers

922 tests   674 ✅  31m 56s ⏱️
  4 suites  248 💤
  4 files      0 ❌

Results for commit de821d7.

♻️ This comment has been updated with latest results.

@simonrw simonrw force-pushed the test-selection/include-cfn-tests-in-service-tests branch from fddfda9 to de821d7 Compare August 29, 2025 20:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@dominikschubert dominikschubert Awaiting requested review from dominikschubert dominikschubert will be requested when the pull request is marked ready for review dominikschubert is a code owner

@joe4dev joe4dev Awaiting requested review from joe4dev joe4dev will be requested when the pull request is marked ready for review joe4dev is a code owner

@gregfurman gregfurman Awaiting requested review from gregfurman gregfurman will be requested when the pull request is marked ready for review gregfurman is a code owner

@tiurin tiurin Awaiting requested review from tiurin tiurin will be requested when the pull request is marked ready for review tiurin is a code owner

@viren-nadkarni viren-nadkarni Awaiting requested review from viren-nadkarni viren-nadkarni will be requested when the pull request is marked ready for review viren-nadkarni is a code owner

@baermat baermat Awaiting requested review from baermat baermat will be requested when the pull request is marked ready for review baermat is a code owner

@thrau thrau Awaiting requested review from thrau thrau will be requested when the pull request is marked ready for review thrau is a code owner

@bentsku bentsku Awaiting requested review from bentsku bentsku will be requested when the pull request is marked ready for review bentsku is a code owner

@macnev2013 macnev2013 Awaiting requested review from macnev2013 macnev2013 will be requested when the pull request is marked ready for review macnev2013 is a code owner

@k-a-il k-a-il Awaiting requested review from k-a-il k-a-il will be requested when the pull request is marked ready for review k-a-il is a code owner

@aidehn aidehn Awaiting requested review from aidehn aidehn will be requested when the pull request is marked ready for review aidehn is a code owner

@giograno giograno Awaiting requested review from giograno giograno will be requested when the pull request is marked ready for review giograno is a code owner

@alexrashed alexrashed Awaiting requested review from alexrashed alexrashed will be requested when the pull request is marked ready for review alexrashed is a code owner

@silv-io silv-io Awaiting requested review from silv-io silv-io will be requested when the pull request is marked ready for review silv-io is a code owner

@pinzon pinzon Awaiting requested review from pinzon pinzon will be requested when the pull request is marked ready for review pinzon is a code owner

@steffyP steffyP Awaiting requested review from steffyP steffyP will be requested when the pull request is marked ready for review steffyP is a code owner

@dfangl dfangl Awaiting requested review from dfangl dfangl will be requested when the pull request is marked ready for review dfangl is a code owner

@maxhoheiser maxhoheiser Awaiting requested review from maxhoheiser maxhoheiser will be requested when the pull request is marked ready for review maxhoheiser is a code owner

@cloutierMat cloutierMat Awaiting requested review from cloutierMat cloutierMat will be requested when the pull request is marked ready for review cloutierMat is a code owner

At least 1 approving review is required to merge this pull request.

Assignees

@simonrw simonrw

Labels

docs: skip Pull request does not require documentation changes semver: patch Non-breaking changes which can be included in patch releases

Projects

None yet

Milestone

Playground

Development

Successfully merging this pull request may close these issues.

1 participant

@simonrw