Download as PDF, PPTX
Uploaded byDevOps.com
Optimize your CI/CD with GitLab and AWS
The document discusses best practices for accelerating the transformation to mature software manufacturing using DevOps principles. It recommends adopting modern approaches like microservices, serverless architectures, infrastructure as code, and event-driven architectures to increase speed and agility. Automating testing and deployments through continuous integration/delivery (CI/CD) pipelines is key. The document advocates treating infrastructure like code and using tools like AWS CodePipeline and GitLab to enable self-service platforms and faster delivery through automation.
More Related Content
![[JAZUG Tohoku Azure DevOps] Azure DevOps](https://cdn.slidesharecdn.com/ss_thumbnails/20181130jazugsendaiazuredevops-181201031328-thumbnail.jpg?width=640&height=640&fit=bounds)
Similar to Optimize your CI/CD with GitLab and AWS
![[CPT DevOps Meetup] Developing Modern Applications in the Cloud](https://cdn.slidesharecdn.com/ss_thumbnails/devopsdeveloping-modern-applications-in-the-cloud-no-notes-190718062311-thumbnail.jpg?width=640&height=640&fit=bounds)
![Agility and Control from AWS [FutureStack16]](https://cdn.slidesharecdn.com/ss_thumbnails/agilityandcontroldougleeaws-161201210211-thumbnail.jpg?width=640&height=640&fit=bounds)
![Coded Agents – with UiPath SDK + LangGraph [Virtual Hands-on Workshop]](https://cdn.slidesharecdn.com/ss_thumbnails/codedagentsdeck-251215155422-5497c599-thumbnail.jpg?width=640&height=640&fit=bounds)
Optimize your CI/CD with GitLab and AWS
- 1. 1 Born for DevOpsCI CD: Accelerating the Transformation to Mature Software Manufacturing Darwin Sanoy, Senior Solutions Architect, GitLab James Bland, Ph.D., Sr. Solutions Architect, AWS
- 2. © 2020, AmazonWeb Services, Inc. or its Affiliates. All rights reserved. What is DevOps? DevOps is the combination of cultural philosophies, processes, people, and technology that increases an organization’s ability to deliver applications and services at high velocity: evolving and improving products at a faster pace than organizations using traditional software development and infrastructure management processes. This speed enables organizations to better serve their customers and compete more effectively in the market.
- 3. © 2020, AmazonWeb Services, Inc. or its Affiliates. All rights reserved. What is DevOps today DevOps = Modernization
- 4. © 2020, AmazonWeb Services, Inc. or its Affiliates. All rights reserved. • Shrink the scope • Offload the undifferentiated pieces • Choose the right tool for the job • Automate everything A modern approach allows you to:
- 5. © 2020, AmazonWeb Services, Inc. or its Affiliates. All rights reserved. Why are customers choosing to build modern apps? Faster to Market More reliable applications Increased rate of innovation Reduced costs
- 6. © 2020, AmazonWeb Services, Inc. or its Affiliates. All rights reserved. Best practices from AWS
- 7. © 2020, AmazonWeb Services, Inc. or its Affiliates. All rights reserved. Two-pizza teams are fast & agile • Full ownership & autonomy • You build it, you run it • DevOps – small, nimble teams • Focused innovation
- 8. © 2020, AmazonWeb Services, Inc. or its Affiliates. All rights reserved. Structure apps as collections of microservices When the impact of change is small, release velocity can increase Monolithic application Does everything Shared release pipeline Rigid scaling High impact of change Hard to adopt new technologies Microservices Does one thing Independent deployments Independent scaling Small impact of change Choice of technology
- 9. © 2020, AmazonWeb Services, Inc. or its Affiliates. All rights reserved. Structure apps as collections of microservices APIs and decoupled communications enable automation and improves reliability FUNCTION API API MICROSERVICE FUNCTION EVENT API MICROSERVICE EVENT API MICROSERVICE APPLICATION
- 10. © 2020, AmazonWeb Services, Inc. or its Affiliates. All rights reserved. Use code to model applications and infrastructure Treating everything as software increases the speed and agility of infrastructure deployments Design Write application code Create infrastructure templates Create stacks Iterate
- 11. © 2020, AmazonWeb Services, Inc. or its Affiliates. All rights reserved. LEVELOFABSTRACTION FOCUS ON BUSINESS LOGIC SERVERLESS P H Y S IC A L M A C H IN E S V IR T U A L M A C H IN E S C O N T A IN E R IZ A T IO N AWS Lambda AWS Fargate • Continuous scaling • Fault tolerance built-in • Pay for value • Zero maintenance • Focus on business value Serverless architectures provide the greatest agility with the least effort
- 12. © 2020, AmazonWeb Services, Inc. or its Affiliates. All rights reserved. Directed Commands Observable Events Event-driven architectures Events are observable, not directed
- 13. © 2020, AmazonWeb Services, Inc. or its Affiliates. All rights reserved. Event-driven Benefits: Break up the work Decoupled and distributed Decrease waste Pay for value Event-driven architectures
- 14. © 2020, AmazonWeb Services, Inc. or its Affiliates. All rights reserved. Creating powerful self-service platforms that enable builders to use the right tool for the right job Self-service platforms without gatekeepers
- 15. © 2020, AmazonWeb Services, Inc. or its Affiliates. All rights reserved. developers delivery pipelines services monitorreleasetestbuild Deployment: Monolith development lifecycle
- 16. © 2020, AmazonWeb Services, Inc. or its Affiliates. All rights reserved. developers services monitorreleasetestbuild delivery pipelines monitorreleasetestbuild monitorreleasetestbuild monitorreleasetestbuild monitorreleasetestbuild monitorreleasetestbuild Deployment: Microservice development lifecycle
- 17. © 2020, AmazonWeb Services, Inc. or its Affiliates. All rights reserved. Security in the CI/CD pipeline Artifact validation Static code analysis Security of the CI/CD pipeline Access roles Hardening build servers/nodes 1 2 Automated and Secure Pipelines
- 18. © 2020, AmazonWeb Services, Inc. or its Affiliates. All rights reserved. A shift to microservices • Single-purpose • Connect only through hardened APIs • Largely “black boxes” to each other • Business logic and data only accessible through APIs
- 19. © 2020, AmazonWeb Services, Inc. or its Affiliates. All rights reserved. Deployment at scale 1,000s of teams Micro- services CI/CD Serverless = 190Million Deployments / Year
- 20. © 2020, AmazonWeb Services, Inc. or its Affiliates. Want to learn more
- 21. © 2020, AmazonWeb Services, Inc. or its Affiliates. Architecture, software delivery, and operations By Amazon’s senior technical executives and engineers Real-world practices with detailed explanations Content available for free on the website The Amazon Builders’ Library How Amazon builds and operates software
- 22. © 2020, AmazonWeb Services, Inc. or its Affiliates. AWS Modernization Workshops Curated workshops created by AWS Partners https://awsworkshop.io
- 23. 23 How To Havea Tortured Software Manufacturing Transformation Avoiding The DevSecOps Russian Nesting Doll Born for DevOps CI CD Accelerating the Transformation to a Mature Culture of Software Manufacturing Parallel Worlds: The Evolution of Automated Manufacturing Inherent DevOps Transformation Throughput Limits The Russian Nesting Doll Effect (Matryoshka) GitLab and AWS – Better Together!
- 24. 24 Everywhere Else DoEngineered Automation (as Much as Possible) Focus Craftsmanship On Innovations That Delight Customers Henry Ford: Manufacturing Transformation
- 25. 25 Benefits of PullingA “Henry Ford” On Software Manufacturing ● Speed of innovations to market ● Improving predictability and quality ● Lowering cost of production and maintenance (after surge investment) ● Encoding of tacit human knowledge into Intellectual Property ● Removal of drudgery Let’s look at some key elements affecting the transition from the practice of software craftmanship to the practice of software manufacturing.
- 26. 26 Car Manufacturing asCraft Challenges: ● Skilling Workers (Skill Breadth=Time) ● Scaling Production ● Consistent Quality ● Serviceability of Product ● Know How as a Corporate Asset Integrated: Know How + Manufacturing Labor
- 27. 27 Automation Maturity Wave1 Making product from engineered plans using an assembly line Product Engineer
- 28. 28 Key Craft Transformationsand Limits - Tacit Knowledge - Individual Knowledge - Implicit processes - Product Designed for Craft Assembly - Integrated Manufacturing Labor (Drudgery) - Explicit Knowledge - Shared Knowledge - Explicit processes - Product Re-Designed for Automated Assembly - Externalized & Automated Manufacturing Labor Maximum Bandwidth of Deep Know How Extraction Maximum Bandwidth of Reskilling Product Refactoring for Assembly Line Production Crafting Engineering & Automation Inherent Limits (While Continuing To Deliver Customer Value with Current Means)
- 29. 31 The Russian NestingDoll Effect Maturity Level 1 Product Engineered From Standard Parts Maturity Level 4: The Above for Infrastructure as Code (IaC) Product Craft Iterate! Maturity Level 2 Parts Build Automation Machine Op Craft Maturity Level 3 Assembly Lines Engineered from Standard Components or Functional Templates Factory Craft (DevOps ToolChain)
- 30. 32 DevSecOps / AppSec- Skip the Russian Dolls ● Many companies efforts are in a Craftsmanship stage ○ Limited automation ○ Throughput limited by size of Security team ○ Manual reports, reviews and processes. Do a 3 Year DevSecOps Integrated Maturity & TCO Plan to compare Mature DIY DevSecOps to GitLab’s Prefabricated DevSecOps Assembly Line (Maturity Level 3) ● Multiple tools knitted together with complex manual processes and some hand off automation Late discoveries due to non-planned journey to maturity: ● Consumption based scanning license models ● Tool implementation & training costs ● Cost of non-integrated CI code review Why Play With Dolls At All?
- 31. 33 GitLab Prefabricated Sub- Assemblies(e.g. SAST Scanning) Primitives to Build Sub-Assemblies and Lines Templates Your Prefabricated Sub-Assemblies & Lines Prefabricated Partner Sub-Assemblies Prefabricated Assembly Lines (AutoDevOps, DevSecOps) Skip the Russian Dolls by starting as high in this stack as you can. Unboxing The GitLab Factory Making Kit
- 32. 34 Self-Managed SaaS 3 Paid EnterpriseTiers Free Enterprise (Public, OSS, Edu, Startup) Free Community Edition (100% OSS) Lots of Options To Get Involved 30 Day Trials - Self-Run - Sales Assisted
- 33. 35 Production Grade CI/ CD Automation Prefabricated Modular Scalable Highly Available 100% CI CD Automated Enables Building Your Own CI CD Primitives When Required Pipeline As Code
- 34. 36 How to Havea Tortured Software Manufacturing Transformation ● Not accounting for both coding domains: Application Coding (CI), Infrastructure Coding (CD) ● Not skipping the DevSecOps Russian Doll ● Not allocating surge time and resources for culture, skill and technical transformation ● Not allocating refactor time for product code that is not DevOps assembly line friendly (12- factor.net) ● Not driving knowledge extraction, skilling or product refactoring to done-done. ● Not automating all the Russian dolls (Is QA or AppSec still the semi-manual bottleneck?) ● Not using off the shelf tooling to skip Craft => Engineering cycles as much as possible ● Not remembering craft is what creates all innovation - you must have it - but not where industry has or is reducing it to engineered automation.
- 35. 37 All Branches SCM CI Job BranchA CIJob BranchB CI Job CI Job CD Integration Job BranchA Background AWS Services Config (Per-Service + Per-Branch / Per-Pipeline) IAM Role Per Service BranchA CodePipeline Per-Branch / Per-Pipeline) Code Pipeline CodeBuild CodeCommit CodeDeployCodeBuild CodeBuild S3 CloudWatch CloudFormation IAM CodeDeploy 3. Repo Mirroring Trigger - Full CodePipeline Integration - Leverage Superior GitLab SCM - Phased Migration Step (migration) 1. CodeDeploy Direct Integration - for Superior AWS Specific CD - for true Build/Pipeline as Code - No IaC for pipelines - Dynamic, per-branch pipelines Integration Options ECR 2. Artifact Depositing Trigger - CodePipeline for Deploy Only - Leverage Superior GitLab SCM - for Superior AWS Specific CD - S3 or ECR triggers S3 AWS Amplify AWS Elastic Beanstalk
- 36. 38 Feature (many) (withbuilds) Rich Workflow Gating Controls for “Code Review” and “Environment Change Control” Push Rules (entire repo) Pre-prod (Deploy) Environment Deployment Branches Integration (Code Review & Validate) Code Review Branch Protections w/ Custom Groups Stage (Deploy Branch) Independent Branch Protections by Role by Users & Custom Groups by CODEOWNERS Prod (Deploy) Security & Compliance Scanning & Dashboard Custom Groups BS GU Verified Committers Signed Commits SP Lock Files / Dirs SP BSFC SP Repo-Wide Merge Approvals Tag Protections w/ Custom Groups BS BS Multiple Merge Approval Rules w/ Custom Groups SP GitLab CI FC BS AWS CodeDeploy Amazon ECS AWS CodeDeploy AWS CodePipelin e Amazon ECS AWS CodeDeploy AWS CodePipelin eAmazon ECS AWS CodeDeploy AWS CodePipelin e Amazon ECS ECS is built in, CodeDeploy and CodePipeline are Easy to Add
- 37. 39 Resources Wire GitLab and AWSCodePipeline Together Integrate GitLab with AWS CodePipeline https://docs.gitlab.com/ee/user/project/repository/repository_mirroring.html Rich Change Controls for Building Workflows You Can Trust https://www.youtube.com/watch?v=uW95PV8d-w8
- 38. 40 Solution Architects Partnering InYour Success With GitLab Need Help Understanding or Trying Out GitLab CI? https://about.gitlab.com/sales/ Darwin Sanoy, Senior Solutions Architect, GitLab James Bland, Ph.D., Sr. Solutions Architect, AWS DarwinJS @DarwinTheorizes missionimpossiblecode.io linkedin.com/in/DarwinSanoy linkedin.com/in/jamesbland123