Thursday, June 22, 2023

Devops: Demystifying CI vs. CD and the "Delivery-Deployment" Differences

This image has an empty alt attribute; its file name is cicd_test_deploy.png


Intro


In the space of software delivery, CI/CD has emerged as a game-changing framework that accelerates development life cycle
. While most users consider practice a handy shortcut to knowledge, diving headfirst into the CI/CD universe without a solid grasp of the fundamental bases is like trying to learn salsa moves without understanding tempo or foot work. It's crucial to understand the core principles that underpin a framework and their nuances to avoid confusion when talking to team members & clients.You first need to get your terminology right.


In this short post, we will revisit the basic definition of few core components of Devops CI/CD, bridging the gap between what you might have thought you knew, and what you truly comprehend.


Back to basics, what's the difference between CI and CD really?

Implementing QA in a CI/CD Pipeline - Parasoft

Both are related to the continuous delivery of software applications, but serve different purposes.

    • CI (Continuous Integration):

      The most widely known, it is a process of regularly integrating code changes into a shared repository, allowing teams to detect and resolve issues quickly. The goal of CI is to ensure that each change made to the codebase is automatically tested, verified, and integrated into the main codebase as soon as possible.
      This is where the compiling/building/packaging of your software happens including unit tests.

    • CD (Continuous Delivery/Deployment):

      Is a process of deploying software changes to a production environment in a reliable, efficient, and repeatable manner. The goal of CD is to automate the entire software delivery process, from building the application to testing, deploying, and monitoring it in production ( or “beta” / “staging” environments).


    • In summary, CI focuses on testing and integration, while CD focuses on the automation of software delivery.


    How different is testing within CI and CD?

    CI/CD What is it and Why is it Important?


    • CI pipeline, testing

      CI testing is focused on ensuring that the changes made in the codebase do not introduce new bugs and that the codebase meets the predefined quality standards. The testing is done as early as possible in the development process, typically after every code commit or at least daily.

    • CD pipeline testing

      CD testing, is focused on ensuring that the changes made in the codebase can be reliably deployed to the production environment with minimal or no disruption. The testing is done after the code is built and packaged, and it is performed in an environment that is as close to the production environment as possible. The CD pipeline also includes testing for non-functional requirements such as performance, scalability, and security.


    • In summary, while testing is a critical part of both CI and CD pipelines, the testing strategies and objectives may differ depending on the pipeline's goal.


    Continuous delivery vs Continuous deployment

    This image has an empty alt attribute; its file name is cicd_deploy-3.png

    Here again, CD has a double meaning as it encompasses both Delivery & Deployment, serving a dual purpose in the CI/CD framework.

    • Continuous delivery
      A continuous delivery pipeline automatically tests the application, but keeps the deployment decision as a manual step (manual trigger) .

    • Continuous deployment
      A continuous deployment pipeline, on the other hand, will automatically deploy this working version ( no manual approval).


    Which one to choose 

    There is no “right” or “wrong” way. Different organizations have different needs. Use the one that delivers the most value to your customers!  As the author of the manifesto of agile development Martin Fowler says,

    "Continuous Deployment means that every change goes through the pipeline and automatically gets put into production,resulting in many production deployments every day."


    Not to be confused with continuous delivery, which is getting the application in a state that is ready for deployment (Developers don’t necessarily deploy it immediately). Continuous deployment finishes off that last mile.



    CONCLUSION

    • I hope that the above bits into the fundamentals of CI/CD, allowed you to unlock a deeper understanding of their significance in the DevOps landscape if you’re starting in this journey.

    • CI & CD aren’t just buzzwords, but powerful methodologies that empower software development.

    • It is crucial for us, practitioners, to grasp these small yet important nuances.

    • As said before , understanding the terminology is key to making sense of every CI/CD practice.

    Stay tuned    

    Friday, June 16, 2023

    My corrected OCI-IAM-compartment Terraform Module

    How to Get the Latest OS Image in Oracle Cloud Infrastructure (OCI) using  Terraform | by Guillermo Musumeci | Medium

      Oracle Logo

    Intro


    So, turns out I've been living in the Stone Age of Terraform. I thought keeping my code lean and mean was the way to go, but apparently, the cool kids are all about terraform modules. They're all like, "Modularity! Reusability! It's the secret sauce of Infrastructure as Code!". Up until now, with my resources being simple, I learned to just leverage dynamic blocks (for_each) or variable interpolations through locals to keep it light, but not for long it seems.



    Are Modules just hype or a real necessity ?

    This is what I wanted to figure after getting my HashiCorp certification to see what all the hype was about.
    And for that, I picked a simple OCI Iam module available in the terraform registry of Oracle provider

    What’s wrong with the OCI IAM module anyway ?


    Just like any software, Terraform is constantly evolving. Staying up to date with the latest versions is crucial, as new releases can bring significant changes to the syntax and supported attributes. Failing to proof your code against these updates could break your stack. This applies to Terraform modules maintainers too (Oracle TF Modules).

    GitHub - oracle-terraform-modules/terraform-oci-iam: Terraform Module for  managing Oracle Cloud Infrastructure Identity and Access Management (IAM)  resources

    The module in question is iam-compartment under: https://github.com/oracle-terraform-modules/terraform-oci-iam

    Disclaimer

     
    A
    t the time of my correction back in Feb, the following errors were not fixed and I was too lazy for a pull request, hence my local module creation. Fortunately for OCI users, recent fixes were added to the iam module.
                                                                   I didn’t try it yet but feel free to do so.


    1. Duplicate Required providers configuration (versions.tf & main.tf)

      

    I faced this `iam-compartment` issue with Terraform v1.0.3 but it occurs even with v0.13. Right from the init step.
    # terraform init

    Initializing modules... Downloading oracle-terraform-modules/iam/oci 2.0.2 for iam_compartment_x... - iam_compartment_x in .terraform/modules/iam_compartment_x/modules/iam-compartment There are some problems with the configuration, described below. The Terraform configuration must be valid before initialization so that Terraform can
    determine which modules and providers need to be installed.
    Error: Duplicate required providers configuration │ on .terraform/modules/iam_compartment_x/modules/iam-compartment/versions.tf line 2,
    | in terraform: │ 2: required_providers {.. │ A module may have only one required providers configuration.
    | The required providers were previously configured at
    .terraform/modules/iam_compartment_x/modules/iam-compartment/main.tf : line 5,3-21


    CAUSE:  See issue #33 

    • In the above error terraform complains that the required_provider block was defined twice

      • versions.tf => line 2  & main.tf  => line 5

    This image has an empty alt attribute; its file name is image-3.png

    Reason
    : In 2022 Oracle moved their terraform provider out of HashiCorp which led to the introduction of required_provider block, to update oci provider source to "oracle/oci". 

     

      Solution:
      Only keep one required_provider block in the compartment module (i.e: version.tf  over main.tf)

      # vi ~/modules/iam-compartment/version.tf

      terraform { required_providers { oci = { source = "oracle/oci" <---- Provider maintained/hosted by Oracle not Hashicorp version = ">=4.67.3" } } required_version = ">= 1.0.0" }


      2. Deprecated list and map function calls from iam-compartment

        

      Another issue due to deprecated functions (list/map) that were no longer supported by terraform 
      main.tf
      .. compartment_ids = concat(flatten(data.oci_identity_compartments.this.*.compartments), list(map("id", ""))) …

      Error > main.tf
      │ Call to function "list" failed: the "list" function was deprecated in │ Terraform v0.12 and is no longer available; use tolist([ ... ]) syntax to │ write a literal list.

      Error > output.tf
      │ Call to function "map" failed: the "map" function was deprecated
      | in Terraform v0.12 and is no longer available; use tomap({ ... }) syntax to
      | write a literal map.

      CAUSE:  See issue #27 

      • As described in the docs, list() and map() function are deprecated in v012 & removed in v015 (read here)

        

        Solution:
        Terraform can infer the necessary type conversions automatically from context. In those cases, you can just use the [...] or {...} syntax directly, without a conversion function.


        Another option would be to just convert the functions into their new equivalent (map=> tomap, list=>tolist).

        Example: main.tf line 27-28

        --- Before
        compartment_ids = concat(flatten(data.oci_identity_compartments.this.*.compartments),
        list(map("id", "")))
        ...

        --- Option 1
        compartment_ids = concat(flatten(data.oci_identity_compartments.this.*.compartments),
        [{ id = "" }])
        ...


        --- Option2 : using tomap and tolist

        compartment_ids = concat(flatten(data.oci_identity_compartments.this.*.compartments),
        tolist( [tomap({"id" = ""})]))


        My corrected version (local module)


        It was easier to create a corrected version in my git repo that you can use locally. see folder iam-compartment

        This image has an empty alt attribute; its file name is image-4.png


        How to call it
         

        Same as if you would for the official registry module (oracle-terraform-modules/terraform-oci-iam)

        module "My_compartments" { source = "./modules/iam-compartment" #tenancy_ocid = var.tenancy_ocid # optional compartment_id = var.comp_id # Parent compartment compartment_name = var.comp_name compartment_description = var.comp_description compartment_create = true enable_delete = true }

        What you need to know about modules

        • Registry modules

          • One single resource block per module: you’ll have as many folders as module blocks

          • Even if you use for_each loop say i=10, you will have 10 downloads under .terraform folder

          • You depend on the maintainers good will for code sanity/integrity checks

        • Local modules

          • Reuse a single instance of the module for all your module calls, including dynamic blocks

          • Code proofing responsibility against new updates falls on you


        Conclusion:

        • This goes to show how important it is to keep the code up to date & compatible especially for tf modules

        • Whether you choose a local or 3rd party module, there are positives and negatives as with all things.

        • Due diligence is always welcome to decide if the code is high quality, secure and is maintained.

        • The same is just as true for modules you make yourself.

        • There’s a style of Terraform programming where people try to make everything a module. Some consider this to be an antipattern.

        • As for me, I’m glad I always tried tuning with dynamic blocks and locals to reduce my code footprint first 


        next up  “multilevel compartment deployment using this local module”

        Stay Tuned