Friday, August 11, 2023

Your GitHub Secrets Ain't That Secret: (GitHub actions hack)


Intro


Sometimes secrets in the real world aren't as secret as we think! Although it can be scary, it’s really easy to show the content of your secret in GitHub actions log. You might not mind if your repo is private but if it is public and your Open Source project still needs things to stay private like secrets, It’s a real liability.


In this blog post, we’ll reveal how seemingly hidden GitHub secrets can sometimes slip out in plain sight. All it takes is a tiny code snippet in my workflow. But first, let’s review some fundamentals about GitHub secrets. down pointing left hand index (brown)

 


GitHub Actions and Secrets 


  • Secrets in public repositories are automatically masked in build logs & show as strings of  "*" characters.

  • There are organization, repository and environment level secrets (contexts).

  • In case of  conflict, organization is overridden by repository and repository is overridden by env values.

    • secrets.GITHUB_TOKEN is a temporary token for each workflow run.

  • Beware: GitHub Actions logs of your public repo are visible to anyone as opposed to private ones.


Let’s crack your secret

cheers meme

                           What if the secret is already visible in your workflow log of your public repo? 


In order to run this test and show your secret in your build log. You can either

  • Clone my repo then load it back to your own GitHub.

  • Just reuse the code in my Workflow.

                                                                PREREQUISITES

  • A repository + branch (optional)

    Example   Repo: brokedba/githubactions_hacks  Branch: git_actions

     

  • An environment
    Name: lab_tests , With deployment branch set to `selected branch`: i.e git_actions 


  • A Secret

    secret name:
    TEST_SECRET             
    secret value: MYPassword

  • A workflow


You can download test_secret.yml located under .github/workflows as below display might not be properly indented.

# example: test secret” action that shows the spaced content of your secret

name: 'My_secret_test_Workflow'
on:
  push:
    branches: [ "git_actions" ]
    paths:
      - '.github/workflows/test_secret.yml'   

jobs:
  job_test_secrets:
    runs-on: ubuntu-latest
    environment: test-labs
#Use default shell of the GitHub Actions runner     defaults:      run:
       shell: bash
   steps:  
    # Checkout the repository to the GitHub Actions runner
    - name: Checkout
      uses: actions/checkout@v3  
   

# Show how to print unmasked GitHub secrets to the console
     - name: Step 1 - Echo out a GitHub Actions Secret to the logs
      run: |
# ==> show masked value ******
        echo ${{ secrets.TEST_SECRET }}

        echo "Trick to echo GitHub Actions Secret:  "  

        # ===> show the real content with spaces 'M Y P a s s w o r d'
        echo ${{secrets.TEST_SECRET}} | sed 's/./& /g' 

  • The content has nothing more but a check out action and a tiny code snippet that consumes the secret.

  • We chose the push as trigger event and the git_actions as target branch

  • We set paths so anytime the yaml workflow is changed a pipeline is triggered 

  • Finally a simple sed expression piped into the “echo secret” command  



Workflow Execution result


                                               A little change commit for test_secret.yml and Voila!   
                    

                                               My secret is now readable in plain sight on the build log


You are never safe


Now, this is just a small example of how your secret can be revealed in the logs and although you’d argue that no one is going to add this code in their workflow in purpose but there are many ways to infiltrate your Github pipelines and inject malicious code from an external PR to either extract secrets or damage your deployments or artifacts.
It could be due to misconfiguration of your workflows or by using untrusted actions from Marketplace.


What we already know


More Vulnerabilities


Exploitability and impact of untrusted input such as supply-chain compromise is real, Here are few examples
 

  • Any maintainer can update a branch or a tag, hence a corrupted Public actions@v1 can have an updated tag that has a malicious code inside & everyone will call the wrong release. Read more Here 

  • There a list of event context data that might be attacker controlled like issue, title, body, & comment

  • The attacker server can use the GitHub API to modify repository content, including releases.

  • The attacker can exfiltrate repository and organizational secrets other than the ones of the workflow by creating a new workflow via personal access token (PAT).

  •  By default, pull requests from first-time contributors require approval from a maintainer but following PR might be malicious

  • pull_request_target trigger event: will allow any PR submitted from a forked repo to run in the context of your main repository getting access to secrets and write access to the original repo


Detailed studies on GitHub actions Vulnerabilities

Best practices


Again if your workflow is in a public repo, remember below safety measures  ⏬  

  • Use commit hash(@sha1) as version tag when calling third-party actions to shield yourself.

  • You must only invite trustworthy outside collaborators.

  • Don't have any workflow with a pull_request_target trigger.

  • Avoid using untrusted public actions and if you have to always test them in dummy repos first.

  • Prefer OpenID Connect to access cloud resources 

  • GITHUB_TOKEN and PAT should always be granted the minimum required permissions

  • Don’t reference directly values that you don’t control like

  • Avoid run steps (inline script) and use external actions instead

  • Sanitize your input using environment variables

  • scan for secrets in the source code itself using ggshield-action

  • If you create a credential from a secret (e.g. base64 an API key), you should mask the new value so it doesn't leak.

  • Have the "Require approval for all outside collaborators" option for GitHub Actions turned on

Below is an excellent cheat-sheet on available security best practices



                                    
Conclusion:

  • This blog post aimed to shed light on potential security pitfalls in GitHub Actions.

  • Even masked secrets can be uncovered with some clever tricks as shown in my workflow code snippet .

  • There are even more vulnerabilities out there as mentioned in the second part of this article

  • Again if your workflow is in a public repo, remember to apply the mentioned safety measures

  • You should always be careful about how you set up your workflows  

  • Next, I will write a post about cleaning up your GitHub action cache after use.

Stay tuned




Deploy multilevel OCI sub-compartments with terraform (using local module)

 

Intro


In my previous blogpost, we explored the challenges with relying on terraform registry modules for code sanity checks. Additionally, we reviewed OCI iam-compartment module and I shared my revised version in my Github. 
Today, we'll demonstrate how to deploy multi-level OCI compartments using my local module (iam-compartment), which you can easily clone from my GitHub Repo.


Why multi level compartments  ?
 

Oracle Cloud landing zone reference architecture recommends a multi-layer compartment design, reflecting the functional structure observed across different organizations. With this approach, IT responsibilities are efficiently separated among networking, security, application development, and database administrators.



Use case and Topology


Let's say our business is a Mediterranean fast food franchise called "La Dolce Pita". Their application goal is for customers to find nearby restaurants, order ahead, receive updates, and enjoy rewards.


Following Cloud Adoption Framework (CAF) Landing Zone of OCI, we'll need a robust foundation, ensuring full control and isolation over the tenancy. Which is why multi level compartments comes into play.



The workload compartment is our LDP application which will share common infrastructure resources such as identity domains, FastConnect, Cloud Guard targets, and so on.

Overview


The final design includes 3 level compartments segregating operational duties and resources .

  • Level 1: - under the root compartment

    • LDP Top enclosing compartment for the workload sitting

  • Level 2: - under the LDP compartment

    • LDP Shared compartment hosting shared resources such as monitoring VMs, Bastion hosts, Windows Domain Controllers (DC), backup endpoints, etc..

    • LDP-Network compartment hosting all network resources like VCNs, subnets, DRGs, etc.

    • LDP-Security compartment hosting security related resources (Key Vault, cloud guard, audit,..)

    • LDP-App compartment : enclosing all the application resources & compartments (front/back) 

  • Level 3: - under the LDP-app  compartment

    • LDP-app-prod the production environment resources

    • LDP-app-dev  the development environment resources (we can have many more like QA/test) 

    • LDP-app-dr  literally the DR tier in a neighbor region

    • LDP-app-db backend databases (DBCS, Exadata, Autonomous DB, MySQL, PostgreSQL) 


Which module are we using?


It’s my revised version of the official OCI iam-compartment, located in my GitHub under iam-compartment folder.
Local modules are good as it allows a single copy use for all your calls, including dynamic blocks. 

This image has an empty alt attribute; its file name is image-4.png
See below call example with the module attributes.

module "My_compartments" { source = "./modules/iam-compartment" #tenancy_ocid = var.tenancy_ocid # optional compartment_id = var.comp_id # Parent compartment compartment_name = var.comp_name compartment_description = var.comp_description compartment_create = true enable_delete = true }


Deploying Multi level compartments  

Clone the repository

  • This is my own github repo, Pick an area on your file system and run the clone command

$ git clone https://github.com/brokedba/terraform-examples.git

You will find our configuration under a subdirectory called terraform-provider-oci/compartments


  • Cd Into the subdirectory where our configuration resides and run the init

$ cd   ~/terraform-examples/terraform-provider-oci/compartments
$ terraform init

  • Here’s a tree of the files composing our configuration

$ tree . |-- variables.tf ---> Resource variables needed for the deploy including locals
|-- compartments.tf ---> Our main compartment resource declaration
|-- output.tf ---> displays the compartments detail at the end of the deploy
|-- terraform.tfvars.template---> environment_variables needed to authenticate to OCI
|--
modules | ├── iam-compartment
     | ├── main.tf
   | ├── output.tf
   | ├── variables.tf
  │   | ├── versions.tf └── modules.json --> file mapping the modules and their corresponding resource blocks.


Using locals and dynamic blocks


By using local variables, I stored the display names of all my sub compartments. This allows for a single dynamic block per compartment level and a to use for_each loop to create all the compartments efficiently.

  • The enclosing compartment “LDP” will use a normal resource block and not the module to keep it simple.

  • But all sub compartments will leverage our module by using one dynamic block per level 

    • level_1_sub_compartments: for all level 1 compartments

    • level_2_sub_compartments: for all level 2 compartments



1. Execution plan 


  • Under the working directory (terraform-provider-oci/compartments) update terraform.tfvars file

  • Run terraform plan (see below excerpts of a level1 resource block referencing the module).

#Adjust terraform.tfvars.template with authentication parameters & rename it to terraform.tfvars

$ terraform plan .Terraform will perform the following actions:
… snip
# module.level_2_sub_compartments[LDP-app-db].oci_identity_compartment.this[0]
will be created
+ resource "oci_identity_compartment" "this" {
    + compartment_id = (known after apply)
    + defined_tags   = (known after apply)
    + description    = "Database instances and resources"
    + enable_delete  = true
    + freeform_tags  = (known after apply)
    + id             = (known after apply)
    + inactive_state = (known after apply)
    + is_accessible  = (known after apply)    
+ name           = "LDP-app-db"
    + state          = (known after apply)
    + time_created   = (known after apply)
   }
...
--- OTHER remaining resources

Plan: 9 to add, 0 to change, 0 to destroy.

2. Deployment (Apply)

  

And here you 9 resources created among which 1 main compartment and 8 sub compartments in two levels.   

# Original output was truncated for more visibility

$ terraform apply -–auto-approve

oci_identity_compartment.iam_compartment_main: Creating...
module.level_1_sub_compartments["LDP-network"].oci_identity_compartment.this[0]:Creating..
module.level_1_sub_compartments["LDP-app"].oci_identity_compartment.this[0]: Creating...
module.level_1_sub_compartments["LDP-security"].oci_identity_compartment.this[0]: Creating...
module.level_1_sub_compartments["LDP-shared"].oci_identity_compartment.this[0]: Creating...
module.level_2_sub_compartments["LDP-app-dev"].oci_identity_compartment.this[0]:Creating...
module.level_2_sub_compartments["LDP-app-prod"].oci_identity_compartment.this[0]:Creating...
module.level_2_sub_compartments["LDP-app-dr"].oci_identity_compartment.this[0]:Creating…
module.level_2_sub_compartments["LDP-app-db"].oci_identity_compartment.this[0]:Creating...


Apply complete! Resources: 9 added, 0 changed, 0 destroyed.
Outputs:
l1_sub_compartment = {"comp_name" = "LDP-app"} l1_sub_compartments= {
"LDP-app" = ocid1.xx => Desc: Parent compartment for all application resources"  
"LDP-network" = ocid1.xx => Desc: for all FW, VCNs and LBRs"  
"LDP-security" = ocid1.xx => Desc: for Security related resources like Vaults, keys"  
"LDP-shared" = ocid1.xx => Desc: for shared services like AD DC, Monitoring" }
l2_sub_compartments = {  
"LDP-app-db" = ocid1.xx => Desc: Database instances and resources"  
"LDP-app-dev" = ocid1.xx => Desc: non-production VMs"  
"LDP-app-dr" = ocid1.xx => Desc: DR VMs"  
"LDP-app-prod" = ocid1.xx => Desc:  production VMs" }
main_compartment = {  
"Comp_name" = "LDP"  
"comp_desc" = "Enclosing compartment at root level"  
"comp_ocid" = "ocid1.xx"
}

You can see the end result in the below console view



Conclusion:

  • We have just demonstrated how to quickly deploy a multi level compartments using terraform in OCI 

  • Remember that all used attributes in this exercise can be modified in the variables.tf output.tf  files.

  • This also allows to follow OCI landing zone recommendations for control & isolation of each workload

  • local modules not offer the same modularity in registry modules, but is even more powerful as since it allows to  reuse a single module copy for all the calls, including dynamic blocks.

  • Using local variables to feed the the dynamic blocks allowed to have only 2 blocks for 8 compartments reducing drastically the foot print of our code (on top of  reusing a unique copy of the module)