Monday, December 14, 2020

Deploy a webserver vm using Azure CLI and bash scripts (Linux/windows)

                                        
Intro

Azure cli was another tool I had to try right after passing Az-900 azure certification, because what are certifications for if it’s not to give a little itching to get your hands a little dirty. Thus, az-cli became my third victim after oci-cli and aws-cli.
As in my previous cli tutorials, the aim is to demonstrate how the CLI tool can enrich your IaC experience when combined with powerful shell scripts. What comes along when you test your third cli tool in a row though, is how easy it is to spot differences with other cloud platforms (expect to see some in this post).

Here’s a direct link to my Github repo with all the scripts: https://github.com/brokedba/az-cli-examples
Here’s also a gif demo to see things in motion: https://bit.ly/3hvTwVu 

I. Bash still rocks

With a little patience and few days debugging bash errors, my interactive scripts can now automate the provisioning of a webserver on 6 different OS (RHEL, Centos, Oracle Linux, Ubuntu, SUSE, Windows server 2016)
In total, 6 interactive BASH scripts & userdata files were used with Windows turning out to be the nastiest to deploy (I’ll explain why). The code also ensures that required network components like subnet or security group are created if missing during the launch. This will hopefully give beginners a glimpse on az-cli capabilities that make it so handy.  
 
A big shout out to Stackoverflow community that made my life easier while bash scripting my way to pull this off.

 
Topology

The following figure shows the layers involved between our workstation and Azure while running the cli commands.

This image has an empty alt attribute; its file name is az-cli_instance-1.png

  CLI setup and assumptions

   If your cli environment is not configured yet, go check my previous post (az-cli installation) it’s very easy.  

   I will assume that the below elements are present/configured in your workstation:

  • ssh key pair to attach to your azure vm. below is the PEM based key pair I generated for this lab
  • $ ssh-keygen -P "" -t rsa -b 2048 -m pem -f ~/id_rsa_az
    Generating public/private rsa key pair.
    Your identification has been saved in /home/brokedba/id_rsa_az.
    Your public key has been saved in /home/brokedba/id_rsa_az.pub.
  • Default output is set to table
  • $  az config set defaults.output=table
    +-----------------------------------------+---------+
    |  Name    | Source                       | Value   | 
    +----------+------------------------------+---------+ 
    | output   | /home/brokedba/.azure/config | table   |
    +----------+------------------------------+---------+
  • Note: The default region used in this lab will depend on your resource group or the configured region parameter 

II. Clone the repository


III. Deployment

  • Before starting the deployment let’s first talk about what makes azure architecture different from other cloud platforms

    Network:

    In Azure every subnet is a public subnet because as soon as you associate a public IP to a Vm’s VNIC, you'll magically have internet access.Thus, the usual internet gateway is not needed here because system routes are taking care of that. Besides the CIDR range in azure is slightly larger than aws ( from /8 to /29).

    ID:
    Azure doesn’t provide regular alpha numeric ids for its resources but a sort of path based identification (see below)
  • $ SUBNET ID  
    /subscriptions/xx/resourceGroups/my_group/providers/Microsoft.Network/virtualNetworks/MY-VNET/subnets/My_SUBNET

    Naming: Naming is
    - Case insensitive (Windows baby ;)!!)     
    - Unique: A resource group can’t have 2 resources of the same type having the same name    

  • Takeaways:
    We can imply from the above that
    1- We won’t need to handle ids but just the resource names in our scripts as they are unique within a resource group
    2- Route table and internet gateway setting will be replaced by the Public IP and VNICs

  1. CREATE A VNET/SUBNET

  • Below script creates both Vnet and Subnet using default or custom values for their names and CIDR block. I also added checks on the IP/CIDR format. You can click on the Script to see the content.
  • Network security group menu will offer to open few ports for you but will also be checked in the vm creation script
    brokedba@ ./create_az_vnet.sh
      Name          Location
    ------------  ---------
    brokedba       eastus
    select the resource Group you wish to set for your resources []:
    Enter the VNET name you wish to create
    [CLI-VPC]: selected VNET name : CLI-VCN
    selected group name: brokedba
    Enter the subnet name you wish to add [CLI-SUB]:
    Enter the VNET CIDR to assign '/8-To-/29' [192.0.0.0/8]: 192.0.0.0/8
    Enter the Subnet CIDR to assign within 192.0.0.0/8 to '/29' [192.168.0.0/16]:

      ==== Created VPC details ==== +---------+------------+---------+--------------+---------------+--------------+ | VNET    | Vnet_CIDR  | Subnet |  SUB_CIDR   | Resource_group| Region | +---------+------------+---------+--------------+---------------+--------------+ | CLI-VNET|192.0.0.0/8 | CLI-SUB |192.168.0.0/16| brokedba    | canadacentral| +---------+------------+---------+--------------+---------------+--------------+ Note: make sure all bytes beyond network prefix length are always zeroed or you'll have an error
    ************ Security Group ! ************
    1) SSH port Only             3) HTTP,RDP, and HTTPS
    2) SSH, HTTP, and HTTPS

    Select a security group ingress rule and press Enter: 2
    ******************* Security Group detail  ******************
    +---------------+---------+-------------------+----------------------------+
    | Name       | Source |   PORT            | Type    | Priority |
    +---------------+---------+-------------------+---------+------------------+
    | Allow-WEB-IN  |Internet | ["22","80","443"] | Inbound |   100 |
    +---------------+---------+-------------------+---------+------------------+


    SG delete command  ==>
    az network vnet delete –g brokedba -n CLI-VNET
    VPC delete command ==> az network nsg delete  -g brokedba -n sg_CLI-SUB_WEB
    Disassociate NSG from its Subnet => az network vnet subnet update --vnet-name CLI-VNET --name CLI-SUB –g brokedba --network-security-group ""

    Note:
    - If no resource group is detected in your account the script will ask you to create one using create_az_rg.sh
    - Delete commands are added in each script so you could destroy/repeat without using the Web Console.



  1. CREATE SUBNET (Optional)

  • You can also create a subnet for an existing vnet using create_az_subnet.sh script but it’ll only allow it if the vnet has no existing subnet (sorry, too much exception handling due to CIDR overlap in case of existing subnets)

  1. LIST VM SIZES (Optional)

  • Below menu returns the available vm sizes per picked vcpu number If you to want to replace the default size defined in the create_az_instance.sh script ( $vm_size)

    brokedba@ ./check_az_vmsize.sh
    ******* azure vm shape Selecta ! ************
    list all vm sizes in eastus region depending on the CPU and Series selected.
    ...>> Pick vm size = CPU#
    1) 1 VCPU
    2) 2 VCPUs
    3) 4 VCPUs
    4) 8 VCPUs
    5)16 VCPUs
    Select a number of cores and press Enter:
    3
    >> Vm compute Series
    1) A Series (Entry-level)              3) D Series (General purpose)
    2) B Series (burstable)               4) E Series (Optimized for in-memory)
    Select a VM series and press Enter: 3
    VM              VCPUS    Memory_MB    MaxDisks   OSDisk_maxMB  UserDisk_maxMB
    --------------- ------ -------------- ----------- ------------- ----------------

    Standard_D4s_v3   4        16384        8           1047552         32768
    Standard_D4_v3    4        16384        8           1047552         102400
    Standard_D4d_v4   4        16384        8           1047552         153600
    Standard_D4_v4    4        16384        8           1047552         0
    Standard_D4ds_v4  4        16384        8           1047552         153600
    Standard_D4s_v4   4        16384        8           1047552         0
    Standard_D4a_v4   4        16384        8           1047552         102400
    Standard_D4as_v4  4        16384        8           1047552         32768

  • Note: Default location is eastus.To change it, modify the variable $location at the top of the script.

  1. LIST IMAGES (Optional)

  • Below menu returns the last available image per OS type. This will help you decide which OS to choose for your vm
    brokedba@ ./check_image.sh 
    ******* AWS Image Selecta ! ************
    Choose your Destiny ||{**}||
    
    1) RHEL          3) Oracle Linux  5) Windows       7) Exit? 
    2) CentOS        4) Ubuntu        6) Suse  
    Select an option and press Enter: 2
    Name   Publisher    Sku         Urn                              Version
    -----  ----------- ------  -----------------------------------  -------------- CentOS  OpenLogic    7.7   OpenLogic:CentOS:7.7:7.7.2020111300  7.7.2020111300

  1. CREATE INSTANCE

  • After running the create Vnet script, we can now launch our new vm using 6 possible OS’ to choose from.
  • The default vm type is “Standard_B1s” but you can give the variable another value if you want.
    You’ll be asked to retry if the Vnet and Subnet names don’t match and if a subnet is missing, it’ll be created for you. 
  • brokedba@ ./create_instance.sh
    ******* Azure VM launch ! ************

    ********** Resource Group ***********
    Name         Location ----------- -----------
    brokedba     eastus

    select the resource Group you wish to set for your resources []: brokedba
    Enter the name of your new Instance [Demo-Cli-Instance]: Demo-vm ----- selected Instance name : Demo-vm Vm size : Standard_B1s
    ********** Network ***********

             **** VNET **** VNET      Vnet_CIDR    Resource_group    Region --------  -----------  --------------- ---------- CLI-VNET  192.0.0.0/8  brokedba       useast select the VNET Name for your new instance []: CLI-VNET selected VNET name : CLI-VNET          

    **** SUBNET *** Subnet    CIDR            ResourceGroup --------  --------------  -------------- CLI-SUB   192.168.0.0/16  brokedba Select The Subnet for your new instance []: CLI-SUB selected subnet name :  CLI-SUB

    Subnet exist => Checking  the OS menu and Security group rules ...

    ************ Network security Security Group ! ************ 
    Checking the associated  NSG : sg_CLI-SUB_WEB
    == Creating the instance with the below NSG.
    { "Combo_rule_Ports": "["22","80","443"]",
      "Name": "sg_CLI-SUB_WEB",
      "resourceGroup": "brokedba",
      "single_rule_Ports": "[]",
      "sub": [ /subscriptions/…/virtualNetworks/CLI-VNET/subnets/CLI-SUB" ]
    }
    ************ Azure Image Selecta ! **************
    Choose your Image ||{**}||

    1) RHEL          3) Oracle Linux  5) Windows       7) Abort?
    2) CentOS        4) Ubuntu        6) Suse
    Select an option and press Enter: 2
    Name    Publisher    Sku    Urn                                 Version ------  -----------  -----  ----------------------------------- --------------
    CentOS  OpenLogic    7.7    OpenLogic:CentOS:7.7:7.7.2020111300 7.7.2020111300

    ===== Instance Deployment Detail ========

    Selected Subnet name : CLI-SUB Selected Instance name : Demo-vm Selected instance Type : Standard_B1s Selected Security Group: sg_CLI-SUB_WEB UserName   : centos Selected OS : CENTOS ... Public IP creation... Name           Location       Stat       Allocation -------------  -------------  ---------  ------------ Demo-vm_pubip  canadacentral  Succeeded  Dynamic Network interface creation... Name         State      PrivateIP -----------  ---------  ----------- Demo-vm_Nic  Succeeded  192.168.0.4
    =========================================== Check the status of the new Instance =========================================== The compute instance is being created. This will take few minutes ... Enter the Path of your ssh key [~/id_rsa_az.pub]:
    selected public key: /home/brokedba/id_rsa_az.pub

    ResourceGroup PowerState PublicIpAddress PrivateIpAddress MacAddress  Location
    ------------- ---------- --------------- ---------------- ----------------- -------
    brokedba      VM running  40.85.21
    8.246     192.168.0.4   00-22-48-3B-BE-DB useast

    ssh connection to the instance ==> ssh -i /home/brokedba/id_rsa_az centos@40.85.218.246
    Your website is ready at this IP :) : http://40.85.218.246
    VM termination command ==> az vm delete –g brokedba -n Demo-vm --yes VNIC termination command ==> az network nic delete –g brokedba -n Demo-vm_Nic IP termination command ==> az network public-ip delete –g brokedba -n Demo-vm_Pubip VNET termination command ==> az network vnet delete –g brokedba -n CLI-VNET Disk termination command ==> az disk delete –g brokedba -n Demo-vm_OsDisk_1_fxxxxxxxx

    Below is the vm view on your Azure Portal once the instance is provisioned.

    This image has an empty alt attribute; its file name is az-cli_vm.png
  1. CONNECTION TO THE VM AND CHECK THE PUBLIC WEBPAGE

  • Grab the public IP and enter the private key to connect to your vm using ssh
  • $ ssh -i /home/brokedba/id_rsa_az centos@40.85.218.246 
  • Type the Public IP in your browser & check the customized Homepage using the custom-data run during bootstrap
    =>  http://40.85.218.246

    This image has an empty alt attribute; its file name is image.png

    The curious case of Windows vms 

The “--custom-data” parameter which takes the cloud-init file and sends its encoded content along with the provisioning configuration in azure does it only for Linux vms.You’ve read right, when providers like aws offer full cloud-init support in their cli-tool for all available images including windows, Microsoft doesn’t for it’s own OS !!

Azure proposed cheap option
The only way  to replace custom-data run when spinning windows vms is to remotely invoke commands on the vm after creation (which sucks). I had to add the below line to run my Powershell custom-data on the vm

    az vm run-command invoke -g "$rg_name" -n $instance_name --command-id SetRDPPort
    az vm run-command invoke -g "$rg_name" -n $instance_name --command-id RunPowerShellScript
    --scripts @cloud-init/Win_userdata.ps1

 

   CONCLUSION 

    • This exercise helped to know more about azure architecture than just practicing on a cli tool and JMESPATH tricks.
    • Az cli is still powerful, but I have to say that I felt quite a latency all along my lab even on simple queries where nothing was created. AWS and OCI for instance are way faster in terms of responsiveness (based on my useast tests) 
    • The fact that a custom route and internet gateway weren’t necessary a reduced my code footprint by almost 30%.
    • One caveat for windows provisioning is that azure doesn’t support cloud-init, but custom code can still run remotely  
    • Feel free to fork my repository and adapt your own version of these scripts
    • Improvement: add the option to add a subnet to vnets with existing subnet to create_az_subnet.sh script   

Thanks for reading!

4 comments:

  1. This information really helped me a lot. It was very informative.
    Cloud Advisory Services

    ReplyDelete
  2. Wow, amazing block structure! How long
    Have you written a blog before? Working on a blog seems easy.
    The overview of your website is pretty good, not to mention what it does.
    In the content!
    Window 12 Pro Crack
    Pixarra TwistedBrush Pro Studio Crack
    Clone Files Checker Crack
    Window 12 Pro Crack
    EasyMorph Crack
    AngelicVibes Thump Multi Effects Crack
    Antivirus Zap Pro Crack
    AAS Strum GS Crack

    ReplyDelete
  3. I guess I am the only one who came here to share my very own experience. Guess what!? I am using my laptop for almost thepast 6 years, but I had no idea of solving some basic issues. I do not know how to Download Cracked Pro Softwares But thankfully, I recently visited is a website named filehippos crack
    AngelicVibes Thump Multi Effects Crack

    ReplyDelete