Prerequisites

• ✅ ACR deployed
• ✅ Hands on lab Variables loaded
• ✅ Resources groups deployed
• ✅ Virtual network deployed
• ✅ User Managed Identities deployed

Cluster identity

To access others resources, AKS Cluster requires either a Service principal or a managed identity. Managed Identities are the recommended way mainly because the don’t need to manage the password.

Managed identities can be either system managed identities or user managed identities.

No Hands-on lab for Service principal

There is no interest to show a sample with service principal in the AKS section because requirements for permissions will be viewed with User Managed identity. A service principal and a User managed identity must be created and theirs permissions applied on the same resources before the AKS provisioning.

Identity type decision criteria

Comparison conclusion

• Don’t use service principal to avoid password storage, usage and renewal management

• Use user Managed Identity to not give “User Access Administrator” roles on other automation accounts

• Use System Managed Identity only for Proof of Concept to have less configuration tasks

User Managed Identity drawback for Ansible provisioning

As It is not yet supported in Ansible azure.azcollection v1.15.0 and a conflict exist between azure.azcollection v1.15.0 and Azure-cli v2.46.0¹ :

• It is mandatory to break the provisioning from Ansible
• Use an other execution environment with only Azure-Cli
• Wait more time during the cluster update to switch from System Managed Identity to User Managed Identity

Remarks

Configuration notes

1. Nodes resources must not exists when creating a new cluster
2. Services cidr must not be an existing subnet cidr
3. Dns service ip must be within the Kubernetes service address range specified in services cidr
4. ACR and AKS should be in the same location

Create Aks with user managed identities

Hands-on lab

1. Get Subnet and Identities Id

    # Get subnet id
    $subnetNodeId = $(az network vnet subnet show -g $aksresourceGroup --vnet-name $vnetName -n $nodesSubnetName --query "id" -o tsv)
    write-host "Subnet node Id : $subnetNodeId"
   
    # Get Control plane Identity Id
    $aksControlPlaneIdentityId =$(az identity show --name $aksControlPlaneIdentity --resource-group $managedIdentitiesResourceGroup --query "id" -o tsv)
    write-host "Aks Control Plane identity Principal Id : $aksControlPlaneIdentityId"
   
    # Get kubelet identity Id
    $aksKubeletIdentityId =$(az identity show --name $aksKubeletIdentity --resource-group $managedIdentitiesResourceGroup --query "id" -o tsv)
    write-host "Aks Control Plane identity Principal Id : $aksKubeletIdentityId"
   
   

2. Create cluster With User Managed identities

   Source : https://learn.microsoft.com/en-us/cli/azure/aks?view=azure-cli-latest#az-aks-create

• Azure-Cli
• Ansible

• az aks create `
      --resource-group $aksresourceGroup `
      --name $aksName `
      --kubernetes-version 1.24.9 `
      --node-resource-group $aksNodesResourceGroup `
      --node-count 1 `
      --generate-ssh-keys `
      --attach-acr $acrName `
      --load-balancer-sku Standard `
      --network-plugin azure `
      --vnet-subnet-id $subnetNodeId `
      --service-cidr $servicesSubnetAddressprefix `
      --dns-service-ip 10.240.4.2 `
      --enable-managed-identity `
      --assign-identity $aksControlPlaneIdentityId `
      --assign-kubelet-identity $aksKubeletIdentityId
  
  

• Create AKS Cluster with User Managed Identities Not Yet Supported

  Ansible module azure.azcollection.azure_rm_aks only supports System Managed Identity or Service Principal.

Create Aks with system managed identities

Hands-on lab

Get Resources dependencies Id

Before creating the cluster, It is mandatory to retrieve some dependencies resources id.

• Azure-Cli
• Ansible

• # Get subnet id
  $subnetNodeId = $(az network vnet subnet show -g $aksresourceGroup --vnet-name $vnetName -n $nodesSubnetName --query "id" -o tsv)
  write-host "Subnet node Id : $subnetNodeId"
  
  

• # Retrieve subnet info to retrieve its id
  - name: Get facts of specific subnet
      azure.azcollection.azure_rm_subnet_info:
      resource_group:       '{{ resource_group }}'
      virtual_network_name: '{{ virtual_network_name }}'
      name:                 '{{ subnet_name }}'
      register: _subnet_info
  
  

Create cluster With System Managed identities

• Azure-Cli
• Ansible

• az aks create `
      --resource-group $aksresourceGroup `
      --name $aksName `
      --kubernetes-version 1.24.9 `
      --node-resource-group $aksNodesResourceGroup `
      --node-count 1 `
      --generate-ssh-keys `
      --attach-acr $acrName `
      --load-balancer-sku Standard `
      --network-plugin azure `
      --vnet-subnet-id $subnetNodeId `
      --service-cidr $servicesSubnetAddressprefix `
      --dns-service-ip 10.240.4.2 `
      --enable-managed-identity
  
  

• - name: Create a managed Azure Container Services (AKS) cluster
      azure.azcollection.azure_rm_aks:
      name:               '{{ cluster_name }}'
      location:           '{{ location }}'
      resource_group:     '{{ resource_group }}'
      dns_prefix:         '{{ cluster_name }}'
      kubernetes_version: "{{ _aks_versions_info.azure_aks_versions[-1] }}"
  
      linux_profile:
        admin_username: "{{ username }}"
        ssh_key:        "{{ ssh_key }}"
  
      agent_pool_profiles:
          - name: default
            count: 1
            vm_size: Standard_D2_v2
            vnet_subnet_id: '{{ _subnet_info.subnets[0].id }}'
            mode: System
      node_resource_group: '{{ nodes_resource_group }}'
      enable_rbac: yes
      network_profile:
        load_balancer_sku: standard
        network_plugin: azure
  
      tags: '{{ cluster_tags | default({}) }}'
  
  

Apply permissions for cluster System Managed Identity

For Proof of concept, You can choose to let all resources in the cluster nodes resources groups and no permissions grant are needed.

The Hands on lab is just to show how retrieve the cluster identity to apply permission if it is required for the Proof of Concept.

Hands-on lab

# Get Aks Identity
$aksIdentity = $(az aks show --resource-group $aksresourceGroup --name $aksName --query "identity.principalId" -o tsv)
if ($null -eq $aksIdentity) { throw "Unable to retrieve aks $aksName identity in resource group $resourceGroup"}
write-host "Aks identity : $aksIdentity"

# Get resource group Id
$hubResourceGroupId = $(az group show -n $hubResourceGroup --query "id" -o tsv)
if ($null -eq $hubResourceGroupId) { throw "Unable to retrieve hub resource group $hubResourceGroup Id"}
write-host "Hub Resource group Id : $hubResourceGroupId"
$hubResourceGroupId

# Assign network contributor to AKS Identity on resource group Hub
az role assignment list --scope $hubResourceGroupId
az role assignment create --assignee $aksIdentity --scope $hubResourceGroupId --role "Network Contributor"

Connect to AKS Cluster

# Get cluster configuration
az aks get-credentials --name $aksName --resource-group $aksresourceGroup --overwrite-existing 

# Check if access is well configured
kubectl get nodes

# Check if ACR access is well configured
az aks check-acr --resource-group $aksresourceGroup --name $aksName --acr $acrName

Sources

• Use Managed Identities
• Use Service Principal

Notes