Service mesh integration with Application Gateway for Containers

Service meshes handle east-west traffic between services and often use mutual TLS (mTLS) for security. When services need external access (north-south traffic), an ingress gateway simplifies connectivity. Application Gateway for Containers provides a service mesh extension that automates certificate lifecycle management and reduces configuration complexity.

While defining mTLS in ingress is possible, definition of mTLS configuration can become redundant and difficult to manage during certificate rotation.

To simplify communication from outside the Kubernetes cluster to the service mesh, Application Gateway for Containers has an optional service mesh extension that can be installed alongside ALB Controller. This extension can greatly decrease operational overhead by automating certificate lifecycle management and simplify ingress configuration.

Here's a diagram of Application Gateway for Containers integrating with Istio service mesh.

ALB Controller Service Mesh Extension

The ALB Controller Istio Extension consists of two pods, deployed in active / standby configuration to allow resiliency during node failure, handle certificate lifecycle management between Application Gateway for Containers and Istio, and implicitly handle mTLS configuration to the services participating in the service mesh.

Install the ALB Controller Service Mesh Extension for Istio

Install the ALB Controller Service Mesh Extension using the Helm chart after provisioning ALB Controller (Add-on or Helm). The ALB Controller Service mesh Extension is installed using a separate helm chart.

The following helm command can be used to install the extension.

HELM_NAMESPACE='<namespace for deployment>'
CONTROLLER_NAMESPACE='azure-alb-system'
RESOURCE_GROUP='<your resource group name>'
AKS_NAME='<your aks cluster name>'

az aks get-credentials --resource-group $RESOURCE_GROUP --name $AKS_NAME

helm install alb-controller-servicemesh-extension oci://mcr.microsoft.com/application-lb/charts/alb-controller-servicemesh-extension \
     --namespace $HELM_NAMESPACE \
     --version 1.10.28

Verify the ALB Controller installation

Verify the ALB Controller Service Extension pods are ready as prefixed by alb-controller-istio-extension.

kubectl get pods -n azure-alb-system

You should see the following output:

Integration with Istio

Application Gateway for Containers supports integration with Istio v1.24 and greater.

After the ALB Controller Service Mesh Extension is installed, if Istio is installed, the service mesh integration will attempt to provision a sidecar for integration into the mesh.

If ALB Controller Service Mesh is installed prior to Istio, you must restart the ALB Controller Service Mesh extension deployment. Restart of the ALB Controller Service Mesh deployment can be done with the command: kubectl rollout restart deployment/alb-controller-istio-extension

Routing traffic to Istio service mesh

Follow these five steps to configure Istio service mesh with Application Gateway for Containers:

• Define a namespace for istio sidecar injection
• Enable mTLS for the services in that namespace
• Deploy a sample application for testing (optional)
• Define a Gateway resource
• Define an HTTPRoute resource

Define the namespace

Depending on how Istio is installed, you'll need to configure the label used to enable Istio sidecar injection.

apiVersion: v1
kind: Namespace
metadata:
  name: test-infra
  labels:
    istio-injection: enabled

kubectl get configmap -n aks-istio-system | grep asm-

istio-asm-1-27                        2      90m
istio-sidecar-injector-asm-1-27       2      90m

az aks show --resource-group <rg> --name <cluster> --query "serviceMeshProfile"

{
  "istio": {
    "certificateAuthority": null,
    "components": {
      "egressGateways": null,
      "ingressGateways": [
        {
          "enabled": null,
          "mode": "Internal"
        },
        {
          "enabled": null,
          "mode": "External"
        }
      ]
    },
    "revisions": [
      "asm-1-27"
    ]
  },
  "mode": "Istio"
}

apiVersion: v1
kind: Namespace
metadata:
  name: test-infra
  labels:
    istio.io/rev: asm-1-27 # Istio version obtained previously

Enable mTLS handling for services in the namespace

In this example, we define a PeerAuthentication Istio custom resource to require mTLS for incoming connections to the services in the test-infra namespace.

apiVersion: security.istio.io/v1beta1
kind: PeerAuthentication
metadata:
  name: default
  namespace: test-infra
spec:
  mtls:
    mode: STRICT

Deploy a sample application (optional)

In this step, we provision some test resources so we can validate the end-to-end scenario. If you have your own application, you can skip this step.

Apply the following deployment.yaml file on your cluster to create a sample web application to demonstrate sending traffic to an application part of the Istio service mesh.

kubectl apply -f https://raw.githubusercontent.com/MicrosoftDocs/azure-docs/refs/heads/main/articles/application-gateway/for-containers/examples/traffic-split-scenario/deployment.yaml

This command creates the following on your cluster:

• A namespace called test-infra
• Two services called backend-v1 and backend-v2 in the test-infra namespace
• Two deployments called backend-v1 and backend-v2 in the test-infra namespace

Define Gateway resource

kubectl apply -f - <<EOF
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
  name: gateway-01
  namespace: test-infra
  annotations:
    alb.networking.azure.io/alb-namespace: alb-test-infra
    alb.networking.azure.io/alb-name: alb-test
spec:
  gatewayClassName: azure-alb-external
  listeners:
  - name: http
    port: 80
    protocol: HTTP
    allowedRoutes:
      namespaces:
        from: All
EOF

Once the gateway resource is created, ensure the status is valid, the listener is Programmed, and an address is assigned to the gateway.

kubectl get gateway gateway-01 -n test-infra -o yaml

Example output of successful gateway creation.

status:
  addresses:
  - type: Hostname
    value: xxxx.yyyy.alb.azure.com
  conditions:
  - lastTransitionTime: "2023-06-19T21:04:55Z"
    message: Valid Gateway
    observedGeneration: 1
    reason: Accepted
    status: "True"
    type: Accepted
  - lastTransitionTime: "2023-06-19T21:04:55Z"
    message: Application Gateway For Containers resource has been successfully updated.
    observedGeneration: 1
    reason: Programmed
    status: "True"
    type: Programmed
  listeners:
  - attachedRoutes: 0
    conditions:
    - lastTransitionTime: "2023-06-19T21:04:55Z"
      message: ""
      observedGeneration: 1
      reason: ResolvedRefs
      status: "True"
      type: ResolvedRefs
    - lastTransitionTime: "2023-06-19T21:04:55Z"
      message: Listener is accepted
      observedGeneration: 1
      reason: Accepted
      status: "True"
      type: Accepted
    - lastTransitionTime: "2023-06-19T21:04:55Z"
      message: Application Gateway For Containers resource has been successfully updated.
      observedGeneration: 1
      reason: Programmed
      status: "True"
      type: Programmed
    name: gateway-01-http
    supportedKinds:
    - group: gateway.networking.k8s.io
      kind: HTTPRoute

Define HTTPRoute resource

Once the gateway is created, create an HTTPRoute

kubectl apply -f - <<EOF
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: traffic-split-mesh-route
  namespace: test-infra
spec:
  parentRefs:
  - name: gateway-01
    namespace: test-infra
  rules:
  - backendRefs:
    - name: backend-v1
      port: 8080
EOF

Once the HTTPRoute resource is created, ensure the route is Accepted and the Application Gateway for Containers resource is Programmed.

kubectl get httproute traffic-split-mesh-route -n test-infra -o yaml

Verify the status of the Application Gateway for Containers resource has been successfully updated.

status:
  parents:
  - conditions:
    - lastTransitionTime: "2023-06-19T22:18:23Z"
      message: ""
      observedGeneration: 1
      reason: ResolvedRefs
      status: "True"
      type: ResolvedRefs
    - lastTransitionTime: "2023-06-19T22:18:23Z"
      message: Route is Accepted
      observedGeneration: 1
      reason: Accepted
      status: "True"
      type: Accepted
    - lastTransitionTime: "2023-06-19T22:18:23Z"
      message: Application Gateway For Containers resource has been successfully updated.
      observedGeneration: 1
      reason: Programmed
      status: "True"
      type: Programmed
    controllerName: alb.networking.azure.io/alb-controller
    parentRef:
      group: gateway.networking.k8s.io
      kind: Gateway
      name: gateway-01
      namespace: test-infra

Test access to the application

Now we're ready to send some traffic to our sample application, via the FQDN assigned to the frontend. Use the following command to get the FQDN:

fqdn=$(kubectl get gateway gateway-01 -n test-infra -o jsonpath='{.status.addresses[0].value}')

Curling this FQDN should return responses from the backends/pods as configured on the HTTPRoute.

curl http://$fqdn

Next, let's delete the PeerAuthentication resource to simulate removal of mTLS.

kubectl delete PeerAuthentication default -n test-infra

When you run the curl command again, you should see a 503 HTTP response code, indicating failure of mTLS negotiation between Application Gateway for Containers and the test application (as the test application is no longer expecting mutual authentication).

Pricing

There's no extra price to enable service mesh integration.

Limitations

• Ingress API isn't supported; consider migrating to Gateway API.