ZPA App Connector Deployment in AWS Using Terraform

Automating the deployment of ZPA App Connectors have become a lot easier recently, with the introduction of new API endpoints, where you can now automate the creation of both App Connector Groups and Provisioning Keys. As some of you are aware, I have recently released an Unofficial ZPA Terraform Provider, which have been updated to support these new API resources. The provider allows for easy and fast deployment of ZPA App Connectors across multiple clouds, since Terraform is agnostic.

Deploying a ZPA connector involves a few tasks, which must be accomplished securely and in a specific order. When deploying the App Connector in AWS, “placing the provisioning key in the user-data may be considered a security risk, and manually copying/pasting is not scalable. Placing the Provisioning Key in a CloudFormation template, or copying a text file are equally insecure.” In this article, I will describe how to use the newly updated ZPA provider in conjunction with AWS Terraform provider to automate and secure the end to end deployment of App Connectors in AWS cloud.

:warning: Attention: As a reminder, the ZPA Terraform Provider, is not affiliated with, nor supported by Zscaler in any way, and is being offered as a community effort AS_IS.

The ZPA Terraform Provider, will perform the following tasks:

  1. Create an App Connector Group
  2. Create a Provisioning Key
  3. Retrieve the Certificate Enrolment ID type.

App Connector Group
The first task is to create the App Connector Group. The below configuration provides the detailed steps of the resource setup.

// Create an the App Connector Group
resource "zpa_app_connector_group" "aws_app_connector_group" {
  name                     = "USA  App Connector Group"
  description              = "USA  App Connector Group"
  enabled                  = true
  city_country             = "San Jose, CA"
  country_code             = "CA"
  latitude                 = "37.3382082"
  longitude                = "-121.8863286"
  location                 = "San Jose, CA, USA"
  upgrade_day              = "SUNDAY"
  upgrade_time_in_secs     = "66600"
  override_version_profile = true
  version_profile_id       = 0
  dns_query_type           = "IPV4_IPV6"
}

Provisioning Key
The following configuration will create a provisioning key in the ZPA portal, and associate it with the previously created App Connector Group and Enrolment Certificate ID type.

// Create Provisioning Key for the App Connector Group
resource "zpa_provisioning_key" "aws_usa_provisioning_key" {
  name                  = "AWS Provisioning Key"
  association_type      = "CONNECTOR_GRP"
  max_usage             = "10"
  enrollment_cert_id    = data.zpa_enrolment_cert.connector.id
  zcomponent_id         = zpa_app_connector_group.aws_app_connector_group.id
}

Retrieve the ZPA Connector Enrolment Cert ID
Finally, we must retrieve the ID of the enrolment cert ID, which tells ZPA, what type of provisioning key is being generated.

data "zpa_enrolment_cert" "connector" {
    name = "Connector"
}

Note: The same process above can be used to onboard a Service Edge Group by using resource "zpa_service_edge_group"

In this next part, we will use the AWS Terraform Provider to perform the heavy lift.

  1. Create an IAM Policy Document
  2. Create an IAM Policy
  3. Create an AWS IAM Role
  4. Create an Instance Profile
  5. Create an AWS KMS Key
  6. Create a SSM parameter containing the provisioning key
  7. Start an AMI with the IAM role
  8. Securely retrieve Provisioning Key to AMI

Note: In Terraform, the steps 1 to 3 are considered separate AWS resources, which then need to be attached to each other using a policy attachment resource. To keep this article short, I will only show the configuration steps 5-6. If you want to see the complete configuration, please visit my GitHub repository.

IAM Policy Document
This policy specification allows the IAM role to retrieve the provisioning key from both KMS and SSM parameter store.

data "aws_iam_policy_document" "zscaler_ssm_kms_policy" {
  statement {
    effect = "Allow"
    actions = [
      "kms:ListKeys",
      "tag:GetResources",
      "kms:ListAliases",
      "kms:DescribeKey"
    ]
    resources = ["*"]
  }
  statement {
    effect    = "Allow"
    actions   = ["ssm:GetParameter"]
    resources = ["arn:aws:ssm:*:*:parameter/ZSDEMO*"]
  }
}

data "aws_iam_policy_document" "app_connector_assume_role" {
  statement {
    actions = ["sts:AssumeRole"]
    effect  = "Allow"
    sid     = ""
    principals {
      type        = "Service"
      identifiers = ["ec2.amazonaws.com"]
    }
  }
}

Create an AWS KMS Key
The AWS KMS key will be used by the SSM parameter store to encrypt the provisioning key that was created by the ZPA Terraform Provider.

# Creates/manages KMS CMK
resource "aws_kms_key" "zscaler_ssm_kms" {
  description              = Zscaler_SSM_KMS
  customer_master_key_spec = SYMMETRIC_DEFAULT
  enable_key_rotation      = true
  multi_region             = true
}

# Add an alias to the key
resource "aws_kms_alias" "zscaler_kms_ssm_alias" {
  name          = "alias/Zscaler_KMS_SSM"
  target_key_id = aws_kms_key.zscaler_ssm_kms.key_id
}

Create a SSM Parameter Store

The SSM parameter store is where the ZPA Terraform Provider will store the provisioning key value. Because, we want this value to be stored securely, we are using Parameter Store “SecureString”.
The SecureString, essentially, encrypts sensitive data i.e (provisioning key) using the KMS keys we created previously.

Create Parameter Store

resource "aws_ssm_parameter" "zscaler_parameter_store" {
  name        = "Zscaler_Parameter_Store"
  description = "Zscaler Parameter Store"
  type        = "SecureString"
  value       = zpa_provisioning_key.aws_provisioning_key.provisioning_key
}

In the above HCL snippet, notice the line “value”. In this line, we are referencing the ZPA Terraform Provider provisioning_key resource/parameter, to store the provisioning key value to the parameter store, which in turn will use the KMS key to encrypt the provisioning key value.

Securely retrieve the Provisioning Key

To securely retrieve the provisioning key from parameter store, the below scripts have been adapted for this article. The ZPA App Connector provided by Zscaler does not have AWSTools installed by default. These will need to be installed first. Alternatively, launching an AWS Linux2 AMI and installing ZPA App Connector on top of it is an also an option.

Zscaler AMI
Start the AMI with the following user_data.

#!/usr/bin/bash
yum install unzip -y
curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o /tmp/awscli-exe-linux-x86_64.zip
unzip /tmp/awscli-exe-linux-x86_64.zip -d /tmp
/tmp/aws/install
REGION=$(curl http://169.254.169.254/latest/meta-data/placement/region)
URL="http://169.254.169.254/latest/meta-data/network/interfaces/macs/"
MAC=$(curl $URL)
URL=$URL$MAC"vpc-id/"
VPC=$(curl $URL)
key="ZSDEMO-"$REGION"-"$VPC

#Stop the App Connector service which was auto-started at boot time
sudo systemctl stop zpa-connector

# Create provisioning key file
sudo touch /opt/zscaler/var/provision_key
sudo chmod 644 /opt/zscaler/var/provision_key

# Retrieve and Decrypt Provisioning Key from Parameter Store
aws ssm get-parameter --name $key --query Parameter.Value --with-decryption --region $REGION | tr -d '"' > /opt/zscaler/var/provision_key

#Run a yum update to apply the latest patches
sudo yum update -y

#Start the App Connector service to enroll it in the ZPA cloud
sudo systemctl start zpa-connector
#Wait for the App Connector to download latest build
sleep 60
#Stop and then start the App Connector for the latest build
systemctl stop zpa-connector
systemctl start zpa-connector

AWS Linux AMI
If you’re using the default AWS Linux2 AMI, start the AMI with the following user_data.

#!/usr/bin/bash
sudo /etc/yum.repos.d/zscaler.repo -R
sudo cat > /etc/yum.repos.d/zscaler.repo <<-EOT
[zscaler]
name=Zscaler Private Access Repository
baseurl=https://yum.private.zscaler.com/yum/el7
enabled=1
gpgcheck=1
gpgkey=https://yum.private.zscaler.com/gpg
EOT

REGION=$(curl http://169.254.169.254/latest/meta-data/placement/region)
URL="http://169.254.169.254/latest/meta-data/network/interfaces/macs/"
MAC=$(curl $URL)
URL=$URL$MAC"vpc-id/"
VPC=$(curl $URL)
key="ZSDEMO-"$REGION"-"$VPC

sudo yum install zpa-connector -y
sudo yum update -y
sudo sleep 60
sudo systemctl stop zpa-connector
sudo touch /opt/zscaler/var/provision_key
sudo chmod 644 /opt/zscaler/var/provision_key
aws ssm get-parameter --name $key --query Parameter.Value --with-decryption --region $REGION | tr -d '"' > /opt/zscaler/var/provision_key
sudo systemctl start zpa-connector

For the complete Terraform template visit the following GitHub repository. Special thanks to Mr. @mryan for providing instructions for similar deployment using CloudFormation.

3 Likes