Terraform and Snowflake Storage Integrations

Terraform is an infrastructure scripting tool many DevOps teams use, and it pays to understand how to use it. Some Background articles:

• Key Pair Authorization (Snowflake)

• Snowflake Labs Terraform (Snowflake Labs)

• Securing the Snowflake Stored Integration on AWS (Burns)

A few minor improvements are in my "Big Data Github Project."

Comments on the User Creation for Terraform

Use the role to manage the permissions rather than grant them directly to the user.

Some quick call-outs are in order. First, we must pull the environment variable from the OpenSSL public and private keys.

export TF_PUBLIC_KEY=$(awk 'NR>1{a[++k]=$0}END{for(i=1;i<k;i++)printf "%s",a[i]}' snowflake_tf_snow_key.pub)

export TF_PASSWORD=$(awk 'NR>1{a[++k]=$0}END{for(i=1;i<k;i++)printf "%s",a[i]}' ~/.snowsql/snowflake_tf_snow_key.p8)
export TF_USER=tf-snow

When we create the user, use SnowSQL and replace the passwords programmatically.

snowsql --connection ${CONNECTION} \
      --rolename ${DATABASE_ADMIN_ROLE} \
      -D TF_USER=${TF_USER} \
      -D TF_PASSWORD=${TF_PASSWORD} \
      -D TF_PUBLIC_KEY=${TF_PUBLIC_KEY} \
      -f src/snowflake/terraform_create_user.sql

Now we have the Terraform user available with Key Pair Authorization.

Terraform

Terraform is a popular DevOps tool for scripting the deployment of cloud services.

The Makefile in my project outlines the steps to deploy a role in Terraform that we will then use to create a STORAGE_INTEGRATION in Snowflake.

Here is what you need to jumpstart Terraform.

tf-install:
   brew tap hashicorp/tap
   brew install hashicorp/tap/terraform

tf-init:
   terraform -chdir="./src/terraform"  init

tf-plan:
   terraform -chdir="./src/terraform" plan -var "storage_integration_name=${S3_TF_STORAGE_INTEGRATION}"

tf-validate:
   terraform -chdir="./src/terraform" validate

tf-apply:
   terraform -chdir="./src/terraform" apply -var "storage_integration_name=${S3_TF_STORAGE_INTEGRATION}" -auto-approve

The Terraform components are here: storage_integration/src/terraform.

A snippet illustrating how Snowflake Storage Integrations AWS Principal gets added into a trusted role on Terraform.

resource "aws_iam_role" "snowflake-storage-integration" {
  assume_role_policy = jsonencode({
    Version = "2012-10-17"
    Statement = [
      {
        Action = "sts:AssumeRole"
        Effect = "Allow"
        Principal = {
          AWS = "${var.snowflake_storage_aws_iam_user_arn}"
        }
        Condition = {
          StringEquals = {
            "sts:ExternalId" = "${var.snowflake_storage_aws_external_id}"
          }
        }
      }
    ]
  })
  name               = "snowflake-storage-integration"
}