Tim Burns
Terraform and Snowflake Storage Integrations
Updated: 6 days ago
Terraform is an infrastructure scripting tool many DevOps teams use, and it pays to understand how to use it. Some Background articles:
Key Pair Authorization (Snowflake)
Snowflake Labs Terraform (Snowflake Labs)
A few minor improvements are in my "Big Data Github Project."
Comments on the User Creation for Terraform
Use the role to manage the permissions rather than grant them directly to the user.
Some quick call-outs are in order. First, we must pull the environment variable from the OpenSSL public and private keys.
export TF_PUBLIC_KEY=$(awk 'NR>1{a[++k]=$0}END{for(i=1;i<k;i++)printf "%s",a[i]}' snowflake_tf_snow_key.pub)
export TF_PASSWORD=$(awk 'NR>1{a[++k]=$0}END{for(i=1;i<k;i++)printf "%s",a[i]}' ~/.snowsql/snowflake_tf_snow_key.p8)
export TF_USER=tf-snow
When we create the user, use SnowSQL and replace the passwords programmatically.
snowsql --connection ${CONNECTION} \
--rolename ${DATABASE_ADMIN_ROLE} \
-D TF_USER=${TF_USER} \
-D TF_PASSWORD=${TF_PASSWORD} \
-D TF_PUBLIC_KEY=${TF_PUBLIC_KEY} \
-f src/snowflake/terraform_create_user.sql
Now we have the Terraform user available with Key Pair Authorization.
Terraform
Terraform is a popular DevOps tool for scripting the deployment of cloud services.
The Makefile in my project outlines the steps to deploy a role in Terraform that we will then use to create a STORAGE_INTEGRATION in Snowflake.
Here is what you need to jumpstart Terraform.
tf-install:
brew tap hashicorp/tap
brew install hashicorp/tap/terraform
tf-init:
terraform -chdir="./src/terraform" init
tf-plan:
terraform -chdir="./src/terraform" plan -var "storage_integration_name=${S3_TF_STORAGE_INTEGRATION}"
tf-validate:
terraform -chdir="./src/terraform" validate
tf-apply:
terraform -chdir="./src/terraform" apply -var "storage_integration_name=${S3_TF_STORAGE_INTEGRATION}" -auto-approve
The Terraform components are here: storage_integration/src/terraform.
A snippet illustrating how Snowflake Storage Integrations AWS Principal gets added into a trusted role on Terraform.
resource "aws_iam_role" "snowflake-storage-integration" {
assume_role_policy = jsonencode({
Version = "2012-10-17"
Statement = [
{
Action = "sts:AssumeRole"
Effect = "Allow"
Principal = {
AWS = "${var.snowflake_storage_aws_iam_user_arn}"
}
Condition = {
StringEquals = {
"sts:ExternalId" = "${var.snowflake_storage_aws_external_id}"
}
}
}
]
})
name = "snowflake-storage-integration"
}