Daniel Eyer

February 03, 2020

Using AWS STS on Windows to populate Terraform credentials

When using the AWS provider in Terraform, one of the methods of authentication is the shared credentials file. The notion of keeping a long-term AWS Access key and Secret key in here is unsettling, but it’s a great place to keep a short-term token issued by the AWS Security Token Service (STS). There are posts about automating the population of tokens with Linux and with JQ but when supporting clients in a vanilla Windows environment, a little old-fashioned Windows scripting comes to the rescue.

The following bat file uses AWS CLI to issue an assume-role command to obtain a short-term token for authentication and parses the JSON response for the elements we need. It uses those elements to populate a shared credential file that we specify in our provider.tf file.

The following assumptions are made:

Your AWS CLI installation is in the default location and your default credentials have authority to (hopefully, ONLY) run sts assume-role.
This script is run from the same directory that contains your terraform.tfvars file. It should contain variables with values for the profile name you’d like to use, the target AWS account ID and (in this case) the AWS partition (aws> for commercial and aws-us-gov for GovCloud).
Your target AWS account has an IAM role named TerraformAdmin that has all the permissions you need to provision infrastructure and a trust policy that allows you to assume it.

Bat file:

										set AWS_DEFAULT_PROFILE=default
										set AWS_CONFIG_FILE=%USERPROFILE%\.aws\config
										set AWS_CREDENTIAL_FILE=%USERPROFILE%\.aws\credentials
										REM echo %AWS_CONFIG_FILE%
										echo Checking for terraform.tfvars in the current directory.
										IF NOT EXIST terraform.tfvars (
										echo No terraform.tfvars file present. Make sure you are in the right directory.
										pause
										exit
										)

										for /f "tokens=1,3" %%a in ('type terraform.tfvars ^| find "aws_profile"') do (
										IF /I "%%a" == "aws_profile" set AWS_PROFILE=%%b
										)
										for /f "tokens=1,3" %%a in ('type terraform.tfvars ^| find "aws_account_id"') do (
										IF /I "%%a" == "aws_account_id" set AWS_ACCOUNT_ID=%%b
										)
										for /f "tokens=1,3" %%a in ('type terraform.tfvars ^| find "aws_partition"') do (
										IF /I "%%a" == "aws_partition" set AWS_PARTITION=%%b
										)

										SET ADMIN_ROLE_ARN_ON_TARGET=arn:%AWS_PARTITION:"=%:iam::%AWS_ACCOUNT_ID%:role/TerraformAdmin
										SET AWS_PROFILE=%AWS_PROFILE:"=%
										REM Now assuming role and writing to file.

										echo Now running: aws sts assume-role --role-arn %ADMIN_ROLE_ARN_ON_TARGET% --role-session-name %username% --output json
										aws sts assume-role --role-arn %ADMIN_ROLE_ARN_ON_TARGET% --role-session-name %username% --output json > "%USERPROFILE%\.aws\tf-shared-creds.txt"

										echo Setting shared credentials file...
										SET AWS_SHARED_CREDENTIALS_FILE=%USERPROFILE%\.aws\tf-shared-creds.txt

										for /f "tokens=1,2 delims={},: " %%a in ('type %USERPROFILE%\.aws\tf-shared-creds.txt ^| find "AccessKeyId"') do (
										IF /I %%a=="AccessKeyId" set STS_AWS_ACCESS_KEY_ID=%%b
										)
										for /f "tokens=1,2 delims={},: " %%a in ('type %USERPROFILE%\.aws\tf-shared-creds.txt ^| find "SecretAccessKey"') do (
										IF /I %%a=="SecretAccessKey" set STS_AWS_SECRET_ACCESS_KEY=%%b
										)
										for /f "tokens=1,2 delims={},: " %%a in ('type %USERPROFILE%\.aws\tf-shared-creds.txt ^| find "SessionToken"') do (
										IF %%a=="SessionToken" set STS_AWS_SESSION_TOKEN=%%b
										)
										for /f "tokens=1,2,3,4 delims=T:Z " %%a in ('type %USERPROFILE%\.aws\tf-shared-creds.txt ^| find "Expiration"') do (
										IF /I %%a=="Expiration" set STS_EXPIRATION="%%c:%%d% UTC"
										)

										echo [%AWS_PROFILE%] > %AWS_SHARED_CREDENTIALS_FILE%
										echo aws_access_key_id = %STS_AWS_ACCESS_KEY_ID:"=% >> %AWS_SHARED_CREDENTIALS_FILE%
										echo aws_secret_access_key = %STS_AWS_SECRET_ACCESS_KEY:"=% >> %AWS_SHARED_CREDENTIALS_FILE%
										echo aws_session_token = %STS_AWS_SESSION_TOKEN:"=% >> %AWS_SHARED_CREDENTIALS_FILE%
										echo last_updated = %date% >> %AWS_SHARED_CREDENTIALS_FILE%

										echo Your STS Token for the %AWS_PROFILE% account expires at %STS_EXPIRATION:"=%
										echo You may now run Terraform against the target account.

October 02, 2018

AWS SSO from Commercial to GovCloud

AWS Single Sign-On (SSO) is integrated with AWS Organizations to allow quick and easy access to cross-account IAM roles to manage all of your AWS accounts with ease. This simplifies cross-account management and relieves password fatigue for users, however this functionality hasn’t been extended from commercial accounts to GovCloud accounts. GovCloud will allow you to assume a role with SAML, so with a little heavy lifting on the admin side, you can add GovCloud cross-account access from your commercial AWS SSO account.

My last post discussed how to add MFA to SSO and assumed that we were using an AWS AD Connector to connect to our AWS Managed Microsoft AD directory. We’ll keep that same assumption for this scenario, although your AD Connector could be reaching out to virtually any identity store. One caveat: You’ll need to create a custom application for each GovCloud account/role you’re trying to access, if you’re using AWS Managed Microsoft AD. You could get a little creative if you’re connecting to a true Active Directory store, but for now, let’s not get too creative.

Let's get building!

We'll start with the following steps:

First, you’ll need to download the GovCloud SAML Metadata file: https://signin.amazonaws-us-gov.com/static/saml-metadata.xml
Now, go to your AWS SSO page and click on Applications. Click Add a new application to get started and click on Add a custom SAML 2.0 application on the following page.
Provide a name for the application — it would be good to name it after your GovCloud account and the role being used. At the bottom of the page, you’ll be asked to provide an Application SAML metadata file. This should be the file you downloaded in Step 1. Click Save changes and you’ll be taken to your complete application.
Download the application metadata file by clicking the Download metadata file link. Save this file and open it in a text editor.
You’ll see a line near the bottom that looks like . Let’s change that line to look like this: urn:oasis:names:tc:SAML:1.1:nameid-format:unspecified
Save your file and let’s head over to GovCloud. Go to the IAM section of your GovCloud account and let’s click on Identity providers, where we will create a new provider (IdP). Select SAML as your provider type, give it whatever name you prefer and provide the metadata document that you just edited that was created with your custom application.
Create an IAM Role while you’re here, if you need to. Make sure that SAML is the trusted identity for the role and provide whatever permissions are necessary.
Now, we get to mapping attributes. This part can differ depending on your directory, but however you map them, SSO can only support the ones listed here. Make sure your directory has attributes mapped and let’s create some SAML attributes for the custom application we’ve created.
Click on the custom SAML application you’ve made and click on the Attribute mappings tab. We want our attributes to look like this:

User attribute in the application	Maps to this string value or user attribute in AWS SSO	Format
Subject	${user:subject}	unspecified
NameID	${user:subject}	unspecified
https://aws.amazon.com/SAML/Attributes/RoleSessionName	${user:subject}	unspecified
https://aws.amazon.com/SAML/Attributes/Role	SEE NOTE	uri

For the final entry, instead of SEE NOTE, you’ll want to add the ARN of the IdP you made in GovCloud followed by a comma and the ARN of the GovCloud role you want to assume. So, the value of that field may look like: arn:aws-us-gov:iam::12345678912:saml-provider/IdpSSO,arn:aws-us-gov:iam::12345678912:role/MySSORole
Now, go to the Assign users tab to provide access to this application to the users and groups necessary.

That should do it! When you access your SSO page now, you’ll see a new icon named after your custom application. Click to assume your new role.

October 01, 2018

Connect AWS SSO to Duo MFA

Amazon's Single Sign-On (SSO) service allows you to connect your directory / user store to several applications and AWS accounts. The benefits are pretty obvious as you start to pile on lots of accounts and apps, but the AWS documentation on configuration is a little vague, so I thought I'd throw this out there for whoever it might help.

My MFA solution is Duo. I often use Google Authenticator, but one of the requirements for MFA in my situation is to run a RADIUS server. AWS SSO can be configured in many ways, but for the purposes of this article, I’m using an AWS AD Connector to connect to AWS Managed Microsoft AD. I’ll assume that:

You know your AD.
You are mildly familiar with Duo.
You have established network connectivity from your AWS SSO account to your AD user store (whether through VPC peering or Direct Connect, etc.).
You have created a secret key to be used between RADIUS and SSO.

Let's get right to it!

AWS CONFIG

View the AD Connector that is associated with your SSO service. Take note of the AD Connector IP addresses. You need to make sure that these IP addresses can reach your RADIUS / Duo server.

Find or create the Security Group that allows UDP 1812 access to your RADIUS / Duo server. Add an Inbound Rule that allows UDP 1812 from your AD Connector. Create another rule for the second AD Connector. Make sure this Security Group is applied to the EC2 instance running your RADIUS / Duo server. Take note of the private IP address of that server because you’ll need it soon.

Now that your AD Connectors are allowed to speak to the RADIUS / Duo server, let’s take a look at the Duo config.

DUO CONFIG

Because AWS SSO is only looking for “it worked” or “it didn’t work” response from RADIUS, we don’t have to worry about Duo carrying any of the authentication payload that a Duo authentication proxy might normally carry. This mode is called “Duo only” or it should be. Further information can be found on Duo’s site and they do a fantastic job with their documentation. Our authproxy.cfg will look something like this:

											[duo_only_client]
												[radius_server_duo_only]
												ikey=WhateverYourIntegrationKeyIs
												skey=WhateverYourSecretKeyIs
												api_host=api-yourcodehere.duosecurity.com
												radius_ip_1=AD Connector #1 IP address
												radius_ip_2=AD Connector #2 IP address
												radius_secret_1=YourSecretKey
												radius_secret_2=YourSecretKey
												port=1812
												client=duo_only_client

BACK TO AWS CONFIG

Now, we’re ready to enable MFA. Find the Multi-factor authentication section of your directory and enable it. The settings you’ll need to provide are:

IP address of the RADIUS / Duo server
Port 1812
The secret key you have created for your connection
Select the PAP protocol

Save your changes when you’re ready and you should be able to view the registration activity in the authproxy.log file on your RADIUS / Duo server.

Happy Hunting!

July 15, 2016

Adding aws-cfn-bootstrap (CloudFormation helper scripts) to custom Linux AMIs

As more and more of my CloudFormation (CF) stacks use a base image and CloudFormation::Init magic, it’s become imperative to have an AMI that has the helper scripts (cfn-signal, cfn-init, etc.) built-in. This isn’t a problem if you use the Amazon Linux AMI, but if you’re playing with things like immutable infrastructure or baking your own custom AMIs for CIS hardening or some other regulatory requirement, it can become a big issue quickly. There’s a little documentation out there on installing the CF helper scripts (https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/cfn-helper-scripts-reference.html) but the installation process is not quite so straightforward as one would hope.

The solution to this issue varies depending on your OS. I’ve had no issue with Windows AMIs because the Ec2Config service takes care of everything, but in CentOS and RHEL, there are a few extra steps. I’ll break them down by OS. Note that you may need to search for updated version of things like epel-release to make sure it matches your OS or you’re using the most current version.

CentOS 6

This was relatively painless, thanks to the contents of the cloudformation-examples bucket being publicly visible. The latest version of the helper-scripts requires some Python elements/versions that are a pain to set up, but you can use an older version of the helper scripts without any issues. As of CentOS 6.8, you can use aws-cfn-bootstrap-1.4-5.amzn1.noarch.rpm without having to juggle any prerequisites.

CentOS 7

Not as easy. The following code will download the latest helper scripts, unpack them, build them and create symbolic links in the directory in which AWS expects them to exist (/opt/aws/bin).

										
											cd /opt
											curl -O https://s3.amazonaws.com/cloudformation-examples/aws-cfn-bootstrap-latest.tar.gz
											tar -xvpf aws-cfn-bootstrap-latest.tar.gz
											cd aws-cfn-bootstrap-1.4/
											python setup.py build
											python setup.py install
											ln -s /usr/init/redhat/cfn-hup /etc/init.d/cfn-hup
											chmod 775 /usr/init/redhat/cfn-hup
											cd /opt
											mkdir aws
											cd aws
											mkdir bin
											ln -s /usr/bin/cfn-hup /opt/aws/bin/cfn-hup
											ln -s /usr/bin/cfn-init /opt/aws/bin/cfn-init
											ln -s /usr/bin/cfn-signal /opt/aws/bin/cfn-signal
											ln -s /usr/bin/cfn-elect-cmd-leader /opt/aws/bin/cfn-elect-cmd-leader
											ln -s /usr/bin/cfn-get-metadata /opt/aws/bin/cfn-get-metadata
											ln -s /usr/bin/cfn-send-cmd-event /opt/aws/bin/cfn-send-cmd-event
											ln -s /usr/bin/cfn-send-cmd-result /opt/aws/bin/cfn-send-cmd-result

RHEL 6

RedHat 6 uses the same version of aws-cfn-bootstrap as CentOS 6, but it gets there in a different way.

										
											rpm -ivh http://dl.fedoraproject.org/pub/epel/6/x86_64/epel-release-6-8.noarch.rpm
											yum install -y python-pip
											cd /usr/bind
											pip install pystache
											pip install argparse
											pip install python-daemon
											pip install requests
											yum install -y https://s3.amazonaws.com/cloudformation-examples/aws-cfn-bootstrap-1.4-5.amzn1.noarch.rpm

First, we have to download epel-release to download python-pip so we can build some dependencies. After installing the dependencies, we download the 1.4-5 version of the scripts.

RHEL 7

The hardest of them all! Or the one that requires the most steps. We have to download an epel-release, install pip, install python packages, download the scripts and create symbolic links.

										
											rpm -ivh http://dl.fedoraproject.org/pub/epel/7/x86_64/e/epel-release-7-7.noarch.rpm
											yum install -y python-pip
											cd /usr/bin
											pip install pystache
											pip install argparse
											pip install python-daemon
											pip install requests
											cd /opt
											curl -O https://s3.amazonaws.com/cloudformation-examples/aws-cfn-bootstrap-latest.tar.gz
											tar -xvpf aws-cfn-bootstrap-latest.tar.gz
											cd aws-cfn-bootstrap-1.4/
											python setup.py build
											python setup.py install
											ln -s /usr/init/redhat/cfn-hup /etc/init.d/cfn-hup
											chmod 775 /usr/init/redhat/cfn-hup
											cd /opt
											mkdir aws
											cd aws
											mkdir bin
											ln -s /usr/bin/cfn-hup /opt/aws/bin/cfn-hup
											ln -s /usr/bin/cfn-init /opt/aws/bin/cfn-init
											ln -s /usr/bin/cfn-signal /opt/aws/bin/cfn-signal
											ln -s /usr/bin/cfn-elect-cmd-leader /opt/aws/bin/cfn-elect-cmd-leader
											ln -s /usr/bin/cfn-get-metadata /opt/aws/bin/cfn-get-metadata
											ln -s /usr/bin/cfn-send-cmd-event /opt/aws/bin/cfn-send-cmd-event
											ln -s /usr/bin/cfn-send-cmd-result /opt/aws/bin/cfn-send-cmd-result

Voila! Your custom AMI can now leverage CloudFormation::Init instructions.

Create Refine Repeat

Create
Refine
Repeat