Invoke Pester Tests Serverlessly with AWS Lambda and SSM

Pester and CI

If you’re doing Windows scripting in 2016, you’d better be using PowerShell. And if you’re writing PowerShell scripts, you’d better be checking them into source control and covering them with Pester tests.

It turns out that you can do more with Pester than just run tests manually at the console. As part of a continuous integration (CI) process, you may want to invoke Pester tests on a remote server and report the results up through the build chain. Handily, you can export Pester test output in an NUnit XML format that modern CI systems like Jenkins understand.

But what if you’re not using a build server to invoke Pester? What if your CI setup is … dun dun dun … “serverless”?

Invoking Pester tests with SSM from AWS Lambda

AWS Lambda, the original “function as a service” offering, allows you to define and execute small chunks of code  without managing any infrastructure. EC2’s newish Run Command feature lets you run arbitrary code on remote EC2 systems via API calls. When we combine these two features, we suddenly have a way to serverlessly invoke anything we want on an EC2 Windows box – even Pester tests.

Let’s assume we have some Pester tests saved in a folder called “C:\tests” on an EC2 Windows server. In order to execute these tests remotely without an additional build server, we’ll need to write a Lambda function that sends commands to our test server via Run Command. Unfortunately, Lambda functions don’t currently support PowerShell, so we’ll write our utility code in Python.

Let’s create a Python file, “InvokePesterTests.py”, as follows:


import boto3
import time
import json
 #The lambda_handler Python function gets called when you run your AWS Lambda function.
def lambda_handler(event, context):

    ssmClient = boto3.client('ssm')
    s3Client = boto3.client('s3')

    instanceId = event['id']
    bucket = event['bucket']
    keyPrefix = event['keyPrefix']

    ssmCommand = ssmClient.send_command(
        InstanceIds = [
            instanceId
        ],
        DocumentName = 'AWS-RunPowerShellScript',
        TimeoutSeconds = 240,
        Comment = 'Run Pester Tests',
        Parameters = {
            'commands': [
                "cd C:/tests",
                 "Invoke-Pester -OutputFile test.xml -OutputFormat NUnitXml",
                 "aws s3 cp test.xml s3://" + bucket + "/" + keyPrefix + "/test.xml"
            ]
        },
        OutputS3BucketName = bucket,
        OutputS3KeyPrefix = keyPrefix
    )
#poll SSM until EC2 Run Command completes
    status = 'Pending'
    while status == 'Pending' or status == 'InProgress':
        time.sleep(3)
        status = (ssmClient.list_commands(CommandId=ssmCommand['Command']['CommandId']))['Commands'][0]['Status']

    if(status != 'Success'):
        print "test failed with status " + status
        return

    xmlOutput = s3Client.get_object(
        Bucket = bucket,
        Key = keyPrefix + "/test.xml"
    )['Body'].read()

Now let’s talk through what’s happening in the code.

  1. The “event” parameter

The “event” parameter is a dictionary built from a JSON struct that contains user-defined input. In our case, we want three input values: the s3 bucket and key location (subfolder) where we’ll save our remote command output, and the unique “instance ID” assigned to the Windows server we want to test. Our JSON input to the function would look like this:

{
    "id": "i-12345678",
    "bucket": "myBucket",
    "keyPrefix": "tests/Pester"
}

 

Note: we don’t necessarily need to pass the EC2 instance identifier as a parameter to this function. If the instance is tagged with information that’s meaningful to us, we could run a “DescribeInstance” call to the EC2 API and look up the instance ID based on the tag name.

We will use boto, AWS’s SDK for Python, so we also need to set up boto clients here for EC2 Run Command and S3.

  1. The SSM “send command” call
  ssmCommand = ssmClient.send_command(
        InstanceIds = [
            instanceId
        ],
        DocumentName = 'AWS-RunPowerShellScript',
        TimeoutSeconds = 240,
        Comment = 'Run Pester Tests',
        Parameters = {
            'commands': [
                "cd C:/tests",
                 "Invoke-Pester -OutputFile test.xml -OutputFormat NUnitXml",
                 "aws s3 cp test.xml s3://" + bucket + "/" + keyPrefix + "/test.xml"
            ]
        },
        OutputS3BucketName = bucket,
        OutputS3KeyPrefix = keyPrefix
    )

Now that we know what server to talk to and where to send our output, we want to invoke our Pester tests via SSM. (SSM = Simple Systems Manager, the AWS configuration management solution that includes Run Command). SSM uses the EC2Config service running on your Windows server (and its credentials) to execute commands that you specify.

Note: Before using SSM, make sure that your EC2 instance is running at least version 3.10.442 of the EC2Config service.

The SSM send command needs a few parameters:

  • A list of instance IDs (we’ll specify the one we passed into our function)
  • The SSM document to use. SSM documents let you specify exactly what commands SSM is allowed to execute on your system. In this case, I chose to use the default “AWS-RunPowerShellScript” document that lets me run arbitrary PowerShell commands. In real life, you might want to define something a bit more restrictive.
  • The number of seconds to wait before giving up on a response from the server
  • Some comment about the command that’s meaningful to you
  • The s3 bucket name and key path to save the command output
  • The list of Powershell commands to execute

The commands to execute in our case are simple: we just want to run any Pester tests underneath our C:\tests folder, outputting the results as an XML file, and upload the XML to our specified S3 location. (The Invoke-Pester command makes this task really easy because it will look recursively under C:\tests for any files with the *.Tests.ps1 naming convention.)

  1. Response polling
#poll SSM until EC2 Run Command completes
    status = 'Pending'
    while status == 'Pending' or status == 'InProgress':
        time.sleep(3)
        status = (ssmClient.list_commands(CommandId=ssmCommand['Command']['CommandId']))['Commands'][0]['Status']

    if(status != 'Success'):
        print "test failed with status " + status
        return

SSM commands are run asynchronously: you have to keep checking the status of the command to see when it finishes. I chose to poll the “list_commands” API every three seconds. Of course, Lambda functions have a maximum runtime of five minutes, so you can’t poll forever; that’s why I set the SSM call timeout to four minutes. If your Pester tests take too long for Lambda to get the final SSM status using synchronous polling, you might need to set up an asynchronous trigger to fire a second function when a call completes.

  1. Get the output
 xmlOutput = s3Client.get_object(
        Bucket = bucket,
        Key = keyPrefix + "/test.xml"
    )['Body'].read()

SSM commands save two text files to your designated s3 bucket by default: one containing the standard output of your commands and another containing any error text. As part of our PowerShell code, we uploaded a third file to s3 containing the Pester output in XML format. The last line of the lambda function just streams the XML from s3 into a string variable. I stopped here, but now it’s easy to parse the XML using Python’s “xmltodict” library for any additional data you need.

Running the function

You can skip this section if you know how to create and run Lambda functions, but I’ll go ahead and include it here for the sake of completeness.

  1. In the AWS Lambda console, click “Create a Lambda Function”.
  2. Skip the “Select blueprint” page
  3. Skip the “Configure triggers” page
  4. Fill out the “Configure function” section as follows:Screen Shot 2016-07-30 at 7.15.44 PM.png
  5. Paste the full Python code into the “Lambda function code” section
  6. In the “Lambda function handler and role” settings, specify the handler as shown below. You will also need to specify an IAM role for the function as discussed here. In our case, your function will need at least access to SSM’s send and list command and read access to your s3 bucket path. Your Windows server that runs the Pester tests will also need access to write to the s3 bucket.Screen Shot 2016-07-30 at 7.18.52 PM.png
  7. In the “Advanced settings” field, set the lambda function timeout to be greater than the timeout of your SSM command.
  8. Review the details on the next screen and click “Create function”
  9. When the function is created, you can run it by configuring a test event with the JSON struct shown above, substituting your instance ID and bucket details.

That’s it! Now you can remotely invoke Pester tests on any Windows instance in your AWS environment without maintaining a build server. I hope this is helpful to you! Let me know if you have any questions in the comments.

Invoke Pester Tests Serverlessly with AWS Lambda and SSM

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s