Adventures in AWS: Choosing a Windows Scripting Environment

In this series, I explore some of the everyday challenges facing an AWS developer/sysadmin. Today: is Powershell always the best choice for AWS scripting on Windows?

THE PROBLEM
Welcome to AWS, Windows sysadmins! There are many wonderful services here for you to use and a nice API to help you automate them. You could use the AWS CLI to access this API, but more likely you want to integrate AWS commands into your favorite scripting language. Luckily for you, AWS provides options. Like, ALL the options. There is a Java SDK, a .NET SDK, a Python SDK and more, plus some higher-level toolsets built on top of them. Since you’re a Windows scripter, let’s focus on two of these toolsets: the AWS Tools for Windows Powershell and the boto library for Python. Like any good Windows admin, your first instinct is to choose Powershell as the default scripting environment for every task. But as Cicero once said: “The wise are instructed by reason, average minds by experience, the stupid by necessity and the brute by instinct.” So let’s dig a little deeper and see whether Cicero would make a good cloud developer.

THE BACKGROUND
The AWS Tools for Windows Powershell comprise a set of Powershell cmdlets built on the .NET SDK. They support Powershell goodness like pipelining, but supposedly expose the same feature set as the underlying .NET SDK layer. (As we will see, this is not always the case.)

The boto library for Python is a project produced in cahoots with the open-source community, so it’s a little bit shaggy around the edges. In addition to the original project, there’s a new version called boto3 that supports Python 3 and seems to have fixed some of the old boto’s issues and inconsistencies.

Using boto makes a lot of sense if you are developing a cross-platform script that needs to run on Windows and Linux, but is there any reason to avoid the Powershell Tools in a pure Windows environment?

THE SOLUTION
It really does depend on what you are trying to do. If your script mixes AWS API calls with, say, Windows registry operations, Powershell should be your go-to environment. But if your script is mainly calling AWS services and is more or less OS-agnostic, the choice isn’t so clear-cut, because there are some significant feature differences between the various AWS toolsets. Forget Cicero; George Orwell’s Animal Farm had this one right.

Orwellian AWS Truth:
All SDK’s are created equal, but some are more equal than others.

This should make sense when you consider AWS’s overall development philosophy, which gives small teams a lot of autonomy over individual services and supposedly doesn’t much care if the same problem is solved multiple times by different teams using different approaches. The upside of this philosophy is that the individual AWS services and SDK’s tend to be developed in an internally coherent fashion and patched quickly. The downside is a sometimes-ghastly lack of consistency between services. Compare the random differences in the AWS Console layouts for RDS, DynamoDB, EC2 and CloudFormation sometime and tell me you don’t want to punt a small animal. (Gotta watch those brute instincts, Cicero.)

Understandably, this means that certain toolsets are going to support access to certain new AWS features before others. Likewise, sometimes one toolset lags behind for no apparent reason. One example involves DynamoDB, AWS’s NoSQL database service. Here’s a boto snippet to add a record to an existing DynamoDB table called animal_farm:

db_conn = boto.dynamodb.connect_to_region('us-east-1')
farm_animals = db_conn.get_table('animal_farm')
item_data = {
'intelligence': '8',
'ego': '10',
'empathy': '0'
}
item = farm_animals.new_item(
hash_key='pigs',
attrs=item_data
)
item.put()

And here’s the same action using the Powershell Tools:

<#crickets#>

Yep, that’s right. You can’t do any get or put operations on DynamoDB data in the Powershell Tools; you have to to dip into the underlying .NET SDK or embed an AWS CLI call to get this essential functionality, which boto has provided from the beginning. And that makes Powershell a pretty bad choice for any DynamoDB-heavy script at present.

The good news is that if you find a feature gap like this in one of the toolsets, you might not have to wait long for a fix. I wasn’t kidding when I said that AWS’s agile approach allows them to patch services quickly. Here are the release notes for the Powershell Tools. At last count, they are averaging a new version increment about once a week. They work hard not to push “breaking changes” in these updates; more often I’ve found that if I have a version problem, it’s because I’m trying to use deprecated features in an older version.

So, to recap: don’t assume that the Powershell Tools are the best choice for every AWS scripting task on Windows. Use the tool that provides the cleanest access to the services you are dealing with. And whatever scripting environment you choose, make sure it’s up to date every time you use it. Feel free to share your favorite (or least favorite) AWS SDK in the comments!

Adventures in AWS: Choosing a Windows Scripting Environment

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s