Adventures in AWS: Understanding the Price List API

The AWS Price List API

In December 2015, AWS announced a new Price List API that exposes access to current prices for several of their products, including EC2 and RDS. If you’re thinking about building some automation to track AWS spend, and you’re used to working with AWS’s generally marvelous APIs and language SDKs, this sounds like great news. At least, that’s what I thought when I sat down last week to build a tool that would track cost savings for stopped EC2 instances in some of my company’s AWS environments.

Let’s just say the Price List API didn’t match my initial expectations. In fact, calling it an “API” is a bit generous; basically it’s a giant JSON or CSV file you can download for various AWS services that contains price and product data indexed by SKU.

What’s a SKU, you ask? The short answer is it’s a “Stock Keeping Unit“; the even shorter answer is that it doesn’t matter, because nowhere else in the AWS API ecosystem are SKUs for individual resources made available, so you’re not going to be cross-referencing them with anything else. But don’t despair; it is in fact possible to automate price lookups inside the AWS Price List data dump. Let me walk you through the approach I used.

Continue reading “Adventures in AWS: Understanding the Price List API”

Adventures in AWS: Understanding the Price List API

Lambda calculus: talking Serverless with Florian Motlik

The open source Serverless project, which currently has nearly 10,000 stars on Github, provides tooling around AWS’s “Function as a Service” ecosystem that includes Lambda and API Gateway. I recently had the opportunity to chat with Florian Motlik, CTO of Serverless, about his thoughts on serverless architectures and the future of the Serverless framework.

The following interview has been edited and condensed. 

Forrest: Although AWS Lambda is less than two years old, we’re already seeing a robust tooling ecosystem appear around it, including the Serverless Framework. How did the Serverless project get started?

HeadshotFlorian: Austen Collins, our founder, started Serverless about a year ago. In his previous life as a consultant, he worked with AWS Lambda while building various applications. Austen saw two things about Lambda that made a huge difference for him. First, it enables you to build applications without having to maintain infrastructure. And as someone who had to maintain infrastructure in the past, he saw that was a really interesting direction for the industry to go. Second, Lambda enables an event-driven architecture, where you just react to events that can be fired from anywhere to anywhere. Austen also saw that although Lambda was very powerful, its lack of tooling made it hard for new users to get started. So, about a year ago he started building the Serverless framework. The project took off right away, and towards the end of last year, he decided that this is not just an open source framework; it’s something we can build a company around. So that’s when I was brought on as the CTO to lead our engineering team, and we grew from there.

Continue reading “Lambda calculus: talking Serverless with Florian Motlik”

Lambda calculus: talking Serverless with Florian Motlik

Invoke Pester Tests Serverlessly with AWS Lambda and SSM

Pester and CI

If you’re doing Windows scripting in 2016, you’d better be using PowerShell. And if you’re writing PowerShell scripts, you’d better be checking them into source control and covering them with Pester tests.

It turns out that you can do more with Pester than just run tests manually at the console. As part of a continuous integration (CI) process, you may want to invoke Pester tests on a remote server and report the results up through the build chain. Handily, you can export Pester test output in an NUnit XML format that modern CI systems like Jenkins understand.

But what if you’re not using a build server to invoke Pester? What if your CI setup is … dun dun dun … “serverless”?

Continue reading “Invoke Pester Tests Serverlessly with AWS Lambda and SSM”

Invoke Pester Tests Serverlessly with AWS Lambda and SSM

AI, automation and the merry-go-round of the mind

Is there such a thing as too much automation?

At a family wedding the other weekend, I fell into conversation with a relative who has several decades of experience in the aerospace industry. He bemoaned a growing problem among the younger engineers who work for him. It seems that some of these highly-paid professionals have not developed the ability to look at a finished piece of work and say – “That doesn’t seem right” – because they rely on their advanced computer systems to do the validation. When the computer makes a mistake, they do not have the breadth of experience to realize it.

This point resonated with me for the simple reason that I experience it every day. I’m a professional automator – I automate software processes for a living – and I spend a lot of time inside the Amazon Web Services cloud. AWS handles the compute, storage and networking details for me so I can focus on higher-level tasks, which is both nice and worrisome. Nice because I can get more done in less time, worrisome because I don’t get the opportunity to grapple with the implementation details of server and network virtualization. I understand those things on a theoretical level, but I don’t get to play with them much, and this sometimes hampers my grasp of what’s really going on beneath all the automation.

Continue reading “AI, automation and the merry-go-round of the mind”

AI, automation and the merry-go-round of the mind