Serverless Workflows on AWS: My Journey From SWF to Step Functions

AWS Lambda functions can only run for a maximum of five minutes. This must be distinctly understood, or nothing wonderful can come of the story you are about to hear.

The Problem

This past summer, my team and I set out to build an internal software system used for deployment testing on AWS. The application would run a large number of workflow executions in parallel each night and might perform a few one-off executions during the day – maybe six hours total use out of every twenty-four, with only a small fraction of that time spent doing actual compute tasks. Trying to scale, manage and spend money on EC2 instances for that workload didn’t interest us. We wanted to run our whole workflow process end-to-end on AWS Lambda.

And we did. Heaven help us, we did. This is our story.

Continue reading “Serverless Workflows on AWS: My Journey From SWF to Step Functions”

Serverless Workflows on AWS: My Journey From SWF to Step Functions

What if you managed your workday like an OS scheduler?


The longer you work in a knowledge-related job, the more responsibilities you have in a given day and the less likely you are to get any of them done. There are the five tasks you’ve been working on for two weeks that are due today, the ten things that were really due last week, the hot-button item that your boss invented unexpectedly this morning, and the ever-present backlog of stuff that you have to do sometime if you ever get the chance. Plus there are constant meetings, hallway conversations, instant messages, and emails. So…many…emails.

The worst part of this phenomenon is a nagging terror that of all the things you could be working on, you have chosen exactly the wrong one, and you will never get on top of anything ever again. (As you sit worrying about this, eleven urgent emails roll in.)


All this existential workplace angst boils down to a question of priority: what is the one best thing you should be doing right now?

Continue reading “What if you managed your workday like an OS scheduler?”

What if you managed your workday like an OS scheduler?

Loving The Cubicle, Part II


So I’ve had some interesting feedback since posting about how I learned to love my cubicle.  While the majority of readers seemed to agree with the majority of the points in that article, I also heard from some vocal work-from-home proponents. The readers tended to fall into three categories:

Category 1. People who work remotely for an external company (the “classic” work-from-home situation)
Category 2. People who work remotely for themselves, as consultants or other business owners
Category 3. People who may or may not work from home themselves, but work with and/or employ those who do.

These people all offered interesting opinions about how working from home has impacted their professional effectiveness. Some of their ideas were ones I hadn’t considered. So here are a few of the new thoughts I’ve been chewing on – let’s keep the conversation going!

Continue reading “Loving The Cubicle, Part II”

Loving The Cubicle, Part II