15 Microservices in 15 Days

A Molasses Migration

Avvo.com started out as a Rails app and, like many people, we used Capistrano for deployments with server configurations managed through Chef. This worked for some time but as our stack grew, the architecture headed into SOA and more servers were added. We quickly ran into environment variable dependencies and conflicting system libraries(i.e different ruby versions, libv8, or imagemagick). This created an un-maintainable infrastructure and limited our self service capability.

At the same time Docker came onto the scene.

Contained dependencies and environment variable management through a 3rd party application(racher.io)?

Sounded great to us. Sign us up!

Welcome to Docker-ZORK! aka DOCK. Your adventure begins with only a console and your wits...

Adventures in Docker

We set off to migrate all of our capistrano deployed applications to docker. After many iterations and countless trials of different software solutions, we decided to use rancher.io. There was even a "Dockerize all the things" bash day to get all our applications dockerized. Even though it took a few weeks after to get completely ready for docker, we did it. Our stack was finally ready to deploy using rancher....except we still relied on Chef to set up our infrastructure containers. Environment variables were stored securely and relied on our Infrastructure team to update. A few services were deployed in rancher and others still using Capistrano.

The deployment pipeline was not as fluid as we wanted it to be.

A Foot in Each Room

With a production environment mostly still in Capistrano, and our development stacks completely dockerize, we stood with one foot in the old way, and one foot using our new pipeline infrastructure. Each new application was always a struggle to get out the door. Be it global configs, database connections settings, creating new databases, setting up our CI, or integrating into our new development enviroments with Rancher, each time there was mised results of success. A few times it would go smoothly, while most of the time it was like wandering around a dark basement bumping your head continuously.

Shake Out the Snakes

One day while discussing ACLs on a new product, which involved another service to create, Donald Plummer says "While you're at it can you please break out sales regions, resolve and location from our location service?" (paraphrase) That was followed up with two more services mentioned in our dev chat. I off-handedly said "whelp, time to start on the 15 service apis Donald was suggesting..". At the time it seemed like a joke, but then we thought about it more. * What if we could build and stand up services in one day in all environments? * What if each of the engineers had no barriers to releasing services? * What if we did move to super small easily rewritable services?

Then the most awesome Hunter Davis had a wonderful conversation with another awesome engineer, Jacob Kemp, and came up with an even better addition. "Let's pair program our way to success"

Thus 15 apps in 15 days was born.

Here’s how it will work: * Starting June 5th, I will pair with one engineer every day for 15 days. * The services we build would be small (a few endpoints and small in scope) * The morning will include API design and building out the endpoints. (break for lunch) * The afternoon would be dedicated to standing up the services in our development environment, Staging and Production. * By the end of the day we should have services running in all environments * After each iteration, we’ll have a quick retrospective

Our goal: 1. Discover new tools and libraries 2. Uncover deployment impediments 3. Broaden the engineering team’s knowledge for creating and releasing new services. 4. Move closer to the engineering goals: * 10 minute deployments * 1 day to create and release a service

Stay Tuned

I'll be blogging about what we learned each day, including the stumbling blocks we uncovered and improvment suggestions