Unicorns and Rainbows

My employer has been in the process of reorganizing for several months, now. A couple of weeks ago, as part of this reorganization, I was moved to a “new” team. In actuality, this team is simply a small subset of the people I already worked with. It was an All-Star team of three. Our mission was … whatever we saw fit, related to everything in our domain of skill, interest, and concern. Mostly we would direct our efforts at release engineering, build engineering, systems architecture, and deep-investigation type troubleshooting. Stuff we already did anyway, albeit much less formally.

When I first learned of the new team (and the team reorg plan in general), I was in a bad state of burnout. So bad, in fact, I just couldn’t even get excited about the new team. I just didn’t care. However, when the three of us got into a room with our PO for the first time, the energy was palpable. We are like-minded engineering types, and I have the utmost admiration for my teammates’ skill, professionalism, and opinions. I felt much more positive after that first meeting.

Continue reading

Bootstrapping A Simple Openstack Lab

In my last post, I waved my hands and assumed you had a deployment of Openstack to run the CoreOS cluster that was the subject of that post.  I felt a little absurd talking about autonomous, self-assembling infrastructure-in-a-box while choosing to forego a simple tutorial on setting up an all-in-one lab environment, at a minimum. So this time around, we’ll discuss a way you might bootstrap a simple Openstack deployment.  This setup will result in an RDO all-in-one Openstack system where the floating ip pool will be assigning valid IP addresses on your “public” network.  In this example, we pretend your lab subnet is  When you assign a floating IP address, any host that can route to that network will be able to talk to those floating IPs (assuming your security group allows it).

In my personal home lab, the first thing I set up was a Cobbler server.  I performed a minimal installation of CentOS 6.5 x86_64 with a netboot ISO, installed the EPEL repo, and set up Cobbler using the instructions on the Cobbler website.  I configured Cobbler to manage DHCP and DNS, and also installed cobbler-web.  I imported one distro: centos-6.5-x86_64.

Continue reading

A Simple CoreOS and Etcd Cluster

In my idle moments, I contemplate such things as self-assembling infrastructure-in-a-box.  What I mean by that is how do you go from an an empty room (conceptually) to doing useful work for the business in as few steps as possible, as quickly as possible?  What minimum complement of services do you need, and how do you bootstrap it into existence, autonomously?  The goal is to perform as few manual steps as possible to get to a state where you can easily build out the rest of your infrastructure and start deploying applications right away.  Bootstrapping.  Inception.  Whatever.

In this post, at a high level, I want to lay out a simple application platform bootstrapping exercise.  We’re going to make use of a HOT template for Openstack to bring up a three node CoreOS cluster which performs auto-discovery to assemble an etcd cluster.  This is not a tutorial on Openstack.  I am not covering the obvious question of how to get Openstack running.  The point of this exercise is only to lay the foundation for running clustered applications in Docker containers.

Continue reading

Continuous Agile

Even though my organization has been practicing “agile” for several years, I have witnessed a seismic shift in what that means over the last 15 months.  We have shifted from Sprint Teams running two-week Sprints and releasing every other Sprint cycle to an interrupt-driven Kanban “pull” model with no strictly time-boxed development cycle where we release twice a week, whatever is ready.  This is just a step along the road to fully automated Continuous Delivery for us, though.  We’re learning as we go.  Our focus right now is ruthlessly stripping our manual release process to its essence, then automating what’s left.

Even though we’re focused on optimization of a semi-manual release process, we’re thinking about what else we need to do to improve our game.  We’re not just thinking about release process.  We’re still defining our development philosophy is, how we can create an infrastructure in which we can experiment, and what techniques we can employ now that improve our flexibility and give us more options for future experimentation.

Continue reading