In my last post, I spent a long time describing some recent successes at work. At the end, I had the nerve to close the post on a very unsatisfying allusion to a ‘discussion for another post about “the process”‘. Let this post be the first installment in a recurring series discussing “the process”, by which I mean “Engineering process”.
I want to start by addressing something that wasn’t acknowledged in my previous post, and is really only obvious in retrospect: While the successes were pretty spectacular, they were made possible by a situation of extremely pent-up need. That string of rapid, extreme successes was the dam breaking more than anything. We were just there to ride the wave. We made it look easy and sound amazing, but the truth is the situation made that easy for us. Since that initial burst of productivity, things have settled down to a steady plod as we continue to iterate on improving things.
Success stories are fun to tell, but, we tend not to talk about daily reality. For example, there’s always a long tail of effort when solving a hard problem. Right about now I usually mutter something about the 80/20 rule (which I am certain I misuse and misrepresent, but the principle is conceptually apt). Most of the value from a big effort gets realized in the early phase when you have expended relatively little of the total effort required to completely solve the problem. Thus, for 20% of the total effort, you get 80% of the solution, and most of the value of having the problem solved – a big gain for relatively little effort. That remaining 80% of effort is yet to be spent, will still yield value, and is (probably) [mostly] necessary; but it’s going to be proportionally more work for the value realized.
There is a trap here, though: using this rule appropriately. I’d say this rule is appropriate and useful for helping you understand the qualitative succession of your efforts during an undertaking, but would not be useful in predicting a timeline for your progress. It would also not be appropriate or useful in helping you decide when you’ve achieved an optimum outcome and can stop.
This introduces us to a fundamental element of Engineering process: using heuristics. Heuristics go by many names such as rules-of-thumb, rules, patterns, procedures, processes, approximations, sayings, mantras, maxims, or even techniques. We are drowning in them in the fields of Computer Science and Computer Engineering. To make matters worse, not all heuristics are compatible. They conflict or outright contradict each other at times. They are not all equally applicable in a given scenario, nor are they equally effective even when they are applicable. And they change. As we gain more experience, learn new things, progress as a society, our heuristics evolve. Even still, heuristics underpin everything we do in Engineering. They are the foundation upon which the entire Engineering process is built. As engineers, our challenge is to figure out when to use the correct ones.
These are a few common heuristics we use around the office:
- Make small changes
- If you can verify it, do so (don’t guess)
- No in-place app/config changes (i.e. outside build or config management)
- Use short hostnames in your config
I won’t get into the specifics of how we arrived at the items on that list right now, nor is this list exhaustive in any way. I merely wanted to draw attention to the variety and prevalence of heuristics to get you started thinking about others you probably use and might not even be aware of, both in normal life and at work.