3 Questions to ask before automating

If you’re a DevOps engineer who is constantly fighting fires and trying to keep your head down, it’s easy to get stuck in a rut of “You ask, I build”. Heck, it’s easy to think that way even when everything is running smoothly.

But that’s not what the job should be, or at least what the job can be. Not asking the right questions leads to more firefighting down the road and smooth operations devolving into mediocrity.

If you have a desire to be better at your job and build truly helpful, effective tools, before any code is written, there are (at least!) three questions that must be answered.

1. Are we solving the right problem?

Unfortunately, this question almost never gets asked and when it is asked, the answer is often “no”. A Dev or PM will send an email that says “We need to automate X.” and the DevOps engineer will asks a couple of technical questions before starting the build.

If you’re a DevOps engineer and you aren’t asking questions about process and goals, you are a bit pusher, not an engineer. You have unique insight into how the systems you work on interoperate and feed one another, you think in terms of systems and dataflows— you have a contribution to make that’s more than writing playbooks and scripts.

If the teams you’re working with aren’t volunteering their goals, you need to ask for them. “What are you trying to accomplish?” Having this context is critical if you want to provide real value. What you were asked to build is often not the right solution to the problem that needs solving.

Before you build tools, build relationships with the developers you work with and get in the habit of asking questions that aren’t just “what version of this plugin do you need?” Ask questions even when you’re offering feedback.“What if we did it this way? Would that work?”

2. Is building a tool the right solution?

If you’re confident in the process, the next question you should ask is “Is building a tool the solution to this problem?” In a lot of cases the answer is “yes”, but thousands of custom tech fixes have been built to solve problems that could have been solved more appropriately with an email, a Google form, or God forbid, two people actually talking to one another.

This is when it helps to get someone in the room who isn’t an engineer and can provide a sanity check against the “when you have a hammer, every problem looks like a nail” problem.

Being competent with Python or Ruby is not an excuse to try and solve every problem with a script. I don’t know how many discussions I’ve sat in on where people were arguing about how to build a custom tech fix and I wanted to scream “JUST HAVE THEM GO SIGN UP FOR A DOCUSIGN ACCOUNT! WE DON’T NEED TO SOLVE THIS PROBLEM WITH CODE!” or something similar.

Find someone who will call you on your BS and tell you when you’re being myopic. That person is your new best friend.

3. Who is this tool for?

Hopefully the answer to this question will come out of defining the process, but that’s not always the case.

If you find that you’re only building tools that you or your DevOps team can use, you have a problem. There are certainly circumstances where having an internal-only tool is appropriate, but the majority of what you build should be usable by the people you’re trying to help.

Don’t build a script that you run on your local machine to deploy servers that the developers ask for. Build a portal that allows the devs to deploy without your involvement. That local script may be a first step, but fight tooth and nail to remove you and your team as a dependency for getting something done.

Some engineers think they’re being helpful by responding to requests with “Sure, I’ll build that.” But if you’re doing the same thing over and over (deployments, updates, etc.) you are not being helpful. Likely, you are slowing everyone down and have the potential to bring the dev process to a complete halt.

If you want to be a SysAdmin, keep on keepin’ on. If you want to be a DevOps engineer, build tools that others can use, then get out of the way.

Originally posted on BestTech.io

Automation is not DevOps

A few years ago, if you would have asked me to define DevOps, my answer would have sounded something like “Mumble mumble automation, mumble, automation, mumble mumble, infrastructure-as-code, mumble mumble, strategery.”

Thinking that DevOps equated to automation had mostly to do with the fact that most of the DevOps people I talked to and articles I read really only spoke about automation with only indirect references to anything else.

Implementing automation is definitely a part of what it means to be practicing DevOps, but it’s maybe 5-10%.

The reason that so much of what’s been written about DevOps is focused on automation is that it’s easy to write and talk about tools. Relative to implementing a true DevOps practice, automation is the easy part.

What DevOps actually represents

I’ll use Gartner’s definition to provide a common basis:

DevOps represents a change in IT culture, focusing on rapid IT service delivery through the adoption of agile, lean practices in the context of a system-oriented approach. DevOps emphasizes people (and culture), and seeks to improve collaboration between operations and development teams. DevOps implementations utilize technology — especially automation tools that can leverage an increasingly programmable and dynamic infrastructure from a life cycle perspective. — Gartner.com

Notice that automation is the last thing mentioned. The main components of DevOps are people and process.

This is the hard stuff, the boring stuff, the stuff no one likes talking about because it involves dealing with people and managing work. But it’s also the stuff that gets shit done.

Automation is a force-multiplier, a lever. That’s all computers and software are in general — levers. Without people and process, automation is just a rusted-up socket-wrench— you can use it as a blunt instrument, but you’re not getting its full utility.

Automation alone can have an impact, but mostly in that it allows you to do stupid stuff faster.

People

At the heart of a successful DevOps practice is a culture that accepts failure, doesn’t focus blame, and demands collaboration and knowledge sharing.

If your team members are afraid of how their manager reacts when they make a mistake, they will not attempt new ways of doing things and the team and business will not move forward. Full stop.

Screwing up is a necessary part of learning and improving, but most companies have a culture that doesn’t allow even minor failure. I don’t know how many times I’ve heard executives earnestly say “Failure is not an option.” completely missing that they were driving their company into the ground as the industry changed and competitors sprinted past them, having embraced failure as an opportunity to learn and do something new.

Accepting failure doesn’t mean not having high expectations or accepting mediocrity. Accepting failure prevents mediocrity.

The expectation for a DevOps team member should be: “Mistakes happen, don’t try to hide your screw ups. Communicate what happened, work on fixing it, and most of all… learn from it.”

Accepting failure directly feeds into reducing the desire to place blame. Worrying about the repercussions of failure and working to deflect blame take away from doing actual work and fixing the original problem, aside from poisoning the work environment.

When co-workers trust that they’re not going to get thrown under the bus, they collaborate and are more productive. Playing the politics of avoiding and placing blame are distractions that need to be snuffed out. If you’re a manager and you allow your team to sit around pointing fingers at one another, shame on you. If you encourage it (and oh, God have I met those managers), I hope your house gets invaded by bees.

If your goal with implementing DevOps was to speed up delivery and become more agile, you have to aggressively remove roadblocks. Anything that stands in the way of the team collaborating with the business, developers, and each other needs to be bulldozed. Fear, politics, mistrust — gone.

It might feel like wrangling a kindergarten class, but you have to get your team to share. There is no such thing as “too busy to train” or “too busy to document”. If only one person knows how to do a thing, they become a bottleneck that can shutdown your DevOps factory.

Knowledge sharing has to be the expectation. If there is someone on your team who can’t be coached to not keep things secret, they need to work somewhere else.

Process

You can have amazing automation and culture and still not be practicing DevOps, because practicing DevOps is almost entirely about process.

If you want to “do DevOps”, process is where you start and will give you a much higher return on investment than the automation that follows. Without process, there will be too much chaos to maintain a healthy culture and any tooling you build will likely be solving the wrong problems.

Start by getting visibility to the work your team is being asked to perform. This isn’t just for managers, the entire team needs insight into the work backlog, if for nothing else than to prevent duplication of work (“I already have a script for that.”) and provide time-saving context (“Doing X will cause Y to fail.”)

If there are team members who won’t share what they’re working on or only offer vague details, that has to end. Letting people slide by with “I’m working on server things…” during your daily standups (and yes, you should be doing standups), doesn’t fly. “I’m doing X, Y, and Z, today.” is the answer you’re looking for.

Without that transparency, work goes into a vortex of suck. If work is (or isn’t) being done, the team needs to know, because every unknown status compounds delays, reduces quality, and plants the seeds of distrust. A successful DevOps practice requires accountability, not for the sake of holding someone’s feet to the fire, but just so everyone knows WTF is going on and can plan accordingly.

All this requires commitment and continual reinforcement from teammates and management. There is no “We set up a kanban and no one used it so we stopped.” Failure is an option. Not knowing the answer is an option. Working ad-hoc and being lazy about process is NOT an option.

To summarize, if you are not managing work in an Agile way, using Scrum or something similar, you are not practicing DevOps.


It turns out that there are lots of great resources on the non-automation facets of DevOps. Two good places to start are The Phoenix Project and Effective DevOps. The reviews for Effective DevOps are particularly fun because most of the complaints are “the author didn’t talk about automation”, which means she got it right.

The trick when you’re doing research on DevOps is to not search for “DevOps”, because you’ll mostly get 1.) articles that are about automation, and 2.) “DevOps” job postings that are really just sysadmin jobs.

Instead, read about Agile and Lean. Read about Personal Kanban. Read things that make you a better person who can treat other humans with empathy. That’s what will make you a “DevOps Ninja”. Focusing entirely on automation just makes you a code monkey.

Originally posted on BestTech.io