I’ve read dozens of DevOps success stories, tales of bold IT leaders transforming their business and steering big corporate ships into the future. It’s hard to avoid all these stories about “DevOps transformation” and how some guy named Jim pulled the IT department out of the stone age. The trades, blogs, and conference presentations are filled with them.

No one talks about the failures though, very few even write about their struggles implementing DevOps. It’s all sunshine and rainbows, which sucks, because that isn’t real.

Success teaches us little other than survivorship bias and how to feel bad about what we haven’t achieved. Failure and hardship are where we learn. That’s where the good, meaty stuff is.

So here are a few dirty secrets of DevOps.

Most companies that say they are “doing DevOps”, aren’t.

Because of all the success stories (real or imagined) that have wiggled their way into the minds of CXOs, “we should be doing DevOps” became an empty corporate directive that inspired thousands of executives to start calling their IT infrastructure groups “DevOps” instead of “Infrastructure” or “Systems” or “Operations”.

Unfortunately, this seems to often play out as “We’ve renamed the group, so we’re going to be letting most of the team go, because we’re DevOpsing all the things now and being lean and mean. Also, the developers are still a separate group and they’ll be throwing more stuff over the wall to you.”

So you end up with a few overworked, traditional Ops folks trying to keep the wheels on the bus with zero changes to the way work is managed or how the IT group functions day-to-day. Their manager is shouting down the hall about automation while the poor Ops teamis trying to pivot a SAN-installing, server-racking skill set into something that looks like a cave-man version of coding.

The only metric that improves in this scenario is “personnel cost”, and only temporarily because burnout and churn spike, driving up staffing costs a few quarters down the road. But it looks good long enough for someone to say “See, we did it!” and feel validated.

Even if you get the “IT folks” on board, getting plugged into the business so DevOps practices can benefit other groups and the overall bottom line comes with its own challenges.

Fixing this issue requires a lot of skill managing upward and sideways. Often times, it’s not worth trying to change and moving on is a better option. Your mileage may vary.

Implementing DevOps is Really Fucking Hard™

DevOps is all about people and process, getting everyone working together to do fewer dumb things, and smart things faster.

Historically, getting people to work together and not be jerks to one-another has been a bit of a challenge. Humans achieve awesome things when we collaborate (like spaceships and lasers), but we usually suck at working together. Because of that, I’m always impressed when I come across an excellent people-whisperer, someone who can motivate different groups to work towards a common goal without burning down each other’s village.

Problem is, there’s like five of those people on the face of the planet and chances are, they don’t work for your company. You might have lucked out and have 1 or 2 folks who are kind of OK at people-wrangling and peace-keeping, but most businesses (especially the bigger ones) seem about three seconds and a passive-aggressive sticky note in the office kitchen away from an all out blood bath.

Assuming you can get people working together, you’re now faced with the challenge of implementing process. You probably have one person on your team who loves process. Everyone else hates process and that person.

You’re never finished

There’s no such thing as “we achieved DevOps”. It’s a practice like healthy living or Buddhism. There has to be a champion(s) on your team who pushes every day to make things better.

When someone talks about DevOps success, what they’re really talking about is achieving flow, that there is a functional work process in place that is continually measured and improved upon. It’s an ouroboros value pipeline.

That’s not something you can arrive at and stop tending to. Without constant care and feeding, the processes you worked so hard to implement will start to die off.

No champion, no DevOps.

All that being said…

None of this means that DevOps isn’t worth doing, just that you need to be realistic about the challenges you’re going to face. I’ve leaned on hyperbole pretty hard to swing the pendulum away from sunshine and rainbows, because reality is somewhere in the middle.

Getting Dev and Ops (and Security), groups who have traditionally walked down the hall waving their middle fingers at each other, to 1.) work together and 2.) implement and adhere to process, will likely be one of the most difficult things you’ve attempted in your career. You have to put a lot of work into making the right things the easy things, reducing friction wherever you can. Setting mandates or badgering doesn’t work, you have to sell the value.

Getting top-down buy-in (and understanding!) of true DevOps/Agile practices is hard. It requires reorganizing groups and a sustained sales pitch to all involved. The need for this trails off a little once the business and IT staff start seeing value, but expect it to be a long, sustained effort. I’m always a bit dubious when I hear something like “we transformed IT in three months” – either that group really has their shit together, someone is lying, or we’re not using the same dictionary.

For practitioners and evangelists, these are the things we need to start talking about more. There’s a slick consultant vibe that’s weaved throughout discussions about DevOps that glosses over the practical and prescriptive. Too many of these conversations focus on high-level what-to-dos and not enough on concrete how-tos and context, especially when it comes to people-centric issues.