Some people are quite messy by nature - in many cases it’s because they don’t give a … care. In some because they’re overconfident (they believe they are more organized than they actually are) and sometimes it’s a nothing more than an old, crude lack of time to keep things more tidy. Sadly, everything has its limits and human mind can’t keep things under control when:
- the sheer number of them grows too much
- they are so inter-connected and inter-dependent that any single item gets untraceable
- there are many sources of information and sometimes it’s unclear which one of them is the right (and must trustworthy) one
IMHO inventory is crucial especially for DevOps. Without such data repo, how would they tell:
- how close is it to reach the overall capacity?
- is the new setup really coherent? are all the connections well set-up and do they reflect the dependencies between software components?
- what has to be decommissioned (cleared out) after some stream of work has finished?
That’s why each time I see DevOps openly neglecting the need to catalogue and inventory their stuff, something dies screaming within me. This week I had this situation twice, so I couldn’t resist the need to write a bit about that.
Why do we need inventory?
That should be quite obvious: to put numerous items in control. To keep the most important information about them up-to-date. To extend the knowledge base with some operational info - like current utilization. To have a base for further processes and activities: like monitoring and work planning.
A smart individual in theory can keep such data within his sole skull, but:
- who will guarantee that he won’t forget something important?
- what if he gets sick? or finds a new job?
- is he really able to answer any realistic (the one that actually may appear) question immediately?
- what about transparency? It’s cool that he knows that (if he does), but how is this knowledge accessible for others?
Why inventory usually fails?
Because it’s collected manually - it’s an ad-hoc effort that is performed once and the inventory repository is not kept up-to-date after then. Why? Because it’s manual, it’s tedious, people keep forgetting about such stuff.
In the end, the effort is wasted. And inventory finds its way to a trashbin or is updated once a year (usually by being re-created from scratch).
That’s why inventory should be the best friend with automation - make it a vital element of your delivery automation process -> this way you’ll also make sure that it fits its purpose (it doesn’t get out of sync with reality).
What can one do to make it work then?
There are two solutions: both have their challenges, but each of them is equally good, so feel free to pick one that fits you most:
- Create your inventory automatically.
What does it mean? Make the things that you want listed to “announce” or “register” themselves to inventory. Find common points / places / junction-points in your software / infrastructure and then place there a component that will do the job. Examples?
> make your deployment scripts register services being installed
> put metadata in AssemblyInfo to digest it directly from the libs
> mark your services / certificates / machines in a distinguishable way (naming convention) and periodically dump the list to refresh the inventory - Validate your manually-created inventory by making it crucial for a core function
Make sure that your maintenance procedures / operations will work only on the stuff that is present in the inventory. For instance:
> won’t update / start any service that is not in the inventory
> any DB that is not in the inventory, gets auto detached (during everyday’s backup window)
> only services that exist in the inventory may successfully connect to the configuration server
First case is more brain-less, but in some cases it may actually be far more difficult.
Second case is simpler (as a concept), but it seems to require (in general) more effort -> on the other hand, such checks won’t hurt anyway, so it looks like an effort well spent. And once there’s something (with a short feedback loop) that will cover your ass in case of forgetting inventory update, the duty of keeping it current is far less painful.
Your inventory (CMDB in ITIL naming convention) is your treasure map. Would you throw a treasure map out through the window?
