Like my friend the IT Skeptic, I am interested in getting more emprical in my ITSM approaches. In particular, I want to focus on metrics that indicate results, not just activity. (Results don't have to be financial.)
In the area of configuration management, a variety of operational reports and metrics are suggested by ITIL v2, many of them in the "activity," not "results," category.
An interesting "results" metric to me is requests for change that are not successful.
Another one is the cycle time to reduce incidents/problems.
I have an intuitive sense that a CMDB, in its role as knowledge repository, can "move the needle" on both of these metrics. But can I prove it? What if I implement a CMDB, along with a bunch of other effort, and availability improves. What do I say to the skeptic who challenges, "Prove that your CMDB contributed"?
One means might be a form of post-mortem change/problem analysis in which the contribution (positive or negative) of the CMDB data to specific changes or incidents/problems is assessed. That is, for each change or incident/problem, questions would be asked such as
- Was the CMDB consulted and found to be accurate?
- Did the CMDB data contribute in any way to the success or failure of the change or incident/problem resolution?
This is a difficult and subjective assessment for which we'd need to train the potential assessors in order to achieve any kind of consistency.
Pre-implementation baselining would be especially difficult, because you would have to ask the question in terms of "if we had a CMDB, would it have helped?" - an even harder judgement call for an analyst to make.
But without asking such questions prior to implementation, how do you prove the CMDB really helped? "Because ITIL says it's best practice" is not sufficient, I think...
Very interested in any thoughts.
-ctb
