lean4it

Defining Demand Management

The posts yesterday were the "patterns." I suppose the definitional material would help...

Accept Demand

The most successful firms we've found have learned how to “deselect” projects, despite the enthusiasm of parts of the organization, in order to bring the number of projects into line with the available resources.

Womack and Jones, Lean Thinking

Far from being the rational and structured process we would like it to be, demand management is usually a nebulous combination of decibel management and organizational politics

Michael Gentle, CIO Magazine

The request (or demand) for IT services is managed through an intake function known appropriately as demand management. Leveling demand and balancing it with the supply of productive capacity is an essential management principle, yet too often the IT services organization struggles badly with this.

Demand management patterns

The weekend's work... and a rough 15 pages it was.

Posted my Accept Demand Patterns section, broken down thus:

• Understand aggregate demand
• The promise of Critical Chain for IT demand management
• Clarify Accept Demand relationship to other processes
• Clarify service entry points
• Integrate Demand Management System
• Accept Demand - IT Lifecycle implications

This was a hard section. IT demand management is not a well understood process area, and there are still major differences of interpretation between the project management derived views and ITSM.

Comments on these drafts are GREATLY appreciated.

Charlie

Accept Demand - IT Lifecycle implications

The IT process of Accept Demand is core to the Application Service Lifecycle. The demand for new functionality may arise with particular urgency from business partners recognizing a new revenue opportunity, or responding to environmental forces. The ability to quickly capture, assess, and prioritize dynamic demand is essential as the speed of business accelerates, as is the ability to understand what is truly essential for business value in a new service and how to deliver this in smaller, more iterative chunks.

Demand management should track the ordered and delivered IT service through what can be called the “inspire to retire” lifecycle of IT solutions/services. Silos are often seen between

• Governance, architecture, planning and authorization,
• systems development and integration, and
• systems operation functions.

Novel applications may impose net new demand on the infrastructure service lifecycle, but that is only part of the equation. Existing services may see gradual increase or sudden spikes in demand, which do not affect the application lifecycle per se (no software need be rewritten), but do impose new demands for infrastructure. Demand management is essentially equivalent to the concept of Capacity management when considered for the infrastructure lifecycle.

The IT operations area in too many organizations does not have visibility into “what’s coming,” and the result is sub-optimized from an operations point of view.

Dialog: Advance notice

Pat: Why would you need this?

Kelly: Have you ever heard of “over the wall”?

Pat: Yes, our data center and applications support people are always mumbling this.

Pat: Right – because they don’t have enough warning that a new system is coming, and aren’t able to influence its development sufficiently to meet their needs: application architectures that are compatible with data center operations, re-use of existing platforms and skills, instrumentation for availability, support documentation and procedures, and all the rest.

Most of this gets sorted out sooner or later, but it seems like we’re always painfully re-inventing the wheel here, and when you dig into the root cause the fundamental issue always seems to be one of visibility.

Sometimes, demands for increasing infrastructure service capacity can be met in house. Other times, they require the acquisition of new physical assets. A key concern is how backup capacity, also known as “white space,” is managed and in particular funded – a common topic in discussions of IT Financial Management. Such capacity can be construed as a buffer and therefore waste from a Lean perspective, but IT service demand spikes can be instantaneous and insufficient capacity installed and ready “on the wire” can lead directly to service degradation and even loss of revenue.

The Asset Lifecycle may also drive demand, if a large contingent of assets is approaching a refresh date, that effort itself may be handled as a project. The impacts on Applications need to be considered but the effort is not driven by functional Application needs.

Technology products – reproducible combinations of hardware and/or software, not yet acquired or pressed into service – are also subject to demand. Demand for increased processing power translates into demand for new kinds of infrastructure capabilities. Demand for ongoing platform services translates into demand for current, well supported versions of key infrastructure technologies (OS, data management, middleware).

New forms of security hazards translate into demand for new technical solutions. And new application services may require new technical products developed and marketed by specialist firms. All of this demand translates into value engineering kinds of activities, as the technical requirements are translated into functions and candidate solutions assessed for best fit at lowest cost.

The Technology Product Lifecycle may also drive demand; product obsolescence (e.g. due to a new version) is a frequent demand driver, as noted elsewhere.

The promise of Critical Chain for IT demand management

Eli Goldratt has written many novels beyond The Goal. One of the most important for IT professionals is his book Critical Chain.

Critical Chain is ostensibly about project management, but in reality it’s about all of demand management in a services context. The book identifies and examines key causes of project failure:

Bad multitasking. This is the frequently remarked upon churn experienced when tasks cannot be completed but instead must be juggled, with attendant “context-switching” costs.

Student syndrome. This is the tendency for work to get done at the last minute. A corollary is Parkinson’s law that “work expands so as to fill the time available for completion.” In a project context, it means that (for example) a task truly requiring three days may be estimated at fifteen days, with the task not starting in reality until day 12.

When considered as a systemic, cultural practice, student syndrome is hugely wasteful and an obstacle to execution success in many areas. Because of it, delays propagate while early deliveries do not.

Conservative local estimation, and local buffer control. Task durations are typically variable. For example, provisioning a new Unix server may typically take an average of 1.5 days effort, but 20% of the time may take four days effort. However, because of overly strong incentives in many cultures for “on time delivery!,” the server engineer will always estimate a full week to provision the server.

Then, because of student syndrome, they wait until day 3 to start, because the majority of the time they can get it done.  In the meantime, the “buffer” or valuable lead time is lost to the project. Even if the server is turned over early, typically rigid project plans will still not take advantage of this. Again, negative variation propagates but positive variation does not (a key insight of The Goal as well).

Critical Chain addresses these problems through accepting variability in task execution, and capitalizing on early deliveries by centralizing buffers under project management. The concept of project critical path is extended to include the resources working on the key tasks, and the Critical Chain is the critical path combined with resource availability. The critical chain resource is expected to work on nothing but the task at hand and deliver it as soon as possible, with no local buffer management. It’s not “due by the end of the week;” the server is due NOW. And there must be a strong defense against any other demands being placed on that resource (prevent bad multitasking) until that critical chain task is completed, as fast as reasonably possible.

Ongoing re-planning of the project is therefore also needed, since taking advantage of early delivery may mean that timelines across the project need to shift. This is a complex challenge and true Critical Chain project software needs to solve a thorny non-linear problem known as the Resource-Constrained Scheduling Problem.

Critical Chain is especially applicable to environments with large numbers of small, semi-repeatable projects competing for skilled resources – very typical of a large, service request-driven IT environment, especially across the “hosting zone of contention.”

In an environment where these same resources may get called into service requests, incidents, changes, and continuous improvement efforts, the concept is promising indeed and few if any IT organizations to this author’s knowledge have extended their demand and service catalog practices to handle this challenging general case. Instead, resources find themselves at the receiving end of multiple queues with little or no guidance for prioritization.

Integrate Demand Management System

The previous patterns lead to a more technical, system level pattern. They call for implementing some or all of the possible interactions below:

 

(click for larger)

There are many variations on how these integrations might be implemented. In this version, the demand system feeds the service request, project management, and continuous improvement systems, all of which in turn feed the consumption of employee time or indication of effort to the time tracking system.

This resource consumption is then replicated to the IT data warehouse and combined with other activity indicators that typically do not originate out of a demand management system: Change and Incident Management for activities that may not go through any demand qualification, and IT Finance and Capacity for resource views.

Having the Demand system drive Service Requests might surprise some Service Desk managers. This pattern does not preclude end users interacting directly with the service request system for prequalified services. But (as discussed in previous sections) if Agile methods imply a move away from formal Project management, it is reasonable that enhancement requests might be tracked as a pipeline of specialized Service Requests (methods such as Scrumban imply this, while not using the term Service Request Management per se). In such cases, these perhaps-discretionary requests do go through some form of demand qualification prior to acceptance.

Patterns like this illustrate how gray the boundaries between these systems can be. But whether it is called an Agile workflow system or a Service Request Management system, there is a common core of workflow and in the interests of parsimony this book sees them more the same than different.

Clarify service entry points

When viewing IT as a single value chain, the incoming stimuli that initiate the end to end process are of primary concern. They need to be captured as early and systematically as possible, to eliminate missed handoffs, redundancy, and rework.

A challenging aspect of demand management is therefore the service entry point, an interface or avenue of contact between IT and its customers and/or users. There are three major types by which customers (internal and external) experience value with these processes:

• Professional service
• End user service
• Application service interface

The processes map to the entry points as follows:

  
Table 4. IT processes and Service Entry Points

All of the services may be represented in a modern service catalog, but the fundamental differences between Professional Services, Service Desk services, and Application services must be reflected in the structure of any Service Catalog.

The high profile activities of Project, Release, and Change have specialized entry points requiring trained professionals and sometimes longer standing relationships on both sides of the interaction. Service Requests and requests for service restoration may be more commoditized. And Demand, while often seen as entry to professional services, needs to encompass the other types of services as well.

From the senior executive down, the business/IT interactions can be defined:

Business	What	IT entry point
Senior executive	Discussions of largest-grained needs; final escalation of most serious issues. Ideation/ exploration.	Senior IT leader (Customer Relationship Management function)
Unit executive	Requests for major new systems. Large project level.	Accept Demand
Functional area owner	Requests for new systems, additional functionality, improvements	Accept Demand, Execute Project, Deliver Release, Improve Service
User	Requests for orderable IT services; reporting of incidents	Fulfill Service Request, Restore Service

Service consumers and entry points

These service entry points need to be broadly understood in the IT organization and every IT staff person should be educated that any contact with business customers or users should be assignable to one of these categories.

Software development is an interesting service entry point. The primary service entry point of concern to the software development capability is requirements capture. The Agile movement in particular sees this as an ongoing interaction.

Such requests for enhancement are still demand requests, and depending on the service model for the application service, might be prequalified or discretionary. As noted elsewhere, this book views Agile pipeline systems as compatible with a Service Request Management system platform.

	Professional Services	Service Desk	Application Service interface
Accept Demand	X
Execute Project	X
Deliver Release	X
Complete Change	X
Fulfill Service Request		X
Deliver Core Transactional Services			X
Restore Service		X
Improve Service	X
Retire Service	X

Clarify Accept Demand relationship to other processes

The following processes would probably be considered tightly related to Accept Demand, as Demand Management is traditionally understood from a project portfolio management approach:

• Execute Project
• Deliver Release
• Complete Change
• Retire Service

But the relationship of Accept Demand to these processes is less clear:

• Fulfill Service Request
• Deliver Transactional Service
• Restore Service
• Improve Service

These are more about ongoing operational capacity, and less about fundamentally new/changed services. Yet service delivery competes with service development and evolution, and the two need to be understood in tandem.

Demand management therefore has a special role with respect to the other IT processes; at its most mature it provides a common view of all calls for IT resources.

Demand management can be seen as a governing capability thus:

 

(click for larger)

Notice the containment of processes within processes; this is to show that smaller processes are really sub-states of the larger processes and the larger grained Demand process cannot be considered complete until the smaller grained process is. Alternately, one could view the Demand process as complete when the work is authorized and assigned to the Execute Project process:

 

The actual approach here may be dictated by the system implementation, and either form can work equally well. It should be very clearly understood on a cultural level however whether the demand request ends at the point of routing onward, or logically encompasses all the work implied. This book in general will use the second pattern. In this approach, the Demand Request should carry a clear ID into the downstream processes. See the Justify Change pattern.

The above patterns represent a very typical project-centric understanding of demand and project management. However, the Lean dynamics towards more agile, demand pull IT development may mean the diminution of formal IT projects.

This can already be seen in reporting and simple Web site environments, where significant new functionality may be requested without a full project lifecycle. Similarly, even “major” applications may have an initial project to set up the core platform, and then move into a non-project “business as usual” cycle of iterative enhancements.

 

Notice that in the above example the demand requests do not all route to a Project. New functionality may simply be handled as a Release, or even a reporting Service Request. This may become more and more the pattern, with the increasing penetration of Agile methods.

Similarly, service restoration can also result in demand:

 

The flow from Restore Service back to Release and Change shows the essentially circular nature of the IT processes. In considering the above flow, where does Accept Demand fit? Is it invoked only after the identification of an Improvement Opportunity?

 

Or has demand actually been accepted at the point of identified need to restore the service? Or both?

 

This last pattern would appear unusual from the point of view of a traditional incident manager, as one key aspect of demand management does not usually enter into incident management: whether or not to accept the demand. Service restorations by definition are in general pre-approved.

However, as established in the “Manage entitled vs. discretionary demand” pattern, the demand management capability needs visibility into the ongoing volume of preapproved requests and its forecasted trending, so that larger scale demand requests can be prioritized against the resources available. And, the ongoing organizational commitment to the service, to fulfill requests against it, run it, restore it, and improve it, IS fundamentally demand!

To put tersely: the existence of a service IS demand.

Understand aggregate demand

The “Accept Demand” process is defined in this book as

Capture, prioritize, and authorize and track response to requests and identified needs for IT services. Example: Accept request for new functionality in an HR system module.

This seemingly simple definition obscures a fundamental tension in the concept of demand management: is it about the service lifecycle, transactional service delivery, or both? In researching the second edition of this book, the author found diametrically opposed industry guidance coming from the project management and ITSM communities.

Demand for new and changed services is clearly part of demand management, but how do we consider the ongoing demand for delivery of those services? Demand for the delivery of a given service profoundly affects whether and how that service can be enhanced, or other services created or enhanced.

 

Figure 47. Integrated Demand Management (click for larger)

Figure 47 graphically shows the relationship between the ideas of “entitled” demand versus “discretionary” demand.

From a business school perspective, the above diagram might be painfully obvious – of course one must fulfill operational commitments before embarking on discretionary initiatives! But IT management is fragmented, and too often demand is qualified and dispatched with no concern for resource availability, especially if the same resources are working disparate processes with no aggregate visibility. This leads to the notorious problem of overburden in IT work.

For example, a systems administrator might find themselves fulfilling service requests from development teams for ongoing capacity enhancements, called upon to resolve a critical service outage, and also tasked with building out a server cluster for a project. There are few if any IT product suites that would provide an aggregated view into the consumption of that individual’s effort.

Aggregate demand means all of the processes:

• Execute Project
• Deliver Release
• Complete Change
• Fulfill Service Request
• Core Transactional Delivery
• Restore Service
• Improve Service
• Retire Service

It is not possible to manage the IT portfolio holistically unless all of this demand is understood. Yet, modern practice tends to build distinct queues for each with the individual IT contributor required to coordinate and prioritize across the incoming workstreams. This leads to overburden, poor morale, and many forms of IT waste. (In this author’s opinion, this is the biggest crisis facing the large IT organization.)

What's the object?

Many discussions of IT improvement, and continuous improvement and Lean more generally, start with core process management objectives of shrinking cycle time and improving quality (or at least consistency) of results.

Improving these both 1) improves the process value itself and also 2) improves the lifecycle entities the process affects (applications, infrastructure services, assets, and technology products).

To understand this second kind of value, however, one needs to ensure that the processes are properly linked to what they affect.

IT lifecycles and processes

Here is an overview of the 2nd edition's process architecture.(click for larger)

 

Lifecycle definitions:

Application Service Lifecycle	The cradle to grave existence of a direct application of computing technology assets to a business problem. Example: Payroll system, Email service.
Infrastructure Service Lifecycle	The cradle to grave existence of a service which indirectly supports the application of technology to one or more business problems. Example: Network service, distributed hosting service, mainframe transaction processing subsystem
IT Asset Lifecycle	The forecasting, acquisition, operation, and disposition of a tangible IT asset. Example: Dell server w/particular serial #, Oracle database software license.
Technology  Product Lifecycle	The specification, choice, operation, maintenance, and removal of a class of assets. Example: SQL Server 2000, independent of any particular license. Compaq DL580, as a server type, independent of any specific unit.

Process definitions:

Accept Demand	Capture, prioritize, and authorize and track response to requests and identified needs for IT services. Example: Accept request for new functionality in an HR system module.
Execute Project	Within a boundary of time, scope, and resources, complete a set of activities (typically unique or at best semi-repeatable) resulting in a value-added end state delivery.  Example: Define requirements, build, test, and release new HR functionality.
Deliver Release	Coordinate the assembly of IT functionality into a coherent whole and deliver this package into a state where the customer is getting the intended value. May be a sub-process within Project, but not all Releases are governed by Projects, and some Projects may have multiple Releases. Example: 1st iteration of new HR functionality.
Complete Change	Capture, assess impact and risk, track approvals, and coordinate service requests (as needed) to change some controlled aspect of the IT environment. Example: Move new HR code from development source control library onto production server.
Fulfill Service Request	Capture, track, and ensure delivery of routine requests (typically not involving change control) for IT services.  Example: Provision access to the HR system.
Deliver Transactional Service	An individual, granular delivery of a given IT transaction. Example:  Look Up Employee.
Restore Service	Identify IT service outages, assign/escalate, and track through at least tactical remediation. Example: Restore HR System by rebooting application server.
Improve Service	Identify enhancement opportunity for IT system or capability, prioritize, assign, and track through completion. Example: Deploy HR System on redundant cluster.
Retire Service	Identify retirement opportunity for IT service, prioritize, assign, and track through completion. Ensure critical risk and compliance objectives (data preservation and/or media destruction). Example: Retire current HR system from vendor X as part of overall strategy to move to service from vendor Y.