A sound investment?

Analysing and maximising ROI from knowledge management tools

Before committing to the implementation of an expensive knowledge management system, it is essential your organisation knows where the return on its investment will come from. Andre Valente and Thomas Housel present a framework aimed at making it easier for companies to identify potential returns, and to select the right tool for the right job in the first place.

The effective selection and deployment of knowledge management software tools is a critical success factor in any knowledge management initiative. KM tools are frequently very expensive, and therefore demand careful selection and detailed justification. As such, it is critical to develop a firm understanding of where the ROI for any investment is coming from, and how to maximise it.

At the same time, the broad and multidisciplinary nature of the problems surrounding knowledge management allows a wide variety of software products to be marketed as KM tools. As a result, there is a deluge of tools claiming to provide a KM solution. It can be a daunting task to analyse these tools and select the best one – and, in particular, the one that provides the best ROI.

Knowledge management tools and ROI

There are many problems associated with forecasting and calculating the ROI for an investment in KM tools. Two specific issues we address in this article are:

Identifying where the returns can come from. There is a tendency to look for cost savings in, for example, spending less time performing search operations. However, this provides only a limited view. The return on KM tools has to be found in the specific processes the tools have changed, and tied to the specific goals aimed for by the KM initiative the tools are supporting. The problem is, it is not easy to identify the processes that are changed when a KM tool is made available;
Aligning the tool selected (or built) with the problems that are being solved in the organisation. Far too often, tools are selected based mostly on their technical capabilities and features or their cost. However, even a tool with great technology and low cost can fail to provide any returns if it does not match closely with the objectives and needs the business is looking to fulfil.

In this article, we present a framework called Knowledge Structure and Services (KSS) that can support a business in meeting these two challenges. To tackle the first challenge, our framework provides a structured way to understand and describe the problems the KM tool is supposed to solve. This highlights the processes that are touched (and hopefully improved) by the tools. This is a key factor in identifying where the return for the investment on the tool should come from. It also frequently reveals places where one may forget to look for returns.

To help in overcoming the second challenge, the KSS framework can increase the understanding of the capabilities of the tools available on the market. In this way, using the framework to analyse both problems and tools can help a business to match the right tool to the right job. The final result is a better alignment between KM problems and KM tools, and thus improved ROI.

A framework for understanding KM tools

The KSS framework is centred on the concepts of knowledge structure and knowledge services. These two dimensions can be used to form a matrix in which specific KM tools are positioned.

Knowledge structure

There is a wide range of levels of formalistion or structure in the way knowledge is represented in KM systems, as detailed in the list below. From top to bottom, we increase the formalisation and precision of knowledge, while from bottom to top, we accommodate more informality and ambiguity. Knowledge forms towards the top are relatively easy for people to create and update, while knowledge forms listed at the bottom demand knowledge engineering and incremental analysis.

From the least structured to the most structured, the knowledge forms are:

Knowledge in people’s heads is intrinsically non-formalisable, yet many organisations rely on this kind of knowledge and the support of tools to find out who knows what, where, within an organisation;
Audio and video contain multiple ‘streams’ of knowledge such as music, voices, faces and objects. Humans have a much easier time than machines in interpreting and indexing this kind of knowledge, but recent advances have allowed improved automated categorisation;
A raw text document is the formal equivalent of an audio track. Its complexity is comparable natural language, making it equally difficult for machines to process;
In contrast, an HTML document with mark-up tags can reveal the text’s structure. Patterns and regularities in the document structure can aid in interpreting its content. For example, there are now tools that ‘wrap’ structural patterns in HTML text into semantic descriptions;
Structured documents using formats like XML (or its ancestor, SGML) explicate the semantics implicit in HTML mark-ups. For example, an XML document may contain a tag such as USA that clearly indicates that ‘USA’ should be understood as the name of country;
The next kind of knowledge structure is ‘tuples’ of data, the essence of information stored in databases. For example, databases may contain lists of relationships between countries and their populations. Most databases are designed for efficient storage and retrieval of this kind of information, but as a result they are usually incomprehensible to humans. Recently, there has been a trend towards using XML documents as a readable form of database. For instance, a sequence of tags can contain a tag inside a tag to indicate a relationship between a country and its population;
Categorised information has roughly the same level as structured information in databases. Taxonomies such as the ones we use in biology are examples of categorised information. This kind of knowledge is used extensively by directory sites such as Yahoo! to provide taxonomies of concepts, ideas or subjects;
Formal knowledge is used here in the mathematical sense, meaning logical statements such as theorems and equations. This kind of knowledge can be used in a rigorous way to ensure that all semantics are explicit and that all rules are followed.

The level of structure in the knowledge directly affects the amount of automated processing that can be performed because more structured knowledge employs powerful semantics. As a result, it is much easier to process and manage the contents of an XML document than the contents of an HTML document.

Managing highly unstructured knowledge requires more structured descriptions of the content. For example, because it is very hard to recognise faces or voices directly, many tools that manage audio and video knowledge employ simpler forms of knowledge to index the content, such as keywords, categories or close-captioned text.

Knowledge services

Another useful dimension is the range of services KM tools provide. By services, we mean tasks or activities in handling knowledge that can be at least partially automated. To make sense of the disparate services available, knowledge services may be divided into three main types: infrastructure services, core services and packaged services. These services build on one another: packaged services make use of core services, which in turn rely on infrastructure services, etc.

Infrastructure services

Infrastructure services are usually needed to implement any KM solution. We have identified five basic types of infrastructure services:

Communication services enable electronic communication between users. Examples are e-mail, file transfer and chat rooms;
Collaboration services allow for groups of people to communicate through online meetings, shared whiteboards, discussion groups, as well as directory services. Building upon communications services, these tools are also known as groupware, the best known example of which is Lotus Notes;
Translation services transform knowledge from one file format to another or from one language to another;
Workflow management services define processes and support their online execution and control. Typical applications allow users to execute and enter the results of subtasks, and view the status of other subtasks. Workflow management services frequently build upon collaboration services;
Intranets and extranets are web-based applications that streamline communication within an organisation (intranets) or between different organisations (extranets). Intranets and extranets use the web to provide access to content and services. They extend or aggregate other infrastructure services, adding other services such as user management, personalisation and configuration.

Core services

Core services are central to defining KM problems and solutions, as these are the services that explicitly and directly access knowledge repositories. Different core processes involve people or systems with different roles, including knowledge producers, holders, organisers and users. Knowledge producers create knowledge, while knowledge holders learn from other sources. Knowledge organisers work like librarians and allow producers to add knowledge in an orderly fashion to facilitate retrieval by users. Knowledge users consume knowledge to execute their tasks and processes.

Key features of the five core services include the following:

Knowledge generation services produce knowledge in forms that can be stored in a knowledge repository. Used by knowledge producers, these tools distil, refine or simply create new knowledge that is then entered into the repository. These tools frequently involve some kind of automated learning, such as data mining or pattern recognition. Commercial versions include Monterey 1.0 from Inferscape;
Knowledge capture services facilitate addition to repositories. For example, capture tools allow users to add documents to repositories as well as meta-information to support indexing. A simple example of meta-information is the ‘document properties’ mechanism of Microsoft Word, which allows a user to manage information about the document being edited, such as author, revision number, subject and date;
Knowledge organisation (indexing) services help knowledge managers arrange items in a repository to facilitate retrieval and use. Typical knowledge organisation services manage knowledge about a repository and its items – for instance, indexes, taxonomies and directories;
Access management services determine who can access elements of the repository. They control access to the knowledge repository and are usually based on directory services. A typical mechanism to manage access is to define permission levels for a set of roles that are assigned to users;
Retrieval services include searching and navigating functions, as well as translation, visualisation and integration. They create value by making knowledge available for specific uses and may provide personalisation and configuration services.

Packaged services

Packaged services aggregate lower-level services to solve specific types of problem such as customer relationship management. A significant part of the existing knowledge management literature concentrates on packaged services. This focus is attributable to the fact that these types of problems are clearly connected to end-user needs. For example, it is easier for a CIO to justify the purchase of customer relationship management tools than of search engines.

Three classes of packaged services that have received a great deal of attention are:

Customer relationship management services, which provide integrated management of all information relating to a company’s clients. They typically allow internal channels to share and add to the same central knowledge base. Siebel and PeopleSoft are among the leading providers of CRM services;
Business intelligence services, which manage knowledge about competitors and partners. They usually aggregate and provide unified interfaces to information from news agencies, public and private databases, economic and social information, and the world wide web. They also filter and classify information into categories;
Enterprise information portals, which act as specialised gateways to internal and external sources of knowledge. A typical example is My Yahoo!. There have been a large number of EIPs developed over the past few years, many of which have achieved considerable success.

The KSS matrix and the KSS service checklist

Different tools provide distinct arrays of services and manage specific types of knowledge. We visualise the relationships between KM tools in terms of the types of knowledge they handle, and the types of services they offer. Two diagrams display these relationships: the KSS Matrix (figure 1) and the KSS Service Checklist (figure 2). These diagrams position the kinds of solutions provided by given products or vendors. (Of course, a more complete analysis could include additional elements such as hardware and software platforms, the quality of customer support and price.) They can (and should) also be used to analyse the problems an organisation is trying to solve with KM tools, facilitating the match between problem and solution.

The KSS Matrix

The KSS Matrix ensures that the types of knowledge handled are intimately connected with the core services provided. Tools may support different sets of services for each type of knowledge. The KSS Matrix is displayed in figure 1. The horizontal axis recognises the five core knowledge services while the vertical axis displays the eight basic levels of the knowledge structure dimension.

One KSS Matrix is used for each tool analysed, as well as for each problem that needs to be solved. A KSS Matrix is completed by adding small or large squares to each of the cells. Filling a cell indicates that the tool provides a specific service that manipulates knowledge with a given level of structure. The size of the square filling a cell represents the scope of the service offered by the tool. A large square denotes a major offering with a comprehensive set of features, while a small square marks a service that is offered in either a restricted scope or with restricted functionality.

The KSS Service Checklist

The KSS Service Checklist recognises services beyond the core services. A checklist is employed because infrastructure and packaged services are independent of the types of knowledge managed. The KSS Service checklist in figure 2 shows the five infrastructure services and the three packaged services. To the right, we add squares indicating that a service is provided. As with the KSS Matrix, the size of the square represents the scope of the service offered, with a large checkmark indicating major offerings and small checkmarks representing incomplete or restricted offerings.

Using the framework to enhance ROI

An important problem confronting a KM project is how to find the best match between the specific problems and needs of a given enterprise and the KM tools available on the market. The KSS Matrix and the KSS Service Checklist provide a methodical means to perform this task. First, the framework can be used to specify what kinds of knowledge structures will be handled, and what kinds of services are needed for each of them. Second, the KSS framework can be used to analyse prospective tools for specific uses. The KSS diagrams built in the first step work as a target diagram to be matched with the capabilities of specific tools. The analysis may point to a single best tool, or show that the best solution may lie in a combination of two or more tools. It also helps in conducting cost/benefit analysis, for example by focusing first on the services and structures that provide the highest returns.

While the framework can help identify where the returns come from and ensure alignment of problem and solution, the issue of quantifying that return remains. Estimating the value added by a particular KM tool requires a new way of thinking about measuring ROI. We have found that the knowledge value added (KVA) method provides a fast and efficient way to estimate the returns associated with a given tool (see the International Engineering Consortium KVA tutorial at http://www.iec.org/online/tutorials/kva). KVA provides a methodology for proportionately allocating revenue and cost to a company’s core processes and their supporting information technology tools.

Conclusions

The KSS framework provides a convenient way of disentangling the confusion caused by the over-use of the term knowledge management in the context of tools that support KM. The framework allows each tool to be characterised by defining the types of knowledge it handles and the types of services it provides in supporting KM processes.

The KSS Matrix and the KSS Services Checklist help with the visualisation of the coverage offered by a specific tool, and are therefore useful means to quickly compare and distinguish different tools. They can be used to evaluate specific needs and match them to the services provided by available tools. They also serve to separate different types of KM tools. Finally, when the matrices are used to help identify the most promising KM tools, the potential levels of return they will provide can be estimated using the KVA method. This also will help answer the question most often asked by management: “So what’s the bottom line?”

This article is based on chapter 8 of Housel, T. & Bell, A., Measuring and Managing Knowledge (McGraw-Hill, 2001), which was written by Andre Valente and Thomas Housel.

Andre Valente is the CEO of KS Ventures, a California-based KM consultancy. He can be contacted at: avalente@ksventures.com

Thomas Housel is a professor of information management at the Naval Postgraduate School. He can be contacted at: housel@marshall.usc.edu