Contents
 Why measure the performance of KMS
 Case study
 Assignment
Why measure the performance of KMS
As an intangible asset
Knowledge results are difficult to measure
and quantify
The question is “how can organizations
measure KMS benefit?”
Why measure the performance of KMS
 Provide a concrete way of defining what a
KMS will achieve and whether it has met its
goals
 The need to improve the transparency of
value creation to stakeholders – for
governance and financing person.
Benefit of KMS measure
 Representation a way of learning what works and what
does not
 Securing funding for KM implementation
 Providing targets and feedback on implementation
 Assessing the success of implementation
 Deriving lessons for future implementation
Non existing of standardized method for evaluating KM
results in general and specially in monetary terms
Managers often use easily estimable quantitative factors – ROI to
justify the dollar invested and gain support from management
in times of tight IT budgets
 While these benefits have not been tagged.
 better informed employee
 Streamline d communication with customers
 Repeatable and consistent process
If change can be measured, it has to be cross-checked
whether these are the result of KM activities
The question, then
How can organizations measure KMS benefits?
Case study
Case study: Wilson Sonsini Goodrich & Rosati
 Law firm with 600 lawyers
 Provide legal services to technology and growth
business enterprises at all stages of development.
 Its headquarters are in California, with offices in six
other US cities.
Knowledge intensive
firm
Why WSGR needs KM, KMS
 12 lawyers – 100 lawyers – 600 lawyers
 One office – 7 offices in 6 different locations
 Knowledge dispersion
 Clients’ expectation increased
Need a strategy to response to client
faster, and better quality
How WSGR developed KM
 KM Program started in 1998
 Report to IT group until 2001
 In 2001, Director of KM was nominated and formal
team of KM was put in place
 One team – 2 staff – one focused on the litigation
aspect (the court), the other focused on transaction
and advising clients (negotiation)
Case study: Wilson Sonsini Goodrich & Rosati
The firm runs and manage KM program by a
small full time team.
Support
To sell and deliver the services as effectively
and efficiently as possible.
Business
objective
Case study: Wilson Sonsini Goodrich & Rosati
The KM program in WSGR focus on
1. getting new lawyers and other professionals up to
speed quickly, and
2. continuously developing their legal knowledge and
professional skills
What KMS supports
Support client service by putting the essential expertise experience of the
whole firm by a fingertips - readily or instantly available ,of each
lawyer.
To
Deal with different knowledge challenges that the
organization is facing via KMS
How WSGR developed KMS
 Three types of knowledge are focused:
2.
Work product
Project databases
3.
People, Expertise locator
1.
How WSGR developed KMS
Work product collections
Model and precedent documents to guide lawyers as
starting point for drafting new deal or case documents.
This includes sample document , patterns, high quality
templates, forms, and transactional documents for using
as guideline for lawyers.
How WSGR developed KMS
Project databases
Principle deal types are profiled such as private and
public financing, mergers and acquisition
Attach profiles as possible
Lawyers can search profiles for deal similar to those they
are working on.
Good stating point
To show a client that they have valuable experience with
a certain deal type, industry, geography
How WSGR developed KMS
People, Expertise locator
Q & A process to find internal and external
experts
Dedicated e-mail addressees are provide for tough.
Provide practical and related questions and answers to
all staff via online data bases.
The expert location process
Expert 1
Online
submission
of problem
Automatically
directed to the best
available experts
Expert 2
Expert 3
Solutions
Why WSGR evaluate KM program
KM team in WSGR runs program to sell and deliver legal
services as effectively and efficiently as possible.
KM team costs the firm money for salaries, technology,
and fee-earners’ time
No direct revenue
Why WSGR evaluate KM program
To justify the firm’s investment
KM team needs to find the way to show how the
program benefit
Output, Usage, Results are employed
to measure KM benefit
How WSGR evaluate KM program
Three models used to measure KM benefits
1. Outputs
2. Usage
3. Results
How WSGR evaluate KM program
 Output: measure
 quantity of easily accessible
 High quality knowledge
provides an answer to the
essential question
Answers the essential
questions
“Have you built it?”
The firm KM resource must have critical mass of
material before lawyers use it
How WSGR evaluate KM program
 Usage: measure
 Who use the KM resource
 In what way they use
 How often they use
Answers the essential
questions
“Did they come?”
Heavy usage of a resource generally indicates
that lawyers value it highly and low usage means
it is worthless
How WSGR evaluate KM program
 Results: measure
 The outcomes of using the KM resource answers
the most important questions,
“How did the clients and the firms benefit?”
Great benefit KM should be replicated and
expanded wherever possible.
One provides little benefit should be either
repaired or terminated
Measuring output
 Work product collection
 Project data based
 Expertise locator
Measuring output : Work product collection
 Work product collection - precedent documents
 Measure : number of items - briefs in the brief bank ,
- form in the form collection
Is the firm collect more or less, relative to the pace of over all
firm and the specific practices?
How does it help?
Too low number of documents relative to the size or growth of
the practice force the firm to figure out why, and should do
something to increase them.
Measuring output : Work product collection
 Work product collection – precedent documents
In early 2003 – too low number of briefs in brief bank compare to the
peace of litigation practice which is quite busy.
In March 2003, the firm design to increase the number of
briefs in brief bank by improving the collection process.
Inform leader to tell everyone to file each brief with the KM
department at the same time that it was filed with the court.
Having secretaries responsible for file maintenance (rather than
lawyers) send the briefs.
These two changes caused double of briefs in every months,
without major change in the pace of litigation business.
Measuring output : Work product collection
Jan
Feb
Mar
Apr
May
Jun
Jul
Aug
Sep
Oct
Nov
Dec
Measuring output : Project data based
 Project data based- collections of deals that are profiled.
 Measure : percentage of deals closed in a given
month for which profile meet a minimum standard.
 Percentage are analyzed over all and by the practice
groups
How does it help?
Number that are too low force the firm to figure out why, and
to do something to increase them.
Measuring output : Expertise locator
 Expertise locator – e-mail based Q&A facility. Use by lawyers to
locate internal and external experts and examples of their
expertise.
 Measure : number of questions sent and responses each week
How does it help?
Too
many easy and poorly draft questions
The channel becomes devalued because recipients lose interest and
through out
To keep the quality of e-mail high
 lawyers are trained to explain clearly both reason for the
question and the question its self.
 Send reminder when necessary – cooling off effect.
Measuring usage : site and document usage
 Site usage
 Measure : a website counter facility is used to track how
many unique users visit KM site each week
How does it help?
It represents the awareness of and demand for ,
 each online KM resource,
 reaction to training ,
 where the KM team should be investing their time and
expertise.
Measuring usage : site and document usage
 Document usage
 Measure : number and type of users who view each
KM document
How does it help?
They show
 which KM documents are the most demand and
should be revised most frequently
 which lawyers can be called for feedback.
Measuring result
In WSGR results are measured only
anecdotally – user stories that show how a
layer has used a KMS to get client better,
faster, more efficient results.
Assignment
 ประเมินระบบ KMS ปริญญาโท โดยใช้
 Output
 Usage
 Result
Outputs
Work product
collection
ต้องมี เอกสาร
อะไรบ้าง
ปัจจุบ ันมี
อะไรบ้าง
ั ว
่ นทีม
สดส
่ ี
ควรปร ับปรุง
อย่างไร
Project data
based
ควรจ ัดกลุม
่
เอกสารอย่างไร
ปัจจุบ ันจ ัด
อย่างไร
ความยาก/ง่าย
ของการค้นหา
เอกสาร
ควรปร ับปรุง
หรือไม่ อย่างไร
Usage and Results
A
Who use the
KM resource
In what way
they use
How often
they use
Benefits
B
C
D