Proceedings of the 40th Hawaii International Conference on System Sciences - 2007
Building a Research-Practice Partnership:
Lessons from a Government IT Workforce Study
Sharon S. Dawes
[email protected]
Center for Technology in Government,
University at Albany/SUNY
Abstract
For decades researchers have sought ways to make
their work have more impact on the world and
practitioners have wished for research that actually
helps them solve pressing problems. This is
particularly true for digital government research
which emphasizes positive change. Differences in
professional culture and lack of mutual understanding
about the nature and uses of research lead to this gap.
A few studies have outlined ways to bring research and
practice closer together in mutually useful ways. This
paper describes the development and operation of a
research-practice partnership focused on a skills
assessment of a government IT workforce. It illustrates
how these two communities can collaborate to conduct
rigorous research that is also readily usable. The
advantages are considerable for both rigor and
relevance, but there are also significant costs
associated with working in this way.
1. Introduction
For the past twenty years, ongoing development of
new technologies coupled with powerful economic and
demographic trends have elicited strong concerns about
recruitment, retention, and succession planning for the
government information technology (IT) workforce.
These concerns stem from several sources including
government’s increasing reliance on information
technologies to deliver all kinds of programs and
services to citizens. In addition, relatively rigid civil
service systems limit the ability of public employers to
act quickly or flexibly to meet workforce needs and
often also limit employees’ individual career
aspirations. Moreover, the burgeoning growth of IT in
the private sector, both within established businesses
Natalie Helbig
[email protected]
Center for Technology in Government,
University at Albany/SUNY
and as a business itself, has created strong competition
for skilled IT professionals. During the “internet boom”
young people with these skills overwhelmingly chose
private sector jobs over government jobs for better pay,
career mobility, and flexible working conditions. All
of these trends make IT workforce planning and
development especially problematic for governments
today [6, 14, 23, 24, 26].
These workforce trends are also a major concern for
many large employers outside government. General
management or worker surveys to assess trends in
hiring, compensation, training, and attitudes towards
work environments have become routine. The Gartner
Group, for example, surveyed 1,300 CIOs around the
world to determine the top business and technology
priorities in 2005 [9]. Gartner found that only 39
percent of CIOs believe they have the right people to
meet current and future business needs. More than half
(51 percent) of CIOs who responded to the study also
reported concerns about the aging of the IT workforce
and associated difficulty in attracting new people with
the right skills to meet the new requirements.
The Information Technology Association of
America (ITAA) conducts an annual survey to monitor,
assess, and communicate broad market conditions for
IT employers and employees [13]. ITAA conducts
telephone interviews with 500 randomly selected
managers who oversee the hiring of IT workers within
their organizations. The 2004 survey’s major findings
revealed that the overall IT workforce is growing
slightly, non-IT companies added the overwhelming
majority of IT workers, programmers represent the
largest single group of IT workers, and the highest
future demand was for technical support workers.
Governments also conduct IT workforce studies.
The purpose of surveys conducted in 2003 and 2004 by
the US Office of Personnel Management is to help
federal government agencies address broad IT
management guidelines put forth by the Clinger-Cohen
1530-1605/07 $20.00 © 2007 IEEE
1
Proceedings of the 40th Hawaii International Conference on System Sciences - 2007
Act, the E-government Act, the President’s
Management Agenda, and Office of Personnel
Management (OPM) and Government Accountability
Office (GAO) guidelines on human capital
management [8, 22]. The goal of the federal surveys is
to compare the skill sets of existing employees to the
competency needs of federal agencies. The 2004
survey attempted to collect data from over 75,000
federal IT employees through a voluntary web-based
survey. The main findings reveal that three quarters of
the respondents were over the age of forty, and 13% of
them intend to retire within the next three years. Due
to the low response rate of 28%, however, these figures
may not reflect the workforce as a whole [8].
In the State of Hawaii, a public-private partnership
was created between trade associations, a university,
and private companies to survey managers and
employees in over 300 organizations in all sectors.
The purpose was to assess the current information
technology skill sets in Hawaii’s high-tech workforce
and to project the IT skill sets needed for the future
[10]. The final report stated that due to a small
response rate of 24% they were not able to draw any
substantive conclusions or projections from the data.
However, they did provide general information on the
types and levels of training respondents preferred, and
the types of certifications held and desired.
While the information generated by these studies
can be useful, it is often quite general, pertaining to the
overall IT workforce as in the case of the ITAA and
Gartner studies, or limited by low response rates as
happened in both the OPM and Hawaii studies.
Methodologies are not always described (Gartner’s is
not) and reports may not discuss material weaknesses
in data or methods. Perhaps more important, while the
broad industry trends reported from these studies are
often consistent and usually worrisome, none of the
studies provides the kind of data that can support
specific actions that will address these trends, which is
most often what IT leaders need and want to do.
This paper presents and discusses a researchpractice partnership that helped to overcome these
problems and is generating detailed, high quality
information that can be used as the basis for action
planning, policy making, and workforce investment
decisions. The rest of this paper is divided into four
sections. The first section presents arguments from the
research literature that illustrate the value of active
partnerships between research and practice. We then
describe in detail a partnership-based study of the IT
workforce in New York State government. This is
followed by a discussion of the benefits and costs of
the partnership approach and conclusions about how
they might be sustained.
2. Value of research-practice partnerships
The lack of connection between research and
practice is a matter of long concern and debate in the
academic community. In 1990, Ernest Boyer called for
a scholarship of engagement which comprises four
kinds of scholarship: (1) discovery or the creation of
knowledge, (2) integration of knowledge across
disciplines into broader contexts of understanding, (3)
the scholarship of practice or the purposeful generation
of issues and the application of knowledge to the world
around us, and (4) teaching or the conveyance of
knowledge in the service of both action and future
scholarship [4]. This four-part definition of scholarship
strongly values traditional academic research, but
argues that it is not the sole means, or always the most
useful means, of building knowledge. Van de Ven
(2000) contends that the declining usefulness of
academic research as a means of solving practical
problems may be addressed by emphasizing “action” or
applied science in addition to “normal” or academic
science [27].
The introductory article in a special research forum
in the Academy of Management Journal traces the long
history of the gap between organizational research and
managerial practice going back to the 1980s. It
outlines the persistent chasm that results from
researchers and practitioners belonging to separate
communities with different values and ideologies [3] as
well as the lack of empirical research to back up mostly
anecdotal evidence of what to do about it [24]. The
forum strongly emphasized the social nature of the
knowledge transfer process and the importance of
actual working relationships among the parties (e.g.
[16]. Boland and co-authors focus on the way
knowledge is represented. They present an
experimental study that shows how different forms of
knowledge representation are associated with different
decision-making processes and managerial outcomes.
They speculate that “knowledge itself is being limited
through current academic writing and publishing
practices,” which miss a significant opportunity to
represent knowledge in a variety of ways more useful
to managers [16].
Similar concerns are expressed in the information
systems literature. Benbasat and Zmud, for example,
argue that research relevant to practice must address
current problems and challenges in a timely way, make
implementable recommendations, synthesize a larger
body of knowledge, prompt critical thinking that
2
Proceedings of the 40th Hawaii International Conference on System Sciences - 2007
challenges the practitioner’s causal assumptions, and
convey findings in a style and tone that uses everyday
language in a direct and actionable way [1].
In a rejoinder essay, Lyytinen [15] takes some issue
with the narrowness the Benbasat and Zmud
prescription, urging an even broader perspective that
includes the academic institutional policies and
incentives necessary to form good research relations
with practitioners and their organizations.
In a report that helped define the National Science
Foundation’s Digital Government Research Program,
[7], the gap between research and practice was
explicitly addressed and the report made specific
recommendations for matching research resources to
government needs including, 1) the need to understand
what research-based products are useful to government,
2) to develop innovative methodologies that can speed
the dissemination of useful knowledge, and 3) to
experiment with new partnerships that benefit both
researchers and government.
Despite these observations, research programs still
very seldom include a strategy for involving
practitioners in the research process or for making
research findings available and useable by government.
Action researchers have offered some insights into
this problem. Davison et al. [5] offer a set of principles
for conducting action research that is both rigorous and
relevant.
These include attention to an explicit
researcher-client agreement, conducting the work in a
cyclical process from diagnosis through reflection,
accomplishing
change
through
action
and
understanding of the organizational context, and
learning through reflection. Other research in fields
ranging from education [11] to natural hazards [28] to
organizational sociology [2] all indicate that “sustained
interactivity” [12] before, during, and after a research
project is the best way to assure that the research is
relevant to, and understood and used by practitioners.
According to Yin and Moore [28] such research should
identify at the design stage who will use the findings,
be open to modifying designs in ways that meet user
needs without compromising integrity, and plan
explicitly to produce major products especially for a
practitioner audience. This kind of approach rests on
rich social interactions between investigators and
research users in a continual exchange of ideas in
which both parties learn from the expertise and
viewpoint of the other.
The design and execution of the study reported here
illustrates how a partnership approach can make use of
these different kinds of knowledge and expertise in the
design, execution, and impact of a research project.
3. The State IT Workforce Study
In 2002, New York State created a Chief
Information Officer (CIO) Council to oversee and
collaborate on IT policies and strategies. Senior
executives from more than 70 state agencies, local
governments and authorities came together to address
issues and policies that impact statewide, agency, and
interagency IT initiatives. Seven standing committees
were established including groups focused on
leadership, fiscal/procurement issues, human resources,
security, strategic planning, technology, and
intergovernmental communications [19]. The human
resources (HR) committee is a 20-member
multidisciplinary team composed of agency chief
information officers, human resource directors, staff
development experts, civil service representatives, and
labor-management specialists.
IT workforce issues have been a concern in New
York since the 1980s and were designated high priority
areas in the 2004 and 2005 New York State Enterprise
Information Technology (IT) Strategic Plans [19, 20]
and statement of Enterprise Architecture Principles
[21]. The HR Committee was charged with soliciting
and understanding the multiplicity of agency views on
IT workforce issues and to create a unified NYS IT
workforce development plan.
As a first step, the HR Committee determined that
no hard data existed about the skills of New York
State’s IT workforce. Committee members agreed that
this lack of data limited the state’s ability to identify
gaps between existing IT workforce skills and agency
IT needs; hindered sound agency planning in training,
retention and succession planning; prevented strategic
use of training monies, and limited the ability of
employees to identify and build their own career
development plans. Consequently, the committee began
investigating methods for collecting the needed
information.
A sub-committee identified and evaluated different
alternatives and best practices for obtaining good
quality workforce information. After an extended
search of approaches used in both the public and
private sectors, they recommended adoption of the
employee-oriented survey tool developed by the US
Federal Chief Information Officers’ Council (CIOC), in
partnership with the US Office of Personnel
Management (OPM). This instrument was favored
because it had been specifically created for use in
government and had already been tested and used once
at the national level, with a set of lessons learned that
could be used by others. To carry out the federal effort,
OPM had relied heavily on a private sector contractor
3
Proceedings of the 40th Hawaii International Conference on System Sciences - 2007
to create the instrument, and collect and analyze the
data. The New York sub-committee members reached
out successfully to the federal project manager to
acquire the survey questions and methodology,
including information on plans to create not only
employee-related findings, but also agency-related skill
needs and a way to understand the gaps between them.
Due to differences in HR terminology and focus,
significant work was then needed to tailor the federal
survey to meet the needs of New York State. However,
this initial collaboration between state and federal
entities saved considerable development cost and
reduced the length of time it would have taken to create
a survey from scratch.
3.1. Partners
In early 2005, the HR Committee recommended that
they organize a partnership of State agencies, labor
unions, and an independent research organization at the
state university to help design and administer two
surveys, one to be completed by employees and a
companion survey to be completed by agency CIOs.
The committee considered using the same contractor
that had supported the federal effort, but decided for
reasons of cost, management control, and transparency,
that a state university research unit would better serve
their needs. The state labor relations agency provided
funding for the study and acted as a conduit for
bringing in representatives from the major employee
unions. The university research center worked with the
committee to develop a detailed plan outlining the
scope of the project, timeline, deliverables, working
assumptions, participant roles, and communication
mechanisms. An iterative discussion process between
the committee, other state agencies, and the research
center set the stage for a research-practice partnership
in which all parties contributed expertise to the process
and products of the effort.
3.2 Project research methodology
The goal of the government workforce IT skills
assessment project was to gather information to help
the State better meet the training and development
needs of its IT professionals, and to identify future
needs for IT skills. The project included the design,
administration, and analysis of two voluntary on-line
surveys. The first was directed to IT employees in New
York State and the second to Chief Information
Officers (CIOs) in state agencies.
The employee survey, conducted during
March and April 2006, was sent to nearly 5000 IT
professionals employed in 54 state agencies,
authorities, and boards. The survey population
consisted of all state employees who held one of a
specified set of technical job titles as well as other
employees in non-technical titles who were identified
by their employing agencies as doing technical work.
The first list comprised 4,586 employees and was
provided by the Department of Civil Service at the
formal request of the state CIO. The additional
employees were identified by their agencies during a
process of list validation in which each agency
designated a liaison who reviewed the Civil Service
listing, made additions and corrections and added email
addresses so that all employees could be contacted
directly by the university center administering the
study. The CIO survey occurred during the same time
period. This survey was also conducted on line and
most communications took place via email.
A formal Human Subjects Review was prepared by
the research center in cooperation with the committee.
Many discussions were conducted about ways to assure
confidentiality for individual respondents while making
the data usable at both the statewide and agency levels.
In addition, the effect of the state freedom of
information law needed to be assessed and considered.
The final protocol that resulted was approved by the
university Institutional Review Board. It included
statements of informed consent assuring individual
respondents of their rights as research participants, and
stating how their identities and data confidentiality
would be protected and informing them how the data
would be used in the analysis.
A formal communications and outreach plan was
also jointly developed and included letters from the
state CIO to all agency heads, CIOs and IT employees
informing them of the goals of the survey and
encouraging them to participate. Posters designed by a
state agency staff member were printed and distributed
to work sites and several large meetings were held with
employee groups to discuss the survey before it took
place. A project description, list of agency liaisons and
Frequently Asked Questions were posted on the
university center’s web site and several professional
organizations published articles in their newsletters.
Agency liaisons continued to assist in assuring that
technical problems with email administration could be
avoided or quickly addressed. They also provided
information to employees, and answered questions.
The on-line surveys addressed a total of 126 skills
organized into eleven categories ranging from
programming to system design and development, to IT
management, to general administration. The employee
survey asked respondents to indicate their current
4
Proceedings of the 40th Hawaii International Conference on System Sciences - 2007
proficiency in specific skills within each group, as well
as their need for training in these skills. Demographic
questions concentrated on business knowledge,
retirement intentions, and education.
The specific skills and associated skill categories
were derived first from the federal survey and then
simplified and refined in iterative discussions between
the university researchers and committee members so
they would accurately and comprehensively reflect
New York State terminology and job functions. Several
versions of the online survey were reviewed by the
committee, and numerous changes were made, before
a pre-test by volunteers from the agencies and the
unions.
The CIO survey followed a similar development
process, but without union involvement, and asked
agency IT leaders to forecast the need their individual
organizations would have for the same skills three
years into the future.
The researchers analyzed the two data sets
separately and then compared them to produce a
statewide employee skills profile, IT forecast, and gap
analysis.
Agency-level data files and interactive
analysis tools were also produced for the agency CIOs.
A statewide report, reviewed in draft by all the
partners, was prepared as were various public
presentations of the findings. While the researchers
prepared these products, the HR committee considered
the findings and the underlying data to produce a set of
policy and action recommendations for workforce
development investments. Both the formal report and
the associated recommendations were designed to be
released together by the State CIO.
4. Partnership approach
The project was designed to take full advantage of
the different kinds of knowledge and capability
represented among the partners.
The roles and
responsibilities were divided based on the partners’
special skills, knowledge and attributes as shown in
Table 1 below.
For example, the Office of the Chief Information
Officer served as the executive sponsor of the project.
The State CIO used the legitimacy of his position to
communicate with the heads of other agencies and to
obtain the employee population list for use by the
researchers.
He also personally reviewed the
methodology and draft report and committed publicly
to make use of the findings in future decisions about
Table 1. Partners roles, responsibilities, and contributions
Organization
Roles & Responsibilities
Office of State CIO
Executive sponsor
CIO Council’s HR
Committee
Project leadership
Communication planning & outreach
Relationship management among partners
Governor’s Office
of Employee
Relations
Official linkage to unions
Funding for the project
Agency Liaisons
Refine population lists
Direct communication with employees,
serving as contacts for employee questions
and technical difficulties
State Employee
Unions
State University
Research Center
Project planning & management
Survey design, development and
administration
Assurance of confidentiality and validity
Communication planning & outreach
Analysis, presentation & report writing
Representing employee interests
Endorse survey and project
Input on survey development
Communication with member-employees
Specific Contribution to the Project
Strategic overview of statewide IT needs
Coordination among agency CIOs and other
executive agencies
Legitimacy
Expertise in IT, HR management, and training
Understanding of how agencies use IT
Understanding of staffing and recruitment processes
Advice on survey design, execution, and analysis
Access to restricted data such as employee lists
Expertise in labor management relations
Direct relationship with employee unions
Responsible for funding major training programs
Knowledge of own agency IT staffing and internal
technology infrastructure
Familiar face for employees with concerns
Legitimacy, especially in the eyes of their members
Official endorsement of the effort
Extensive experience working with state agencies
Professional credibility among agencies and unions
Research expertise
Facilitation and community building skills
5
Proceedings of the 40th Hawaii International Conference on System Sciences - 2007
training and development investments that would go
into the statewide strategic IT plan.
The HR committee of the CIO Council was
responsible for managing the whole partnership and
developing and leading the communication plan. The
committee members were also the people with the most
knowledge about which managers, task forces, and
employee groups the committee should talk to about
the project. Through their collective contacts, meetings
were set up with major stakeholders to address
questions and concerns about the survey from not only
a logistical perspective (e.g., When will it happen?
How long does it take? Can I take the survey from
home?) but also from a policy standpoint (e.g., Who
will see the results? Are my answers confidential? How
will this study affect training investments and future IT
developments?) Moreover, the committee members all
had long careers in IT positions and had a deep
understanding of IT workforce dynamics as well as the
existing civil service system and training programs
The unions played an important legitimizing role for
their members. Union representatives participated in
survey design and testing, and actively encouraged
members to respond by sending them letters, as well as
hosting breakfasts and lunch meetings.
These
individuals also reviewed and commented on the draft
report. Union involvement was tied to the role of the
Governor’s Office of Employee Relations which has
the mission and authority to negotiate labormanagement agreements and also administers a very
large training budget for state employees.
The university research center brought research
expertise and a reputation for professionalism that had
been built on long-term relationships with state
agencies in previous projects, as well as the
independence and formal protocols that allowed both
managers and employees to build confidence in the
survey process and results. The researchers set forth
good practice requirements for the length, format, and
presentation of the survey that helped assure a high
response rate, something that was a major concern
because the federal survey had achieved only a 28
percent response rate, creating major concerns about
what could be learned from or done with the results. At
the center’s recommendation, the whole team adopted a
goal of at least 50 percent employee response and built
its communication and outreach program with this goal
(and its associated validity benefits) in mind. In the
end, the survey achieved a 58 percent overall response
rate with many individual agencies exceeding 80
percent.
The project began with many long discussions about
both the feasibility and validity of various approaches.
The participants (many of whom had known and
worked with one another on other projects for many
years) adopted a set of working assumptions about
roles, responsibilities, and deliverables that served as a
guide to the work plan and timeline. This explicit, but
rather informal, set of agreements was the only
backdrop when work was started. It took several
months for a formal memorandum of understanding
(MOU) to be executed between the research center,
GOER (which was financing the effort and concerned
with labor-management implications), and the Office
for Technology (which represented the interests of the
CIO Council), and several more months before funds
were actually provided. The work proceeded while
these official formalities were carried out. Some of
these processes posed major challenges to the project.
For example, the initial MOU language (which
consisted mostly of standard provisions used in
contracts) would have stripped the researchers of all
intellectual property rights. Serious negotiations with
administrators and attorneys who were not directly
involved in the project but who played more distant
official institutional roles were needed to come to
closure on this and other issues.
This juxtaposition of familiarity and close working
relationships within the partnership against the
necessary but more distant institutional relationships
outside the partnership introduced similar challenges at
other points in time. For instance, at the point that the
data were being analyzed for the first time, and again
when the early draft report was being considered, one
organization changed the person assigned to the
project. These new players had no history with the
group and had much more traditional expectations
about their own roles and the roles that each of the
other parties would be or should be playing. They
expected to see and participate in a traditional
contractor relationship, not in the easy informality and
give-and-take of the project partner discussions that
were taking place. They were not entirely able to
accept that the committee members were actually
involved in doing the project work, not just exercising
oversight regarding the work of the researchers.
Nevertheless, the strength of the partnership as a whole
and the quality of the research process and results were
sufficient to overcome these concerns.
4.1 Benefits of the partnership approach
The partnership approach offered significant
advantages over traditional methods in which studies
are driven by the interests of investigators and results
are disseminated within the academic community. It
6
Proceedings of the 40th Hawaii International Conference on System Sciences - 2007
also has advantages over contract research in which the
government and the research organization operate at
arms length and the relationship is one oversight rather
than collaboration. Three benefits from the partnership
clearly enhanced the rigor of the work:
First, the partnership produced an authoritative
population list that included all IT employees in the
state. It is unlikely that any research organization
would have been given access to this information
without extensive use of the Freedom of Information
Act, and then it would have contained only names and
titles, not the email addresses that were critical to being
able to conduct a cost-effective survey. Contract
researchers might have been given this information, but
it is likely that employees would not have been as
willing to respond.
Second, the survey instruments were finely tailored
to the needs of New York State through painstakingly
detailed discussions between the researchers and the
HR committee members. These instruments were also
vetted with IT leadership and union representatives
before being implemented.
Third, every IT employee in the state was wellinformed about the purpose and process of the study.
There were many coordinated communication vehicles,
and ample opportunities to ask and receive the answers
to questions. The state, the researchers, and the unions
all had the same information and gave consistent
answers to these questions.
Three additional benefits of the partnership served
to enhance the relevance and usability of the work:
The analysis was been conducted through a process
that involved detailed discussions of data quality,
preliminary findings, and underlying conditions in the
state workforce that impinge on data interpretation.
These discussions are open dialogs in which the
practitioners learned more about the limits and
possibilities of quantitative analysis and the researchers
learned more about the organizational, operational, and
political context of the survey data. Trust among the
players, which emerged from their constant interaction,
allowed for careful adjustments to the analysis plan that
enhanced both understanding and confidence in the
data and the analysis. This confidence will help to
assure the results will be used effectively by
government decision makers.
The research products provided both scientific rigor
and practical value.
They included a rigorous
presentation of the findings (the statewide report
developed by the researchers), a data-driven set of
policy and action recommendations (developed by the
the HR committee), and data sets with associated
practical analytical tools for CIOs to conduct their own
further analyses (also produced by the researchers).
In addition, the entire effort was completely
transparent. Anyone interested in how it was designed
and carried out can readily understand it. Again, this
engenders confidence in the results. Moreover, the
results will be made public and shared directly with
employees as well as their unions in addition to the
state government. With so much visibility, and shared
investment in the outcome, there is great pressure to
make good and immediate use of the results in
upcoming decisions about budget allocations and
training priorities.
4.2. Costs of the partnership approach
As with any other collaborative endeavor, project
management demanded special attention to complex
relationships, unconventional practices, and multiple
interdependencies among the players. While the HR
committee chair took on responsibility for leading the
effort, the research center was responsible for day to
day management of the entire project.
Weekly
meetings, sometimes in person, sometimes by
conference call, were accompanied by detailed
agendas, specific goals, and associated materials.
Many specific discussions also took place via email
among the team members. The members all devoted
considerable time to these frequent discussions.
Communication costs were also significant. With a
goal of at least 50 percent employee response across 59
separate agencies, the partners needed to engage in an
extensive outreach and communication campaign. This
meant drafting and orchestrating letters from the state
CIO to the commissioners and HR directors of all 59
state agencies to introduce the project and ask for their
support. Another letter was sent from the CIO to all
4,833 state IT employees encouraging them to respond.
Posters were distributed to all state agencies and
liaisons were appointed to serve as the connection
between the researchers and individual agencies.
Liaisons played a crucial communication role in
assuring the accuracy and completeness of employee
listings, checking the compatibility of agency networks
with the planned mass email distributions, assuring and
employees had desktop access to the internet-based
survey. Briefing meetings with union representatives
and several professional associations were also
conducted and newsletter articles were written for their
regular publications. All of this paid off in consistent
messages, easy access to the survey for employees,
complete employee coverage and a high response rate.
7
Proceedings of the 40th Hawaii International Conference on System Sciences - 2007
Time is a third major cost element. When the
project was conceived by the HR committee of the CIO
council, the group hoped the project could be
completed in about three months. In fact, it took more
than a year from conception to completion of the final
report. Much of this time was spent in the pre-survey
communication phases and in assuring a reliable and
relevant survey instrument. Analysis of both statewide
and agency-level results are a third major time
segment. Successive drafts of the statewide report and
work needed to produce agency-specific data sets and
user-friendly analysis tools consumed additional weeks.
However, despite this heavy workload, all the
partners shared a sense of urgency to keep moving so
that the window of opportunity for using the survey
findings in upcoming budgets and strategic plans would
not be not missed. Toward this end, the partners agreed
early on the essential elements of the analysis and
delayed consideration of special requests that would
slow down the core work. For the researchers, this
means academic publications based on the survey
results will come last.
5. Lessons learned
Some aspects of this experience appear to be quite
generalizeable to other research efforts.
Other
elements may be idiosyncratic or applicable only in
certain circumstances.
For any research effort that aspires to be a serious
partnership, two aspects of Davison’s principles appear
to be essential [5]. First, such efforts demand mutual
agreement between the researchers and practitioners
about roles, goals, expectations, and balance. A jointly
developed working agreement (apart from any formal
contract) that is openly discussed and amended as
needed is an essential underpinning for a research
partnership. In this case, explicitly stated working
assumptions served this purpose. They helped the
project get started with clear expectations and kept it
on course when new knowledge or circumstances made
changes desirable or necessary.
Second is the need for both researchers and
practitioners to have a clear, shared understanding of
the “organizational situation which doubles as the
research context” [5, p.75]. This project demonstrated
the value of this shared understanding in every phase
from design of the survey instruments to consideration
and use of the final results. This was especially
valuable when it became necessary to find solutions to
unexpected problems such as the execution of the
MOU. The project could not have succeeded without
shared understanding and mutual willingness to discuss
and confront the organizational, institutional, and
political context of the work.
Similarly, the three recommendations of Yin and
Moore [28] appear to hold true: identify at the design
stage who will use the findings and work accordingly to
be sure their needs are met; keep an open mind about
making design modifications that will meet practitioner
needs without compromising integrity; and plan
explicitly to produce major products expressly for a
practitioner audience.
In addition, we suggest that project partners need to
be prepared for the possibility that new participants
with traditional expectations will enter a project midstream without the history and socialization that has
taken place among the original partners. To avoid
miscommunication and misunderstanding, these new
entrants need to be oriented to the approach,
relationships, and history of the project as soon as they
become involved.
Finally, the partners need to understand that the
character of their interaction may be mostly
unconventional and informal, but it plays out in the
context of powerful institutional constraints. Effective
as the informal relationships may be, they need to
accommodate institutional imperatives in order to
maintain the legitimacy of the effort. In our case, for
example, interaction with the employee unions took
place mostly under the aegis of an established labormanagement committee rather than in informal settings
where most of the other work was done.
Some aspects of this project are less generalizeable.
For example, the participants for the most part already
had long, positive working relationships.
These
relationships made it possible for them to undertake a
project that was broad in scope, politically sensitive,
and highly visible.
All of these characteristics
presented risks to the researchers, the HR committee,
and the sponsors. Long association and mutual trust
and respect were necessary for the project to even be
contemplated, much less carried out.
These
relationships are unlikely to exist in the typical research
situation and need to be built and nurtured as part of
the research process. In addition, in this project the
research center had the financial stability to work at
first on the promise of funding. Many researchers are
not in that position and need funding to be available
when work begins. For these reason, smaller projects
with fewer players, up-front funding, and more modest
goals would be a better place to start.
8
Proceedings of the 40th Hawaii International Conference on System Sciences - 2007
6. Conclusion
Digital government is one research area in which
strong connections between research and practice are
encouraged, and are in fact necessary to the
investigation of many topics. Today, however, we have
largely independent communities of digital government
research and practice, with few mechanisms that
connect them in mutually satisfying ways. In effect,
these cross-community relationships are quite weak
compared to the ties within each community. The
typical situation is illustrated in Figure 1.
Government
community of
practice
Research
community of
practice
Research &
knowledge needs
Research results
Consultants &
Private sector
solutions
Academic
literature
Strong, structured,
on-going, many to many
relationships
Weak, unstructured,
time-limited,one-to-one,
relationships
Figure 1. Traditional Research and Practice communities
By contrast, research-practice partnerships, while
demanding some special resources and acceptance of
non-traditional roles, offer the possibility of more
fruitful and impactful research, informed by a better
understanding of context and practical needs. The
relationships would look more like Figure 2.
could foster improved needs assessment to identify key
problems, development of a comprehensive research
agenda of mutual interest, synthesis of research results
that could improve practice, and the creation of
education resources that rest on rigorous research
findings that address the knowledge needs of
practitioners. Clearinghouses and discussion forums
could provide venues for sharing problems, solutions,
knowledge, and approaches that cross the boundaries
of the two communities. None of these efforts would
damage the quality of research or the relevance of the
results. In fact both could be enhanced.
At the same time, it is important to acknowledge and
plan for the costs of working in this more collaborative
way. The strong traditions that separate the
communities of research and practice will not be
changed without changing the way we think and
behave. Project management, open and frequent
communication, and sufficient time for them to play out
are all necessary conditions for success.
The IT workforce study reveals some of the
strategies that can be applied as well as the investments
that need to be made in order to to help achieve this
more integrated approach to advancing digital
government.
References
1.
2.
3.
Government
community of
practice
Research
community of
practice
kn Res
ow ea
led rch
ge &
ne
Re
ed
se
s
arc
hr
esu
lts
Public Domain
Partnership
s
ult
res
rch
a
e
s
Re
& s
rch eed
sea e n
Re ledg
ow
kn
needs assessment
comprehensive research agenda
research synthesis
educational resources
partnership broker
information clearinghouse
discussion forum
4.
5.
6.
7.
Figure 2. Research-practice partnership
In this situation, the possibilities for benefit in both
directions are improved. A public domain partnership
8.
Benbasat, Izak and Robert Zmud, “Empirical research in
information systems: the practice of relevance”. MIS
Quarterly, 1999, 23(1), 3-16.
Beyer, J.M., and H.M. Trice, “The utilization of
process: A conceptual framework and synthesis of
empirical findings”, Administrative Science Quarterly,
1982, 27, 591-622.
Boland, Richard, et al., “Knowledge representations and
knowledge transfer”, Academy of Management Journal,
2002, 44(2), 393-417.
Boyer, Ernest L., Scholarship reconsidered: Priorities
for the professoriat, New York: Carnegie Foundation,
1990.
Davison, Robert M., Maris G. Martinsons, and Ned
Kock, “Principles of canonical action research”,
Information Systems Journal, 2004, 14(1), 65-86.
Dawes, Sharon S., “Human Resource Implications of
Information Technology in State Government”, Public
Personnel Management, 1994, 23(1), 31 – 46.
Dawes, Sharon S., Peter A. Bloniarz, Kristine L. Kelly
and Patricia D. Fletcher, Some Assembly Required:
Building a Digital Government for the 21st Centur,.
Center for Technology in Government, University at
Albany/SUNY, On-line at
http://www.ctg.albany.edu/research/workshop/dgfinalre
port.pdf, 1999.
Federal Chief Information Officer (CIO) Council, Office
of Management and Budget, Information Technology
9
Proceedings of the 40th Hawaii International Conference on System Sciences - 2007
9.
10.
11.
12.
13.
14.
15.
16.
17.
18.
(IT) Workforce Capability Assessment Survey (2004):
Analysis of Survey Results December 2004, retrieved
online November 2005 at
http://www.whitehouse.gov/OMB/inforeg/2004_egov_r
eport.pdf.
Gartner Group 2005, Gartner EXP, January 2005,
retrieved online June 2006 at
http://www.gartner.com/press_releases/asset_117739_1
1.html.
Hawaii Information Technology Development
Workforce Survey, Prepared for the High Technology
Development Corporation, December 2002, retrieved
online June 2006 at
http://www.htdc.org/2002_IT_Workforce_Survey.pdf.
Huberman, M., “Linkage between researchers and
practitioners: A qualitative study”, American
Educational Research Journal, 1990, 27(2), 363-391.
Hutchinson, J. and Huberman, M., Knowledge
dissemination and use in science and mathematics
education: A literature review. National Science
Foundation, NSF 93-75, 1993.
Information Technology Association of America
(ITAA), Adding Value… Growing Careers: The
Employment Outlook in Today’s Increasingly
Competitive IT Job Market, September 2004, retrieved
online June 2006 at
http://itaa.org/workforce/studies/04wfstudy.pdf.
Kiyonaga, Nancy B., “Today is the tomorrow you
worried about yesterday: meeting the challenges of a
changing workforce”, Public Personnel Management,
2004, 33(4), 361.
Lyytinen, Kalle, “Empirical research in information
systems: on the relevance of practice in thinking of IS
research”, MIS Quarterly, 1999, 23(1), 25-28.
Mohrman, Susan, Christina Gibson and Allan
Morhrman, “Doing research that is useful to practice: a
model and empirical exploration”, Academy of
Management Journal, 2002, 44(2), 357-375.
National Research Council, Computer Science and
Telecommunications Board, Committee on Computing
and Communications Research to Enable Better Use of
Information Technology in Government, Information
Technology Research, Innovation, and E-Government.
Washington, DC: National Academy Press, 2002, Online at
http://www7.nationalacademies.org/cstb/pub_egovernm
ent.html
New York State Office for Technology, Office of the
State CIO, The New York State Enterprise, June 2003,
19.
20.
21.
22.
23.
24.
25.
26.
27.
28.
Vol. 1. No. 1. retrieved online June 2006 at
http://www.cio.state.ny.us/newsletters/2003_06_newslet
ter.htm.
New York State Office for Technology, Office of the
State CIO, 2005 NYS Enterprise Information
Technology (IT) Strategic Plan, retrieved online
November 2005 at
http://www.cio.state.ny.us/NYS%20IT%20Strategic%2
0Plan%202005.htm.
New York State Office for Technology, Office of the
State CIO. 2004 NYS Enterprise Information
Technology (IT) Strategic Plan, retrieved online
November 2005 at
http://www.oft.state.ny.us/Policy/p04004/NYSInformationTechnologyStrategicPlan.pdf.
New York State Office for Technology, Office of the
State CIO, Technology Policies: Principles Governing
The New York State Information Technology Enterprise
Architecture, retrieved online June 2006 at
http://www.oft.state.ny.us/Policy/p04001/principles.htm.
Office of Management and Budget. FY 2004 Report to
Congress on Implementation of the E-Government Act
of 2002. March 1, 2005. retrieved online June 2006 at
http://www.whitehouse.gov/OMB/inforeg/2004_egov_r
eport.pdf.
Pawlowski, S. D., D. Pratim, and A. L. Houston, “The
(Gradually) Changing Face of State IT Jobs”,
Communication of the ACM, 2005, 48(5), p. 87 – 91.
Peterson, Shane, “Findings a way”, Government
Technology, 2005, 18(9) 18 -34.
Rynes, Sara, Jean Bartunek, and Richard Daft, “Across
the great divide: Knowledge creation and transfer
between practitioners and academics”, Academy of
Management Journal, 2001, 44(2): 340-355.
Selden, Sally Coleman, Patricia Wallace Ingraham and
Willow Jacobson, “Human Resources Practices in State
Government: Findings for a National Survey”, Public
Administration Review, 2001, 61(5): 598 – 607.
Van de Ven, Andrew, Professional science for a
professional school: Action science and normal science.
In Beer, M and Nohria, N, eds., Breaking the code of
change: Boston: Harvard Business School Press, 2000.
Yin, R. and Moore, G., The utilization of research:
Lessons for the natural hazards field, Washington, DC:
Cosmos Corp., 1985.
10