VDRDC DRAFT ONE September 2, 2011
Center to Improve Project Performance
Logic Model Template
A logic model provides a starting point for developing an effective evaluation plan. CIPP’s goal is to assist projects in developing logic models that are precise and that include
features and content that give the models utility for evaluation purposes. The logic models serve as both ends and means. As ends, they are stand-alone representations of projects that
provide an overall visual summary. They also provide descriptive information that can be used to catalogue or compare features across multiple projects, if needed. As means, the use of
logic models is central to (1) defining outcomes that are meaningfully connected to project activities and (2) supporting evaluations so that the process will improve projects’ overall
performance.
CIPP’s Logic Model Scheme
A project logic model portrays a project’s overall plan. It serves to clarify the relationships among a project’s goals, activities, and outputs and to lay out the connections between
them and the project’s expected outcomes. Therefore, a project logic model depicts a program theory and accompanying hypotheses, highlighting (1) the resources or inputs dedicated to
an effort, (2) the planned activities to be carried out with those resources, and (3) the specific outputs and outcomes the activities will generate. Evaluation can then be viewed as a test of
the logic model’s hypotheses, and a logic model can be used by evaluators and the grantee to refine and guide data collection and analysis for assessing both process and performance.
We find it helpful to begin with a summary chart that contains the information that will populate the logic model. The chart outlines the OSEP priority, assumptions, external
factors/context, and inputs. It then displays, in table format from left to right, the project’s goals and objectives, strategies/activities, outputs, and outcomes. From the chart, a logic
model is prepared. The logic model is less comprehensive than the chart in its content but it uses lines and arrows to connect specific project elements and provides a dynamic display.
Both the chart and the logic model are continuously updated as the content of specific elements changes, such as when planned activities are revised or when unintended outcomes
occur. The logic model will also change as the relationships among the components develop over time, mostly likely by becoming more complex and interactive. For CIPP, we use the
following definitions of the logic model components:
Goals/Objectives – The goals capture the overarching purposes of the project. Goals make clear the anticipated impact on systems or individuals. Goals imply gaps or deficits that
will be remedied when the project produces its long-term outcomes. Objectives, if used in a logic model, are targeted sub-goals.
Strategies/Activities – Strategies are the broad approaches to addressing the goals. They include multiple activities. Activities, which may or may not be listed in the logic model, are
the specific actions funded by the grant or supported by other resources under the umbrella of the project.
Outputs – Outputs are the direct results of the project activities, including project products and programs. Most outputs will be quantifiable, including tallies of the number of
products and programs or counts of the customer contacts with those products and programs.
WESTAT/COMPASS
Center to Improve Project Performance
1
VDRDC DRAFT ONE September 2, 2011
Direct/Intermediate Outcomes – Direct outcomes are what customers do or become as a result of outputs. Usually, direct outcomes are changes in the customers’ actions or
behaviors based on knowledge or skills acquired through project outputs. Intermediate outcomes result either directly from outputs or indirectly through direct outcomes. They generally
come later in time than direct outcomes and often represent a step between direct outcomes and long-term outcomes.
Long-term Outcomes – Long-term outcomes are the broadest project outcomes and follow logically from the direct and intermediate outcomes. They are the results that fulfill the
project’s goals. Outputs, direct outcomes, and intermediate outcomes all contribute to the achievement of the long-term outcomes. Although the long-term outcomes represent
fulfillment of the purpose of the project, they may or may not represent the achievement of a desired larger project impact. That is, the project may have an anticipated impact that is
beyond the immediate scope of the project, either temporally or conceptually, and thus beyond the scope of the logic model.
The Summary Chart
The summary chart contains all the information that will populate the logic model, plus additional details about strategies, activities, and outputs. The chart begins with the
OSEP priority and then states assumptions about how and why the project will be successful. External factors and context provide a brief description of the environment in which the
project will be operating. Inputs are specific resources available to the project. The table itself displays, from left to right, the project’s goals and objectives, strategies/activities, outputs,
and outcomes. Note that strategies/activities are aligned with outputs but that goals and outcomes may cut across multiple strategies/activities.
The Logic Model
The content of the logic model is taken entirely from the summary chart, but the logic model content is condensed to fit into the flow chart format. In the logic model, lines and
arrows are used to depict the temporal and causal connections among the various project elements. Not surprisingly, multiple lines or arrows come to or from most of the boxes,
indicating the complexity of the relationships that are expected. Also depicted are the anticipated results in the form of direct, intermediate, and long-term outcomes. The outcomes are
themselves interconnected. Thus, direct outcomes, as well as outputs, lead to the higher level, more distal outcomes.
WESTAT/COMPASS
Center to Improve Project Performance
2
VDRDC DRAFT ONE September 2, 2011
SUMMARY CHART
OSEP Priority: OSEP’s stated purpose1 for the Video Description Research and Development Center (VDRDC) is to (1) Improve results for children with disabilities by promoting the
development, demonstration, and use of technology; (2) support educational media services activities designed to be of educational value in the classroom setting to children with disabilities,
and (3) provide support for captioning and video description of educational materials that are appropriate for use in the classroom setting. In the context of this notice, educational materials for
use in the classroom setting include television programs, videos, and other materials, including programs and materials associated with new and emerging technologies, such as CDs, DVDs,
and other forms of multimedia.
Assumptions: The VDRDC is managed by the Smith-Kettlewell Institute (SKI, or the Institute). This non-profit research Institute was founded in 1963 and is located in San Francisco at the
California Pacific Medical Center. The Institute’s mission is to “create a unique environment for research on human vision-- basic scientific research to increase the understanding of normal
vision and of eye disorders, clinical research to develop new diagnostic procedures and treatments for visual and other sensory disorders, and engineering research to produce better
techniques for aid and rehabilitation of the blind and partially sighted”. The Institute supports this mission by engaging in research with a high degree of collaboration among staff that represent
varied medical and scientific arenas, including ophthalmology, neurology, engineering, physics, and audiology, to name a few. The Institute’s current research agenda includes:
 Clinical studies which relate directly to the diagnosis and treatment of eye diseases and disorders.
 The development of devices and vocational programs to aid the partially sighted, blind and hearing impaired.
 Basic research to understand how the eye and brain work, providing a fundamental background for both the clinical and rehabilitation programs.
OSEP funding for the VDRDC combined with SKI technical expertise and ability to draw in collaborators will further the state of knowledge regarding video description and emerging and
promising technologies that promote access to visual content among blind and visually impaired students.
Stakeholders are brought together in the Institute’s Description Leadership Network (DLN) and include major contributors and users for video description technology: The American Council of
the Blind (ACB), The American Foundation for the Blind (AFB), CaptionMax, The Described and Captioned Media Program (DCMP), Dicapta, Ideal Group, The Metropolitan Museum of Art
(MMA), The National Federation of the Blind (NFB), and the Narrative Television Network (NTN). Together with SKI, the DLN will vet promising technologies (such as Crowd-Sourced
Description for Web-Based Video (CSD) , Remote Real-Time Description (RRTD), Algorithmic Automated Description (AAD), Separate-Channel Audio-Based Synchronization (SCABS), and
Descriptive Video Exchange (DVX) ), further the development of the most promising of these, and facilitate access to these technologies by the public. The impact of these services will be felt
not only in educational arenas but in a wide range of professional, public, and personal activities.
External Factors/Context:
(1) This initiative brings together most major stakeholders in video description and is committed to providing its products as “open source” code, where applicable. Thus, there is the
potential for this project to focus and stimulate many independent researchers to “evolve” promising technologies.
(2) Currently, although emerging technologies exist for research and development (such as those listed above), there are few reliable and well-sourced options for public consumption. The
VDRDC therefore is in the position of focusing the field on the most promising of technologies.
1
http://www2.ed.gov/legislation/FedRegister/announcements/2010-3/081210e.html
WESTAT/COMPASS
Center to Improve Project Performance
3
VDRDC DRAFT ONE September 2, 2011
Inputs: Project staff are leveraging OSEP funding with funding from other sources, such as the National Eye Institute which has provided funding for video description for DVD media—the
VDRDC will allow expansion of this initiative (the Descriptive Video Exchange, or DVX) into internet-based media. The project’s Principle Investigator (Dr. Josh Miele) is a member of SKI’s
Rehabilitation Engineering Research Center and brings an extensive knowledge of improving information-accessibility for blind/visually-impaired users. Dr. Miele will facilitate and lead the
Descriptive Leadership Network, which will create opportunities for field testing and disseminating promising technologies and free, open-source software.
Goals/Objectives
1. Research, development, and
innovation
Strategies and Activities
Outputs
(A) Complete a systematic review and synthesis of existing research on
technology that provides accessibility to educational program content
that is delivered via the Internet or through other technological
devices for children who are blind or visually impaired.

Two written literature syntheses on technology that
provides access when (a) delivered as streamed or
downloaded video via the internet and (b) delivered
through other technological devices
(B) Conduct research and develop technology on emerging alternatives
to video description that could improve the accessibility of educational
program content, such as (1) crowd-sourced tagging (i.e., Descriptive
Video Exchange, or DVX) for internet-based media (as opposed to
DVD media funded through the National Eye Institute) and video
annotation; (2) remote, real-time description; (3) algorithmic
automated description; (4) separate-channel audio-based
synchronization
(C) Collaborate with researchers and technology experts to enhance or
develop new open source technologies that make educational
program content—appropriate for use in the classroom and delivered
via the Internet or through other technological devices (e.g., smart
phones, cell phones, and digital video cameras)—accessible through
video description. For promising technology, collaboration with The
IDEAL Group to implement the technology as a simple app for an offthe-shelf smart phone


Number of technologies evaluated
Number of technologies/ applications produced for
dissemination

Number of collaborative meetings and events
conducted with the DLN
Four focus groups conducted by the American Council
of the Blind and National Federation of the Blind
(D) Systematically field-test and evaluate the efficacy of the technology
developed under paragraph (C) of this priority.

2. Outreach and dissemination


WESTAT/COMPASS
Number of participants in field tests (break out in our
measures the different types of field tests); Number of
stakeholders involved in field tests of Spanish video
description
Number of applications (i.e., products) field tested
Center to Improve Project Performance
Outcomes
Direct:
• Increase in technology (e.g.,
software or code) and
applications that facilitate
video description (By # of
apps/web applets)
• Increased knowledge of
emerging and promising
technology among
stakeholders (i.e., the DLN)
(by # of mtgs & events, # of
participants at focus groups
and their demographics)
Intermediate:

Increased access to
accurate and effective
technology for video
description (by # of
downloads, online
webapplets, comments,
views (youtube videos)
Long-term:
• The degree to which the
Center’s activities contribute
to changed practice and
improved implementation of
technology that provide
educational program content
delivered via the internet and
through other technological
devices accessible to
children who are blind or
visually impaired.
4
VDRDC DRAFT ONE September 2, 2011
Goals/Objectives
Strategies and Activities
Outputs
(E) Provide all technology developed under paragraph (C) as open
source materials (i.e., the source codes for the developed technology
are freely available to the public).




(F) Develop and implement—upon review and approval from OSEP—a
strategy for disseminating research and development findings, in
open source format, to key stakeholders including educators,
technology developers, vendors, researchers, and federally funded
technical assistance and dissemination projects (e.g., the National
Center on Technology and Innovation and the Family Center on
Technology and Disability).








WESTAT/COMPASS
Outcomes
Number of apps or web applets produced
Number of downloads of apps/web applets
Number of downloads of source code
Including phone app developed for the IDEAL group
Dissemination plan and strategies
Number of publications (of experimental results)
submitted for peer-review
Number of technical trainings at annual meetings of
the Description Leadership Network
Number of participants attending the four focus groups
at the annual conventions of the American Council of
the Blind and the National Federation of the Blind
Number of webinars and participants of the webinars
for educators, provided in collaboration with the
Described and Captioned Media Program
Number of museum workshops, provided in
collaboration with the Metropolitan Museum of Art
Number of events and participants of youth experience
and mentorship opportunities and provided in
collaboration with the National Federation of the Blind
Center to Improve Project Performance
5
VDRDC DRAFT ONE September 2, 2011
VIDEO DESCRIPTION RESEARCH AND DEVELOPMENT CENTER LOGIC MODEL
Outcomes
Goals
Strategies
Outputs
A. Systematic review and synthesis
of existing research
1.
Research, development, and
innovation
C. Conduct research on emerging
alternatives to video description
F. Provide all technology
developed) as open source
materials
Inputs
 Staff time and
expertise
 SKI research and
development
resources
 Collaboration with
the DLN
WESTAT/COMPASS
Number of collaborative meetings and
events conducted with the DLN
Four focus groups
Collaboration with The IDEAL Group




G. Develop and implement a
strategy for disseminating
research and development
findings



Outreach and dissemination
Number of technologies evaluated
Number of technologies produced for
dissemination

E. Systematically field-test and
evaluate the efficacy of the
technologies
2.




D. Collaborate with researchers and
technology experts to enhance or
develop new open source
technologies




Long-term
Two written literature syntheses

B. Develop methods for providing
video description that can be
used in conjunction with the
Internet and other technological
devices
Intermediate
Direct
Increase in technologies
(e.g., software or code)
and applications that
facilitate video
description
Increased access to
accurate and effective
technologies for video
description
Number of field tests conducted
Number of technologies field tested
Number of free, open-source software
(FOSS) units produced for
dissemination
Number of English and Spanish versions of
technologies, as appropriate
Dissemination plan and strategies
Number of publications (of experimental results)
submitted for peer-review
Number of technical trainings at annual
meetings of the Description Leadership Network
Number of presentations at annual conventions
of the American Council of the Blind and the
National Federation of the Blind
Number of webinars for educators, provided in
collaboration with the Described and Captioned
Media Program
Number of museum workshops, provided in
collaboration with the Metropolitan Museum of
Art
Number of youth experience and mentorship
opportunities, provided in collaboration with the
National Federation of the Blind
Center to Improve Project Performance
Changed practice and
improved
implementation of
technologies that
provide educational
program content
delivered via the
internet and through
other technological
devices accessible to
children who are blind
or visually impaired.
Increased knowledge of
emerging and promising
technologies among
stakeholders (i.e., the
DLN)
External Factors/Context


Absence of “industry-standard” video description
technology
Vibrant community of users and advocates
6