If You Are Walking Down the Right Path
and You Are Willing to Keep Walking*
A Participatory Evaluation of Community MusicWorks
Dr. Dennie Palmer Wolf
Dr. Steven J. Holochwost
WolfBrown
December 2009
´,I\RX·UHZDONLQJGRZQWKHULJKWSDWKDQG\RX·UHZLOOLQJWRNHHSZDONLQJHYHQWXDOO\\RX·OO
PDNHSURJUHVVµLs a quote from a speech at Lincoln College in 2005 by then-Senator Barack
2EDPD7KLVTXRWHZDVIHDWXUHGLQWKHO\ULFVIRUWKH´$QWKHPµYLdeo, composed and
performed at Community MusicWorks in 2009.
If You Are Walking Down the Right Path: An Evaluation of Community MusicWorks
TABLE OF CONTENTS
Acknowledgements
Section One: Executive Summary
Community MusicWorks
The Design of the Evaluation
Major Accomplishments
Major Challenges
Recommendations for the Further Development of CMW
Creating Support for this Expanded Work
Moving Forward: Developing Action Steps
3
4
5
5
6
10
11
13
14
Section Two: Introduction
15
Section Three: Design of the Evaluation
23
The Context for the Evaluation
Community MusicWorks in Context
$)RFXVRQ<RXQJ3HRSOH·V0XVLFLDQVKLS3HUVRQDO$JHQF\
and Participation in a Wider World
A Developmental Model
Capacity Building and Utilization Approach
The Evaluation Process
16
21
24
25
25
26
Section Four: Methods
29
Section Five: Major Findings
35
Section Six: Recommendations
56
Data Collection
Scoring and Analysis
Enrollment and Persistence
Personal Agency
Participating in a Wider World
Challenges for CMW
Recommendations for the Further Development of CMW
Creating Support for this Expanded Work
Moving Forward: Developing Action Steps
30
31
36
47
49
57
58
60
61
Cover photo credit: Jori Ketten
WolfBrown
Page 2
If You Are Walking Down the Right Path: An Evaluation of Community MusicWorks
ACKNOWLEDGEMENTS
We would like to thank the many people who participated in this evaluation and whose time and
thoughtfulness have greatly enriched it.
For their ongoing support of the process
Liz Cox
Chloe Kline
Sebastian Ruth
For serving on the Advisory Team
Shirley Brice Heath
Liz Hollander
Jori Ketten
Chloe Kline
Eileen Landay
Karen Romer
For interviewing family members
Tehani Collazzo
Paromita De
Adrienne Gagnon
Liz Hollander
Jori Ketten
Chloe Kline
Jacqueline Russom
For being interviewers and interviewees
The Faculty and Fellows of CMW
The Families and Students of the CMW Community
For contributing photographs
Jori Ketten
WolfBrown
Page 3
If You Are Walking Down the Right Path: An Evaluation of Community MusicWorks
SECTION ONE:
EXECUTIVE
SUMMARY
Photo credit: Jori Ketten
WolfBrown
Page 4
If You Are Walking Down the Right Path: An Evaluation of Community MusicWorks
COMMUNITY MUSICWORKS
Based on the conviction that musicians can play an important public service role, Community
MusicWorks (CMW) has created an opportunity for a professional string quartet to build and
transform the urban community of which it has become an integral part ² Providence, Rhode Island.
Through a permanent residency of the Providence String Quartet, CMW provides free after school
instruction and performance opportunities that build meaningful long-term relationships among
musicians, young people, and their families in the West End and South Side neighborhoods of
Providence ² a hard pressed city seeking to resurface by harnessing the creative energies of
institutions, neighborhoods, and young people.
CMW is an ambitious program. While focused on string instrument instruction, its definition of
musicianship involves students in many string traditions and urges them to develop improvisational
and composing skills. In addition, musical expression is taught as a way of developing personal
agency and as a way of developing a voice in a larger civic and cultural world.
THE DESIGN OF THE EVALUATION
Such bold goals are challenging to assess. Recognizing this, the staff and Board of Community
MusicWorks undertook an eighteen-month participatory evaluation in 2008 with funding from the
Rhode Island Foundation. In the words of Sebastian Ruth, founder and executive director, the
purpose was to examine whether and how ´EHFRPLQJDPHPEHURIWKH&0:FRPPXQLW\DIIHFWVVWXGHQWV·LQQHU
lives: their conceptions of themselves, their sense of themselves as effective, their willingness to try new things, or to be
expressive.µ
The evaluation process was steered by an Advisory Team composed of staff, Board members, and
researchers, and was designed and executed by Dennie Wolf and Steven Holochwost of WolfBrown,
an international consulting firm specializing in arts, culture, and communities. The resulting
evaluation was designed to:
$VNDQGDQVZHUWKHFHQWUDOTXHVWLRQ´$V\RXQJSHRSOHSHUVLVWLQWKH&0:FRPPXQLW\
how does their musicianship, their sense of themselves, and their participation in a wider
cLYLFDQGFXOWXUDOZRUOGFKDQJH"µ
Engage many sectors of the CMW community in thinking about the current strengths of and
frontiers for the program: musicians, staff, Board members, families, and students.
Develop actionable knowledge for the organization, providing clear and close to the ground
implications for strengthening programs and their effects.
Yield a set of tools that CMW could use to generate ongoing data about the program and its
effects.
To accomplish these goals, members of the CMW community, along with a team of evaluators,
collected several, complementary forms of data:
WolfBrown
Page 5
If You Are Walking Down the Right Path: An Evaluation of Community MusicWorks
SOURCES OF DATA
Contextual Data
Observations of afternoon lessons, rehearsals, and Performance Parties
Ongoing discussions with faculty, fellows, Board members
Student Outcome Data
Persistence Data
Who enrolls and persists in CMW over time
developed with
Who leaves the program and for what reasons
CMW staff
Teacher and
Student
Lesson-based
Interviews
Student
Year-end
Interviews with
Outside
Evaluators
Parent
Interviews
conducted by
CMW Board
Musicianship
ƒ Practice
ƒ Musical
knowledge
ƒ Involvement
ƒ
Diary day: Role
of music in daily
lives of students
ƒ
(Not a dimension
many parents felt
comfortable
commenting on)
Personal Agency
ƒ Self-esteem
ƒ Selfmonitoring
ƒ Motivation
ƒ Accounts of
doing
something
difficult at
school/CMW
ƒ
Highs and
lows in
VWXGHQW·VWLPH
at CMW
Participation in Wider World
ƒ Developmental network
ƒ Broader aspirations
ƒ Application to other
spheres
ƒ
Maps of the worlds created
around CMW and a
comparison activity
ƒ
Distinctive consequences
of attending CMW
MAJOR ACCOMPLISHMENTS
ENROLLMENT AND PERSISTENCE
In the contemporary United States, classical music and classical music training have yet to reflect the
range of cultures and ethnicities that could enrich it. More than 95% of symphony players are
Caucasian or Asian, and an equally large proportion come from families and neighborhoods where
incomes are middle class and above. Even in community schools of the arts, the numbers of young
African American and Hispanic students who participate in classical music training do not yet reflect
WKHLUSUHVHQFHLQWKHSRSXODWLRQ$JDLQVWWKLVEDFNJURXQG&0:·VHQUROOPHQWVWDQGVRXWIRUIXOO\
reflecting the economically and culturally diverse families of the West End and South Side
neighborhoods of Providence.
)XUWKHULQDFLW\DQGDVFKRROV\VWHPZLWKKLJKPRELOLW\UDWHV&0:·VFDSDFLW\WRDWWUDFWDQGKROG
students and families is also remarkable. Of the students who were enrolled in 2005, more than half
(54%) are still playing or have graduated from the program. Students stay in the program for an
average of 4.5 years ² longer than an entire high school career. Of those who leave the program,
QHDUO\KDOIOHDYHIRUFLUFXPVWDQFHVEH\RQGVWXGHQWV·DQGRUIDPLOLHV·FRQWUROHJ., moving, family
illness, or the difficulty of getting to and from lessons and work in a city without efficient public
transportation).
WolfBrown
Page 6
If You Are Walking Down the Right Path: An Evaluation of Community MusicWorks
In part this persistence, and the effects associated with it, comes from the prograP·VDELOLW\WRDWWUDFW
and hold not just students but entire families:
I play, my sisters play, and some of my cousins.
-- Phase II student
I realized the significance of CMW in our lives when I caught myself scheduling events around CMW
events. The concerts and the Performance Parties FRPHILUVW´2KQRZHKDYHWRJRWRWKHFRQFHUWµ7KH\
come before the after school events. I thank God for CMW. I thank God they decided to locate in
Providence, right in the neighborhood. This way we can walk, we can ride a bike, within the community
we see the people that we live with that we work with, I see the kids that go to school with my kids, their
families.
-- Parent of two long-term students at CMW
MUSICIANSHIP
When a student persists in the program and engages in its many offerings, the result is often a
distinctive young musician with strong ² though basic ² technique, a keen sense of the
responsibilities of ensemble playing, a taste for a wide range of musical traditions, and a willingness
to go beyond the score to improvise and compose. This is a quality that begins to develop in Phase I
students and matures in the most engaged Phase II and Phase III students.
1RZ,DPZRUNLQJRQD9LYDOGLSLHFH:KHQ,EHJDQ,ZDVZRUULHG,FRXOGQ·WSOD\LW1RZ,DPMXVW
thrilled with it. When I play I PDNHDOLWWOHVWRU\LQP\PLQGFDOOHG´&RQFHUWRµ7KHVWRU\OLQHVWDUWV
fast. I imagine antique bows going furiously, I feel jumpy on account of it. Vivaldi is listening;; I can
picture him watching all the players. He is jumpy too, to see all the players there. I see the politics of the
time;; a lot of the issues, the common and the rich people all listening to the music. I try to play it as I
enjoy it as if to make them enjoy it.
-- A graduating CMW student
PERSONAL AGENCY
Again, when students persist and engage, the result is a young musician who takes responsibility for
his or her own continuing development by practicing and thinking about music. This young person
understands the personal agency and effort that it takes to engage, persist, and make progress. In
Phase I this takes the form of joint child-family excitement over having and playing an instrument. In
highly engaged Phase II and III students this agency grows into an investment in mastery and the
development of a distinct musical voice.
Though I had played in an orchestra before, I got my voice musically [playing at CMW].
Maybe you could say I got certain preferences about how I wanted to play. I really wanted to
express myself through the music not just out of the book.
-- A graduating CMW student
6KHUHDOO\ZDQWHGWRDXGLWLRQIRUWKLVRWKHURUFKHVWUD6KHVSHQW,GRQ·WNQRZKRZPDQ\ZHHNV
getting ready practicing a lot at home. She asked me if I would drive her when the day came.
All the way there I could tell she wanted it to go well. Driving home she was disappointed, she
GLGQ·WWKLQNVKHSOD\HGZHOOHQRXJK:HWDONHGDOOWKHZD\KRPHDERXWKRZPD\EHLWZDV
better than she thought, how she could always try again.
-- Grandfather of a CMW student
WolfBrown
Page 7
If You Are Walking Down the Right Path: An Evaluation of Community MusicWorks
PARTICIPATION IN A WIDER WORLD
Students at CMW have the rare experience of a wide and continuous network of support that
includes family, consistent teachers, older students, and practice mentors.
At dinner my whole family all talked about the concert on Saturday. I had a solo on Saturday,
so we were reflecting on how people thought I did.
-- A highly engaged Phase II student
As students persist at CMW and mature as young adults, CMW introduces them to a still wider civic
and cultural world that includes concerts, opportunities to work with and talk with professional
musicians, and the rare experience of performing pieces composed for CMW musicians. Highly
engaged Phase II and III students who take up these opportunities develop a keen sense of their
musical identity in a wider world:
The best day of my life, the day I will remember forever, is the day that we worked with DBR1
DQGSHUIRUPHG7KDWZDVWKHNLQJRIGD\VZKHQ,VDLGWRP\VHOI´7KLVLVZK\,SOD\µ,
FRXOGQ·WJRWRVOHHSWKDWQLJKW
-- A Phase III student
They also develop a network of connections and experiences that gives them new knowledge and
opportunities and expand their horizons:
Today at the concert I saw the girl who had been here for nine years, and she got this scholarship
and she is furthering her education, whether or not the scholarships comes from her being here
alone or not. ,WUHPLQGHGPHRIP\IULHQG·VGDXJKWHUWKHRQH I came to see. In nine years, she
PLJKWEHLQWKHVDPHVSRWJUDGXDWLQJZLWKDVFKRODUVKLS0D\EH\RXGRQ·WNQRZ7KDWZDVP\
favorite. Seeing someone stick with the program for nine years. It is awesome, after working for
nine years, college will be nothing.
-- Friend of a CMW family
ENRICHED AND SUSTAINED INNER LIVES
Persistence, musicianship, agency, and participation in a wider world are the separate faces of the
kind of motivated, curious, and engaged inner life that the CMW program helps young people forge.
When students persist, when they engage deeply with the musical training and take advantage of the
many opportunities presented, remarkable things occur.
There is evidence that even outside of CMW lessons, music becomes a force organizing their lives,
focusing their activities and relationships. Here, for instance, is a Sunday in the life of a highly
engaged Phase II player (with the instances of musical activities highlighted):
Daniel Bernard Roumain, a contemporary composer and violinist, who composed and rehearsed an original piece with
CMW students.
1
WolfBrown
Page 8
If You Are Walking Down the Right Path: An Evaluation of Community MusicWorks
Where
Activity
Home
Breakfast ² Ate a bowl of cereal
Got myself and brother ready for
church
Listened to music on my stereo
Went to church
Service ² I played viola
About 30 min
Mother had meeting, watched my
brother play outside
Listened to iPod
1:30 ² 2:30
Church
(Bethel)
)ULHQG·V
house
Home
Took my instrument, she plays
too, we played music, then
watched TV with her little sister
Got on Facebook with CMW and
others ² a friend from CMW got
me on last summer
Dinner ² We talked about the
concert on Saturday, I had a solo
on Saturday, so we were
reflecting on how people thought
I did
Got ready for school
Watched MTV
Listened to music on stereo
How Long?
With
How
regularly?
Family
11:00 ² 1:30
About 3:00
30 - 45 min
Family
Listen every
day when I get
ready
Family
Congregation
Twice a
month
Another
friend, a cellist,
was going to
come
Get together
to play about
once every
two months
About 30 ² 45
min
10 ² 15 min
Every day
Family
Whenever I
perform
About 30 min
About 30 min
Every day
The program can also result in young people learning habits of hard work, investment, and mutual
responsibility:
The senior CMW quartet I got to coach in the 2008 camp simply made my week. They had a
number of attributes which made them one of the best groups there ² they were older than most,
they had known each other for a long time, they had a highly developed (and very funny) rapport
and common language, and they seemed 100% personally invested in being musicians and
SOD\LQJWRJHWKHU«In no way were they looking to their coach for motivation ² they brought
their own, and they just wanted to play better. Their Haydn quartet movement was a fast,
energetic piece and it pleased them greatly «the first violinist said when they got the piece up to
tempo and played it off the string and made it to the end, ´I love this piece so much I want to
PDUU\LWµ
-- A coach from a partnering summer program
In sum, the experience of growing up at CMW can provide an anchor for a core identity:
,FDQ·WJRWRFROOHJHZLWKRXWP\FHOOR «,·OOSUREDEO\JRFUD]\ «,FDQ·WLPDJLQHP\VHOIQRW
playing music
-- Phase III student
WolfBrown
Page 9
If You Are Walking Down the Right Path: An Evaluation of Community MusicWorks
MAJOR CHALLENGES
The key question for CMW is how more than the most persistent and most engaged students can be
affected in deep and lasting ways.
ENROLLMENT AND PERSISTENCE
Such remarkable outcomes for young people depend on students enrolling early enough, becoming
highly engaged, and persisting in the program. However, less than half (47%) of CMW students
enroll by age 9, with 88% enrolled by 12. Moreover, during high school (particularly after age 16)
students are increasingly likely to drop out voluntarily.
MUSICIANSHIP
Samples of students playing in lessons suggest that CMW students are technically young for the
number of years that they have been playing (in intonation, rhythm, fingering, etc.). Part of the
reason may be that students report that their lesson is the only time that they play. This means that
despite their interest they simply do not have the hours they need for fluency on their instruments.
Given [WKH&0:VWXGHQW·V] incredible motivation and rapport and their growing identity as
musicians, I'd like to see them get all the tools they need to play at an even better level.
-- A coach from a partnering summer chamber music camp
This raises the question about whether it would be possible to create social settings (e.g., afterschool
practice sessions), cross-age mentoring, or family-to-IDPLO\´MDPµVHVVLRQVWKDWZRXOGKHOS\RXQJ
players get the hours they need on their instruments.
PERSONAL AGENCY
Students across phases have a limited sense of what they can do to make a difference in their playing.
7KH\UHSRUWWKDWWKH\GRQ·WNQRZKRZto practice in ways that make a difference or how to talk
about where they are struggling.
Teacher: What do you do when you get to a hard part in the piece?
Student: If I am feeling energy, sometimes I keep going over and over it and seeing if I could get
better. Or sometimes I just stop or leave it to my lesson.
-- Phase II student
7KHGDWDRQVWXGHQWV·LQYROYHPHQWDQGLQYHVWPHQWLQSUDFWLFLQJDQGSOD\LQJVKRZDGLSEHWZHHQ
Phase I and later Phase II and III engagement. There is a period when some students and their
families report losing interest or sense of progress.
I would say he has lost steam in this last year. He was very excited when he got his instrument,
ZKHQKHFRXOGJHWDJRRGVRXQGZKHQKHSOD\HGKLVILUVWUHDOVRQJV1RZ,·GKDYHWRVD\WKDW
KH·VORVW+HGRHVQ·WVHHKLPVHOIDVDJood musician who is moving along to bigger things. He
KDVQ·WKDGPXFKRIDSDUWLQSHUIRUPDQFHV,GRQ·WZDQWKLVLQWHUHVWWRIDGH
-- Parent of a CMW student
WolfBrown
Page 10
If You Are Walking Down the Right Path: An Evaluation of Community MusicWorks
These findings suggest that particularly during WKH´ORQJKDXOµRIGHYHORSLQJWKHPDQXDODQGPXsical
skills for playing well, CMW students may need a more articulated sense of what they can do to
progress and how to use their lessons or home practice sessions. It is possible that students also need
clearer markers of their progress (e.g., a process for setting goals, different roles and tasks that they
can take on, and choices of pieces they want to tackle). It is worth considering the model of
development practiced in karate and other martial arts where there is a sequence of clear levels that
students prepare for and take on as they recognize they are ready.
PARTICIPATION IN A WIDER WORLD
An additional challenge for CMW is how to translate the intimate developmental network (i.e.,
students, families, and teachers) into a set of wider opportunities for increasing numbers of students.
Families describe how hard it is to do anything more than lessons and practicing when their children
are also involved in sports, debate, or step classes. Some Phase I and less engaged Phase II students
describe additional CMW opportunities, such as concerts trips or workshops as being like
homework: something that has to be done.
,GRQ·WOLNHGRLQJWKHH[WUDVWXIIOLNHWKHFRQFHUWWULSV,ZDVQ·WJRLQJXQWLOWKH\VDLG\RXKDGWR
go to at least VRPHRIWKHP«
,GRQ·W do the labs ² OLNHPXVLFODEDQGILGGOHODE,GRQ·WOLNHKDYLQJWRPDNHXSWKLQJVLQ
front of other people.
-- A fourth year student at CMW
This student speaks to what may be an underlying challenge for CMW: finding ways to engage
younger and less engaged students in taking risks and trying out the many opportunities that the
organization has to offer. This entails trying new situations and venturing outside familiar routines
and neighborhoods.
RECOMMENDATIONS FOR THE FURTHER DEVELOPMENT OF
CMW
ENROLLMENT AND PERSISTENCE
More than half of the students in the 2005 cohort have persisted in the program ² a retention rate
that is remarkable for a classical music program in a city where economic hardship frequently forces
families to be highly mobile. Even so, there are steps CMW should consider taking:
Enrolling students at younger ages in order to give them the time to develop the skills that
will let them progress.
Helping older students to persist even though there may be strong competition from middle
and high school activities and home-based demands, as well as the lure of or need for afterschool jobs.
WolfBrown
Page 11
If You Are Walking Down the Right Path: An Evaluation of Community MusicWorks
Collecting exit data when families and students leave the program in order to understand
what motivated their leaving.2
MUSICIANSHIP
CMW seeks to share a distinctive kind of mXVLFLDQVKLS:KLOHWKHSURJUDP·VSULPDU\JRDO is not to
prepare professional musicians, the program does aim to give students the rewards of working hard
on a challenging skill, developing a level of personal expression, becoming a contributing member of
a community, and understanding that music can be a powerful force in the life of individuals and
communities. To make this possible for more students, CMW should consider:
Creating an articulated map of expectations, skills, and possibilities for each phase of work at
CMW that can be shared with faculty, fellows, families, and students.
Investing in workshops with skilled teachers of young string players for faculty, fellows, and
mentors, with an emphasis on strategies that work as early as Phase I.
Developing social forms of practice that support progress (e.g., times and places where
students can practice together, get help learning how to practice effectively, and learn from
older students and young adults working on pieces).
Continuing work with families so that they can support, enjoy, and extend practice outside
of lessons.
PERSONAL AGENCY
This set of skills catches fire in highly motivated Phase II and III students. The challenge is how to
build these skills and habits earlier and more widely by:
Building a more active role for students in their own musical development (e.g., picking their
next piece from a range of appropriate choices;; announcing selections and what to listen for
at Performance Parties;; or keeping a digital portfolio.).
Continuing lesson-embedded interviews to support mutual reflection and goal-setting
between teachers and students.
Building a system of electronic portfolios that help students to chart progress (music
samples, interviews, and short personally-relevant samples from Performance Parties) and
yield year-HQG´DOEXPV.µ
Creating paid roles for Phase III students to assist in this work, while they also learn relevant
musical, instructional and technical skills.
PARTICIPATION IN A WIDER WORLD
These capacities are a distinguishing feature of the programs at CMW, expanding its scope of
outcomes beyond what is sought by many other music education programs. Currently, a small group
of students realize these outcomes. The challenge for the organization is to begin to build these
An effort to examine comparison data from other community-based music programs revealed that most institutions track
enrollment but not persistence.
2
WolfBrown
Page 12
If You Are Walking Down the Right Path: An Evaluation of Community MusicWorks
capacities earlier, more explicitly, and in a wider number of young people. In combination with the
strategies outlined above, additional actions could include:
Building on events like WKLV\HDU·VFRPSRVLWLRQDQGSHUIormance of an original piece in
KRQRURI2EDPD·VHOHFWLRQ, which provided many students with a close-to-home
introduction to the wider world.
Expanding the range of places where CMW students can play through faith-based
organizations, schools, neighborhood festivals, etc.
Expanding partnerships with other organizations where older students can play (e.g.,
Philharmonic Music School, Rhode Island Youth Philharmonic Orchestra, Apple Hill, Bay
Chamber Concerts, etc.).
Seeking and more widely distributing opportunities for Phase II and III students to present
the work of the organization, through attending live events or through producing web-based
materials.
,QYHVWLQJLQZRUNVKRSVRUSDUWQHUVKLSVWKDWZLOODOORZVWXGHQWV·digital recording and
documentation skills to mature and improve so that they can contribute reflections to the
website, produce materials for YouTube, etc., at an increasingly high level of quality.
CREATING SUPPORT FOR THIS EXPANDED WORK
Taking on the work outlined above will require considerable organizational and community support.
Staff, Board members, and families will have to:
Build family-based musicianship: Many families want to be better able to support the
musical aspects of practicing (e.g., by asking questions, offering strategies, or turning to
CMW CDs or a website with examples of what the pieces sound like played well or in a
variety of styles/approaches, etc.). CMW could also work with families to create communitybased opportunities for young people·s playing (e.g., at faith-based services or family events,
or by creating CDs to send to distant relatives, etc.).
Establish media partnerships for documenting development: CMW would benefit from
an alliance with a strong media partner to help with the organizational design and
implementation of electronic musical portfolios for students, as well as internships for older
students. The purpose would be to record individual student development, support their
reflection, and allow them to share their musical work (e.g., playing, composing, writing liner
notes) with others.
Staff for a differentiated program: If these investments yield younger students enrolling
and higher numbers of students persisting into Phases II and III, the organization may well
need to think in terms of more explicit specialization in faculty and fellows, with some
individuals concentrating on early years and others focusing on more mature young
musicians.
Create ongoing data systems: As a part of annual intake and year-end interviews with
students and families, it is important to build stronger ongoing data collection systems that
WolfBrown
Page 13
If You Are Walking Down the Right Path: An Evaluation of Community MusicWorks
regularly capture information (e.g., grades, course enrollments and electives, auditions and
additional musical activities, forms of personal or community service, etc.). Together with
longitudinal persistence data, this would provide the foundation for ongoing reflection on
the program and for possible larger scale comparisons on a range of indicators (e.g.,
comparisons with other neighborhood students who also attend Providence Public Schools.)
This information would be an ongoing resource for talking about the program with funders.
Collaborate and learn from a network of colleague organizations: CMW has strong
local organizational colleagues through the Providence Youth Arts Collaborative. There is
also a growing network of music-specific colleagues engaged in similar youth development
and community-building work, including several sister organizations that, like Community
MusicWorks, were featured by Chamber Music America aV´6HYHQ6WDUVµ in the arts
education field. Two of these programs were directly inspired by CMW. Regular exchanges
with these programs will be vital to both problem-solving and innovation.
MOVING FORWARD: DEVELOPING ACTION STEPS
Acting on the above recommendations will take time, reflection, and resources ² even as CMW has
programs to run, new fellows to welcome, and other initiatives to undertake. With that in mind,
together the Board and staff might consider using the evaluation as a stimulus to discussion,
designing options and making choices. Critical steps in such a process might include:
Holding discussions on key points designed to develop a prioritized list of which
recommendations are the most urgent and feasible in the 2009-2010 year and across the next
three years.
Ensuring that the strategic plan for the organization accounts for work toward the key
recommendations that are selected.
Continue to share and substantiate these findings through regular evaluations that can index
progress in all the areas highlighted in this report.
Expand the indices of student success to include dimensions such as academic achievement,
community service, and enrollment in additional forms of music education.
Develop strategies to follow up with graduates of the program following their later
development as students, musicians, and concerned citizens.
WolfBrown
Page 14
If You Are Walking Down the Right Path: An Evaluation of Community MusicWorks
SECTION TWO: INTRODUCTION
Photo credit: Jori Ketten
WolfBrown
Page 15
If You Are Walking Down the Right Path: An Evaluation of Community MusicWorks
For twelve years Community MusicWorks (CMW) has offered musical instruction and experiences to
community children in the West End and South Side neighborhoods of Providence, areas of the city
that are home to many working families who seek to find the best for their children in a city that is
hard-pressed to provide opportunities such as music and art in its schools.
The highly accomplished members of the Providence String Quartet provide private and group
lessons on the instruments of the classical string quartet: violin, viola, and cello. In addition, children
enroll in elective courses such as Music Lab and Fiddle Lab, attend workshops by visiting ensembles
and composers, and perform regularly at parties and salons attended by their teachers, families, and
members of the community.
Unlike the pre-collegiate divisions of many major conservatories, &0:·V purpose is not to prepare
students exclusively for careers as professional musicians. Instead, CMW provides sustained training
in classical music to children in order to develop a broader, holistic musicianship designed to foster a
life-long engagement with and love of music as a source of personal and community identity.
Moreover, the CMW program seeks to harness the power of music as a tool for building dedication
and perseverance, along with a commitment to joint action and social justice.
In 2008, the Rhode Island Foundation provided funding for an evaluation of the CMW program.
The staff of CMW began work on this evaluation with an Advisory Team composed of staff and
Board members, scholars, and evaluators. Subsequently, CMW commissioned Dennie Wolf and
Steven Holochwost of WolfBrown to conduct and write the evaluation. The following document
provides:
context for the evaluation
background on the approach and process of the evaluation
major findings for each of the key outcomes (Musicianship, Agency, Relations to Wider
World)
recommendations and action steps
THE CONTEXT FOR THE EVALUATION
THE PROMISE AND PARADOX OF CITIES
Cities are rich in resources (libraries, museums, magnet and charter schools, parks, colleges and
universities), and many of these are free. Cities have another source of cultural wealth ² they are
places of confluence where languages, cultures, religions, and cuisines meet and fuse. As rich as they
are, cities also host the greatest divides in opportunities for children and youth. Providence is no
exception. Growing up on the affluent East Side or in the poorer West End is dramatically different
in terms of the quality of public schools, the hours the public library is open, the facilities at the Y,
and the number of bus transfers it takes to reach a museum, concert, or a campus class. Even access
to and cost of fresh fruits and vegetables follows this same geography.3
This unequal distribution matters, as it has long fueled the gaps between who graduates from high
school, who attends college, and who is in a position to thrive and give back to their community.
However, these gaps may take on added meaning in current times when earning and thriving will
increasingly be tied to an emerging set of characteristics:
3
Rothman, Robert. (2007). City Schools. Cambridge, MA: Harvard Education Review Press.
WolfBrown
Page 16
If You Are Walking Down the Right Path: An Evaluation of Community MusicWorks
Who is a citizen of the world: someone who can move comfortably and respectfully across
situations, languages, and different cultural communities?
Who thinks critically about existing knowledge and systems?
Who can create and innovate?
Who is willing to take action and work with others to make a difference?4
While some might argue that these capabilities are the responsibility of families and/or schools,
Community MusicWorks has developed a different proposition: even modestly-sized cultural
organizations, if positioned within urban neighborhoods and willing to open their practice to children
and families, can make substantial contributions to:
the development of children and youth
fDPLOLHV·FDSDFLW\WRSURYLGHRSSRUWXQLWLHVIRUWKHLUFKLOGUHQ
the stability and quality of life in a neighborhood
a enlarged mission for arts and cultural organizations situated in urban communities
It is this larger hypothesis that furnishes the broad context for the design and conduct of this
evaluation of Community MusicWorks.
THE CMW NEIGHBORHOODS
By locating its office, teaching spaces, and performances in the West End and South Side
neighborhoods of Providence, and by offering free instruction, CMW stands to challenge both the
racial and economic homogeneity of classical music education and practice. Data from the
Providence Plan make it clear that families in these neighborhoods who seek the best for their
children are doing so under challenging circumstances:5
Diversity
% Public school
children with primary
language other than English
Median Family Income
% Families below Poverty
The CMW Neighborhoods
White 26.5%
Non-Hispanic White 13.8%
Hispanic 51.6%
Black 18.7%
Asian 13.2%
Providence
White 54.5%
Non-Hispanic White 45.8%
Hispanic 30.0%
Black 14.5%
Asian 6.2%
68%
32%
$23,346
36.6
$32,058
23.9
4
Although differently worded, these are the capacities listed by the Partnership for 21 st Skills, under the discussion of
learning and innovation skills. Available online at www.21stcenturyskills.org/index.php?Itemid=
120&id=254&option=com.
5
Providence Plan. Available online at http://local.provplan.org/profiles/wed_main.html. Accessed September 21, 2009.
WolfBrown
Page 17
If You Are Walking Down the Right Path: An Evaluation of Community MusicWorks
THE CONTEXT OF CONTEMPORARY MUSIC EDUCATION
In 2008, the National Assessment of Educational Progress published data from a nation-wide
assessment of arts education. The assessment, which was focused on 8th graders studying music and
the visual arts, provides a sobering picture of the kinds of activities young adolescents engage in and
their resulting skills. Among the most discouraging findings is how infrequently students engage in
making music (33% playing instruments, 28% singing, 17% composing).6 Moreover, these patterns
are stubbornly low across an entire decade.
The percentage of students at 8th grade, by student-reported in-school activities their teachers ask them to do at least once
a month in 1997 and 20087
Listen to music
Sing
Play instruments
Write down music
Work on group assignments
Make up own music
1997
51%
30%
28%
26%
36%
16%
2008
49%
28%
33%
33%
35%
17%
6
U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National
Assessment of Education Progress (NAEP), 2008 Arts Assessment. Available online at
http://nces.ed.gov/nationsreportcard/pdf/main2008/2009488.pdf. Accessed December 17, 2009.
7 Ibid.
WolfBrown
Page 18
If You Are Walking Down the Right Path: An Evaluation of Community MusicWorks
Clearly, the majority of 8th graders in this country have only rudimentary knowledge of how music
works, how to read it, or how to talk about it:
Map of the items that current US 8th graders can accomplish in music8
Scale
score
300
-275
265
239
237
230
228
225
195
195
192
183
182
176
174
172
167
166
163
151
136
128
124
84
49
-0
Question description
Identify piece of music as coming from 09th century and provide limited justification why
,GHQWLI\RQHHOHPHQWRIMD]]SUHVHQWLQ´5KDSVRG\LQ%OXHµ
Describe an emotion or mood created by a composition and describe two ways in which
emotion or mood was created
Provide a partial identification and description of errors in pitch in an instrumental solo
Describe a similarity and a difference between two written vocal parts
,GHQWLI\WKHVRORLQVWUXPHQWEHJLQQLQJ´5KDSVRG\LQ%OXHµ
Provide a comparison between the tone color of two different singers
Select a line drawing reflective of the texture of an example of music
Identify the term for a fermata symbol
Identify the name of a piano dynamic marking and explain its meaning
Either identify the name of a piano dynamic marking or explain its meaning
3URYLGHDOLPLWHGH[SODQDWLRQRIZK\VSLULWXDOVZHUHLPSRUWDQWLQSHRSOH·VOLYHV
Identify a correct time signature for a piece of music
75th percentile
Identify region of origin of African musical excerpt and provide a description of
characteristic related to its style
Identify a bass clef symbol
Identify the type of instrumental ensemble performing an excerpt
Describe one feature of a song that identifies it as a spiritual
Identify the length of the introduction oI´6KDORP0\)ULHQGVµ
Identify directional contour of part of melodic phrase
25th percentile
Identify region of African musical excerpt and provide partial explanation of its style
characteristics
Describe an emotion or mood created by a composition
Identify region of origin of African musical excerpt
Note: Regular type denotes a constructed-response question. Italic type denotes a multiple-choice question. The position of
a question on the scale represents the average scale score attained by students who had a 65 percent probability of obtaining
credit as a specific level of a constructed-response question, or a 74 percent probability of correctly answering a four-option
multiple-choice question.
Additional NAEP data also make it clear that students in city school systems have less access to
music and correspondingly perform at a lower level in music than students from other settings.
Similarly poor children also perform at significantly lower levels than their wealthier peers.9 These
realities are evident in a city like Providence, where there are few music classes available, especially at
middle schools and high schools, and where bands and orchestras have disappeared.
Ibid.
Keiper, Shelley, Sandene, Brent, Persky, Hilary and Kuang, Ming. (2009). 7KH1DWLRQ·V5HSRUW&DUG7KH$UWV
Washington, DC: Institute for Education Science. National Assessment of Educational Progress.
8
9
WolfBrown
Page 19
If You Are Walking Down the Right Path: An Evaluation of Community MusicWorks
Additional data from recent surveys of community schools of the arts indicates that CMW is
distinctive even in the context of organizations whose mission is to make music learning widely
accessible.10 Compared to many other community music schools, CMW serves a much higher
proportion of children of color, especially Hispanic students, and provides universally free music
learning as compared to a mix of scholarship and fee-based lessons.
THE CONTEXT OF CLASSICAL MUSIC
&:0·VZRUNDOVRQHHGVWREHXQGHUVWRRGLQWKHFRQWH[WRIWKHFRQWHPSRUDU\ZRUOGRIFODVVLFal
music, in which Latinos and African Americans comprise only 4% of symphony players,11 but
account for nearly a third of the general population. Though most classical music organizations are
located in urban areas, many of which are primarily inhabited by people of color, audiences are
predominantly white, usually coming from the suburbs. Most classical music audience members are
over sixty years old, despite the huge interest in music among young people. There are, therefore,
serious questions about how the classical music tradition will be shared, but also how it will be
invigorated by the presence of players and audiences who bring different traditions and sensibilities
to their musical experiences.
If a conductor rehearses and performs a given piece wiWKD´QRQ-GLYHUVHµRUFKHVWUDDQGWKHQ
rehearses and performs the same piece with a diverse orchestra, the result will be a completely
different artistic product ² one that I would argue is superior, because the scope of the artistic
input that went into its creation is so much greater. Classical music is an art form that thrives
on new interpretations and cultural influences. Currently, it is thirsting for new oases from
which to sustain its artistic vibrancy.
-- Aaron Dworkin, President of the Sphinx Organization12
In addition, many musicians would point out that the pool of young players is also homogeneous in
terms of their class background. Becoming a skilled player demands enormous resources. There are
the obvious investments, such as a quality instrument, music, the cost of lessons, and concert clothes.
In addition, there are the sidebar costs: tickets to live performances, discs to listen to, and
WUDQVSRUWDWLRQWRDQGIURPOHVVRQV$QGWKHUHDUHWKH´KLGGHQFRVWVµOLNHWKHWLPHVRFLDOFDSLWDODQG
research it takes to identify a good teacher, to find out about and travel to auditions, and to prepare
and qualify for scholarships at special programs.
Thus, the location, enrollment, and programs at CMW challenge the belief that classical music is
inevLWDEO\:HVWHUQHOLWHDQG´DERYHµWKHFRQFHUQVRIDFRQWHPSRUDU\XUEDQZRUOG,QWKLVUHJDUG
&0:·VSURJUDPVUHSUHVHQWRQHFRPSHOOLQJVWUDWHJ\IRUWKHVXUYLYDOIRURUFKHVWUDVDQGFKDPEHU
music: the creation of a diverse body of music-lovers through consistent exposure to and instruction
in the art itself.
National Guild of Community Schools of the Arts. (2009). Field Survey Report: 2007-2008 Academic Year.
Sadykhly, Afa. Diversifying the symphony. Available online at www.polyphonic.org/panels.php?id=8&day=1. Accessed
September 20, 2009.
12 Dworkin, Aaron. (2007). In pursuit of diversity in our orchestras. Available online at www.polyphonic.org/article.php?id=102.
Accessed September 20, 2009.
10
11
WolfBrown
Page 20
If You Are Walking Down the Right Path: An Evaluation of Community MusicWorks
COMMUNITY MUSICWORKS IN CONTEXT
Based on the conviction that musicians can play an important public service role, Community
MusicWorks has created an opportunity for a professional string quartet to build and transform the
urban community of which it has become an integral part -- Providence, Rhode Island. Through a
permanent residency of the Providence String Quartet, Community MusicWorks provides free after
school instruction and performance opportunities that build meaningful long-term relationships
between musicians, young people, and their families in the neighborhoods of the city.
As a music program, CMW is unusual. First, its definition of musicianship is bold, encompassing
more than technical proficiency in classical music by involving students in many string traditions and
urging them to develop improvisational and composing skills. Second, at CMW the intention is that
playing music becomes an experience of developing personal agency and of recognizing both the
power and the responsibility of having a voice in larger civic and cultural conversations. Finally,
music is seen as a way of being in the world: a set of practices for learning from and teaching others,
giving back and contributing, and traveling towards new horizons.
&0:·VFXUUHQWFXUULFXOXPH[SDQGVWKURXJKWKUHHSKDVHVLQZKLFKERWKWKHPXVLFDODQGFRPPXQLW\
work deepen. When they enter the program, all children are assigned to Phase I. Students in this
phase of the program participate in weekly individual lessons, monthly workshops and trips to
concerts, and Performance Parties, which take place five times a year. Once they have achieved a
sufficient level of expertise, and are at least 12 years old, they advance to Phase II, which is intended
for early adolescents. In this phase, students participate in weekly orchestra rehearsals, music lab
classes that teach improvisation, theory, and composition, and discussions on themes of social
justice. In addition, Phase II students organize and help choose the topics for Youth Salons in which
they and audience members discuss an important issue. (In 2009, the Salon focused on the inequities
in arts education for young people growing up in Providence.) Phase III students, who are late
adolescents (ages 15 to 18), participate in all Phase I and II activities, but also rehearse and perform
as a chamber ensemble.
Thus, unlike the pre-FROOHJLDWHGLYLVLRQVRIPDQ\PDMRUFRQVHUYDWRULHV&0:·VSULPDU\SXUSRVHLV
not to prepare students to become professional musicians. Instead, CMW provides sustained training
in classical music in order to develop a broader, holistic musicianship designed to foster a life-long
engagement with and love of music, as well as a keen sense of community and social justice. In
addition, CMW seeks to harness the power of music as a tool in building perseverance, agency and
curiosity, competencies that have applications far outside the arts, and indeed that are critical to
success throughout life.
This mission represents a sharp contrast to typical outreach concerts that many musical institutions
offer WRXUEDQVWXGHQWV,QWKLVVHQVH&0:·VZRUNSURYLGHVDcompelling alternative strategy for
building audiences for classical music. Rather than exposure or lower-priced tickets, CMW has reimagined the survival of orchestral and chamber music as the creation of a diverse body of musiclovers through consistent exposure to and instruction in the art itself. In this regard, CMW inherits
and expands a vital tradition in American learning: the belief that all institutions ² museums,
orchestras, universities, and libraries ² should be open to the public and contribute to the well-being
and vibrancy of their communities.
Finally, CMW holds and shares a distinctive definition of musical excellence. In this view, being a
highly skilled musician combines musical proficiency (both technical skills and expression), personal
WolfBrown
Page 21
If You Are Walking Down the Right Path: An Evaluation of Community MusicWorks
commitment, and an understanding of music as a cultural force that can both express and help to
create a more humane world. Though a static picture hardly captures this dynamic relationship,
&0:·VYLHZRIPXVLFDOH[FHOOHQFHORRNVVRPHWKLQJOLNH
Music as a powerful force
in expressing and
creating a humane world
Musical
Proficiency
WolfBrown
Personal
Commitment
Page 22
If You Are Walking Down the Right Path: An Evaluation of Community MusicWorks
SECTION THREE:
DESIGN OF THE
EVALUATION
Photo credit: Jori Ketten
WolfBrown
Page 23
If You Are Walking Down the Right Path: An Evaluation of Community MusicWorks
A FOCUS ON YOUNG PEOPLE·S MUSICIANSHIP, PERSONAL
AGENCY, AND PARTICIPATION IN A WIDER WORLD
There are many important questions that an evaluation might pose with regard to CMW, such as the
sustainability and scalability of its particular model of music education or the consequences for the
careers of the musicians who choose to work at CMW as quartet members, fellows, or practice
mentors. However, in early discussions of the evaluation, staff and advisors decided that the
foundational question they wanted to pursue was, in the words of CMW founder Sebastian Ruth:
How does becoming a member of the CMW communiW\DIIHFWDVWXGHQW·VLQQHUOLIHWKHLU
conceptions of themselves, their sense of themselves as effective, as well as their willingness to try
new things, or to be expressive?
In order to translate this broad question into an evaluation design, evaluators worked with an
Advisory Team and the faculty of Community MusicWorks to develop a logic model that reflected
the current inputs for each phase of the program and expected short- and longer-term outcomes in
three major areas:
Musicianship includes instrumental ability, dedication to regular practice, and the depth of
involvement in practice and performance, as well as general knowledge about the
mechanics and techniques of playing, musical vocabulary and concepts, and history of
music.
Personal Agency is a concept that encompasses self-HVWHHPPRWLYDWLRQWRDFKLHYHRQH·V
musical goals, and the ability to objectively monitor progress towards those goals, and,
when necessary, the willingness to develop and deploy different strategies toward reaching
those goals. In short, agency is the fuel for persistence and achievement.
Participation in a Wider Civic and Cultural World refers to sWXGHQWV·ZLOOLQJQHVVWR
participate in a world wider than their neighborhood, family, and school. One component
is the existence of a developmental network RILQGLYLGXDOVZKRVXSSRUWD\RXQJSHUVRQ·V
explorations through and beyond music. At an intimate level, this network may include
family members, CMW teachers, practice mentors, and other young musicians. But as
students take advantage of what CMW has to offer, this network branches and expands to
include distant and diverse role models (e.g., musicians like Daniel Bernard Roumain and
WKLQNHUVOLNH0D[LQH*UHHQH$VHFRQGFRPSRQHQWLQFOXGHVVWXGHQWV·GHYHORSPHQWRI
broader aspirations and their will to take on new experiences and challenges.
While these traits can be defined separately, in the lives of young musicians at CMW, they often
overlap and reinforce one another. The following reflections of a music coach at a partner program
in the region reflect how intertwined musicianship, agency, and participation in a wider world can be:
The senior CMW quartet I got to coach in the 2008 camp simply made my week. They had a
number of attributes which made them one of the best groups there ² they were older than most,
they had known each other for a long time, they had a highly developed (and very funny) rapport
and common language, and they seemed 100% personally invested in being musicians and
playing together «In no way were they looking to their coach for motivation ² they brought
their own, and they just wanted to play better. Their Haydn quartet movement was a fast,
WolfBrown
Page 24
If You Are Walking Down the Right Path: An Evaluation of Community MusicWorks
energetic piece and it pleased them greatly «the first violinist said when they got the piece up to
tempo and played it off the string and made it to the end, "I love this piece so much I want to
PDUU\LWµ
They set a good example for our students. I also feel like I got some perspective on my own
teaching situation that serves families and teaches within the dynamics of the family ² I was
inspired by the CMW students· independent ownership of their work in classical music and
their creation of a new community based on music and friendship.
-- Coach from a partnering summer music program
A DEVELOPMENTAL MODEL
Our evaluation used a developmental (or longitudinal) model, in which we examined the changes in
each of these three dimensions over time, looking at the differences between students enrolled in
Phases I, II, and III of the program. We chose this strategy, rather than one that focused on
comparisons to a control group, for a number of reasons. First, there is no easily constructed control
group, as comparably engaged young musicians in the area are, on average, more privileged. The
young people on the CMW waiting list are chiefly younger students with modest training. In addition,
there was no ethical way to hold families on the waiting list throughout the evaluation, given that the
program can have many benefits for families and children and those willing to wait might be far from
a random sample of families. Finally, a design with a control group would have necessitated
collecting data from twice the number of children, over a comparable period of time. We chose
instead to use the available time and resources to design a set of research tools that could become
ongoing strategies for data collection and reflection for the organization and its many stakeholders.
Thus, the core research question for the evaluation became:
Do we see positive changes in musicianship, agency, and participation in activities in and beyond
WKHLUFRPPXQLWLHVDVVRFLDWHGZLWKVWXGHQWV·OHQJWKRIWLPHLQ&0:WKHLUOHYHORIHQJDJHPHQW
and their progression through the phases of the program?
CAPACITY BUILDING AND UTILIZATION APPROACH
,QDGGLWLRQWRDQVZHULQJWKDWFRUHTXHVWLRQDERXW\RXQJSHRSOHV·GHYHORSPHQWWKHHYDOXDWLRQKDG
two further purposes. The first of these was to build the capacity of individuals throughout the
organization to reflect on the investments and results of the program in an ongoing way, constantly
asking questions, seeking evidence, and thinking through possibilities and implications in order to
VWUHQJWKHQWKHRUJDQL]DWLRQ·VSUDFWLFHVThus, the design builds on a tradition of participatory and
empowerment evaluations.13 This approach stresses engaging stakeholders in designing and
conducting their own evaluations using an outside evaluator or a team of advisors in the role of
coach(es) or critical friends. The process emphasizes the following steps: 1) establishing a shared
13
Fetterman, David. (2001). Empowerment evaluation: The pursuit of quality. Advances in Program Evaluation, Vol. 7, pgs. 73
² 106. Fetterman, David, Kaftarian, Shakeh and Wandersman, Abraham. (Eds.). (1996). Empowerment evaluation: knowledge and
tools for self-assessment & accountability. Thousand Oaks, CA: Sage Publications. Suarez-Balcazar, Yolanda and Harper, Gary W.
(Eds.). (2003). Empowerment and participatory evaluation of community interventions. Binghamton, NY: Haworth Press.
WolfBrown
Page 25
If You Are Walking Down the Right Path: An Evaluation of Community MusicWorks
GHILQLWLRQRIWKHSURJUDP·VRXWFRPHVDQGWKHSXUSRVHVIRUWKHHYDOXDWLRQGHYHORSLQJDQGXVLQJD
UDQJHRIWRROVIRUWDNLQJVWRFNRIWKHSURJUDP·VFXUUHQWVWUHQJWKVDQGWKHIURQWLHUVIRULWVJURZWK
and 3) and planning for the future. When conducted jointly these three steps build organizational
capacity and a sense of community. An overarching goal is to create a culture of learning, selfassessment, and continuous improvement that far outlasts the period of the specific evaluation.
The second purpose was to develop a set of practices that would continue to be useful to CMW, long
past the end of the evaluation period. The aim was to develop and field test a set of research tools
that could translate into everyday practices that would allow CMW staff (in concert with the Board,
families, and students) to document, reflect on, and strengthen outcomes in years to come. For
example, during the evaluation teachers regularly interviewed their students about how and what they
were learning. The questions were deliberately designed to work for students of all ages, to fit into
the last minutes of a lesson, and to give both teachers and students a chance to reflect on what was
happening for students both within and outside of lessons. The interviews were also intended as a
WRROWKDWFRXOGEHDGDSWHGWREHFRPHDQRQJRLQJSDUWRI&0:·VSUDFWLFHV
THE EVALUATION PROCESS
As mentioned above, one of the opportunities in the evaluation was to engage faculty, staff, and
Board members in a year-long process of reflection and conversation. The major steps in this process
included:
(VWDEOLVKLQJD6KDUHG'HILQLWLRQRIWKH3URJUDP·V2XWFRPHVDQGWKH3XUSRVHVRIWKH
Evaluation
Step 1: Involving CMW Student Researchers: In the spring and summer of 2008, we worked
with Phase III CMW students to adapt and develop tools that would permit us to examine
the three broad areas of outcomes. First, a number of students volunteered to help us pilot
the interview tools that we had developed. Second, during the summer of 2008, the Phase
III students collaborated on ensuring that the tools would be effective with young people
(i.e., they used them with one another, discussed whether the information reflected their
experience, and helped to ensure that the language and directions worked for young
people).
Step 2: Collaborating with CMW Faculty: Beginning in the fall of 2008, teaching staff at
CMW became actively involved in the evaluation. As an integral part of weekly lessons,
teachers began interviewing a structured sample of their students (selected to vary in the
intensity of their involvement at CMW). These interviews focused in turn on the three
major outcomes the program is seeking: musicianship, agency, and participation in a wider
world. While furnishing data for the evaluation, they are also becoming a part of the
practice at CMW ² another tool that teachers have to work with their students. In
addition, CMW faculty critiqued and refined the logic model that informed the evaluation.
Step 3: Collaborating with CMW Families: A set of data collection tools used with CMW
IDPLOLHVIRFXVHGRQFDSWXULQJWKHZD\VLQZKLFKFKLOGUHQ·VSDUWLFLSDWLRQLQ&0:
DIIHFWVFRQWULEXWHVWRIDPLO\OLIHDQGYLFHYHUVD7KHLQLWLDOSLORWWRRNWKHIRUPRID´GDWD
IHVWLYDOµZKHUHIDPLOLHVDQGWKHLUchildren volunteered to come to a pizza dinner and create
maps of their creative and musical worlds, as well as diaries of the ways in which they were
involved in music. The evening also served to introduce families to the evaluation process.
WolfBrown
Page 26
If You Are Walking Down the Right Path: An Evaluation of Community MusicWorks
Developing and Using a Range of Tools to Capture the Strengths and Needs of the
Program
Step 4: Reflecting on Creating Public Value: In January and again in June 2009, several
members of the CMW evaluation advisory team met with evaluators of similar intensive
arts education programs across the country. The purpose was to discuss how to:
o develop new methods to capture and represent the effects of such programs
o structure evaluations so that they contribute to practices and quality of organizations
o present the resulting information in ways that are convincing to funders and a
broader public interested in positive outcomes for young people
Step 5: Documenting Student and Family Experiences: In the spring of 2009, faculty and
evaluators collected the final round of data. This included a final set of lesson-based
interviews conducted by teachers as well as additional interviews and data collection with
students by a team of outside evaluators. Board members also conducted their interviews
with families. In addition, evaluators attended multiple afternoon lesson sessions, rehearsals
for Performance Parties, and three performances.
Planning for the Future
Step 6: Sharing Initial Findings with the CMW Board: In August 2009, evaluators shared a
sub-set of findings at the CMW Board retreat. This discussion focused on the most
challenging of the findings ² those that could have far-reaching implications for the way in
which the Board used its time and resources in the coming year(s). The discussion and
questions that arose in that meeting were used to develop the discussion draft of the
evaluation.
Step 7: Discussion Draft Shared with Advisory Team and the CMW Board: In
November a discussion draft of the evaluation was shared with the Advisory Team, CMW
faculty, and Board for discussion and feedback.
Step 8: Final Evaluation Document delivered to CMW and the Rhode Island
Foundation.
Step 9: (Pending resources) The Design of Ongoing Reflective Tools for CMW:
Provided that resources are available, the evaluation process and findings will be translated
into a set of tools that will enable students, teachers, and the organization as a whole to
WKLQNDERXW\RXQJSHRSOH·VSURJUHVVWRZDUGWKHWKUHHRXWFRPHVRIWKHSURJUDP
Discussion with teachers and Board members suggest that several practices may be of
ongoing use:
o lesson-based interviews about the meaning and consequences of being a musician
o annual portfolios of student work, SRVVLEO\LQWKHIRUPRI'9'´DOEXPVµWKDW
FDSWXUHD\RXQJSHUVRQ·VSURJUHVVRYHUWLPHLQZRUGVDQGPXVLFLQcluding both
individual and ensemble performances
o potential internships for older students as mentors and technicians in helping to
produce these albums
o augmented data systems that produce a database that can yield regular data on the
demographics of students and families, persistence in the program, and other
potential forms of student information (course-taking and electives in high school,
additional musical activities, community service, etc.)
Throughout the evaluation process, many key CMW constituencies participated, contributing their
expertise and experience:
Evaluation Advisory Team: Liz Hollander, chair of the CMW Board;; Sebastian Ruth,
Executive-Artistic Director of CMW;; Chloe Kline, Program and Evaluation Specialist at
WolfBrown
Page 27
If You Are Walking Down the Right Path: An Evaluation of Community MusicWorks
CMW;; Karen Romer, a former CMW Board chair;; Shirley Brice Heath, Professor at Brown
and Stanford Universities, Eileen Landay, former Professor of Education at Brown
University;; and Jori Ketten, an artist and media documentary specialist who works closely
with CMW to record key events.
Faculty Interviewers: All CMW faculty members participated in interviewing students as a
part of their lessons throughout the 2008-2009 year.
Board Interviewers: Four Board members and three evaluators trained to interview parents
of CMW students.
Student Researchers: Phase III students helped to pilot the questions for the student
interviews.
Staff Researchers&0:·V3URJUDPand Administrative Coordinator, Liz Cox, together
with Chloe Kline, Program and Evaluation Specialist, helped to schedule and organize the
research, as well as reviewing the enrollment and persistence data.
WolfBrown
Page 28
If You Are Walking Down the Right Path: An Evaluation of Community MusicWorks
SECTION FOUR:
METHODS
Photo credit: Don Tarallo
WolfBrown
Page 29
If You Are Walking Down the Right Path: An Evaluation of Community MusicWorks
DATA COLLECTION
Data were collected in several ways.
An examination of the persistence data for the group of students who were at CMW
from 2005 to the present to provide a sense of who progresses and who leaves the program.
A series of semi-VWUXFWXUHGLQWHUYLHZVFRQGXFWHGE\VWXGHQWV·UHJXODU&0:WHDFKHUV
Students were asked to disFXVVWKHLUVSHFLILFH[SHULHQFHVGXULQJWKHSUHYLRXVZHHN·VSUDFWLFH
(i.e., what improved or remained difficult, what practice techniques they employed) and their
more general feelings about how and why they play their particular instrument. There were
three rounds of interviews. The first two rounds concentrated on musicianship and agency
questions, while the final interview asked additional questions about how students saw CMW
connecting them to the wider world.
A year-end interview with students with evaluators: In this interview, evaluators
interviewed a subset of 49 of the 73 focus students. These 49 represented a balanced sample
of ages, phases, levels of engagement, and gender of students enrolled in CMW. These
interviews were designed to supplement the shorter teacher-led interviews, particularly in the
areas of agency and connection to the wider world. These interviews included:
o A journal exercise in which children were asked to detail the SUHYLRXVGD\·V
activities, followed by specific questions aERXWWKHFKLOG·VHQJDJHment with music
during that day.
o A mapping exercise in which students were asked to diagram their musical world,
beginning with their experiences at CMW. This world could include people they
met, music they listened to, other musicians with whom they played, and ideas or
conversations they had about music.
o A series of twelve questions drawn from standard measures of self-efficacy and
agency.
Interviews with family members of focus students: These were conducted by CMW
Board members who were trained on a family interview protocol by evaluators. Interviews
ZHUHFRQGXFWHGHLWKHULQ6SDQLVKRU(QJOLVKGHSHQGLQJRQIDPLOLHV·ODQJXDJHRIFKRLFH
The interviewers collected data on three topics:
o hLJKDQGORZSRLQWVLQWKHLUFKLOGUHQ·Vexperience at CMW
o fDPLO\PHPEHUV·FRQWULEXWLRQVWRFKLOGUHQ·VSDUWLFLSDWLRQDW&0:
o areas where CMW could improve the program for students or outreach to families
The following table diagrams these sources of data and the way in which each contributed to an
overall picture of the outcomes for young people who participate in CMW.
WolfBrown
Page 30
If You Are Walking Down the Right Path: An Evaluation of Community MusicWorks
SOURCES OF DATA
Contextual Data
Observations of afternoon lessons, rehearsals, and Performance Parties
Ongoing discussions with faculty, fellows, Board members
Student Outcome Data
Persistence Data
Who enrolls and persists in CMW over time
developed with
Who leaves the program and for what reasons
CMW staff
Teacher and
Student
Lesson-based
Interviews
Student
Year-end
Interviews with
Outside
Evaluators
Parent
Interviews
conducted by
CMW Board
Musicianship
ƒ Practice
ƒ Musical
knowledge
ƒ Involvement
Personal Agency
ƒ Self-esteem
ƒ Selfmonitoring
ƒ Motivation
ƒ Accounts of
doing
something
difficult at
school/CMW
ƒ
Diary day:
Role of music in
daily lives of
students
ƒ
(Not a
dimension many ƒ
parents felt
comfortable
commenting
on)
Highs and
lows in
VWXGHQW·VWLPH
at CMW
Participation in Wider World
ƒ
ƒ
ƒ
Developmental network
Broader aspirations
Application to other spheres
ƒ
Maps of the worlds created
around CMW and a
comparison activity
ƒ
Distinctive consequences
of attending CMW
SCORING AND ANALYSIS
ENROLLMENT AND PERSISTENCE
All of the analyses of the persistence findings are based on the annual enrollment data that CMW
tracks and were provided to the consultants by Community MusicWorks staff. These data include
information on all those who have enrolled in Community MusicWorks programs since the
SURJUDP·VLQFHSWLRQLQ6HSWHPEHU'DWDLQFOXGHGQDPHGDWHRIELUWKLQLWLDOHQUROOPHQWGDWH,
transition to Phase II or III, end date (graduation or withdrawal from the program), and for those
who were enrolled in 2005 and who subsequently left, the reason for their leaving the program.
The 2005 cohort analysis included all students who were active in the program as of September 2005
(63 students7KHDQDO\VHVWUDFNHGVWXGHQWV·SURJUHVVWKURXJKWKHVHYHUDOSKDVHVRIWKHSURJUDP
until September 2009, also tracking the date and phase for any departing student. The reasons for
students no longer being active in the program were coded into four categories: 1) reasons beyond
control of the family or child (e.g., moving, family illness, etc.);; 2) graduation from the program;; 3)
moving to take up another instrument;; and 4) graduation. WolfBrown
Page 31
If You Are Walking Down the Right Path: An Evaluation of Community MusicWorks
LESSON-BASED INTERVIEWS: MUSICIANSHIP, AGENCY, AND CONNECTION
Thirty first-round interviews were selected at random and reviewed by one of the principal
investigators who was blind to the level and age of the student. Based on these interviews, indicators
of three levels were established each of the dimensions that the CMW program is designed to
improve (see the table below). These dimensions and levels were shared with CMW teachers and
revised according to their suggestions.
Musicianship
Practice
No evidence
Level 2
Level 3
ƒ Practicing 4-5 days in a
week
ƒ 3UDFWLFLQJ´RIWHQµRU
´DORWµ
ƒ Practicing 6-7 days in a
week
ƒ 3UDFWLFLQJ´DOOWKH
WLPHµ
ƒ Knowledge of
concepts, terms (scales,
articulations)
ƒ Knowledge of
composers, repertoire
ƒ ´1HYHUµHQWHULQJWKH
zone ² too distracted
ƒ Nothing is challenging
or difficult (disconnect)
ƒ Paying
attention/listening
when practicing
ƒ Naming specific
exercises practiced
ƒ Sometimes enters the
zone
ƒ Taking initiative ²
seeking out pieces that
DUH´FKDOOHQJLQJµ
ƒ Learning to manage
time ² practicing
before homework, at
consistent time
ƒ Very engrossed ²
´JRLQJWRYLROD-ODQGµ
ƒ Taking initiative ²
composing on own
ƒ Self-professed love for
playing, instrument, or
CMW
NO SCORE
ASSIGNED
ƒ Elementary
ƒ Intermediate
ƒ Advanced
ƒ Practicing 0-1 day in a
week
ƒ ´1HYHUµpracticing
Musical Knowledge
Involvement
Quality (technical
proficiency,
expressiveness)
Agency
Self-esteem
Self-monitoring/
Techniques &
Tricks
No evidence
ƒ ¶1RWJRRG·RU¶EDG·
ƒ No techniques
ƒ Nothing got better
ƒ Cannot articulate what
got better
Level 1
ƒ Practicing 2-3 days in
a week
ƒ Practicing
´VRPHWLPHVµ
ƒ Basic knowledge of
instrumental
mechanics (bow,
stroke, etc.)
Level 1
Level 2
ƒ Okay or pretty good
ƒ Getting better
ƒ Rudimentary pieces
becoming easy or
basic
ƒ Good
ƒ A lot of improvement
ƒ Wanting more
challenging pieces
ƒ Blunt repetition
ƒ Got better generally
ƒ General recognition of
value of practice
ƒ Mistakes as aids to
learning
ƒ Student can name
specific things got
better in playing
ƒ Noticing and
articulating difference
in playing from practice
Level 3
ƒ Great
ƒ Mastery of challenging
pieces
ƒ Change of selfconcept as a result of
CMW involvement14
ƒ Learning to deploy
techniques in service of
goals (varying tempi,
using recordings)
ƒ Extracting broader life
lessons connecting
effort and goal
attainment15
2QHVWXGHQWVWDWHGH[SOLFLWO\WKDWSOD\LQJDQLQVWUXPHQWZDVKHOSLQJWKHP´ILQGVRPHWKLQJQHZLQVLGHµDQRWKHUVDLGWKDW
one of the things they most enjoy about playing the violin is their newfound identity as a musician.
15 Noting, for example, that playing a musical instrument can help with college applications, or the more general sentiment
WKDWSHUKDSVSOD\LQJDQLQVWUXPHQWFDQ´JHWPHVRPHZKHUHµ
14
WolfBrown
Page 32
If You Are Walking Down the Right Path: An Evaluation of Community MusicWorks
Connection to
Wider World
No evidence
Level 1
Level 2
Family
Involvement
ƒ Parent suggested
CMW
ƒ Parent monitors
practice
ƒ Practicing in front of
parent
ƒ Family member as
inspiration to play
Mentoring
ƒ Some rapport between
teacher and student
ƒ Clear evidence of
student-teacher bond
Broader
Aspirations
ƒ Practicing for
performance party
ƒ Practicing for lessons
ƒ Just likes instrument
or wants to play
Carry-over from
Music to Other
Contexts
ƒ Very specific and
literal connections
ƒ 8VLQJPXVLFWR´JHW
VRPHZKHUHµ
ƒ Using music for college
entry
ƒ Inspired by
professional
ƒ Seeing connections to
practice and
performance in other
domains
Level 3
ƒ Parent actively aids in
practice sessions
ƒ Sibling actively aids in
practice sessions
ƒ Student confides in or
relies on teacher for
counsel, advice
ƒ Wanting to continue in
music for fulfillment or
self-actualization
ƒ Seeing music as having
taught them a very
wide range of strategies
for pursuing excellence
These levels were then used to assign a level for each dimension to each child at each interview.
These assignments were done blindly ² the investigator coding the interviews did not know how long
each child had been in the program. These scores were then averaged across the interviews to yield a
typical score (medial-OHYHOIRUHDFKVWXGHQW·VVWDQGLQJIRUHDFKRIWKHHLJKWGRPDLQVOLVWHGLQWKH
table above.16
Reliability was established by selecting a random sample of interviews from 15 children (representing
20% of the full sample of N = 75). The reliability sample was stratified to ensure it represented the
full sample in terms phase, gender, age, and teacher (see the table below). The second principal
investigator then scored these interviews using the guidelines detailed above. Inter-rater reliability was
high, ranging from 93% to 80% for the different categories.
Sample for Establishing Coding Reliability
Phase
I
II
III
Gender
Male
Female
Age (in months)
Mean
Standard Deviation
Teacher
Teacher 1
Teacher 2
Teacher 3
Teacher 4
Teacher 5
Teacher 6
Teacher 7
Teacher 8
Full (N = 75)
Reliability (N = 15)
56
16
3
9
5
1
37
38
7
8
148.7
34.7
160.8
37.8
8
10
14
5
8
16
7
7
2
2
3
1
2
3
1
1
Best- and worst-case scenarios were computed by taking the highest and lowest scores, respectively, across all interviews
for each domain.
16
WolfBrown
Page 33
If You Are Walking Down the Right Path: An Evaluation of Community MusicWorks
Given the highly personal and varied nature of the data on broader aspirations and carry-over to
other contexts and goals, evaluators decided to analyze a subset of the larger sample in depth, rather
than treat the entire sample more broadly. Fourteen Phase II and III students were included in this
subset, as were 14 Phase I students matched for gender and teacher.17
The resulting data were analyzed in the aggregate, to identify comparative strengths and weaknesses
of the program with regard to the three major outcomes: musicianship, agency and connections to a
wider world. 7KHQXVLQJHQUROOPHQWGDWDIURP&0:ILOHVFKLOGUHQ·VOHYHORIDWWDLQPHQWLQHDFKDUHD
was analyzed as a function of their length of time in the program. It was our expectation that levels
RIDWWDLQPHQWZLWKLQHDFKDUHDDQGRYHUDOOZRXOGEHSRVLWLYHO\FRUUHODWHGZLWKFKLOGUHQ·Vlength of
time in the program. On the topic of exposure to a wider world, data were analyzed qualitatively in
order to capture the major contrasts between Phase I students and their more experienced peers in
Phases II and III.
YEAR-END STUDENT AND FAMILY INTERVIEWS: DIARIES AND MAPS
At the close of the year a team of evaluators conducted extended interviews (45 to 60 minutes) with
49 of the focus students. (This was the total number of students whose families were able to add an
interview to their schedule during the final weeks of the program, minus the students who were
unable to keep their appointments.) The purpose was to collect qualitative data that would amplify
WKHILQGLQJVIURPWKHVFRUHGOHVVRQLQWHUYLHZV7KLVZDVGRQHXVLQJWZRWRROV6WXGHQWV·GLDU\
days in which they recorded their activities for the previous 24-hours, and highlighted where music
played a role and 2) Student maps of what and whom CMW connected them to.
7KHVWXGHQWV·GLDU\GD\VZHUHFDWHJRUL]HGE\VWXGHQWV·SKDVHs and levels of engagement. Additionally,
the diary days were analyzed for:
Musical engagement: All instances and types of occasions when students engaged in
musical activities.
Spill-over: The number and types of occasions where students initiated musical activity
on their own beyond lessons, performances, and practice sessions directly tied to CMW
(e.g., listeninJ´FXUDWLQJµDQGFROOHFWLQJRUDFWLYHO\SOD\LQJRUFRPSRVLQJ7KH
evidence of spill-over was seen as an indication that students were taking charge of their
own musical activity and also using it to carry them to other activities and relationship.
The VWXGHQWV·PDSVZHUHDOVRFDWHJRUL]HGE\VWXGHQWV·SKDVHs and levels of engagement. Further, they
were coded for:
Connections: The numbers of people, places, and activities students saw CMW as
connecting them to.
Branching connections: Occasions when CMW linked the student to an activity, place,
or person that subsequently led to additional connections.
Two Phase II students did not complete the final lesson interviews, and therefore were not included among the subset.
One teacher had only Phase II and III students;; the students of another teacher, with the same gender distribution, were
substituted instead. Because age and phase are closely related, it was not possible to match the two halves of the subset for
age.
17
WolfBrown
Page 34
If You Are Walking Down the Right Path: An Evaluation of Community MusicWorks
SECTION FIVE:
MAJOR FINDINGS
Photo credit: Jori Ketten
WolfBrown
Page 35
If You Are Walking Down the Right Path: An Evaluation of Community MusicWorks
ENROLLMENT AND PERSISTENCE
Like many community music schools, CMW provides lessons for students from six to eighteen years
of age. Students who stay with the program have the opportunity to progress through a series of
three phases with increasing levels of opportunity and range of offerings. In order to understand
&0:·VKROGLQJSRZHUHYDOXDWRUVH[DPLQHGWKHIXOOFRKRUWRIVWXGHQWV (i.e., all students who
were active in the fall of that year), examining who persisted and for how long.
Tenure in Program (all of 2005 cohort)
# students
% of total
Less than 1 year
3
5%
1 year
2
3%
2 years
7
11%
3 years
4
6%
4 years
10
16%
5 years
9
14%
6 years
10
16%
7 years
5
8%
8 years
8
13%
9 years
2
3%
10 years
3
5%
63
100%
Thirty-two students moved to Phase II (51% of total cohort).
Seven students moved back to Phase I (21% of those in Phase II).
Three students moved to Phase III (5% of total cohort).18
Tenure of 2005 Cohort in Program Beyond 2005
Active Withdrew
Graduated
As of 6/31/2005
63
0
0
As of 6/31/2006
53
9
1
As of 6/31/2007
36
21
6
As of 6/30/2008
27
26
11
As of 6/31/2009
21
29
13
Twenty-nine of 63 students (46%) in the 2005 cohort withdrew from the program in
subsequent years.
An additional 13 (21%) left the program due to graduation.
18
This is everyone in the database who is currently in Phase III.
WolfBrown
Page 36
If You Are Walking Down the Right Path: An Evaluation of Community MusicWorks
Percent of 2005 Cohort Remaining in Program
100%
90%
80%
70%
60%
50%
Withdrew
40%
Active
30%
20%
10%
0%
As of
6/31/2005
As of
6/31/2006
As of
6/31/2007
As of
6/30/2008
Tenure in Program Beyond 2005 (# of students)
Active
Active
Phase I
Phase II
As of 6/31/2005
56
7
As of 6/31/2006
39
14
As of 6/31/2007
20
16
As of 6/30/2008
15
11
As of 6/31/2009
10
11
As of
6/31/2009
Withdrew
Phase I
0
8
20
26
28
Withdrew
Phase II
0
2
7
11
14
Percent of 2005 Participants Remaining in Program
100%
90%
80%
70%
60%
Withdrew Phase II
50%
Withdrew Phase I
40%
Active Phase II
30%
Active Phase I
20%
10%
0%
As of
As of
As of
As of
As of
6/31/2005 6/31/2006 6/31/2007 6/30/2008 6/31/2009
WolfBrown
Page 37
If You Are Walking Down the Right Path: An Evaluation of Community MusicWorks
Tenure in Program Beyond 2005 (# of students) Excluding Graduates
Active
Active
Withdrew
Withdrew
Phase I
Phase II
Phase I
Phase II
As of 6/31/2005
56
7
0
0
As of 6/31/2006
39
14
8
1
As of 6/31/2007
20
16
17
4
As of 6/30/2008
15
11
21
5
As of 6/31/2009
10
11
23
6
Note: Seven of the 42 who left the program entered Phase II for a period of time, and then
returned to Phase I before leaving (or graduating). Of the 13 who graduated, 4 fell into this
category of student.
WHO LEFT THE PROGRAM?
Gender of Those Who Left the Program
(all reasons, including graduation)
2006
2007
2008
Female
6
15
23
Male
4
12
14
10
27
37
2009
28
14
42
Twenty-eight of the 38 females (74%) in the 2005 cohort withdrew from the program in
subsequent years.
Fourteen of the 25 males (56%) in the 2005 cohort withdrew from the program in
subsequent years.
Gender of Those Who Withdrew
(excluding graduates)
2006
2007
Female
5
10
Male
4
11
9
21
2008
13
13
26
2009
16
13
29
If those who graduated are excluded from the analysis, 16 of the 38 females (42%) in the
2005 cohort withdrew from the program in subsequent years.
If those who graduated are excluded from the analysis, 13 of the 25 males (52%) in the 2005
cohort withdrew from the program in subsequent years.
WolfBrown
Page 38
If You Are Walking Down the Right Path: An Evaluation of Community MusicWorks
Age at Enrollment for Those Who Left the Program
(all reasons, including graduation)
2006
2007
2008
2009
Age 7
1
2
3
4
Age 8
2
4
7
8
Age 9
2
2
4
5
Age 10
3
9
9
9
Age 11
0
1
2
2
Age 12
1
3
5
6
Age 13
0
1
1
2
Age 14
0
0
0
0
Age 15
1
2
2
2
Age 16
0
2
2
2
Age 17
0
1
2
2
Age 18
0
0
0
0
10
27
37
42
Nine of those who withdrew (23%) were age 10 at the time of enrollment, 8 (20%) were age
8, and 6 (15%) were age 12.
Age at Time of Departure (cumulative, all reasons
including graduation)
2006
2007
2008
2009*
Age 8
1
1
1
1
Age 9
1
1
1
1
Age 10
2
2
2
2
Age 11
0
0
1
1
Age 12
1
4
4
5
Age 13
1
1
1
1
Age 14
0
3
4
5
Age 15
0
4
5
5
Age 16
2
4
5
5
Age 17
0
1
2
3
Age 18 or older
2
6
11
13
10
27
37
42
*The total in 2009 represents all those who withdrew during
the years reported (i.e., each year includes those who
withdrew in the previous year(s).)
Twelve of those who withdrew (30%) were age 18 or older, including 5 of the 7 who were in
the program for 8 or more years.
WolfBrown
Page 39
If You Are Walking Down the Right Path: An Evaluation of Community MusicWorks
Age at Time of Withdrawal (cumulative, excluding
graduates)
2006
2007
2008
2009*
Age 8
1
1
1
1
Age 9
1
1
1
1
Age 10
2
2
2
2
Age 11
0
0
1
1
Age 12
1
4
4
5
Age 13
1
1
1
1
Age 14
0
4
5
6
Age 15
0
2
3
3
Age 16
2
5
6
6
Age 17
0
0
1
2
Age 18 or older
1
1
1
1
9
21
26
29
*The total in 2009 represents all those who withdrew during the years
reported (i.e., each year includes those who withdrew in the previous
year(s).)
Phase at Time of Withdrawal
(excluding graduates)
Phase I
Phase II
Age 12
5
0
Age 13
1
0
Age 14
4
2
Age 15
2
1
Age 16
5
1
17
4
Total
5
1
6
3
6
21
Two of those who were in Phase I at withdrawal
had recently returned to Phase I from Phase II.
Eighty one percent (17 out of 21) of those who withdrew did so from Phase I. However, 2
students of the 17 had only recently returned to Phase I, having withdrawn after some time
in Phase II.
WolfBrown
Page 40
If You Are Walking Down the Right Path: An Evaluation of Community MusicWorks
Years in Program for Those Who Left the Program
(cumulative, all reasons, including graduation)
2006
2007
2008 2009**
Less than 1 year
2
2
2
2
1 year
3
3
3
3
2 years
1
7
7
7
3 years
0
1
2
2
4 years
1
6
7
8
5 years
1
3
3
5
6 years
0
2
5
5
7 years
0
0
2
3
8 years
2
3
3
3
9 years
0
0
2
2
10 years
0
0
1
2
10
27
37
42
*The total in 2009 represents all those who withdrew during the years
reported (i.e., each year includes those who withdrew in the previous
year(s).)
Eight of those who withdrew (20%) had been in the program for 4 years. Seven (18%) had
been in for 2 years.
Average (mean) tenure in program is 4.5 years.
Median tenure is 4.1 years.
Modal tenure is 3.75 years.
Years in Program for Those Who Left the Program
(cumulative, excluding graduates)
Less than 1 year
1 year
2 years
3 years
4 years
5 years
6 years
7 years
8 years
9 years
10 years
2006
2
3
1
0
1
1
0
0
1
0
0
9
2007
2
3
4
1
5
3
2
0
1
0
0
21
2008
2
3
4
2
6
3
2
2
1
1
0
26
2009**
2
3
4
2
7
4
2
3
1
1
0
29
*The total in 2009 represents all those who withdrew during the years
reported (i.e., each year includes those who withdrew in the previous
year(s).)
WolfBrown
Page 41
If You Are Walking Down the Right Path: An Evaluation of Community MusicWorks
Average (mean) tenure in program (excluding graduates) is 3.8 years.
Median tenure is 3.75 years.
Mode tenure is 3.75 years.
Gender Analysis-2005 Cohort
100%
90%
80%
35%
33%
46%
70%
60%
50%
Male
40%
30%
Female
65%
67%
Enrollees
Those Leaving-All
Reasons
54%
20%
10%
0%
Those Leaving-Loss of
Interest
REASON FOR LEAVING PROGRAM
2005 Cohort Reasons for Leaving Program
Graduation
Circumstances beyond child/family control
3URJUDPGLGQ·WKROGWKHP
Left to play another instrument
Not available
WolfBrown
As of
6/31/2006
1
4
2
1
2
10
As of
6/31/2007
6
6
9
2
4
27
As of
6/30/2008
11
7
13
2
4
37
As of
6/31/2009
13
10
13
2
4
42
Page 42
If You Are Walking Down the Right Path: An Evaluation of Community MusicWorks
Reasons for 2005 Cohort Leaving Program
(excluding graduation)
100%
90%
80%
70%
60%
NA
50%
Different instrument
40%
Lack of interest
30%
Circumstances
20%
10%
0%
As of
6/31/2006
As of
6/31/2007
As of
6/30/2008
As of
6/31/2009
Number of Years in Program at Enrollment in Phase II
16
14
14
# of students
12
10
8
6
6
4
7
7
5
2
2
0
0
1
2
1
Less than 1 to 1.5 2 to 2.5 3 to 3.5 4 years 5 years 6 years 7 years 8 years 9 years
1
years
years
years
Of the 45 students who participated in Phase II, 7 were in the program for a period of time
and then went back to Phase I. Of those 7, 5 were in the program for less than one year, 1
remained for 1.5 years, and another was in Phase II for almost 3 years.
Male students are more likely than female students to remain active in Phase II programs ²
while 71% of those who enroll are female, only 59% of those who are currently active are
female.
Most students who enroll in the Phase II program have participated for 3 to 3.5 years (14
out of 45, or 31%).
WolfBrown
Page 43
If You Are Walking Down the Right Path: An Evaluation of Community MusicWorks
MUSICIANSHIP
We defined musicianship as having four dimensions: practice, musical knowledge, involvement in
music, and quality of musical performance (both technical proficiency and expressiveness).
The data from the lesson-based interviews between students and teachers presents the following
SLFWXUHRIKRZ&0:VWXGHQWV·PXVLFLDQVKLSGHYHORSV7KHFKDUWSUHVHQWVPHDQVFRUHVIRUHDFKRI
the four dimensions by phase.19)
Musicianship Scores by Phase
Details of Scoring
3
Area
Mean Score
2.5
Practice
2
1.5
Knowledge
1
Musical
Involvement
0.5
Quality
0
Practice
Knowledge
Involvement
Quality
Phase
I
II
III
Mean
.74
.96
1.67
I
II
III
.68
1.24
2.11
I
II
III
.76
1.27
1.89
I
II
III
1.20
1.77
3.00
Area
Phase I (n=55)
Phase II (n=15)
Phase III (n=3)
7KHUHLVDGLVWLQFWEXWXQHYHQSURJUHVVLRQLQVWXGHQWV·XQGHUVWDQGLQJRIWKHUROHRISUDFWLFHIURP
Phase I to Phase III.20 Young people often reported: 1) that they felt they were not as good as they
should be;; 2) that they know practice improves their playing;; and 3) that currently they do not
practice enough to be getting better. However, as the graph shows, Phase II students do not progress
much beyond Phase I.
6WXGHQWV·OHYHOVRIPXVLFDONQRZOHGJHDQGPXVLFDOLQYROYHment develop steadily throughout the
phases of the program. By Phase II, students possessed fundamental knowledge of the mechanics of
their instrument and how it was played;; by Phase III, this knowledge had expanded to include an
understanding of musical concepts and terms (e.g., time signature, key signature, legato, etc.).
The depth of student involvement from Phase I to III underwent a similar steady progression, with
students able to focus on and identify specific exercises they had practiced by Phase II, and
becoming increasingly engrossed in practice and actively seeking out new and more challenging
pieces by Phase III.
Note that in the case of musical quality, if the student did not perform during their lesson interview they were omitted
from the analysis. This prevented artificial deflation of scores that would have resulted from assigning these students scores
of 0.
20 For musical involvement and knowledge, differences in mean scores among all three groups were significant at the level
of p < .06. In the cases of practice and quality, only the differences between Phase III and Phases II and I were significant.
Put alternatively, in terms of practice and quality of playing, Phase I and II students cannot be distinguished from one
another.
19
WolfBrown
Page 44
If You Are Walking Down the Right Path: An Evaluation of Community MusicWorks
Finally, the quality ratings showed uneven progression. If children played during their lesson, they
were rated as elementary, intermediate, or advanced, based on their technical expertise and the
expressiveness of their playing. The data show that on average, children in Phase I play slightly above
the elementary level, Phase II students are playing at slightly below the intermediate level, and Phase
III students are advanced.
The qualitative data provide deeper insight into how musicianship develops for young people
HQUROOHGLQ&0:)RULQVWDQFHDFORVHORRNDWVWXGHQWV·GLDU\GDWDVKRZVWKDWRQDYHUDJH
For Phase I students and less engaged Phase II students their lesson and practice are often
one in the same;; outside of lessons, practicing only peaks around Performance Parties.
Students begin intensive practicing outside of lesson practicing or inventing their own freetime musLFDODFWLYLWLHVHJJHWWLQJWRJHWKHUWRSOD\ZLWKIULHQGV´MDPPLQJµHWFLIWKH\
become very engaged in Phase II and proceed to becoming Phase III students. The diary
from one highly engaged Phase II player shows what this looks like:
Where
Activity
Home
Breakfast ² Ate a bowl of cereal
Got myself and brother ready for
church
Listened to music on my stereo
Church
(Bethel)
)ULHQG·V
house
Home
How Long?
How
regularly?
Family
About 30 min
Went to church
Service ² I played viola
11:00 ² 1:30
Mother had meeting, watched my
brother play outside
Listened to iPod
1:30 ² 2:30
Took my instrument, she plays
too, we played music, then
watched TV with her little sister
About 3:00
30 - 45 min
Got on Facebook with CMW and
others ² a friend from CMW got
me on last summer
Dinner ² We talked about the
concert on Saturday, I had a solo
on Saturday, so we were
reflecting on how people thought
I did
Got ready for school
Watched MTV
Listened to music on stereo
With
Family
Listen every
day when I get
ready
Family
Congregation
Twice a
month
Another
friend, a cellist,
was going to
come
Get together
to play about
once every
two months
About 30 ² 45
min
10 ² 15 min
Every day
Family
Whenever I
perform
About 30 min
About 30 min
Every day
When this kind of invented or independent musical activity occurs, it can become a major focus of a
VWXGHQW·VLGHQWLW\ZKDWGLVWLQJXLVKHVWKHP what JLYHVWKHPFRXUDJHDQGDVHQVHWKDWWKH\FDQ´JR
SODFHVµWKH\PD\QHYHUKDYHLPDJLQHGEHIRUH7KLVLVFOHDUO\IXHOIRUVWXGHQWV·VHQVHWKDWWKH\EHORQJ
to a long line of musicians, across times and continents:
Now I am working on a Vivaldi piece. When I began, I was worried I could not play it. Now
WolfBrown
Page 45
If You Are Walking Down the Right Path: An Evaluation of Community MusicWorks
I am just thrilled with it. When I play I make a little story in my mind callHG´&RQFHUWRµ7KH
story line starts fast. I imagine antique bows going furiously, I feel jumpy on account of it.
Vivaldi is listening;; I can picture him, watching all the players. He is jumpy too, to see all the
players there. I see the politics of the time;; a lot of the issues, the common and the rich people all
listening to the music. I try to play it as I enjoy it as if to make them enjoy it.
-- A graduating CMW student
This is exactly the kind of motivated and expansive musical activity that yields practicing which, in
turn, yields a sense of progress and excitement. Therefore, DQLPSRUWDQWTXHVWLRQLV´+RZFDQWKLV
kind of engagement occur for even more participants and at younger ages?µ 6WXGHQWV·GLDULHVDQG
interviews contain important hints about the kinds of events and circumstances that stimulate
musical activity outside of CMW lessons and performances:
When other youth (e. g., cousins or older role models) are playing beyond what is required.
For example, a number of young people spoke excitedly about the work that they put into
SUHSDULQJIRUWKH´$QWKHPµFRQFHUWZKHQVWXGHQWVRIPDQ\DJHVDQGDELOLWLHVDOO
FROODERUDWHGRQSURGXFLQJDQRULJLQDOSLHFHLQFHOHEUDWLRQRI3UHVLGHQW2EDPD·VHOHFWLRQ
:KHQWKHPXVLFEHFRPHV´DJLIWµWRVRPHRQHZKo matters, for instance, making a tape to
send to a relative who loves music or who got the student started playing.
At faith-based events where musical expression is highly valued as part of worship (e.g.,
instrumental playing as part of the meditation in a service, performing to support a choir, or
EHLQJSDUWRIDFRQJUHJDWLRQ·VFHOHEUDWLRQRI\RXWKRUVXPPHUUHWUHDW.
These are potentially important clues about how and when young people, even as beginners, might
be stimulated to see their music as important beyond the demands of lessons and performances.
%HORZLVRQHSDUHQW·VPDSRIWKHKLJKDQGORZSRLQWVLQKHUGDXJKWHU·VWKUHH-year career at CMW. It
provides a look at the kinds of experiences that can motivate young players (and their families) to
work hard at developing their musicianship:
WolfBrown
Page 46
If You Are Walking Down the Right Path: An Evaluation of Community MusicWorks
PERSONAL AGENCY
Personal agency combines elements of self-esteem and self-PRQLWRULQJDORQJZLWKWKH´JULWµDQd
strategies to pursue a goal. For example, a student may have the goal of playing a particularly
challenging new piece at the next performance party. %XWZLVKLQJLVQ·WSHUIRUPLQJ5HDFKLQJKHUJRDO
takes:
Self-esteem: If she judges herself to be a good-enough violinist, she will see herself as equal
to the challenges of the piece.
Belief tied to action (i.e., actually practicing) so that she can master the piece.
Insight: She has to be able to evaluate her own progress towards mastery, figuring out
exactly what and how to practice so that her hard work makes the difference she is hoping
for.
Personal agency matters. This kind of persistent and thoughtful effort, as much as raw talent, is what
makes for human achievement, in music and in many other fields.
By looking closely at our varied sources of data, we can see the pattern of development for agency in
the students at CMW. Below are the results for the lesson-based interviews:
Agency Scores by Phase
Details of Scoring
3
Area
Mean Score
2.5
Self-esteem
2
Selfmonitoring
1.5
Phase
I
II
III
Mean
.99
.93
1.83
I
II
III
1.12
1.52
2.22
1
0.5
0
Esteem
Monitoring
Area
Phase I (n=55)
Phase II (n=15)
Phase III (n=3)
Again, while there is an overall development, that growth occurs unevenly. Phase I students scored at
Level 1on self-HVWHHPLQGLFDWLQJWKDWRQDYHUDJHWKH\VDZWKHPVHOYHVDV´RND\µRU´SUHWW\JRRGµ
instrumentalists. This is realistic since students are still learning to play a difficult instrument.
Similarly, they have only a modest range of ways to monitor and improve their playing. Chiefly, if
they make a serious mistake or fumble, they begin again at the beginning of the piece.
WolfBrown
Page 47
If You Are Walking Down the Right Path: An Evaluation of Community MusicWorks
On average, however, Phase II students did not respond differently than Phase I students.21 In fact,
they scored slightly lower than Phase I students on self-esteem, and often accompanied this selfassessment with the observation that they were not as good as they should be, given the amount of
time they have been in the program:
,VKRXOGEHEHWWHUFRQVLGHULQJKRZORQJ,·YHEHHQSOD\LQJ
I am not progressing as much as I should be.
$VDJURXSWKH3KDVH,,VWXGHQWV·VHOI-monitoring skills were only slightly better than their younger
peers. Many of them describe their practicing in terms like these:
I figure I can stop practicing if I can get [from] the first note to the last.
Getting all the way through the piece without having to stop and start again.
They think of their progress in terms of:
«having longer pieces, with more and faster notes to play.
This changes among highly engaged Phase II students and Phase III students. At that point
individuals begin to think hard about what constitutes playing well:
Though I had played in an orchestra before, I got my voice musically [playing at CMW] maybe
you could say I got certain preferences about how I wanted to play. I really wanted to express
myself through the music not just out of the book.
-- A graduating CMW student
They also begin to think and act in ways that would get them to the goals they set. For example:
6KHUHDOO\ZDQWHGWRDXGLWLRQIRUWKLVRWKHURUFKHVWUD6KHVSHQW,GRQ·WNQRZKRZPDQ\ZHHNV
getting ready practicing a lot at home. She asked me if I would drive her when the day came.
All the way there I could tell she wanted it to go well. Driving home she was disappointed, she
GLGQ·WWKLQNVKHSOD\HGZHOOHQRXJK:HWDONHGDOOWKHZD\KRPHDERXWKRZPD\EHLWZDV
better than she thought, how she could always try again.
-- Grandfather of a CMW student
As with the earlier case of musicianship, these data raise the question about what could happen early
LQVWXGHQWV·FDUHHUVDW&0:WKDWZRXOGKHOSWKHPGHYHORSWKHLUSHUVRQDODJHQF\DVPXVLFLDQVHYHQ
as they strXJJOHZLWKPDVWHULQJDVWULQJHGLQVWUXPHQW7KHUHDUHLPSRUWDQWKLQWVLQIDPLOLHV·DQG
VWXGHQWV·LQWHUYLHZV
Around the esteem issues, a segment of both students and families described wanting a clearer sense
of progress, particularly in the stretch between the early burst of excitement of getting an instrument
and playing a first song and the regularly marked achievements of being a late Phase II or III player:
He was very excited when he started to play;; we all were. We had a celebration when he could
play his first real piece. But now it feels like things have leveled off. He has been playing the
Only the mean scores for Phase III students were significantly different (p < .05) than those for students in Phases I or II
in the areas of self-esteem and self-monitoring.
21
WolfBrown
Page 48
If You Are Walking Down the Right Path: An Evaluation of Community MusicWorks
VDPHSLHFHVIRUDORQJWLPH5LJKWQRZKHKDVRQHWKDWKHKDGODVW\HDU+HKDVQ·WJRWWHQD
chance to do anything but play in the big ensemble at performances,GRQ·WZDQWKLPWRORVH
LQWHUHVWRUJHWWRWKLQNLQJKH·VQRJRRG,NHHSWKLQNLQJZKDWDUHWKHVPDOOVWHSV,FRXOGSRLQW
WRRUZKDWDUHWKHQHZWKLQJVKHFRXOGEHGRLQJ",NQRZLW·VKDUGVORZZRUNEXWKHQHHGV
mile-markers, you know.
-- Parent of a CMW student
Families also want to know how they can do more than insist on a regular time for practicing. They
have a sixth sense that practicing and playing take explicit musical strategies. They want to be able to
´GLJLQWRµWKHPXVLF-making:
I love that she plays. But I am not a musician. When she gets into a hard place, all I can tell
her is to try again. Or to put it down and come back to it later when she is not so frustrated.
Lots of times I wish I had other suggestions.
I like listening when they play ² it makes me happy that they are doing this music and that
they are doing it together. But I never got to play when I was growing up. 6R,GRQ·WNQRZZKDW
WRVD\RUWRDVNZLWKWKHP,MXVWFDQVD\´1LFHµRU´<RXKDYHEHHQZRUNLQJVRKDUGµ%XW
nothing more. I have been thinking that maybe I need lessons in lessons.
² Parent of a CMW student
PARTICIPATING IN A WIDER WORLD
Throughout the evaluation, we thought about participation in two ways:
%XLOGLQJD&0:FRPPXQLW\RIVXSSRUWIRU\RXQJSHRSOH·s development that included both
families and mentors. One of the responsibilities of this community was to use music and
music-making as a way to change how young people thought about themselves and their
aspirations for the future.
Connecting young people to a network of wider musical experiences that would, in turn,
connect them to a world broader than their neighborhood and to a set of changed
expectations for their lives.
The lesson-EDVHGLQWHUYLHZVSURYLGHEDVLFLQIRUPDWLRQRQ&0:·VZRUNDWFRQVWUXFting a supportive
community that would help students venture forth into a wider world.
WolfBrown
Page 49
If You Are Walking Down the Right Path: An Evaluation of Community MusicWorks
Supports for Connecting to Wider World by Phase
3
Mean Score
2.5
2
1.5
1
Details of Scoring
Area
Family
Involvement
Phase
I
II
III
Mean
.63
.56
.67
Broader
Aspirations
I
II
III
.57
1.02
1.28
I
II
III
.91
1.20
1.67
Mentoring
0.5
0
Family Involmt.
Aspirations
Mentoring
Area
Phase I (n=55)
Phase II (n=15)
Phase III (n=3)
Students described their families as involved in supporting their music in a number of ways:
transporting them, making special arrangements at work to make time for lessons, sitting with them
to practice, and coming to events, even bringing members of the extended family. There were
LQVWDQFHVZKHQDSDUHQWZRXOGDFWLYHO\DLGLQWKHFKLOG·VSUDFWLFHRUDVLEOLQJZRXOGMRLQDFKLOGWR
play, but these were exceptions. As described earlier, many family members feel as though they do
not have the resources, in terms of time or musical knowledge of their own, to do more than to
PRQLWRUWKHLUFKLOG·VSUDFWLFH
Ratings of quality of the mentoring relationship between student and teacher were high overall and
displayed significant differences across phases.22 The average score of Phase I students, which was
slightly below 1, indicates that there was some basic rapport between teacher and student. By Phase
III, students were approaching a mean score of 2, indicating that there is a clear bond between
teacher and student. Not captured in this aggregate data are the occasions in which the student
clearly regarded the teacher as an advisor and confidant, for topics ranging far beyond music.
Students·VFRUHVIRUEURDGHUDVSLUDWLons were surprisingly low. Here a score of 1 indicates that a child
played to avoid embarrassment in a lesson or performance party or because they felt like playing.
6FRUHVRIDQGDERYHHJWR´JHWWKHPVRPHZKHUHµEHLWLn a community orchestra in the shortterm or a competitive college) were rare. This could be due to the way the questions were asked.
$OWHUQDWLYHO\WKHVHUHVXOWVPD\KDYHWRGRZLWKWKHIDFWWKDWWKLVTXHVWLRQWDSVDQ´XQPDSSHGµ
territory. As members of a first generation of musicians in their families, young people at CMW
VLPSO\PD\QRWKDYHDYHU\FOHDUPDSRIZKDWVSHFLILFRSSRUWXQLWLHVDUHRXWWKHUH´VRPHZKHUHµ
Possibly CMW teachers and mentors could begin to fill in this territory for young people by more
regularly and explicitly discussing and connecting students to experiences outside the neighborhood.
Helping students to build portfolios that track their accomplishments in and outside of
CMW.
Differences among means for mentoring were significant across all groups (p < .05). There were no significant
differences among groups for family involvement, and only the mean for Phase I was significantly different from Phases II
and III for broader aspirations.
22
WolfBrown
Page 50
If You Are Walking Down the Right Path: An Evaluation of Community MusicWorks
Creating events in which older students share their journeys into the wider world with
younger students.
Posting graphics of those journeys on the website, so that other families can see how it is
done.
A comment by an audience member at a performance party makes the potential power of filling in
this territory very clear:
Today at the concert I saw the girl who had been here for nine years, and she got this scholarship and she
is furthering her education, whether or not the scholarships comes from her being here alone or not. It
UHPLQGHGP\IULHQG·VGDXJKWer, the one I came to see. In nine years, she might be in the same spot,
JUDGXDWLQJZLWKDVFKRODUVKLS0D\EH\RXGRQ·WNQRZ7KDWZDVP\IDYRULWH6HHLQJVRPHRQHVWLFNZLWK
the program for nine years. It is awesome, after nine years, college is nothing.
-- Friend of a CMW family who came to a performance party to see a student perform
6WXGHQWV·GLDULHVDQGPDSVSURYLGHPRUHGHWDLOHGLQIRUPDWLRQDERXWKRZWKLVV\VWHPRIVXSSRUWV
allows students to connect to a wider world. First, it is clear that CMW is a community that acts as
both a haven and a springboard for students at all levels (see personal sketch on the following page.)
At the same time there is striking developmental evidence about the ways in which younger and less
engaged students, as compared to older students, use CMW to connect to a wider world (see
diagrams).
WolfBrown
Page 51
If You Are Walking Down the Right Path: An Evaluation of Community MusicWorks
Personal Sketch about the meaning of CMW
WolfBrown
Page 52
If You Are Walking Down the Right Path: An Evaluation of Community MusicWorks
Diagrams of Typical Maps for More and Less Engaged Students
6WXGHQWV·PDSVRIZKDW&0:KDVFRQQHFWHGWKHP to help to explain how having a springboard
and/or haven allows them to know about and explore the wider worlds that lie beyond their family
home, neighborhood and school. The illustrations below show maps typical of two groups of CMW
participants: 1) younger and less engaged students and 2) Older, more engaged students.
The Wider World:
$7\SLFDO<RXQJHURU/HVV(QJDJHG6WXGHQW·V0DS
Peers: Other players at CMW
Teacher(s) at
CMW (adults and
older students)
Places: Lessons,
performance parties,
home and family,
sometimes church
Events: Special CMW event(s) (performance party at
church, Anthem performance, DBR, memorable trip
or event (Bug Opera, opera movie)
The Wider World:
$7\SLFDO+LJKO\(QJDJHG6WXGHQW·V0DS
Peers: Other CMW players, in multiple roles:
(ensemble, mentor, model, friend, colleague);; other
players (church, RIPYO, other ensembles, etc.)
Places: CMW, concerts,
events, as well as additional
sites: church, other music
venues, summer music
programs, YUGA, etc.
Teachers at and outside
of CMW Lessons (e.g.,
mentors, visiting
musicians, summer
programs, other
ensembles like RIPYO)
Events: Special CMW event(s) (performance party at
church, Anthem performance, DBR, memorable
concert trips or events) as active participants and
contributors
WolfBrown
Page 53
If You Are Walking Down the Right Path: An Evaluation of Community MusicWorks
Younger and less engaged students portray CMW as the center of a set of radiating direct
connections to peers, teachers, performance venues, and pieces of music. These pictures illustrate
how their time at CMW is teaching that an enterprise like music is fundamentally social and how
others contribute to their emerging skills. At the same time, their maps contain no ideas or activities
beyond music. Nor do their maps branch to indicate that CMW connected them to something or
someone else, where, in turn, they went on to explore different relationships, activities, or ideas.
By comparison, older, more engaged CMW students make very different kinds of maps.
Characteristically, their maps branch, showing how CMW has placed them within a dynamic
network, where musical activities connect them to other art forms, political action, or thinking about
what happens for them after high school.
Looking at a specific example of data from a highly engaged student demonstrates what these
FRQQHFWLRQVFDQPHDQDWWKHOHYHORIDVLQJOHDGROHVFHQW·VOLIH+HUHLVWKHPDSRIH[WHnded
connections drawn by a highly engaged Phase II student. It includes a network of musical links as
well as ties to YUGA (Youth United for Global Action), an international social action group to
which peers at CMW introduced her.
Highly engaged Phase ,,VWXGHQW·VQHWZRUNRIFRQQHFWLRQV
WolfBrown
Page 54
If You Are Walking Down the Right Path: An Evaluation of Community MusicWorks
3KDVH,,VWXGHQW·VQHWZRUNRIFRQQHFWLRQVFRQW
WolfBrown
Page 55
If You Are Walking Down the Right Path: An Evaluation of Community MusicWorks
SECTION SIX:
RECOMMENDATIONS
Photo credit: Jori Ketten
WolfBrown
Page 56
If You Are Walking Down the Right Path: An Evaluation of Community MusicWorks
CMW affects the musicianship, agency, and connectedness of it students in many remarkable ways.
There are two core questions for CMW at this moment:
How can more students become engaged at the level where their musicianship, agency, and
connectedness are lastingly affected?
How can the organization do this demanding work and still allow its members to flourish as
teachers, performers, writers, and thinkers?
CHALLENGES FOR CMW
The data collected in this evaluation underscore several specific challenges:
Enrollment and Persistence: Such remarkable outcomes for young people depend on
students enrolling early, becoming highly engaged, and persisting in the program. But the
enrollment data point out that CMW students enroll as string players at relatively late ages,
with 30% of the cohort enrolling between the ages of 10 and 12. Moreover, during high
school students are more likely to drop out voluntarily.
Musicianship: Samples of students playing in lessons suggest that a number of CMW
students are technically young for the number of years that they have been playing (in
intonation, rhythm, fingering, etc.). This is most pronounced at Phase II. This raises the
question of how to create social settings or compelling motivations for these students to
help them invest the time and develop the strategies that would allow them to make
significant and motivating progress on their instruments.
Personal Agency: Students across phases have a limited sense of what they can do to make
DGLIIHUHQFHLQWKHLUSOD\LQJ7KH\GRQ·WNQRZKRZWRSUDFWLFHLQZD\VWKDWPDNHDGLIIHUHQFH
RUKRZWRWDONDERXWZKHUHWKH\DUHVWUXJJOLQJ7KHGDWDRQVWXGHQWV·LQYROYHPHQt and
investment in practicing and playing show a dip between Phase I and later Phase II and III
engagement. There is a time when students report losing interest or sense of progress. These
ILQGLQJVVXJJHVWWKDWSDUWLFXODUO\GXULQJWKH´ORQJKDXOµRIGHYHOoping the manual and
musical skills for playing well, students may need a more articulated sense of what they can
do to progress, as well as clearer markers of their development.
Participation in a Wider World: A challenge for CMW is how to translate the intimate
developmental network (i.e., students, families, and teachers) into a set of wider
opportunities for increasing numbers of students. Families describe how hard it is to do
more than lessons and practicing when their several children are also involved in sports,
debate, or step classes. Some Phase I and less engaged Phase II students describe additional
opportunities such as concerts trips like homework: something that has to be done.
WolfBrown
Page 57
If You Are Walking Down the Right Path: An Evaluation of Community MusicWorks
RECOMMENDATIONS FOR THE FURTHER DEVELOPMENT OF
CMW
ENROLLMENT AND PERSISTENCE
More than half (54%) of students in the 2005 cohort have persisted in the program ² a retention rate
that is remarkable for a classical music program in a city where economies frequently force families
to be highly mobile. Even so, there are steps CMW should consider taking:
Enrolling students at younger ages in order to give them the time to develop the skills
that will let them progress. This data points out that CMW students enroll as string
players at relatively late ages, with 30% of the cohort enrolling between the ages of 10
and 12.
Helping older students to persist even though there may be strong competition from
high school activities and other demands.
Collecting exit data when families and students leave the program.23
MUSICIANSHIP
CMW seeks to share a distinctive kind of musicianship. While its primary goal is not to prepare
professional musicians, the program does aim to give students the rewards of working hard on a
challenging skill, developing a level of personal expression, becoming a contributing member of a
community, and understanding that music can be a powerful force in the life of individuals and
communities. To make this possible for more students, CMW should consider:
Creating an articulated map of expectations, skills, and possibilities for each phase of
work at CMW that can be shared with faculty, fellows, families, and students.
Investing in workshops with skilled teachers of young string players for faculty, fellows,
and mentors, with an emphasis on strategies that work as early as Phase I.
Developing social forms of practice that support progress (e.g., times and places where
students can practice together, get help learning how to practice effectively, and learn
from older students and young adults working on pieces).
Continuing work with families so that they can support, enjoy, and extend practice
outside of lessons.
An effort to examine comparison data from other community-based music programs revealed that most institutions track
enrollment, but not persistence.
23
WolfBrown
Page 58
If You Are Walking Down the Right Path: An Evaluation of Community MusicWorks
PERSONAL AGENCY
This set of skills catches fire in highly motivated Phase II and III students. The challenge is how to
build these skills and habits earlier and more widely by:
Building a more active role for students in their own musical development (e.g., picking
next piece from a range of appropriate choices, announcing selections and what to listen
for at Performance Parties, making entries in their portfolios, etc.).
Continuing lesson-embedded interviews to support mutual reflection and goal-setting
between teachers and students.
Building a system of electronic portfolios that help students to chart progress (music
samples, interviews, short relevant samples from Performance Parties) and yield yearHQG´DOEXPVµ
Creating paid roles for Phase III students to assist in this work, learning relevant musical
and technical skills.
PARTICIPATION IN A WIDER WORLD
These capacities are a distinguishing feature of the programs at CMW, expanding the scope of
outcomes beyond what is sought by many other music education programs. Currently, a small group
of students realize these outcomes. The challenge for the organization is to begin to build these
capacities earlier, more explicitly, and in a wider number of young people. In combination with the
strategies outlined above, additional actions could include:
%XLOGLQJRQHYHQWVOLNHWKLV\HDU·VFRPSRVLWLRQDQGSHUIRUPDQFHRIDQoriginal piece in
KRQRURI3UHVLGHQW2EDPD·VHOHFWLRQ,WSURYLGHGPDQ\VWXGHQWVZLWKDFORVH-to-home
introduction to the wider world.
Expanding the range of places where CMW students can play through faith-based
organizations, schools, neighborhood festivals, etc.
Expanding partnerships with other organizations where older students can play (e.g.,
Philharmonic Music School, Rhode Island Youth Philharmonic Orchestra, Apple Hill,
Bay Chamber Concerts, etc.).
Seeking and more widely distributing opportunities for Phase II and III students to
present the work of the organization, through attending live events or through
producing web-based materials.
,QYHVWLQJLQZRUNVKRSVRUSDUWQHUVKLSVWKDWZLOODOORZVWXGHQWV·GRFXPHQWDU\ZRUNWR
mature and improve so that they can contribute reflections to the website, produce
materials for YouTube, etc., at an increasingly high level of quality.
WolfBrown
Page 59
If You Are Walking Down the Right Path: An Evaluation of Community MusicWorks
CREATING SUPPORT FOR THIS EXPANDED WORK
Taking on the work outlined above will require considerable organizational and community support.
Staff, Board members, and families will have to:
Build family-based musicianship: Many families want to be better able to support the
musical aspects of practicing (e.g., questions to ask, strategies, website or CMW CDs of what
the pieces sound like played well or in a variety of styles/approaches, etc.). CMW could work
with families to create community-EDVHGRSSRUWXQLWLHVIRU\RXQJSHRSOHV·SOD\LQJIDLWKbased services, family events, CDs to send to distant relatives, etc.).
Establish media partnerships for documenting development: CMW would benefit
from an alliance with a strong media partner that could help the organization design and
implement electronic musical portfolios for students, as well as creating internships for older
students. The purpose would be to record individual student development, support their
reflection, and allow them to share their musical work (playing, composing, writing liner
notes) with others.
Staff for a differentiated program: If these investments yield younger students enrolling
and higher numbers of students persisting into Phases II and III, the organization may well
need to think in terms of more explicit specialization in faculty and fellows, with some
individuals concentrating on early years and others focusing on more mature young
musicians.
Create ongoing data systems: As a part of annual intake and year-end interviews with
students and families, it is important to build stronger ongoing data collection systems that
regularly capture information (e.g., grades, course enrollments and electives, auditions and
additional musical activities, forms of personal or community service, etc.). Together with
persistence data, this would provide the foundation for ongoing reflection on the program
and for possible larger scale comparisons on a range of indicators (e.g., comparisons with
other neighborhood students who also attend Providence Public Schools.) This information
would be an ongoing resource for talking about the program with funders.
Collaborate and learn from a network of colleague organizations: In Providence, CMW
has strong organizational colleagues via the Providence Youth Arts Collaborative. There is
also a growing network of music-specific colleagues engaged in similar youth development
and community-building work, including several sister organizations that, like Community
0XVLF:RUNVZHUHIHDWXUHGE\&KDPEHU0XVLF$PHULFDDV´6HYHQ6WDUVµLQWKHDUWV
education field. Two of these programs were directly inspired by CMW. Regular exchanges
with these programs will be vital to both problem-solving and innovation.
WolfBrown
Page 60
If You Are Walking Down the Right Path: An Evaluation of Community MusicWorks
MOVING FORWARD: DEVELOPING ACTION STEPS
Acting on the above recommendations will take time, reflection, and resources ² even as CMW has
programs to run, new fellows to welcome, and other initiatives to undertake. With that in mind,
together the Board and staff might consider using the evaluation as a stimulus to discussion,
designing options, and making choices. Critical steps in such a process might include:
Hold discussions on key points designed to develop a prioritized list of which
recommendations are the most urgent and feasible in the 2009-2010 year and across the next
three years;; and
Ensure that the strategic plan for the organization aligns with for working toward the key
recommendations that are selected for pursuit.
Continue to share and substantiate these findings through regular evaluations that can index
progress in all the areas highlighted in this report.
Expand the indices of student success to include dimensions such as academic achievement,
community service, and enrollment in additional forms of music education.
Develop strategies to follow up with graduates of the program following their later
development as students, musicians and concerned citizens.
WolfBrown
Page 61
If You Are Walking Down the Right Path: An Evaluation of Community MusicWorks
CMW Phase III member Fidelia Vasquez trying
out Yo-<R0D·VFHOORGXULQJDYLVLWWRD6LON
Road Project rehearsal.
WolfBrown
Photo credit: Sebastian Ruth
Page 62