Data Collection Protocol
CCZone Evaluation
September-December 2015
Materials in Education Office (pick up and return to the shelf by Robyn’s desk):









Bring a phone/watch!
Wear comfy shoes!
Copies of Observation + Interview Form (1 double-sided page per visitor group)
Copies of the Observation Quota Form (you only need 1 per shift)
1 Clipboard
Pens/pencils
Data Collection Protocol for CCZone Evaluation (this document)
Stack of index cards or other token to give visitors
Envelope for “Completed Forms”
Study description and Robyn’s business cards (a few on hand just in case)
Change Log (remains in box at Robyn’s desk)
1) Prior to your shift, review this document, the observation quota form, and the observation + interview form. If you
have any questions, email Lauren at [email protected] or call her cell: 303-704-6236.
2) At the beginning of your shift time, pick up your materials from the shelf by Robyn’s desk (in the Education office
space).
Make sure you have extra pens/pencils and anything you need for your shift, including a watch or cell phone you
can use to check the time.
3) Go to the Cleveland Creates Zone. There should not be a facilitator and/or volunteer working during your
observation. If a facilitator is scheduled at the same time as your observation, please make a note on the
observation sheet.
4) You will be standing in one of two places during data collection. You will intercept visitors and ask them to
participate as they enter the CCZone at the opening at the top of the stairs. Once a visitor agrees to participate,
you will observe from the triangular balcony that juts out above the light sculptures station. Feel free to bring a
stool to the balcony if you need to rest during your shift.
5) Begin with the appropriate papers (observation+ interview forms, the observation quota form, the completed forms
envelope, and index cards) on your clipboard with the observation forms on top, facing up, with interview questions
facing down and have a pen/pencil ready. You will need to quickly reference the observation quota form as your
group enters the CCZone. Keep this sheet somewhere you can quickly access; two possible locations are the last
paper on your clipboard or underneath the top observation sheet. See what works best for you.
Write your name, the date, and a visitor ID number (begin each shift with #1) on the top of the observation
sheet where indicated. If it is the beginning of your shift, do the same on the observation quota form, and mark the
time you are beginning your shift.
Please ensure your handwriting is legible!
6) We want a “relatively random” sample of visitors. When you are free (i.e. not observing or interviewing a group),
approach the first adult visitor you see entering the CCZone. This approach will help make sure we don’t observe and
interview the same “type” of people, even on accident (e.g., only those with happy kids, never those who look
annoyed, etc.). We are approaching adults (those who appear to you to be 18+) and will be observing and recording
responses from youth with adults, but not unaccompanied youth.
7) Approach the adult in the group and say:
“Hi! When you're done exploring this part of the Museum, I would love to talk with you and your
group about it for just a few minutes. Would that be ok?"
If they agree, hand them an index card (or another token they can carry while being observed) and continue with:
"Thanks; I’m giving you this card to hold onto while you’re in the space. When you are finished,
please bring it back to me. I will come meet you here when you are ready. Feel free to spend as long
or as short as you like."
If they do not agree, thank them and let them know you hope they have a great museum visit. Repeat until you find a
consenting adult.
8) Once you have consent from an adult, check your clock and note the “Time In” on your observation form, the
approximate number of people in the CCZone (i.e., how busy it is), and the number of people in the group you’re
observing. Write down the number of adults, number of youth, and list your best guess for the ages of the youth
in the group. Check the school group box if the visitors are attending with a school. You will only observe school
groups, not interview.
Next, choose one individual in the group to observe. The below box explains how to use the Observation Quota
Form to choose the individual to observe. You do not need to inform the group that you are only observing one
individual. The person you observe can be, but does not need to be the same person who took the card/agreed to
participate for the group.
How to select an individual to observe:
Use the table on the Observation Quota Form throughout your shift to determine which individual in
each group to observe. For each individual, check one box in the appropriate quadrant. (A small
example of this form’s structure is pictured at right).
For example, if the first person you observe is an adult female, check the box next to 1 in the adult
female quadrant. If the next person you observe is a young boy, check the box next to 1 in the youth
male quadrant. If you then observe an elderly woman, check the box next to 2 in the adult female
quadrant.
Try your best to keep each of the quadrants equal at any given point during your shift (e.g. 4 adult
males, 4 adult females, 4 youth males, and 4 youth females). This may be difficult, since groups
might not have individuals in the categories you need!
It is okay if these quadrants are a little uneven at the end of your shift, but if you have a choice of who
to observe in a group, pick an individual that best matches the quadrant with the lowest number
checked. This keeps us from accidentally viewing too many of the same type of person (e.g. mostly
moms, only children, etc.).
Because of the target audience for this project, choose to observe youth between 7-14 years old.
Please remember: Only observe youth accompanied by an adult.
Adults
18 yrs
& over
Youth
Ideally,
7-14 yrs
old
# of
MALES
☐1
☐2
☐3
☐4
1
☐2
☐3
☐4
# of
FEMALES
1
2
☐3
☐4
☐1
☐2
☐3
☐4
Fill out the “Individual Observed” box: check whether they are adult or youth, write their approximate age, and
circle their gender.
9) There are columns on the observation form for each of the CCZone components (i.e., rocket launchers,
parachutes, light sculptures, etc.) the observed person in the group visited. There is also a column marked
“OTHER___”. This space is intended for any new stations which are installed during data collection in the space
where the newspaper tube structures are currently located.
First, note the order in which they visited each component (e.g. if they went to the test track first, put 1 in that slot,
rockets second, put 2 there, etc.). If a visitor leaves the component, and then comes back, add another number in
this slot (e.g. If they went to the test track, then rockets, then test track again, the test track would have 1, 3 and the
rockets would be marked 2). If the individual did not visit a component, leave the entire column blank.
Definitions:
Test trial: going to test at race track, parachute fan/drop, or rocket launcher
Iteration: return to work station for refining, revising, adjusting (after a test trial)
Collaborative: providing tips, sharing materials, etc.
Competitive: using comparison with others as motive to improve project (e.g. race)
Solo project: works predominantly alone on project
Shared project: works predominantly with another collaborator as a team
Pride indicators: exclamations of joy (“Hooray!”), high-fives, smiles, etc.
Challenge indicators: taking a step back, “aww”, throwing away project, sigh, etc.
At each component, you will tally the number of test trials the observed individual conducted with their creation.
For Iteration, only mark one box; check if the visitor returned to the work station at least once to make adjustments
or if they did not iterate at that component. If there were other people in the space, indicate whether the project was
collaborative, competitive, or both. If the observed individual was alone in the space, leave the Social Interaction
segment blank. Use the Project Type segment to indicate whether the individual worked on a solo project, a shared
project, or both. If the observed individual did not work on a project, but spent time in the space, leave the Project
Type segment blank and explain what they did (e.g. watched a child work, talked on the phone, took pictures, etc.) in
the Notes segment.
In the segments labeled Pride Indicators and Challenge Indicators, write anything you see or hear that shows that
the individual is proud of their work or encountered difficulty. Use quotation marks to indicate direct quotes.
Leave the segment blank if you do not observe anything that fits into that category.
Use the Notes segment to record anything else you see and hear related to the project, making, and iteration
(persisting, trying again, and making adjustments when something doesn’t work the first time). Again, use quotation
marks to indicate direct quotes and ensure your handwriting is legible! The more information (including detailed
notes) that you are able to record on the observation sheets, the better. It can be tricky to watch closely from a
distance and record every detail. Remember, this is an evaluation about Making and the Engineering Design
Process—prioritize notes about that. It’s ok if you don’t get everything down; just do the best you can.
If a member of the group you are observing asks about the observation, feel free to show/tell them. It is not a secret!
Likewise, if anyone wants to learn more about the study, provide them with the study description paragraph and
Robyn’s business card.
10) When the observed family/group appears to disengage and begins to leave the CCZone, note the “Exit time”
and approximate number of people in the zone on your observation form.
If the individual was part of a SCHOOL GROUP, this completes the observation. School groups will not be
interviewed. Instead, return to the top of the stairs and return to step #5 to begin a new observation.
11) As soon as possible, return to the entry/exit of the space at the top of the stairs. Approach the visitor you gave
the index card at the start of their visit and use the opening line on the top of the interview sheet:
“I’d love to talk with you/your group about your experience in the CCZone.
[Then, if appropriate] Do I also have your permission to talk to those under 18 in your group?”
Feel free to let visitors know you expect it to take 5-7 minutes of their time.
***IF A VISITOR DECLINES THE INTERVIEW, DRAW AN “X” THROUGH THE INTERVIEW FORM*** This will let
Lauren know that they were observed, but not interviewed. If you know why they refused, it is helpful to note it on the
form (e.g. not a native English speaker, on the phone, etc.).
If the visitor refuses, return to step #5 above and repeat.
If the visitor accepts, continue as below.
12) Go through the interview form in the order it is written and ask the questions verbatim.
Please record all answers as clearly and completely as possible and using the visitors’ own words. Use quotation
marks to indicate direct quotes).Capture answers from all visitors in a family/group, and star (*) responses from
the individual you observed.
If the visitor has questions that you cannot answer, please refer them to a staff member/volunteer who can help,
provide them with a copy of the study description paragraph and Robyn’s business card.
13) When you complete the interview, thank them for their time and contributions. You may want to check in to see if
there is anything else they’d like to tell you, offer to help them find where they’d like to go next, etc. Being a great
ambassador for the museum is never a bad thing!
15) Go back over all your notes to ensure they are complete and legible before you move to your next
observation/interview. Take your time with this. Often you think of things you want to add and capture after you’ve
had a minute to decompress.
16) Put the completed form in the “Completed Forms” envelope.
When ready, return to step #5 above. Take breaks as needed! This gets tiring!
17) At the end of your shift, please return all materials to the shelf by Robyn’s desk. There will be a space for
“completed forms.” Feel free to leave sticky notes for the other data collectors based on what you learned or found
helpful during your shift. Also, please log any changes that occur in the space (e.g. new signage, new station,
painted walls, etc.) in the notebook. Please be as descriptive as possible.
THANK YOU FOR HELPING WITH THIS EVALUATION! If there are any urgent concerns—either with the
instruments or with a visitor complaint, please speak with Robyn or email or call Lauren
([email protected], 303-704-6236). Also feel free to email with any thoughts and feedback about the
process. Thanks!