Teaching Social Work Research Methods in the On-line Synchronous Classroom:
An example using Macromedia’s Breeze
A paper submitted to TCEA 2007
ANDREW S. QUINN
University of North Dakota
Department of Social Work
Grand Forks, ND 58202-7135
[email protected]
Teaching Social Work Research Methods in the On-line Synchronous Classroom:
An example using Macromedia’s Breeze
Higher education courses offer at a distance are traditionally delivered using
asynchronous technology. The delivery methods have included email, listservs, threaded
discussion groups, and web pages with links to readings, syllabus and readings. The
asynchronous delivery methods lack of face to face communication and have left students
feeling isolated and out of touch. Technology has become available that has permitted a
shift from asynchronous to synchronous, face to face, delivery.
One product called Breeze allows students and instructors to connect
synchronously using video and voice over ip (VOIP) to participate in educational
experiences. The way the software works is that the instructors and the students log into
a website simultaneously. All students and the instructor see the same screen. Figure 1
describes the layout of a Breeze Session.
Figure 1: A screen capture from a Breeze session.
There is the video pod where students and instructors can see each other (labeled
1). Next is the participant list which identifies the participants by name (labeled 2). This
particular pod has a feature that allows students to indicate responses via a thumb up
thumb down icon. For example, if an instructor wanted to gage the understanding of a
concept, the class could be asked to indicate their comprehension by displaying the
thumb up/thumb down icon. Third is a chat window (labeled 3). The chat window
allows for conversations to talk place publicly and privately away from the audio portion
of the course. The chat window can be used to ask questions, have side bar conversation
and even make humorous asides. The largest window is the sharing pod (labeled 4).
This pod will display documents whether they are websites, lecture notes, or PowerPoint
slides. In addition to the pods, Breeze has a set of buttons that, when pushed, a user can
be heard by all the participants. For example, if a student has a question, they can push
their talk button, the green microphone icon will light up and the student can begin to
talk. In addition to the talking, the instructor can give the student control to manipulate
the sharing pod or the whiteboard pod. In other words, students can resize the screen,
move from slide to slide (if the content in the share screen window is a PowerPoint) or
they can use the drawing features of the whiteboard to draw on the whiteboard. The one
drawback in giving students control is that when one student manipulates the feature,
such as changing a PowerPoint slide, the whole class sees the manipulation.
Systems Theory: A Conceptual Framework for Studying Synchronous Learning
When studying synchronous communication it is helpful to examine the delivery
as an interaction among systems. Dean, Biner, and Coenen (1995) identified five systems
or variables in distance education research: learner, instructor, presentation, content, and
outcome. The learner variable includes demographics, personality, general abilities,
motivation, prior knowledge, learning history and styles, past scholastic achievement,
performance skills, and the learner’s reaction to the course and presenter. The instructor
variable includes personality, subject-matter knowledge, extent of training/experience in
distance education and face-to-face presentation, goals and objectives, and teaching style.
The presentation variable includes time and spacing of materials, modalities and
technologies used, attention/entertainment characteristics of presentation, and the
environment in which the presentation was made. The content variable includes
discipline of course, learning objectives, type of material, and meta-disciplinary skills
used. The outcome variable includes final course grades, objective test results,
simulations, and interviews. An additional variable, the instructional environment was
added to this framework by Schoech (2000). The instructional environment variable
includes administrative support, organizational planning, technology infrastructure,
library support, and other supports and structures that make the course run smooth and
easy to deliver (Schoech, 2000).
Researching Synchronous Delivery: A pilot study
In Spring 2006, Breeze was used to deliver a graduate research methods course to
12 students spread around North and South Dakota. The class met over Breeze 14 times
with each session lasting 2 hours. A typical session consisted of discussion using video
and VIOP, a PowerPoint displaying notes on the week’s topic, polling questions, and the
sharing of complimentary documents. For example, if the topic of the session was on
conducting culturally competent research students saw a PowerPoint that discussed how
to write consent forms, provide culturally specific incentives, how to write surveys,
obtain endorsements from community leaders, and how to work with the community to
collect data. In addition, through the document sharing window, they saw a short clip
from the movie Kinsey that demonstrated how Kinsey taught his researchers to speak to
people of different economic classes. At the end of each session, polling questions were
used as a review of the content. For example, when discussing culturally competent
research one of the polling questions was ‘If you were to conduct research on tribal land,
would you employ a Native American to assist you with conducting research’. Students
were asked to respond yes or no. Finally, at any time, students were able to ask questions
via chat or the VOIP features. In fact, the instructor often verbally posed questions for
the students to consider and verbally respond to.
For the purpose of study, the general research question was “was Breeze an
appropriate method for delivering Social Work Research Methods?”. The interactions
between the systems proposed by Dean et al (1995) and Schoech (2000) allows for
investigation into the success and or failures of the research course. Table 1 describes the
systems and the variables investigated.
Table 1: The System and the respective variables examined.
Learner
• Satisfaction with teaching style
•
Satisfaction with the use the audio
(VOIP) and the video within Breeze
Instructor
• Teaching style
• Knowledge about the use of Breeze
Presentation
• The use of PowerPoint
• The use of polling
• The use of chat
• The use of document sharing
• The use of VOIP to deliver content
Content
• The research methods course
Outcome
• Course grades
• Pre-post test knowledge
Instructional environment
• University and departmental
support of the use of Breeze
Data Collection and Analysis
The outcome variable of knowledge, which was measured quantitatively, was defined by
final course grades and a change score produced by subtracting a comprehensive final
test score to a pretest. Data for the remainder of the system variables were collected using
open ended questions designed to solicit responses about the students experience and
satisfaction with the course, the audio and video capabilities of Breeze, and the other
features of Breeze. The other system variables were measured qualitatively. To facilitate
the discussion several open ended questions were proposed to the students. Table 2 lists
the open ended questions.
Table 2: Questions designed to elicit student responses
What aspects of breeze, as it relates to my teaching mechanism, are you satisfied with?
What aspects of breeze, as it relates to my teaching mechanisms have been frustrating?
In relation to how I use breeze, what would you like to see more?
In relation to how I use breeze, what would you like to see less of?
Do you benefit from seeing your classmates? Explain your answer.
Do you benefit from hearing your classmates? Explain your answer.
Does seeing your classmates increase your confidence in learning the materials? Explain
your answer
Does hearing your classmate increase your confidence in learning the materials? Explain
your answer.
What do you think having the ability to see the instructor adds to the learning experience?
Results
Quantitative Findings: One interesting way to address outcomes, especially with newer
technologies, is to examine the academic performance of the students. To accomplish
this, a pretest that covered the range of the materials discussed in the course was
administered on the first day of class. The pretest score was compared to a
comprehensive final score. The comparison indicated that there was a significant change
between the mean scores on the pretest (m=82.1) and the post test (m=92.9), t (9) =-3.91,
p=.004. The magnitude of change was also examined. The effect size for the comparison
was calculated as 2.45. An effect size of 2.45 is considered strong and significant
(Rubin, 2007). The findings of the significant t and the strong effect size indicate that
students were able to acquire the necessary knowledge when research methods were
taught using Breeze. As for the final grades, the majority of the students made A’s (n=11)
with one B. Interestingly, the one student who received a B reported the most technical
difficulties with using the video and VOIP features of Breeze.
Qualitative Findings: Nine questions (Table 2) were developed to assist with
understanding the effect that teaching through Breeze had on the educational process.
These questions drew on the various system variables described by Dean et al (1995) and
Schoech (2000). The first set of questions combined learner and instructor variables by
asking students to comment on how the instructor’s teaching technique influenced their
satisfaction and dissatisfaction with how Breeze was being used. Overall, students found
that the class was interactive and approximated the real classroom. For example, one
student stated “Breeze as a classroom is a box. Your classes are as interactive as a normal
classroom would permit. The only difference being that we are sitting at a computer
terminal”.
Some of the responses indicated that there were that there were issues with the
presentation variables of VOIP and the video as there were reports of technical difficulty
related to the video and audio delivery. For example, several students reported glitches
with the audio cutting in and out. One interesting theme that did emerge surrounding the
presentation variables was related to the dissatisfaction with Breeze in the edict involved
in talking in real time. For example one student wrote “I think just knowing when to
answer a question” while another one stated that talking over each other was an issue.
The next set of questions focused on the importance of using the VOIP and video
features of breeze. Students were asked for their opinions about seeing and hearing their
peers. As far as seeing each other, students believed that the visual experience made the
learning experience more engaging and real. One student even stated “I seem to forget
that I am not in the same room with my classmates.” Students also felt that hearing each
other enhanced the learning process and allowed for a collaborative learning experience.
For example, one comment stated “Everyone has such different viewpoints and ideas that
it is nice to be able to look at things from a different perspective”.
Students were also surveyed about the relationship between the learning
experience and seeing the instructor. Similar to the other themes, seeing the instructor
made the course more real, added structure, and gave the students a connection. One
student reported seeing the instructor “assists with the understanding of personality
(sarcasm, sense of humor, etc.) which helps understanding in communication, and that
helps to communicate the lesson”.
In addition the open ended questions the instructor was able to offer some self
reflection on the instructor variable and the instructional environment variable. Prior to
teaching in Breeze environment, the instructor had a strong background in educational
technology. In addition to a strong technology background, the instructor was not afraid
to stretch the use of Breeze. For example, the instructor was able to incorporate many of
the features of Breeze, such as the polling, the screen sharing, and the chat, into his class
session.
The Breeze course described here also benefited from a strong instructional
environment. First, the university provided in-dept training on Breeze. Second, a
representative from continuing education (where the Breeze server was housed) sat in on
classes periodically through the semester to observe and note any problems. Third, the
social work department (the department offering the course) hired a graduate student to
provide Breeze support for the semester.
Discussion
The success of an on-line educational experience is dependent on the successful
interaction of several systems. Based on the interpretation of the qualitative and
quantitative data for the semester long research method course this appears to be the case.
Breeze seemed to be a successful mechanism for delivering a social work research
methods course. First, there appeared to be a gain in knowledge about the course content.
This finding is supported by the significant t-test and the strong effect size. Second,
students seemed, aside from some technical difficulties, enjoyed the delivery of research
methods over breeze.
One possible reason for the success was the solid support from the department
and the university. For example, often times, the graduate teaching assistant was log on
along with the person who provided university support. In addition, the instructor
offering the course had a strong background in technology and was not afraid to test the
limits and try all the features in Breeze. In fact, one test of the usability of Breeze came
during the semester when the instructor was on vacation in California and still conducted
class. A second possible reason was that the content of methods might have lent itself to
this type of delivery system. Research methods involve many concepts and ideas that
allow for a lecture format. There is also not much content that lends itself to hands on
experience. For example, while there might be group work assigned, group work can
occur outside the class. Whereas a course such as practice uses group work to simulate
different practice concepts, and this simulation is more than likely to occur in class. A
third reason for success was the ability to replicate the face to face experience through the
synchronous technology. Students reported hearing and seeing the instructors and their
peers were beneficial. In addition students felt like the class approximated the traditional
classroom environment.
There were, however, some minor reports of technical difficulties. These
difficulties occurred with the presentation variables surrounding the VOIP and video.
There were a few times that student’s videos would be dropped and the student would
have to log back into the session. Also, sometimes the VOIP was choppy. The one
student who claimed the most difficulty was often on a different machine each week. In
addition, she claimed her technical skills were minimal and often had difficulty
configuring the webcam and microphone. Once she was able to log in consistently from
the same computer, most of her technical difficulties dissipated.
Conclusion
The pilot findings do need to be taken with some caution. First, this was an
isolated incident in a small social work program. Second, the small sample size can lead
to generalizability issues. Finally, the instructor had a high comfort level with technology
which allowed for the course to go smoother than someone else teaching the course with
a lower comfort level. However, based on a brief pilot, synchronous delivery shows
promise.
References
Dean, R., Biner, P, & Coenen, M. (1995). Distance education effectiveness. A systems
approach to assessing the effectiveness of distance education. Ed Journal, 9(4),
J17-J20.
Schoech, D. (2000). Teaching over the Internet: Results of one doctoral course. Research
on Social Work Practice, 10(4), 467-486.