Illustrating the Potential of an Online Workshop through a Case Study Example
by Judith Zorfass, Arlene Remz, and Denise Ethier
Educational reform initiatives seeking to promote change through professional development, technical assistance, and dissemination have begun exploring the potential of telecommunications as a primary vehicle. As the Internet is becoming more ubiquitous in and outside of schools, those charged with reform efforts are examining the role telecommunications plays in:
Moreover, educators are looking to others who have already made a foray into the world of online professional development, hoping to gather knowledge and guidance from those with grounded experience.
Since its inception in 1992, The National Center to Improve Practice (NCIP) has relied heavily on telecommunications to assess needs, provide a range of professional development opportunities, and gather evaluation data. NCIP's particular focus is on improving the use of technology for students in pre-K to grade 12 who have sensory, physical, cognitive, and social/emotional disabilities.
NCIP was drawn to telecommunications because of its power to create a national (and international) community of learners that shares a common need to know about special education and technology. In 1992, when we first embarked on the telecommunications track, we designed a bulletin board service (NCIPnet) using FirstClass communications software. From the outset, we surmised that our online forums would give members of the NCIP community a venue for expressing opinions, sharing information, and collaborating to solve problems. This is exactly what happened. In addition, an analysis of discussion threads allowed us to assess the needs of the field. Embedded in conversations were compelling calls for assistance and pleas for information around specific topics, such as, hardware, software, curriculum, standards, diagnosis, policy, parent involvement, and much more.
In 1996 we launched our Web site because we recognized that the rapidly expanding capabilities of the World Wide Web would allow us to be even more responsive to the needs of the field. Using newly available conferencing software, we designed a range of online professional development opportunities that varied in format and content. With every new offering, we included an evaluation component that combined online strategies with more traditional offline strategies. We hoped that this multi-faceted approach would make our findings richer and more valuable, not only to us, but also to others engaged in school reform.
This paper presents a case study of an online workshop we conducted entitled, Ready SETT Go!. We describe how we identified the need for a workshop; designed and conducted it; and carried out a formative evaluation of the workshop's effectiveness and impact. Our emerging findings complement and extend other articles about teaching online (Harasim, 1990; Harasim, Hiltz, Teles, and Turoff, 1996); the designs of netcourses and netseminars (Tinker and Haavind, 1997); reflective conversation in online professional development (Spitzer, Wedding, and DiMauro, (1995); and teaching in a virtual classroom (Hiltz, 1995). This expanding knowledge base can guide others who want to make telecommunications a mainstay in their work.
Identifying the Need
Most school districts have only one person or just a small staff with expertise in assistive technology. These specialists tend to feel isolated and, as a result, are eager learners and willing collaborators. They were frequent contributors to NCIP's online discussion forums, often posing variations on a fundamental question: How do you match appropriate systems of technology tools and supports with the needs of a particular student? They revealed their desire to expand their knowledge about assistive technology
Fortuitously, at the same time that NCIP was identifying this specific need, we were also becoming more familiar with the work of Joy Zabala, an educational specialist at the Region IV Educational Service Center in Houston, Texas. We came to know Joy at first through her frequent contributions to NCIPnet, and then by attending her conference presentations, and reviewing her writings. Based on her extensive background in assistive technology, Joy had developed a framework, which she called SETT, for selecting assistive technology for students with disabilities. Each letter in SETT stands for a different element to consider when selecting assistive technology: S=student, E=environments, T= tasks, and T=tools. As a presenter for local education agency workshops and regional and national conferences, Joy had created an outstanding hands-on training session that left participants with a strong working knowledge of the SETT framework. It became apparent to us that the goals of her workshops mirrored the identified needs of our online community.
We wanted to create a formal online workshop about SETT that went beyond our past informal online discussions. When Joy reacted positively to the idea, we decided to proceed collaboratively to design an online workshop for an audience of teachers, specialists, and trainers. These participants would accomplish the following goals:
We sought to preserve the elements that made Joy's face-to-face workshops such a success. Our challenge was to take full advantage of the interactive capabilities of the Web as we designed the course.
Design and Conduct the Online Course
Joy brought to the collaboration the extensive set of materials she had developed over the years, as well as her expertise as a presenter. NCIP contributed its growing expertise in how to design and structure online conversation, as well as a desire to experiment with HyperNews., a new conferencing software. This software would enable us to
Together with Joy, we defined the roles and responsibilities of the online facilitator. Referring to the categories developed by Berge (1996), and reviewing what we had learned in other NCIP offerings, we generated a set of strategies that Joy could use to
In addition to strong content knowledge, Joy brought to her role as facilitator a natural inclination to acknowledge people, a warm sense of humor, passion about her work, and the ability to write clear and articulate messages. One participant later commented on Joy's skills:
Joy was so positive in her feedback and she made me feel very welcome. She also made us into a group and pulled us all together.
Recruiting, Selecting, and Preparing the Audience
We paid considerable attention to recruiting, selecting, and preparing an audience of assistive technology specialists, clinicians, and professional development providers. Our recruitment strategy involved sending announcements to listservs and to the NCIP mailing list, distributing flyers at conferences, and posting notices on NCIP's Web site. In order to attract people who were motivated to actively participate in online discussions, we asked potential participants to fill out an application form prior to registering. Among other questions, they had to explain their professional roles and specify their reasons for wanting to take the workshop.
The response was strong. From the 196 inquiries we received, we selected 21 participants (mostly specialists) for a pilot workshop and an additional 32 people (mostly trainers and education faculty) for the second workshop to be held several months later. These participants met our selection criteria: they had access to and felt comfortable with telecommunications, were willing to learn how to use the conferencing software, and agreed to spend two hours a week reading and contributing to the conversation.
We prepared the audience for the workshop in two ways. First, we requested that they review a set of resources to develop background knowledge on the SETT framework. These background resources, which were modeled after the overheads, handouts, and verbal explanations that Joy typically uses, were incorporated into a series of hyperlinked Web pages. We also mailed printed versions of key resources to all participants.
Second, we provided technical assistance. In addition to the User's Guide, which was available on the Web and was mailed in printed version as well, we created an online discussion conference with technical tips, directions, and a place to ask questions. We also invited participants to contact the NCIP telecommunications coordinator, Denise Ethier, if they needed additional help. Denise used both email and the telephone to guide people through the logistics of the conferencing software and the navigation of the Web site.
During the first week of the workshop, we asked participants to introduce themselves within a special conference called Virtual Introductions. Here they shared information about their professional and personal lives. For example, a speech pathologist from rural Ohio wrote that many of her students were Amish and lived without electricity. She commented: "It makes forays into the world of technology interesting!" Another participant who is an occupational therapist described living in an Australian village in the rainforest near the Great Barrier Reef.
These introductions not only generated a sense of intimacy among people who could not see each other, they also created a safe environment in which people could feel comfortable expressing their ideas and opinions. For example, after the workshop ended, a participant commented:
It was interesting to see how sometimes we were all together on something and sometimes we went off in different directions. The group was wonderfully comprised-lots of different perspective. . . . I felt like I knew Donna (even though I don't) because sometimes she expressed what I was thinking.
Using a Case Study to Frame the Discussion
The essence of SETT is to have key stakeholders(the classroom teacher, assistive technology or other technology specialist, special education teachers, parents, and the student(jointly assess the student's needs, the learning and home environments, and the tasks in which the student is required to engage. By analyzing this information, they collaboratively recommend an appropriate system of technology tools and support. Just as Joy does in her other workshops, we used a case study about Josh to ground the discussion.
We used the following three design principles to simulate this collaborative process.
First, to give participants the feeling that they were attending a team meeting to discuss Josh, we structured the workshop as a Virtual Team Meeting.
Second, to ensure that the conversation sequentially focused on the student, environment, tasks, and tools, we structured the conversation so that people would discuss the student in week 2, the environment in week 3; and the tasks near the end of week 3. Then, during week 4, as a culmination of the previous discussions, participants made recommendations about tools.
Third, to help participants know where they have been, where they are currently, and where they might want to go next, we used a weaving strategy that involved summarizing and synthesizing (Kimball, 1995). Every few days, we distilled key ideas from their messages and listed these "nuggets" within a four-column chart (one each for student, environment, task, and tools). This is similar to what Joy might do in a face-to-face workshop using chart paper and markers. Figure 1 (link to Figure 1) shows the online SETT chart with the distilled ideas. HyperNews allowed us to link each nugget back to the original message so that participants could access the complete text.
In order to strengthen and improve our current and future work, we engaged in ongoing evaluation to answer the following questions about participation, impact, and how to improve the workshop:
To gather data to answer these questions, we employed a variety of online and offline strategies. For example, to study implementation, we asked participants to reflect on their participation, while we gathered quantitative data (e.g., how many messages were posted in total, by week, and by participant).
To determine the impact on the individual, we gathered data at three points. At the outset of the workshop, we established a baseline by asking a series of questions about reasons for taking the course. When the workshop ended, we asked, "In what ways will the information you gained during this workshop be useful? Four months later, we asked participants to revisit this question and explain how they would modify their previous answer.
To gather suggestions for strengthening the workshop, we created a discussion area on the Web called Wrapping Up. Also, since several participants attended the same national conference, we organized an informal focus group to assess the strengths and weaknesses of the workshop.
Findings about participation were highly consistent across the two workshops. In analyzing the readership of background resources, workshop materials, and posted messages, we found that most people read more than half or all of the materials. When we analyzed the number of contributions and the number of different people making contributions across the five weeks of each workshop, we found that while the second week and the last week were the most active, the second-to-last week was the least active.
We found that certain elements of our design contributed to active participation. In particular, the Virtual Team Meeting was an especially powerful construct. Participants reported becoming highly invested in Josh's welfare; and thus, the task of assessing his needs became a compelling goal. Joy's strategy of assuming the "voices" of Josh and his parents served to heighten the feeling that Josh was a real child with real needs. Obviously, she was so convincing that even after we explained that Joy was the voice behind the characters, people still responded as if they believed that Josh existed and was participating in the workshop.
Most participants found that the SETT summary chart (see Figure 1 (link to Figure 1)) was a potent tool for organizing, synthesizing, and referencing the discussion. Just like writing on chart paper or a white board, it served to capture key ideas and focus thinking. One participant noted, "I found it helpful to print out and refer to the chart as I formulated responses."
Our criteria for impact were closely related to the goals of the online workshop. We wanted participants to develop knowledge and skills related to the SETT framework, understand how the framework could be utilized within a collaborative process, and as a result, change their practice by using and/or sharing what they had learned with others.
Almost all participants reported expanding their knowledge base and sharpening their skills. Some indicated that they developed new ways of thinking about the overall process of assessing assistive technology needs. For others, it was a much-needed confirmation of what they had already been thinking and/or doing. For example, one participant found that:
[The workshop] helped to clarify and verify what I have been doing and to realize that we are on the right track in evaluations and ongoing service provision.
SETT helped me see the details so much more clearly. It's something I have yearned for a long time without knowing it.
In terms of valuing collaboration to assess a student's needs, many participants cited the benefits of hearing the diverse perspectives of people with different roles and responsibilities. For example:
I always learn by hearing others' opinions. It is sometimes surprising that what I take for granted is not so by others [in AT (assistive technology) assessment]. It reminded me to value each and every option and to listen to what and why it was said.
[I] gained ideas and perspectives from people I would not be able to work with normally.
As intended, participants articulated their plans for implementing SETT in their workplace and/or sharing what they had learned with colleagues. In a follow-up questionnaire distributed four months after the completion of the workshop, several participants confirmed that they had followed through with those plans. For example, one participant, a coordinator of a regional assistive technology resource center who had purposely taken the workshop to share information with others, indicated that she had conducted a team workshop based on SETT and that the framework was now being used by others in her region. Another participant, who worked as an occupational therapist, stated that she had taken the workshop to improve her decision-making with students, families, and schools. Four months later, she recounted that she had improved her practice, and that her students were the beneficiaries. She described one instance in which she was able to convince a school to grant a student credit for learning a new technology.
Strengthening the Course
The formative evaluation of Ready SETT, Go! has shown us several ways in which this online workshop, as well as other similar online professional development opportunities, can be strengthened. In fact, much of what we learned during the pilot workshop was immediately translated into practice during the second session. The areas that required strengthening fell into three categories: being clearer about expectations and directions; slowing down the pace of the workshop; and utilizing a more effective range of strategies to foster active engagement and participation.
Expectations and Directions
Even when we thought we were being as explicit as possible about our expectations and directions, we found that we were still not clear enough. In a face-to-face workshop, an instructor relies on cues such as nods, puzzled looks, body language, or raised hands to assess if directions have been understood. In contrast, in an online environment, the facilitator has to substitute accuracy and appropriateness of delayed written messages for the immediacy of body language to know if everyone is on the right track. We found in analyzing messages that some people did not understand what they should be doing and/or where they should be posting.
After identifying this problem in the pilot, we tried to rectify it in the subsequent workshop. For example, we included a dedicated place for the instructor to post directions and explanations called Team Meeting: Directions and Reflections. With NCIP's help, Joy posted very specific directions on a regular basis in this conference. Participants were directed to check that conference for any new directions or updates before proceeding to the assigned conferences. We found that this strategy eased confusion.
We found that the pacing of a short workshop (one which lasts only four to five weeks) was critical. It seemed that soon after start up, it was time to wind down. One finding was clear: we did not allow enough time to pull together loose ends and in a way that gave participants a sense of completion. In the future, we will consider how to use the time before the workshop more effectively and how to ensure that we put closure on the conversation.
We found that in a workshop involving 20-25 participants, we needed to employ a range of strategies to ensure active engagement and participation. In the future we will:
0n September 22, 1997, the U.S. Department of Education sponsored a summit of myriad projects aimed at promoting and supporting school reform. The defining question of the conversation was: How can these projects make connections, learn from each other, and coordinate efforts? NCIP was present and heard a pervasive theme about the desire to share information about the design, value, and effectiveness of telecommunications, in general, and online workshops, in particular. With its first-hand experience, NCIP joins ranks with the growing number of others who are excited about breaking new ground in utilizing telecommunications. Articles such as this one, where we present a specific case in point, can offer the field solid examples of what works and lessons learned. Aggregating our data across projects, sharing insights, and formulating recommendations can move the entire field ahead. We hope this paper stimulates continued dialogue.
We want to acknowledge the contributions of our NCIP colleagues: Patricia Corley, Jennifer Gold, and Bonnie Johnson. They helped design NCIP's innovative online courses. They also offered valuable comments on successive drafts of this paper.
The authors present a case study of an online workshop designed for those involved in the selection of assistive technology for students with disabilities and conducted on the Web.
Judith Zorfass (JudyZ@edc.org) is Principal Investigator, Arlene Remz (ArleneR@edc.org) is Associate Project Director, and Denise Ethier (DeniseE@edc.org) is Telecommunications Associate for The National Center to Improve Practice located at Education Development Center, Inc., Newton Massachusetts.
Copyright © 1998 by Judith Zorfass, Arlene Remz, and Denise Ethier. All Rights Reserved.