This is a pre-publication draft.  Do not cite or copy. You are not allowed to link to this site in any manner, nor pass this URL to anyone else without permission from me.

Last modified 30 September 1998 // © 1998 Carolyn Gale // carolyn@relax.com

    Completion data was based on inspection of assignments, using basic workshop criteria discussed in the Method section and summarized in Appendix B. Non-starters are defined as those who did not complete the first assignment. In this case, 37 did not start, or 15.1% of the participants. Figure 3 presents the number of participants who completed each assignment out of the 245 total and the completion percentage.  The completion rate results resemble an exponential decay model. However, a semi-log plot allows one to see the slope (amount of decay) as a straight line. Figure 4 graphs the percent complete from Table 7 on a semi-log plot, while Figure 5 graphs data of number of participants who completed.

Figure 3. Workshop Completion Rate for Required Assignments (n=245)

Figure 4. Percentage Complete of Required Assignments: Semi-log Plot (n=245)

Figure 5. Number of Participants Complete, Required Assignments: Semi-log Plot (n=245)

    Data for the optional assignments is not included in the following analysis. Two distinct slopes appear in Figures 4 and 5: the first slope, from Assignment 1.1 to Assignment 3, shows a relatively small decline in completion.   There was an average drop of 24 participants between each assignment, from Assignments 1.1, 1.2, and 2. If this trend had continued, completion rate by the last assignment would have been between 15 and 20 percent. However, the slope from Assignment 2 to Assignment 7.1 shows a much sharper drop. Calculating this second slope shows on average fewer participants dropping after Assignment 2.

Independent Variable (Survey) Results

Descriptive Statistics of Quantitative Data

    Of the 245 participants, 92 completed the survey, or 37.6%. Among the survey respondents, 77 said they did not complete all 7 parts of the workshop, while 15 indicated that they did. Most survey respondents started the workshop at the same time: 81 started the workshop on the first week, 8 started on the second week, 2 started on the third week, and 1 started after the third week. For 84 (92%) participants, this was their first time taking an online course or workshop.

    Based on this data, the majority of participants used computers at work with large amounts of memory and high connection speeds when "attending" the workshop. Most found the amount of content to be adequate and used the knowledge from the workshop at their work. However, the majority would not be as likely to complete the workshop if credit were offered.  Also, 2/3 of the participants gave two factors low ratings: "the time frame suggested for each part was good" and "I felt part of a supportive community in the workshop".  Only 1/3 of the participants rated these two factors as either good or excellent.

    To determine if any relationships exist between completing the workshop and other factors, a Chi-square test was applied to the quantitative survey. A Chi-square test was chosen because the dependent variable was dichotomous (the answer to "I completed the entire workshop"). The test was applied to each of the independent variables, which are the survey questions from Tables 8 and 9:

• Recommend workshop to a friend
• Online materials easy to understand
• Conferencing system easy to use
• Can now use software at level that meets my needs
• Feel confident that I can design online course
• Simple to access or log on to web server
• Online facilitator support was fast and efficient
• Time frame suggested for each part was good
• Felt part of a supportive community in workshop
• Number of hours spent working in each Part
• I have used the knowledge I gained from the workshop at my own institution
• I completed all assignments that I needed for what I wanted to do
• If credit was offered, would you be more likely to complete the entire workshop

Chi-square tests for independence indicate that none of the variables are statistically associated. There was limited data to find significant results, since only 15 of the 92 survey respondents indicated that they completed all seven parts of the workshop.

Qualitative Data (Survey Open-Ended Questions)

    The survey included nine open-ended questions, and comments for each question were organized by an affinity sort into categories. Categories were created if there were two or more comments of the same affinity. Remaining comments were filed under "General Comments", which are not listed below. The categories were then sorted by the number of comments found. The following is a summary and description of the major categories that responses for each question. Note that the number of comments for each question will not total the number of survey participants (n=92). This is because participants either replied with a "not applicable" answer, or replied with several different comments that needed to be extracted into different categories. The following results describe each open-ended question, a summary of the most important categories for the question, and a more detailed list of categories sorted by number of responses. Complete results are located in Appendix C.

Best Feature

    The following are categories of responses to the question, "What is the best feature of the workshop? Why?" The largest amount of comments (20) were found in the category of feedback of completed work. This is not the same as facilitator assistance on technology problems, which rated six comments. Participants enjoyed interacting with peers with similar interests, struggling with similar problems. The next best feature was Assignment 1.2 – reviewing and rating other online courses. Other categories are listed below.

1. Feedback of Personal Work (20 comments). This means receiving comments from other participants on posted assignments
2. Assignment 1.2: Case Studies of Other Online Courses (17 comments)
3. Organization and Structure (14 comments). Ease of use, clear directions.
4. Learn Content of Course (11 comments)
5. Asynchronous Medium (9 comments). The ability to access materials and conference anytime, from anyplace.
6. Facilitator Help (6 comments). This is the section on assistance with technical problems.
7. Support from Coworkers (2 comments). Having coworkers participate at the same time allowed for help with course motivation and mechanics.
8. Experiencing Course as a Student (2 comments).

Content Difficulty

    The following are responses to the question, "What difficulties, if any, did you have with the content of the workshop?" It is interesting to note that many of the responses did not directly answer the question of workshop content, with 17 responses addressing technical difficulties and time constraints. In general, participants felt that there was too much content for beginners, with not enough time to complete.

1. Technology Problems (10 comments). Generally difficulties publishing materials to workshop web server.
2. Novice/Expert (8 comments)
3. Novice (5) – too much content for beginners
4. Expert (3) – too little content for experts
5. Lack of Time, Not Content (7 comments)
6. Advanced Sections of Workshop (4 comments) – vague directions, didn't appear to be as well developed as prior sections
7. Too Much Content, Too Little Time (4 comments)
8. Printing Course Materials (2 comments)
9. HTML Assignment – Part 2 (2 comments)

    The following are responses to the question, "(What difficulties, if any, did you have with the mechanics (e.g. logging on, sending materials to the Web server) of the workshop?)" Similar to "Content Difficulty", problems with technology garnered the highest number of comments. Technology problems focused on publishing Assignments 2 and 3 (creating your personal and course web pages) to the workshop web server. Also, accessing the materials and web conference would get slow at certain hours of the day or through a home Internet connection.

1. Technology Problems (29 comments). This included using FTP, publishing materials to Web Server
2. Access to Technology (19 comments). Times when accessing materials was slow or difficult, especially when working at home.
3. Navigation between Course Content and Conferencing System (5 comments)
4. Unclear/Incorrect Directions (2 comments)

Other Topics

    The following are responses to the question: "What other topics would you have liked to see included?" The majority felt that the workshop focused too much on technical aspects of building an online course, and not enough in design (i.e. developing content, learning theories).

1. Pedagogy/Educational Issues/Content Development (18 comments)
2. Web Authoring Tools Other Than FrontPage (7 comments)
3. Interactive Features (6 comments). Examples include CGI scripting, JavaScript, Macromedia Shockwave.
4. Graphics (5 comments). Creating graphics, use of Adobe Photoshop.
5. No More Content (4 comments). Too much as it is
6. Novice/Expert (3 comments). Specifically, split course into two parts, one for novice and one for expert
7. More Time With Current Material (2 comments)

Reason For Not Completing

    The following are responses to the question: "If I did not complete the workshop, the reasons are…" The overwhelming comment was "I did not have enough time". Why did participants not have enough time? Answers ranged from added work responsibilities to too much content to master in suggested time frame. A distant second was problems with technology. This included publishing materials to the workshop web server and problems accessing materials (i.e. incorrect passwords). A strong third reason is that participants reasons' for taking the workshop did not match the stated objectives.

1. Time Frame (53 comments). Generally, too much content in too little time.
2. Technology Problems (15 comments). Publishing materials to web server, accessing course materials or conferencing system.
3. Did Not Match Content Prerequisites (10 comments)
4. Poor Course Design (9 comments)
5. Lack of Feedback or Instructor Support (4 comments)
6. Fell Behind, Couldn't Catch Up (2 comments)

What To Improve

    The following are responses to the question: "What would you improve about the workshop?" Issues with time was the top answer, with suggestions of increasing time to complete various sections and being more "realistic" with the amount of time needed for beginners (the workshop time frame suggested devoting 10 hours per week to each part). Other significant categories addressed the need for more instructors and feedback on work completed.

1. Time Issues (27 comments). Make more realistic; takes more time than stated
2. Change Structure (17 comments). Split into workshops appropriate for different skills levels; less content for novices; expand time in earlier sections
3. More Feedback, More Instructors (12 comments)
4. Decrease Number of Participants (8 comments)
5. More Examples and Case Studies (7 comments) . Similar to Assignment 1.2, reviewing and rating online courses
6. Improve User Interface (5 comments)
7. Technology Issues (5 comments). Improve Internet access, login process.
8. Use of Design Principles (5 comments). More discussions on instructional design and learning theories
9. Clarify Directions (4 comments)
10. More Rigid Deadlines (2 comments). Finding a way to curb procrastination

    The following are responses to the question: "What are your reasons for initially taking this workshop?" The largest category was to learn to create an online course, with the next largest dealing with other people influencing the participant to take the course.

1. Develop online course (26 comments)
2. Other motivation (9 comments). Supervisor required and/or paid for participant to take workshop.
3. Create a web page (7 comments)
4. Learn FrontPage (5 comments)
5. Wanted to experience online course as a student (3 comments)

Worst Feature of Workshop

    The following are responses to the question: "What was the worst feature of the workshop? Why?". Not having enough time to finish the amount of content was the top reason, with technology problems second.

1. Time Frame (15 comments)
2. Technology Problems (10 comments). Publishing materials to the workshop web server, difficulties installing software
3. Too Many of Participants (9 comments)
4. Organization of Materials (5 comments). Difficult to easily navigate between course materials and conferencing system; unclear directions
5. HTML Assignment – Part 2 (5 comments)
6. Too Much Content (5 comments)
7. Novice/Expert (4 comments). Too much content for novices
8. Lack of Interaction (4 comments). This includes interaction with other students as well as with course instructors.
9. Access to Materials (3 comments). Being forced to wait for the future parts to "open"
10. Instructor Issues (2 comments) – lack of feedback

Three Student Types: Combining completion rate and survey data

    The observed completion rate of survey participants (Figure 6) suggests three types of participants. Non-starters are those who did not post a message to the conferencing system (Assignment 1.1). Early drop students completed Assignments 1.1, 1.2 (rating other online courses), and 2 (create a personal web page using HTML), but not 3 (course web page using FrontPage). Completers were those who posted a message in at least one of the last two required assignments (6.2 or 7.1). Ideally, completers should be those who finished all required assignments. However, that criterion would result in very few people completing the workshop.

    The above rules were applied to those who completed the survey. Of the 92 survey participants, four would be classified as "completers", 24 as "early drops", and five as "non-starters". The analysis of these different populations comes from completion rate data and other survey results. An additional survey result used in this section answers the question: "If I did not complete the course, the last part I completed was:"

Early Drop

     Content-wise, the majority of the early drops (58%) felt that the amount was too much. No one stated that there was not enough material to cover. Similar to the non-starters, most early drops accessed the workshop from work (58%). Few actually used the workshop web server to publish web materials (8%).

Completers

    In the group of four completers, the most mentioned comment was of poor time structure. These students wanted to work ahead and access other portions of the course; when other parts could not be accessed until a future date, this would interfere with the students' schedule. Most of the completers worked from home (75%), all used the workshop web server to publish their web materials, and all would highly recommend the workshop to a friend (i.e. all gave the workshop the highest rating).

Return to "Thesis Cover Page"