Some Subject, Treatment, and Data Collection Trends in Current CALL Research
In recent years, the number of research studies in CALL has grown
significantly. However, some elements of CALL research methodology remain
problematic. Whether looking at tool uses, such as CMC, or reviewing specific
CALL environments or tutorial applications, a high percentage of CALL research
appears to have the following characteristics:
1) Research subjects, whether students or teachers, are novices to CALL
2) They are also novices to the task or application under study
3) These novices are untrained before the study and undirected during the study
4) They are often studied exclusively during their initial experience
5) The studies may be very short, representing a single event, such as a class or lab session
6) Surveys and questionnaires are used in place of more objective measures, such as tracking or testing
The preceding points, if valid, suggest a disturbing conclusion for CALL research, namely, that much, possibly most, of what we know about CALL comes from studies involving the attitudes and impressions of undirected CALL neophytes in their first encounters with a given task or application. To test this impression, a corpus of over 90 research articles was reviewed. The articles chosen for the corpus were those that had an identifiable research question and involved human subjects interacting with or through a CALL application. To assure that this corpus represented a credible and current sample of the state of the field, it was composed of all such research articles taken from the most recent two or three volumes of four leading refereed CALL journals: CALICO Journal, volumes 18-20 (9 issues); ReCALL, volumes 13-15 (6 issues); Language Learning and Technology, volumes 4-7 (9 issues); and CALL, volumes 15-16 (10 issues). The presentation will provide detailed results regarding the preceding claims. It will conclude with a discussion of some exemplary studies from the corpus and suggestions for considerations researchers should make in research design and data collection. Most notable among these is that while studies of novices still have a place, we also need studies showing what happens after students are given both technical and pedagogical training and have familiarized themselves with the CALL applications being researched.
I. A lot of CALL research seems to have the following characteristics
1) Research subjects are novices to CALL
2) They are novices to the task/application under study
3) They are untrained before the study
4) They are undirected during the study
5) They are studied during their initial experience
6) They often only use the application for a short time
7) There are a small number of subjects
8) Surveys/questionnaires are used in place of more objective measures
II. Data set:
ReCALL 13-15, 2001-2003 (6 issues)
CALICO 18-20, 2000-2003 (9 issues)
LLT 5-7, 2001-2003 (9 issues)
CALL 15-16, 2002-2003 (10 issues)
III. Factors reviewed
1) Research question(s), results, theoretical orientation
2) Tool/tutor; qualitative/quantitative
3) Reporting of computer experience, task experience, and any training
4) Number of subjects
5) Time on task
6) Training recommendations
Barrette, Catherine. 2001. Students' preparedness and training for CALL. CALICO 19.1, 5-36.
2003. Collaborative email exchange for teaching secondary ESL: A case study in
Kol, Sara & Schcolnik, Miriam. 2000. Enhancing screen reading strategies. CALICO 18.1, 67-80
Kötter, Markus. 2003. Negotiation of meaning and codeswitching in online tandems. Language Learning & Technology 7.2, 145-172.
Soboleva, Olga & Tronenko, Natalia. 2002. A Russian multimedia learning package for classroom use and self study. CALL 15.5, 483-499.
Stockwell, Glenn & Harrington, Michael. 2003. The incidental development of L2 proficiency in NS-NNS email interactions. CALICO 20.2.
Xie, Tianwei. 2002. Using Internet Relay Chat in teaching Chinese. CALICO 19.3, 513-524.