Some Subject, Treatment, and Data Collection Trends in
Current CALL Research
Phil Hubbard,
Below is the handout from the
Abstract:
In recent years, the number of research studies in CALL has grown
significantly. However, some elements of CALL research methodology remain
problematic. Whether looking at tool uses, such as CMC, or reviewing specific
CALL environments or tutorial applications, a high percentage of CALL research
appears to have the following characteristics:
1) Research subjects, whether
students or teachers, are novices to CALL
2) They are also novices to the task
or application under study
3) These novices are untrained
before the study and undirected during the study
4) They are often studied
exclusively during their initial experience
5) The studies may be very short,
representing a single event, such as a class or lab session
6) Surveys and questionnaires are
used in place of more objective measures, such as tracking or testing
The preceding points, if valid, suggest a disturbing conclusion for CALL
research, namely, that much, possibly most, of what we know about CALL comes
from studies involving the attitudes and impressions of undirected CALL
neophytes in their first encounters with a given task or application. To
test this impression, a corpus of over 90 research articles was reviewed. The
articles chosen for the corpus were those that had an identifiable research
question and involved human subjects interacting with or through a CALL
application. To assure that this corpus represented a credible and current sample
of the state of the field, it was composed of all such research articles taken
from the most recent two or three volumes of four leading refereed CALL
journals: CALICO Journal, volumes 18-20 (9 issues); ReCALL,
volumes 13-15 (6 issues); Language Learning and Technology, volumes 4-7
(9 issues); and CALL, volumes 15-16 (10 issues). The presentation
will provide detailed results regarding the preceding claims. It will conclude
with a discussion of some exemplary studies from the corpus and suggestions for
considerations researchers should make in research design and data collection.
Most notable among these is that while studies of novices still have a place,
we also need studies showing what happens after students are given both
technical and pedagogical training and have familiarized themselves with the
CALL applications being researched.
Notes
I. A lot of CALL research seems to have the following characteristics
1) Research subjects are novices to CALL
2) They are novices to the task/application under study
3) They are untrained before the study
4) They are undirected during the study
5) They are studied during their initial experience
6) They often only use the application for a short time
7) There are a small number of subjects
8) Surveys/questionnaires are used in place of more objective measures
II. Data set:
ReCALL 13-15, 2001-2003 (6 issues)
CALICO 18-20, 2000-2003 (9 issues)
LLT 5-7, 2001-2003 (9 issues)
CALL 15-16, 2002-2003 (10 issues)
III. Factors reviewed
1) Research question(s), results, theoretical orientation
2) Tool/tutor; qualitative/quantitative
3) Reporting of computer experience, task experience, and any training
4) Number of subjects
5) Time on task
6) Training recommendations
IV. Results
V. Recommendations
References.
Barrette, Catherine. 2001. Students' preparedness and training for CALL. CALICO 19.1, 5-36.
Greenfield, Roseanne.
2003. Collaborative email exchange for teaching secondary ESL: A case study in
Kol, Sara & Schcolnik, Miriam. 2000. Enhancing screen reading strategies. CALICO 18.1, 67-80
Kötter, Markus. 2003. Negotiation of meaning and codeswitching in online tandems. Language Learning & Technology 7.2, 145-172.
Soboleva, Olga & Tronenko, Natalia. 2002. A Russian multimedia learning package for classroom use and self study. CALL 15.5, 483-499.
Stockwell, Glenn & Harrington, Michael. 2003. The incidental development of L2 proficiency in NS-NNS email interactions. CALICO 20.2.
Xie, Tianwei. 2002. Using Internet Relay Chat in teaching Chinese. CALICO 19.3, 513-524.