Some Subject, Treatment, and Data Collection Trends in Current CALL Research

Phil Hubbard, Stanford University

 

Below is the handout from the September 7, 2004 presentation at 11th International CALL Conference in Antwerp. Click HERE for the PowerPoint of the talk or email efs@stanford.edu for a copy

 

Abstract:      In recent years, the number of research studies in CALL has grown significantly. However, some elements of CALL research methodology remain problematic. Whether looking at tool uses, such as CMC, or reviewing specific CALL environments or tutorial applications, a high percentage of CALL research appears to have the following characteristics:
   1)      Research subjects, whether students or teachers, are novices to CALL
   2)      They are also novices to the task or application under study
   3)      These novices are untrained before the study and undirected during the study
   4)      They are often studied exclusively during their initial experience
   5)      The studies may be very short, representing a single event, such as a class or lab session
   6)      Surveys and questionnaires are used in place of more objective measures, such as tracking or testing
 The preceding points, if valid, suggest a disturbing conclusion for CALL research, namely, that much, possibly most, of what we know about CALL comes from studies involving the attitudes and impressions of undirected CALL neophytes in their first encounters with a given task or application.  To test this impression, a corpus of over 90 research articles was reviewed. The articles chosen for the corpus were those that had an identifiable research question and involved human subjects interacting with or through a CALL application. To assure that this corpus represented a credible and current sample of the state of the field, it was composed of all such research articles taken from the most recent two or three volumes of four leading refereed CALL journals: CALICO Journal, volumes 18-20 (9 issues); ReCALL, volumes 13-15 (6 issues); Language Learning and Technology, volumes 4-7 (9 issues); and CALL, volumes 15-16 (10 issues).  The presentation will provide detailed results regarding the preceding claims. It will conclude with a discussion of some exemplary studies from the corpus and suggestions for considerations researchers should make in research design and data collection. Most notable among these is that while studies of novices still have a place, we also need studies showing what happens after students are given both technical and pedagogical training and have familiarized themselves with the CALL applications being researched.

 

Notes

 

I. A lot of CALL research seems to have the following characteristics

1)      Research subjects are novices to CALL

2)      They are novices to the task/application under study

3)      They are untrained before the study

4)      They are undirected during the study

5)      They are studied during their initial experience

6)      They often only use the application for a short time

7)      There are a small number of subjects

8)      Surveys/questionnaires are used in place of more objective measures

 

II. Data set:

ReCALL 13-15, 2001-2003 (6 issues)

CALICO 18-20, 2000-2003 (9 issues)

LLT 5-7, 2001-2003 (9 issues)

CALL 15-16, 2002-2003 (10 issues)

 

III. Factors reviewed

1)      Research question(s), results, theoretical orientation

2)      Tool/tutor; qualitative/quantitative

3)      Reporting of computer experience, task experience, and any training

4)      Number of subjects

5)      Time on task

6)      Training recommendations

 

IV. Results

 

V. Recommendations

 

References.

Barrette, Catherine. 2001. Students' preparedness and training for CALL. CALICO 19.1, 5-36.

Greenfield, Roseanne. 2003. Collaborative email exchange for teaching secondary ESL: A case study in Hong Kong. LLT 7.1, 46-70.

Kol, Sara & Schcolnik, Miriam. 2000. Enhancing screen reading strategies.  CALICO 18.1, 67-80

Kötter, Markus. 2003. Negotiation of meaning and codeswitching in online tandems. Language Learning & Technology 7.2, 145-172.

Soboleva, Olga & Tronenko, Natalia. 2002. A Russian multimedia learning package for classroom use and self study. CALL 15.5, 483-499.

Stockwell, Glenn & Harrington, Michael. 2003. The incidental development of L2 proficiency in NS-NNS email interactions. CALICO 20.2.

Xie, Tianwei. 2002. Using Internet Relay Chat in teaching Chinese. CALICO 19.3, 513-524.