CALL Course Unit 6

Stanford

LINGUISTICS DEPARTMENT - STANFORD UNIVERSITY

An Invitation to CALL

Foundations of Computer-Assisted Language Learning

Home | Unit 1 | Unit 2 | Unit 3| Unit 4| Unit 5| Unit 6 | Unit 7 | Unit 8 | Supplement

Unit 6: CALL Research

OVERVIEW

So far, we have been going through this course with the implicit assumption that CALL works, that teaching language using computers in some way makes learning "better." But what exactly is "better?" Here are some possible interpretations:

learners pick up language knowledge or skills faster or with less effort (learning efficiency)
learners retain language knowledge or skills longer and/or learn more of what they need (effectiveness)
learners can get materials or experience interactions that would otherwise be difficult or impossible (access)
learners can learn with equal effectiveness across a wider range of times/places (convenience)
learners enjoy the language learning process more (motivation)
learners require less teacher time or less expensive materials (institutional efficiency)

There are no doubt other ways of defining "better," but if we just consider these, a question arises: What are we comparing these to? Presumably, we are comparing a CALL activity to some corresponding non-CALL activity to see which gives us superior results for a given language learning target. This comparative approach has strong face value: language teachers and program administrators are reasonable in wanting evidence that CALL is worthwhile before putting time and expense into it, and comparative research seems the only way to provide definitive answers. Unfortunately, after two decades CALL researchers have not been able to provide those answers, and a number of influential researchers long ago came to the conclusion that the type of study that pitted CALL against non-CALL was a dead end, just as happened with "method comparison" (e.g., audiolingual vs. Total Physical Response) in the 1970s and 80s. Ultimately, the more interesting and answerable questions were not about the computer, but about specific applications, specific features of applications, specific types of activities, specific environments, and specific characteristics of learners.

As with other areas of second language learning, there are two ways for teachers to approach CALL research. One is as a research consumer; the other is as a classroom or action researcher. Each of these is briefly discussed below. As with other units, the objective here is to just give a taste of what is an enormous and constantly growing area.

CALL RESEARCH TRENDS

As suggested above, most CAI (computer-assisted instruction) and early CALL research focused on comparing computer users with a control group typically using traditional methods. The results were mixed, often showing no significant difference, sometimes favoring the computer users, and occasionally favoring the traditional approaches (see Dunkel 1991).

Over time researchers began to argue against comparative research (see for example Chapelle and Jamieson 1989), stating that the number of variables was too great. There are now many areas being researched using a variety of quantitative and qualitative methods (although interest in comparative research remains, particularly in comparing face-to-face interaction with CMC). The results vary widely, and according to recent research reviews (Felix 2005; Hubbard 2005; Huh and Hu 2005) the field is hampered by widespread problems with research designs and reporting.

Research has continued in all areas of CALL but recently has focused on several identifiable areas, such as:

Computer mediated communication; especially, interaction in synchronous chat settings and email in tandem settings
Visual, text and sound annotation to promote comprehension and vocabulary acquisition
Effectiveness of online collaborative and constructivist activities, including development of communities

Here are some example studies. Keep in mind that these and other studies are generally conducted on small groups in specific settings and may suffer from methodological flaws of various sorts. As with all research, proceed with caution in connecting any findings to your own setting.

Study	Description	Some Results
Belz, J. (2001). Institutional and individual dimensions of transatlantic group work in network-based language teaching. ReCALL, 13 (2), 213-231.	Investigated tandem learning with German & US university students	Of 3 tandem groups studied, 1 failed & 2 succeeded, showing individual & group differences are important, not just tasks & technology
Darhower, M. (2002). Interactional features of synchronous computer-mediated communication in the intermediate L2 class: A sociocultural case study. CALICO Journal, 19 (2), 249-277.	Explores use of chat in 2 4th semester Spanish classes: analyzed weeks 1,4 & 8.	Off-task discussion occurred when instructor gone; limited use of L1, social cohesiveness
De Ridder, I. (2002). Visible or invisible links: Does the highlighting of hyperlinks affect incidental vocabulary learning, text comprehension, and the reading process? Language Learning & Technology, 6 (1), 123-146: http://llt.msu.edu/vol6num1/pdf/deridder.pdf	Explores whether visible or invisible links are more effective in getting students to check unknown vocabulary and the effect of each on reading process.	Highlighted links are clicked more often but without affecting speed, comprehension or learning of vocab.
Sun, Y.-C. & Wang, L.-Y. (2003). Concordancers in the EFL classroom: Cognitive approaches and collocation difficulty. CALL Journal, 16 (1), 83-94.	Reports on a study of inductive vs deductive approaches to learning collocations: 2 easy, 2 hard ones	Inductive approach was significantly better for easy collocations and almost significant (p=.05) for hard ones
Thorne, S. (2003). Artifacts and cultures-of-use in intercultural communication. Language Learning & Technology, 7 (2), 38-67: http://llt.msu.edu/vol7num2/pdf/thorne.pdf	Reports on 3 case studies demonstrating artifact- mediated practices as cultural practices in US university students	Case 2: a dyad had an initial problem w/email but went to IM (instant messenger) on their own and started with a 6-hour session. Case 3: US students find email inappropriate for social interaction. Prefer IM.
Sengupta, S. (2001). Exchanging ideas with peers in network-based classrooms: an aid or a pain. Language Learning & Technology, 5 (1), 103-134: http://llt.msu.edu/vol5num1/sengupta/default.pdf	Reports on peer exchanges in two partly networked-based classes: observing patterns	Found two common discourse moves were agreeing & praising--helped build community. Accountability gave feeling of heavy workload. Some students were more active, and one small group was consistently lukewarm
Stockwell, G.& Harrington, M. (2003). The incidental development of L2 proficiency in NS-NNS email interactions. CALICO Journal, 20 (2), 337-359.	Reports on a study of proficiency development via email messages: msgs 1, 5, 10 & 15 analyzed	Showed gains in error-free t-units. Recurring pattern was high performance on first msg, then drop on 5th followed by gradual increase to 15th.
Fernandez-Garcia, M. and Martinez-Arbelaiz, A. (2003). Learner interactions: A comparison of oral and computer assisted written conversations. ReCALL, 15 (1), 116-136.	Compared negotiation patterns/numbers across different pairings and different treatments.	NS-NNS group had most negotiation, especially in oral setting.
Schwienhorst, K. (2002). Evaluating tandem language learning in a MOO: Discourse repair strategies in a bilingual Internet project. CALL Journal, 15 (2), 135-145.	Reviewed MOO (an elaborated form of chat) logs for examples of repair strategies in a tandem setting.	Students said they used repetition requests a lot but logs disagreed; negotiation occurred and was more prevalent than avoidance or misunderstanding; Germans preferred paraphrases but their partners gave them translations
Kötter, M. (2003). Negotiation of meaning and codeswitching in online tandems. Language Learning & Technology, 7 (2), 145-172: http://llt.msu.edu/vol7num2/pdf/kotter.pdf	Reports on a tandem MOO project looking for examples/patterns of negotiation and codeswitching	Logs showed different patterns from self reports, especially wrt repetition. Notable differences between MOO & previous F2F studies
Yoshii, M. and Flaitz, J. (2002). Second language incidental vocabulary retention: The effect of text and picture on annotation types. CALICO Journal, 20 (1), 33-58.	Compared 3 groups for vocabulary acquisition during reading: glosses with picture only, text only, or combination. Surprise vocabulary tests at end and 2 weeks later.	Combination group outperformed others overall; picture group was best on 2-week delayed posttest.
Green, A. & Youngs, B. (2001). Using the Web in elementary French and German courses: Quantitative and qualitative study results. CALICO Journal 19 (1), 89-123.	Reports on study of replacing one of four language class days with independent study on the web	No significant difference in performance after replacing 1 of 4 class days with web.

There are many areas of CALL that have been looked at, and we only cover a few of them here. Check the references at the end for resources to continue your CALL research review. It should be noted before continuing that CALL research has long been burdened by a problem which has not plagued most classroom-based SLA research: the technology adds a dimension of complexity and it is constantly changing; consequently definitive answers in any area don't seem to exist.

SURVEY OF UNANSWERED QUESTIONS IN CALL RESEARCH

In the summer of 2002, I sent a survey to 120 CALL professionals around the world asking them to articulate one research question in the field that they would like to see answered. I received 64 responses. A writeup of the results and the actual questions proposed by the contributors can be viewed at www.stanford.edu/~efs/callsurvey

SUBJECT CHARACTERISTICS IN CALL RESEARCH

In 2004 I carried out a study of research articles found in four CALL journals over a 2-3 year period, focusing on subject characteristics and presented at a conference on CALL and Research Methodologies. The overall conclusion was that "CALL research as a whole is unbalanced in the direction of the study of novices working on novel tasks or using novel applications" (Hubbard 2005: 363). Among other recommendations, I suggested that more studies be done using experienced and/or trained learners (see Unit 7) so that we can get a clearer idea of the effectiveness of CALL software and tasks.

DOING RESEARCH

As noted in the introduction, this course is largely aimed at classroom teachers interested in beginning or expanding their use of CALL, and teachers can take the role of researchers themselves. There are several avenues available to teachers in the role of researchers of their own classroom or students.

Observation. When your students are using software or doing a computer-based task in a lab or other venue where you can--watch them. You can look over their shoulder, check their interactions, and make brief notes of what you notice. Interact with the students as they interact with the software. This can give you feedback on the effectiveness of a given piece of software, CALL exercise, or CALL task, and it can also help you determine student training needs.
Tracking. Some software has built-in tracking features. If you are using a discussion board, all student posts can be reviewed. Some chat programs also allow the sessions to be logged for later review.
Student surveys. Ask specific questions about usage--note that it's best to do this as soon as possible after a CALL session sense memories fade rapidly.
Pre- and post-testing.
Student journals. Getting students to keep a reflective journal of their experiences with software or other CALL activities is useful both for them and to the teacher.

REFERENCES

Chapelle, C.. (2001). Computer Applications in Second Language Acquisition: Foundations for Teaching, Testing, and Research. Cambridge, UK: Cambridge University Press.

Chapelle, C. & Jamieson, J. (1989). "Research Trends in Computer-Assisted Language Learning." in Pennington, M. (ed.) Teaching Language with Computers. La Jolla: Athelstan.

Dunkel, P. (1991). “The Effectiveness Research on Computer-Assisted Instruction and Computer-Assisted Language Learning." In Dunkel, P. (ed.) Computer-Assisted Language Learning and Testing: Research Issues and Practice. New York: Newbury House/Harper Collins.

Egbert, J. & Petrie, G. (eds.) (2005). CALL Research Perspectives. Mahwah NJ: Lawrence Erlbaum

Egbert, J. & Hanson-Smith, E. (eds.) (1999). CALL Environments: Research, Practice, and Critical Issues. Alexandria, VA: TESOL.

Hubbard, P. (2003). "A Survey of Unanswered Questions in CALL." CALL Journal 16.2-3.

Hubbard, P. (2005). "A Review of Subject Characteristics in CALL Research." CALL Journal 18.5.

Warschauer, Mark (2000). "Online learning in second language classrooms: an ethnographic study." In M. Warschauer and R. Kern (Eds.), Network-based language teaching: Concepts and practice. New York: Cambridge University Press.

Last modified: February 27, 2008 by Phil Hubbard