Less-Commonly Taught Languages in Selected Elementary and Secondary Schools in California Foreign Langauge Assistance Act Program: Evaluation Project

Chapter 1: Introduction

In recent years there has been considerable interest in the teaching and learning of certain foreign languages that are not the traditional offerings in the curriculum of most public schools. These languages have come to be called the "less-commonly taught languages" (LCTLs) and generally include many of the world languages such as Japanese, Chinese, Russian, Arabic, Korean, etc. The importance of Asian languages, in particular, has been recognized in California by language educators, parents and students. According to the California Basic Education Data System (1995), the number of students enrolled in Japanese language classes has risen from 556 in 1982-83 to 5,439 in the 1994-95 school year (an increase of 861%). While not as dramatic, enrollment in Chinese classes has increased from 1,085 to 2,708 (149.6% increase) during the same time period. The increase in enrollment in the LCTLs is most notable in high school foreign language classes, although there are some elementary school programs that teach a LCTL.

Policy makers at both the state and federal level have recognized the importance of instruction in the LCTLs. Thus, efforts have been instituted to stimulate elementary and secondary schools to offer instruction in the LCTLs. This has occurred through the use of federal grants to school districts to initiate programs for students in the LCTLs. For instance, one such effort to stimulate demonstration projects in the LCTLs came about through the 1991 Foreign Language Assistance Act which provided federal funds to selected school districts who submitted proposals to teach one or more of the LCTLs. In California, grants for these model projects in the LCTLs were administered through the California Department of Education. These grant-awarded projects were called the Foreign Language Assistance Act Program (FLAAP). An important component of the model projects that were funded by the California Department of Education was evaluation. The intent of the evaluation was to independently determine whether the funded projects met their objectives of teaching students the four language skills (listening, speaking, reading and writing) in the target language as well as cultural knowledge. The California Foreign Language Project (CFLP) at Stanford University was designated as the evaluator for the FLAAP model demonstration projects in the LCTLs.

This Final Report summarizes the results of the evaluation study of selected elementary and secondary schools funded by the California Department of Education under the aegis of the Foreign Language Assistance Act of 1991 to teach the less-commonly taught languages of Japanese, Chinese (Mandarin and Cantonese), Korean and Russian. Specifically, 14 projects (6 Japanese, 4 Russian, 3 Chinese, and 1 Korean projects) at nineteen different school sites received four years of funding (1992-96) to develop and implement a demonstration language program in the teaching of one or more less-commonly taught languages. The programs were highly diverse in terms of teacher composition, student enrollment and strategies for implementing instruction in the LCTLs. Programs were situated in five elementary schools, one middle school, and thirteen high schools in Northern and Southern California.

All funded programs had submitted proposals specifying how they intended to implement LCTL instruction in their school. In the proposal, school officials indicated the objectives that they hoped to achieve through the program. The Stanford-based evaluators, which consisted of the authors of this report and native speakers of the languages studied, henceforth to be referred to as the Evaluation Team, began their study by familiarizing themselves with the proposal submitted by each funded program.

Objectives of the Evaluation

The evaluation of each model program was carried out by a team of language evaluators from Stanford University headed by the authors of this report. The Evaluation Team began the process of establishing the evaluation design shortly after the schools received their funding from the California Department of Education. The design was guided by several broad instructional and evaluation questions which pertained to the teaching and learning of the target language and culture. The questions and the research strategies for gathering evidence to answer these questions were developed by the Stanford Evaluation Team in collaboration with the teachers and school administrators responsible for implementing the model language programs. These questions guided the Evaluation Team throughout the four years of funding. Because the programs were all different in grade level of students, languages taught, curriculum and program implementation, no attempt was ever made to directly compare the programs in the evaluation study. Thus, analyses were carried out by comparing languages taught (e.g., Japanese vs. Chinese), levels of instruction (first vs. second year), and other variables (e.g., gender and ethnic background of students), but at no time were specific programs compared.

Instructional and Evaluation Questions

Based on the objectives for funding the LCTL programs, each project's goals specified in the proposals and knowledge from the literature in foreign language education, six instructional and evaluation questions were developed to guide the assessment of the model programs in the LCTLs. The questions are as follows:

In order to conduct the evaluation of the LCTLs using the guiding questions, it was necessary to establish a process that included the development of several new instruments and procedures for gathering information. The most important steps taken by the Evaluation Team to address each of the six instructional and evaluation questions are summarized in the next section of this Report.

Methodology for Addressing Each Question in the Evaluation Design

Because the nature of instruction was so important to the success of this evaluation, the methods used to assess learning of the LCTLs involved the teachers as much as possible. The hope was that by focusing closely on the instructional process with teachers, we could use portions of the evaluation plan and outcomes to foster professional understanding of effective methods for teaching the LCTLs. We also sought to incorporate their assistance in the data collection effort and to make them a part of the Evaluation Team through individual and group feedback sessions at the school sites or through group workshops. These features are included in the description of the process that the Evaluation Team used in addressing each the six questions.

To address this question, the Evaluation Team first devised a system of student portfolios as a form of performance assessment and implemented this in every language program. Student portfolios include samples of students' work that show their performance on the four language skills (listening, speaking, reading and writing) as well as the learning of cultural information about the people who speak the target language.

The Evaluation Team also developed an oral proficiency rating scale, the Stanford Foreign Language Oral Skills Evaluation Matrix (FLOSEM), that could easily be used by teachers to assess students' oral proficiency development. Teachers were asked to rate their students' proficiency three times (September, January and May) on the FLOSEM during the school year. Target language oral proficiency assessment was also carried out on selected case study students by means of the Classroom Oral Competency Interview (COCI) at the end of the 1994-95 and 1995-96 school years. This assessment was conducted by native speakers of the target language who were part of the Evaluation Team. The COCI was developed by a committee under the auspices of the California Foreign Language Project (CFLP) Policy Board.

A final instrument that was used to assess writing proficiency was the Classroom Writing Competency Assessment (CWCA) which is under final development by the California Foreign Language Project. Participating teachers were given training on the CWCA and asked to use the protocol to obtain writing samples from their students for inclusion in the 1995-96 school year portfolio.

Finally, we also asked high school students during school year 1995-96 to provide the Evaluation Team with self-assessed oral proficiency ratings in the four language skill areas of listening, speaking, reading and writing. These data were collected by means of the Sung Language Assessment Questionnaire which is a 40-item scale. Students were also asked to self-rate their proficiency using a slightly modified FLOSEM scale. This was done to correlate students' self-ratings on the FLOSEM with their teachers' ratings.

In order to answer this question, the Evaluation Team met frequently with teachers, made classroom observations, and developed the assessment protocol described above. We began by exploring ways to develop student portfolios during the 1993-94 school year. Then portfolios were fully implemented during the 1994-95 and 1995-96 school years. As part of the portfolio, teachers were asked to use the FLOSEM rating form at least twice a year (late September and late May) and preferably three times a year (end of January). FLOSEM scores were retrieved by the Evaluation Team and recorded for all students in all language programs by all language levels. This procedure allowed us to examine language growth within an academic year (September to May). Further, by collecting FLOSEM ratings across school years for students who continued from one level to the next in a language program, we were able to examine language development longitudinally for some students as they moved across two levels of foreign language instruction. A similar procedure was used with a smaller number of case study students for whom information on the COCI was available.

The value of the design used in this evaluation study was that it was both longitudinal and cross-sectional. The evaluation design was longitudinal because some information (such as student portfolios for selected case study students and every student's oral proficiency ratings) was available across a two-year span of time. When viewed from a cross-sectional perspective, on the other hand, we can examine changes in language growth by comparing different cohorts of elementary students from kindergarten to fifth grade and from high school students enrolled in level 1 (introductory) to level 4 or 5 (advanced) classes.

To better understand why students enroll in the less-commonly taught language classes, the Evaluation Team developed a survey questionnaire which was administered twice, once in 1993 and again in 1995. The questionnaire contained two different scales which asked students various questions regarding their reasons and interests in studying the less-commonly taught language. One scale consisted of items taken from the now classic foreign language motivation questionnaire developed by Richard Gardner and colleagues (Gardner, 1985). Gardner believes that there are two types of motivation that explain second language learning. These are instrumental and integrative motivation for studying a foreign language. Numerous studies by Gardner and others have confirmed the existence of these two forms of motivation for learning a new language. The Evaluation Team believed that in the absence of much information about why students enroll in the LCTLs, it was important to gather information about students' motivation for language study. The second scale consisted of items developed specifically for this evaluation project. Items on this scale asked students to indicate their reasons for enrolling in a LCTL and were designed to access information about ethnic heritage considerations (e.g., want to speak to a relative), school based factors (e.g., friends are taking the class) or other personal reasons (e.g., want to study abroad) for wanting to learn the language.

The survey questionnaire was administered to all students from fourth through the twelfth grade. It was felt that students below the fourth grade were too young to fully comprehend the questions on the survey so that students from Kindergarten to the third grade were excluded from the survey. In order to determine whether a relationship existed between motivation and language attainment, students' scores on the motivational scales were correlated with teacher FLOSEM ratings.

An important dimension seldom examined in second language acquisition research is the role of parents in the learning of the language by their children. In this evaluation study, a questionnaire was prepared for administration to parents that asked questions about their own language learning experiences and attitudes. In addition, questions were also asked about parents' level of involvement in their child's language study. In essence, these questions sought information on parents' active involvement as language facilitator for their children. To determine whether parents' reports of involvement matched their children's perception of parental involvement, the same set of questions were also included in the student questionnaire. The questions were slightly re-worded to assess students' perception of parental involvement.

The parental questionnaire was sent home with all students from kindergarten to twelfth grade with the request that parents complete the survey and return it to school. The questionnaire was prepared in English, Japanese, Chinese, Korean, Russian, and Spanish. The questionnaire was administered in the Autumn of 1993. Since new cohorts of students entered the various language programs at the beginning of academic year 1994 and again in 1995, the survey questionnaire was administered to all new students and parents in Autumn of 1995 following the same procedures used during the first data collection effort in 1993. In order to examine whether parents' attitudes and involvement influence their child's proficiency attainment, parents' responses were correlated with students' oral proficiency ratings.

Given the changing demographic profile of students in California with the rapidly increasing number of Asian heritage students, this question sought to better understand how heritage language background might influence students' attitudes and motivation to enroll and learn one of the four target languages funded by the California Department of Education. Thus heritage language became a critical part of the analyses to be reported on in this Final Report. In some language programs, there were many same ethnic heritage language students and the Evaluation Team felt that it was important to understand whether heritage language learners differed from other learners in the programs we evaluated. We were able to collect self-designated ethnic heritage information from students and parents from the survey questionnaire which asked questions about ethnic and language background. By combining heritage language information and motivational data and analyzing to see what impact these factors have on students' learning as determined by such measures as ratings on the FLOSEM, we are in a good position to understand how personal factors interact in second language instruction. This is especially important given the scarcity of information regarding student characteristics and the LCTLs.

To answer the last question regarding the challenges and difficulties that teachers in the less-commonly taught language programs encounter, the Evaluation Team conduced interviews with classroom foreign language teachers, foreign language department chair persons, principals, and any other foreign language program-related personnel. At the last meeting with teachers and administrators, a survey was also conducted to evaluate the impact and accomplishment of the FLAAP grant for their foreign language programs.

The above six questions serve as a structure to organize the major findings that emerge from this evaluation study. The organization of this report will now be discussed.

Organization of the Final Report

The instructional and evaluation questions discussed above were used to organize the remainder of this Report. Specifically, in Chapter 2 we describe the process engaged in by the Evaluation Team to implement the evaluation plan over the four years covered by this FLAAP Evaluation Project. Because a review of the literature revealed very little information about the teaching and learning of the LCTLs, it was necessary to develop and/or use several new instruments to collect data on target language attainment. In other studies of the learning of foreign languages, it is known that such factors as motivation to learn the target language, learning strategies and parent involvement all appear important in learning a second language. Thus an instrument to assess these personal factors was also created especially for this evaluation study. Chapter 2 provides information about the measurement devices that were developed for use by the Evaluation Team. Chapters 3 through 6 present the major findings from the evaluation which address the issues of students' proficiency assessment and factors influencing target language proficiency development. The chapters which summarize our findings are organized around the six (6) questions addressed at the start of this Report. Chapters 3 and 4 describe how we sought to assess target language proficiency attainment as brought up in Questions #1 and #2 and what we learned from this effort. Chapter 5 presents information about the students' reasons for enrollment in the LCTLs as mentioned in Question #3. Also included in this chapter is a summary of parents' attitudes toward foreign language learning and their involvement in the language study of their child (Question #4). The heritage language background issue in relation to the students' language study including students' motivation, learning strategies, parental involvement and oral proficiency attainment (Question #5) was discussed throughout Chapters 3, 4, and 5. Chapter 6 summarizes our findings with teachers and administrators about their perceptions of the program. This chapter also describes insights on the challenges and difficulties that teachers and students encounter in the teaching and learning of the less-commonly taught language programs (Question #6).

Cover Page and Directory
Table of Contents
Acknowledgement
Executive Summary
Chapter One: Introduction
Chapter 2: Description of Evaluation Activities
Chapter 3: Portfolios as Assessment Devices
Chapter 4: Oral Proficiency Assessment Results
Chapter 5: Student and Parent Questionnaire Survey Results
Chapter 6: Teacher and Administrator Perceptions of the Program
References
Appendices

Home | Organization | Map | Events and Discussions | Resources | Standards and Frameworks

© Copyright 1997 California Foreign Language Project. All rights reserved