Home | Organization | Map | Events and Discussions | Resources | Standards and Frameworks
- © Copyright 1997 California Foreign Language Project. All rights reserved
Description of Project Sites
When the FLAAP grants were first awarded in 1993, 13 project sites received funding. These projects were located in 12 participating schools in both Northern and Southern California. Projects were guaranteed two-year grant support and later the grants were extended for an additional two years. Thus projects were awarded four years of total support. In the 1994-95 school year, one more project was added with four participating schools. This project received two years of funding extending through Spring 1996. Also a school was added to an existing project site during the final project year. In addition, one Russian project changed over the funding period with one school dropping out of the project and new schools added. These new projects and schools were all incorporated into the evaluation study along with the originally funded projects. However, actual participation in the evaluation effort varied widely from project to project. Information on each project site (participating schools, language taught and length of participation with the grant) is shown in Table 1.
In the first year of the evaluation study, Evaluation Team members visited each project site and had numerous phone conversations with teachers whenever necessary. The evaluators discussed with teachers and school administrators the procedures to be used in evaluating each program. The evaluation questions which were listed at the opening of this Report served as the focal questions in organizing the evaluation. It is also important to mention in this chronology that the original 13 projects had all been funded prior to the awarding of the evaluation grant to Stanford. Thus details for evaluating programs were finalized with the site coordinators and teachers after instruction in the less-commonly taught languages had already begun.
The first site visits took placed during the months of May and June 1993. These visits served the purpose of acquainting Evaluation Team members with the program and staff members at each school site. During these visits, members of the Evaluation Team observed and videotaped class teaching for purposes of documenting instruction. In addition, interviews were carried out with teaching staff, school administrators, students, and their parents to collect baseline data for the longitudinal evaluation. At the same time, the Evaluation Team also collected information on existing techniques and instruments for assessing oral proficiency in the target languages (Japanese, Chinese, Korean, and Russian).
During the Fall of the 1993-94 school year, the Evaluation Team re-visited all of the project sites and had discussions with coordinators and teaching staff about the evaluation plan. This visit focused on working with teachers to develop student language portfolios to document language growth. Specifically, the discussions centered on how portfolios could be used to show language growth and what types of language-related information needed to be gathered for student portfolios. In addition, details were worked out with teachers to distribute and collect a questionnaire intended for students and another for their parents. The student and parent survey was deemed essential for obtaining information known to be related to second language study and attainment, so the importance of the survey was emphasized with teachers who were asked to work with their students to ensure a good response return from students and parents. This was especially true for parents of secondary students who, it was believed, might be especially difficult to reach by means of a take-home questionnaire.
During the third and fourth year, members of the Evaluation Team continued the practice of visiting each site twice a year: Once at the beginning of the school year to report the previous year's evaluation results and discuss the coming year's evaluation plan, and again at the end of the school year to collect student proficiency data by means of the COCI.
Other than regular school site visits and on-going phone conversations with teachers, the Evaluation Team planned and carried out annual meetings with teachers and administrators from all participating schools. During these meetings, the evaluation team provided an opportunity to have a group workshop on various professional development topics (e.g., portfolio assessment, COCI and CWCA training, FLOSEM training, etc.) and to discuss general concerns and problems encountered by the teachers of less-commonly taught languages. These workshops and discussions were very important because most of the teachers who participated in the project had few opportunities for professional development.
Since student portfolios were to be used to document students' language growth within a school year (and also across years of instruction), a series of workshops on portfolio development was held for teachers and administrators. At the beginning of the 1993-94 school year, members of the Evaluation Team visited each school site and held an individual training session with language teachers about the basic information on student language portfolios. Teachers and Evaluation Team members discussed the purposes and audiences of language portfolios, kinds of artifacts which could be included in a portfolio to show the students' language learning progress, and other logistics such as where to store artifacts. After this training meeting, each teacher was asked to think about their respective school plan for the student portfolios. A follow-up discussion was held with teaching staff at each school site via phone conversations. To strengthen and broaden the idea of portfolio assessment, the Evaluation Team with the collaboration of the California Foreign Language Project held a workshop on portfolio assessment on November 1, 1994. Every school representative brought a sample portfolio with them and shared it with other teachers. Teachers also exchanged experiences and problems in developing student portfolios and using them as a form of authentic assessment. The invited speaker, Susanne Charlton from UC Irvine, led a workshop on portfolio assessment. The speaker focused her comments on strengthening teachers' understanding of portfolio assessment and identified ³best² practices to support this form of assessment in foreign language education. Emphasis was also given by the Evaluation Team to teachers to follow the guidelines and a table of content format so that there would be consistency of portfolios across projects regardless of language type or grade/level of instruction.
The Evaluation Team also delivered a mid-point report and explained the evaluation procedures for the current year. During this meeting, teachers were encourage to raise any questions or concerns related to their foreign language teaching. All questions were answered by either a member of the Evaluation Team or by Dr. Duarte Silva, Executive Director of the California Foreign Language Project. At this meeting, all schools were informed that the FLAAP would continue to fund them through the 1995-96 school year, thus, what we were really doing at the Workshop was planning for an evaluation process for the next two years.
COCI Training Workshop
In addition to workshops on portfolio development and use, an intensive two-day workshop for LCTL teaching staff was held at Stanford University in February 1995 on the use of the Classroom Oral Competency Interview (COCI). Since it was planned to collect students' oral proficiency data by means of the COCI as part of the evaluation plan, it was decided prior to such data collection that it would be helpful to teachers to offer a professional development workshop on the administration and scoring of the COCI. Through the COCI workshop, all teachers would be informed on how the Evaluation Team planned to conduct oral proficiency assessments with the identified case study students at all school sites. Further, it was hoped that if teachers received training on the COCI, they might subsequently use the COCI in their own assessment of students' proficiency.
Training on the COCI was conducted by Silvia Jones who was on the committee that developed the COCI and who is one of the major trainers for teachers on the COCI. In the workshop she was assisted by our language evaluators, who had previously been trained themselves on the COCI. During the first day, each teacher and language coordinator received COCI training including a demonstration interview with ESL students. ESL students were interviewed to ensure a common language among all the language teachers. On the second day, teachers practiced administering the COCI to learners of their target language (e.g., Japanese, Russian). The COCI training with teachers was carried out with a two-fold purpose. First, the Evaluation Team wanted to let each teacher know what kind of oral proficiency interview instrument the language evaluator would use with the designated case study students at their school during late Spring. The other purpose was to encourage the classroom foreign language teachers to give serious consideration to using the COCI in their oral proficiency assessment of students when called upon to do so.
In the field of foreign language education, the use of student portfolios as a means of documenting and charting student growth in the four language skills (listening, speaking, reading, and writing) as well as cultural knowledge was very much at a developmental stage when the Evaluation Team began to work with project sites. The Evaluation Team viewed working with the project sites as an essential part of its efforts to contribute to the practice-side of foreign language education and to finding useful methods of performance-based forms of assessment. The use of portfolios was one such method of assessment explored as part of the evaluation plan.
We began with the idea of student portfolios by reviewing the literature on portfolio assessment. A search of the relevant literature revealed that most of the published material on portfolios was in the area of science and language arts education. No literature was located that dealt with the use of portfolios in the foreign language classroom. Using the existing literature, we developed a plan for portfolios that incorporated ideas about criteria for inclusion of materials in a portfolio, frequency of information collection, and documentation of materials selected for the file. Once this plan was developed, we discussed portfolios with teachers through an individual training session in the 1993-94 school year. The content of the training focussed on discussions with teachers about the purposes and audiences of portfolios in their classes, how to assemble the portfolio, what kind of artifacts would be included and how often, etc. In thinking about how best to implement a plan for portfolios, we decided to give teachers the freedom to choose their audience and purpose of the portfolio. We believed that if the use of portfolios was to be implemented meaningfully by teachers, it was important to maintain flexibility in how teachers defined the audience and purpose of their portfolios, since our schools varied between elementary and secondary levels and included four different languages. After presenting the options for portfolios to teachers at the various sites, we simply indicated that there would be Guidelines to follow to make portfolios as uniform as possible across sites and to maximize their utility as alternative forms of assessment.
The Evaluation Team asked each school site to select six students from each level (e.g., Japanese 1, Japanese 2, etc.) or grade (Kindergarten, first grade, etc.) in each language program to serve as case study students for the portfolio assessment portion of this project. After reviewing portfolios at the end of the second year of the Project, the Evaluation Team refined their Guidelines and devised a Table of Contents for student portfolios. Based on the Guidelines and Table of Contents, schools continued to assemble student portfolios during the 1994-95 year of the project evaluation. The use of portfolios was well received by participating teachers and also by students in the program. The portfolio guidelines were again revised after review and study of portfolios collected at the end of the third year. The most recent draft of the revised portfolio guidelines is included as Appendix 1.
Based on our experience in guiding language teachers through the process of developing and implementing student portfolios in a foreign language classroom, the Evaluation Team has given presentations on using portfolios in foreign language education at several professional meetings. For example, members of the Evaluation Team made a presentation at a round table discussion on portfolios in the classroom at the 1995 Annual Meetings of the American Educational Research Association (AERA) in San Francisco. Subsequently, the paper on which the presentation was based was submitted for publication to the Foreign Language Annals which is the official publication of the American Council of the Teaching of Foreign Languages (ACTFL). The paper was published in the Fall 1996 issue of the Foreign Language Annals. A copy is included as Appendix 2. In addition, presentations on implementation of student portfolios in the foreign language classroom were also given at the Annual Meetings of the American Council on the Teaching of Foreign Languages (ACTFL) in November 1995, Los Angeles and again in November 1996, Philadelphia.
The Stanford FLOSEM was first piloted with selected students at every school site at the end of the 1993-94 school year. Based on teacher feedback and reflection on the rating matrix by the Evaluation Team, the matrix was subsequently revised and put into use during school years 1994-95 and 1995-96. The Evaluation Team met with teachers at all the school sites and discussed the FLOSEM matrix and how the instrument should be used in rating students oral proficiency in the target language. Instructions for the FLOSEM along with the scoring matrix are attached to this report as Appendix 3.
At the end of the 1995-96 school year, high school students were asked to self-rate their language proficiency level. The self-rated proficiency task consisted of two parts: the Stanford FLOSEM and the Sung Language Assessment Questionnaire. The FLOSEM matrix was modified slightly, including some re-wording to facilitate interpretation by students of some descriptions in the matrix. The Sung Language Self-Assessment Questionnaire consisting of forty questions, ten questions for each of the four language skill areas (listening, speaking, reading and writing), was also used to gather information about students' proficiency attainment in the language they were studying. Items on the questionnaire were answerable using a five-point Likert scale depending on the degree of comfort students had in using the target language in a given situation. This questionnaire was developed and used successfully in an earlier study by Sung (1995) with adult participants. The value of this questionnaire is that it provides self-assessments in the four language skills. A copy of the self-proficiency questionnaire for students is attached as Appendix 4.
During the month of May or June in 1995, all school sites were visited for purposes of conducting oral proficiency examinations with the designated case study students at each grade or language instruction level. The California Oral Competency Interview (COCI) was administered by native speakers of each language for assessing the level of oral proficiency of the case study students. The administration of the COCI was video and/or audio-taped so that the examiners' could review the tape to ensure that they score the students' interview properly. Teachers were provided with copies of the COCI videotapes of their students at the beginning of Fall semester. The results of the COCI proficiency assessments were also shared with all teachers.
In order to examine students' and parents' attitudes and motivation of learning these less-commonly taught languages (LCTLs) as well as parental involvement in their child's language study, the Evaluation Team conducted a questionnaire survey of students from the 4th grade through to high school seniors enrolled in a language class. The questionnaire for students consisted of four parts. Part 1 consisted of 8 items comprising a scale of instrumental and integrative motivation for students enrolled in a LCTL program (e.g., "studying Japanese is important because it will be useful in getting a job in the future"; "it will allow me to meet with a variety of people"). This scale was adapted from a similar scale used previously by Gardner (1985). Part 2 was designed specifically for this study and consisted of 14 items which tapped other motives or reasons for studying these specific languages. These motives included: 1) motivation based on ethnic heritage (e.g., "this is my heritage language"); 2) school-related motivation (e.g., "I heard that the Chinese teacher at our school was good"); and 3) reasons based on personal interests (e.g., "I thought that studying Korean would be more interesting than studying European languages such as French, Spanish, or German"). Part 3 of the questionnaire contained a series of items that asked students about various strategies that they may or may not have used in their language study. There is growing evidence that successful language learners use various strategies to acquire language skills in a second language. Items for this section of the questionnaire were adapted from Oxford (1992) who has provided extensive information about language learning strategies known to be effective in language classrooms. Part 4 of the questionnaire asked students to indicate the level of parental involvement in their foreign language study. Examples of the 9 items include the following (e.g., "my parents encourage me to practice the foreign language as much as possible"; "my parents show considerable interest in my foreign language study").
The questionnaire for parents was divided into two parts. Part 1 consisted of 9 questions and asked parents about their attitudes toward foreign language learning in general (e.g., "studying a foreign language is an enjoyable experience"; "I wish to learn many foreign languages"). The second part which also consisted of 9 items asked parents to indicate their level of involvement in their child's language study (e.g., "I encourage my child to practice the foreign language as much as possible"). These last 9 items of the parental questionnaire were identical to the Part 4 of the student questionnaire. Items on both the student and parent questionnaires were arranged on a 7-point Likert scale from "strongly agree (6)" to "strongly disagree (0)".
The student and parent questionnaires are included in as an Appendix 5 of this report.
Home | Organization | Map | Events and Discussions | Resources | Standards and Frameworks