Stanford
Teacher Education Program Evaluation
Stanford Teacher Education Program Evaluation
Stanford Teacher Education Program Evaluation
The president of Stanford University, Gerhard Casper, requested an evaluation of the Stanford Teacher Education Program.
President Gerhard Casper
A
Report to the President: Stanford Teacher Education Program
1997-98 You will need Adobe Acrobat Reader to open these
files. You can download the free program by selecting Reader.
Stakeholders and Needs Assessment Phase
Given the President's role and request, he represents a key stakeholder in the evaluation. During the needs assessment phase of the evaluation we interviewed as many key stakeholders as feasible, to determine the most significant issues associated with this effort. Gerhard was generous enough to take the time to speak to us about the issues most relevant to him. In addition, Provost Condoleezza Rice provided invaluable insights from a university-wide perspective.
Stakeholders directly responsible for the administration of the program include, School of Education Dean Richard Shavelson, Associate Dean Denis Phillips, Professor Larry Cuban, and the Director of STEP, Dr. Beverly Carter.
Dean Richard Shavelson and Associate Dean Denis Phillips Students also represent major stakeholders in a teacher education program. We conducted focus groups with students to learn more about the strengths and weaknesses of the program from their perspective. This part of the evaluation was customer or client driven.
1997 Graduating STEP Students (members of our focus groups) We asked each stakeholder to identify his or her primary concerns about STEP. We used empowerment evaluation concepts and techniques to elicit student perspectives about the most relevant aspects of the program and their assessment of those components. We also conducted a survey of the 1997 graduating STEP class; (the survey was primarly based on what we learned about the program from individual interviews and focus groups).
In addition, we reviewed past evaluation documents, program material, and relevant literature. Then we listed all the issues identified during the needs assessment phase and looked for common denominators, overarching themes, and critical concerns (often shaped by the role of the key stakeholder in the institution).
NUDIST (Non-numerical Unstructured Data Indexing Searching and Theorizing)
We used NUDIST software in the evaluation to keep our records organized, conduct searches and sorts, test theories, and generate reports including statistical summaries.
The evaluation team used it to establish key themes and common denominators.
This form of analysis helped us make the transition from the needs assessment phase of the evaluation to the plan of action or fieldwork phase. We also used this software extensively to sort data and test propositions throughout the evaluation. A visual display of how NUDIST helps organize data is presented below:
This is a computer screen snapshot of NUDIST software. The "project information list" provides information about the number of documents and nodes (or categories). The "tree display" window provides an image of how the data is classified and categorized. For example, one category of data focuses on the budget, another on professional education schools, the third on the faculty, and so on. The arrow to the right indicates the existence of additional categories. (The entire structure of the data base can be viewed in the small tree in the left hand corner of that window.)
Arrows going down from the "1: budget" and "3: faculty" categories indicate "children" or subcategories. Finally, the information at the bottom of the screen image is the actual interview data that has been coded under the faculty domain or category.
As you can see, this software helped us manage a large data base with visual clarity. It also helped us keep track of the categories that emerged during this phase and throughout the evaluation project.
Plan of Action or Proposal
We generated a list of issues and program components that merited detailed review, given the resources available and time allotted. This plan of action or proposal was shared with key stakeholders. We responded to stakeholder inquiries about the list of items, clarified areas of concern and then began the fieldwork stage of the evaluation.
Critical Steps in the Evaluation: Fieldwork, Data Collection, Analysis, Memoranda, Interim Reports, Formative and Summative Reports
As this evaluation unfolded, the evaluative steps and activities were reported on this page. We focused on activities associated with our fieldwork engagement, including data collection, analysis, writing memoranda (summarizing our preliminary findings and sharing drafts with program staff members), interim reports, and formative and summative reports.
Summer School (Summer Quarter)
This is a picture of the front of the summer school. The evaluation team's first major fieldwork activity involved STEP's summer program. It consisted of two parts: the middle school field experience and the Stanford courses. STEP students participate in a summer middle school shortly after arriving at Stanford - it is half of their Stanford summer program. They observe master teachers, participate in the classroom, and teach some of the classes (under the supervision of a master teacher). The master teacher is a role model for them and provides critical feedback about their teaching habits and style. A "debriefing" time at the end of the school day is devoted to discussing what happened in class in the morning. This time was also useful to help socialize STEP students into the profession.
Select the Peterson page for a few pictures of STEP Summer School activities.
Select Learning With and About Technology to see an on-line article about research conducted on site
during the summer school evaluation.
The second half of the day involved Stanford STEP courses. The evaluation team observed STEP students and master teachers every day at the summer middle school. In addition, the evaluation team observed the Stanford courses with the STEP students. This schedule was grueling. We started the day at 7:00 am., met the STEP students at the middle school at 8:00 am., observed classes until the end of the school day and then observed the "debriefing" period at the end of the day. After that we ran back to Stanford with them to observe the Stanford classes, typically from 2 to 4:15 (with a 15 minute break) and then another class from 4:30 to 6:00. We would often informally interview STEP students throughout the day at the middle school, during lunch, in between classes, and after class.
Stanford course evaluations were consumer or student focused. The evaluation team relied heavily on student perceptions about pedagogy, however, the team made an independent assessment combining both the student perception and the evaluation team perception of classroom activity and performance. Relevant sections of the draft report were shared with faculty for comment. The evaluation team considered all responses and made changes as deemed appropriate. One part of the report involved the sensitive topic of assessing faculty teaching.
This is a picture of the cover of the summer school evaluation report.
Click here to download a copy of the report.
The 1997 Stanford Teacher Education Program Summer School Evaluation Report was issued to the President on October 24, 1997. Dr. Fetterman presented it to the faculty immediately after discussing the report with the president. The executive summary is publically available.
Focus Groups (Autumn to Spring Quarters)
We conducted STEP student focus groups throughout the year to solicit their views about the program, in conjunction with our observations, surveys, and individual interviews. The Autumn 1997 focus groups discussed issues concerning placement, cooperating teachers, supervisors, and curriculum.
Select Focus Group for a few pictures of focus group participants.
Web Surveys
The evaluation team used web surveys to collect and analyze evaluation data. We used FileMaker Pro 4.0 to generate online surveys. They have been used to collect alumni data for the last 3 years, as well as data from current students about the program.
One of the benefits of using a web survey is that respondents can complete the form at their convenience (and it is always available at a specific location when they want to complete it, as compared with a hard copy which often is lost or misplaced). In addition, once the respondent completes the survey, it is automatically entered into the data base. There is no time lost mailing the survey back and entering it into a data base. There is also a significant cost savings since postage is reduced - to and from the respondent - and there are not costs associated with data entry (it is automatic). Another benefit of this approach is that each member of the evaluation team can sort the data as desired at any time from the convenience of their home, office, or any location in which they can connect to the web.
This is a computer screen snapshot of one of our web surveys.
Faculty Interviews
We also interviewed all of the faculty most closely associated with the program. They provided an insight into some of the most significant issues facing the program, focusing on faculty involvement, excellence in teaching, and the mission of the program. Their contributions were invaluable in understanding and assessing the program. Some faculty have been associated with the program for over 20 years and their stories about their experiences 20 years ago were as real and alive as if they occurred yesterday. This attachment and involvement in the program helped the evaluation team appreciate many of the sensitivies associated with the program, while adding in some cases to the degree of difficulty associated with conducting the evaluation.
Year long Placement
The evaluation team completed the most significant phase of the evaluation by the middle of June 1998 - focusing on the year-long placements in local schools (as well as the academic course work).
Final Reports
A draft report of the year long program, as well as a report to the President, was made available for faculty review. Faculty were given a couple of months to review the reports and provide comments and concerns. The reports focused on the following topics and issues: unity of purpose or mission, curriculum, research, alumni contact, professional development schools/university school partnerships, faculty involvement, excellence in teaching, and length of the program. Specific program components were highlighted in the year long program evaluation report including: admissions, placement, supervision, and portfolios.
Here is an except from the draft report that provides some insight into the tone of the report:
STEP stands at a crossroad in its history and development. STEP has some of the ingredients of a first-rate teacher education program, ranging from a world-renowned faculty to exceptional students. As we enter a new millennium, however, faculty in the School of Education have a unique opportunity to raise the standard of excellence in the program and in the field.
Reports are a traditional vehicle in which to disseminate evaluation findings and recommendations. However, we have also provided policy makers, researchers, and program staff members with web-based PowerPoint "slide" presentations.
Select PowerPoint slide presentation for a tour of the presentation.
For a copy of the report to the President, select President.
For a the detailed technical report, select Faculty.
Evaluation Team
The evaluation began January 1, 1997. It will conclude January 1999. It was a three-year project. Therefore, we planned for a flow of students in and out of the evaluation throughout this project. The evaluation team at the beginning of this evaluative journey consists of MA Policy Analysis and Evaluation student Jane Yoo and Ph.D. Language, Literacy, and Culture student Kristy Dunlap, and David Fetterman.
Here are Jane and Kristy in action (Winter Quarter 1997) after interviewing the president below:
Amy Greenfield and Carrie Penner (both members of the 1996-97 MA Policy Analysis and Evaluation Program) joined the evaluation team in the spring quarter. Amy is currently working for WestEd. Gregory Brower, Tom Matos, and Wendy Connors joined the evaluation team in the fall of 1997 along with Kristy Dunlap and David Fetterman.
Gregory Brower, Tom Matos, David Fetterman, Kristy Dunlap, and Wendy Connors
The evaluation team members focused on specific tasks including admissions, placement, supervision, course work, and alumni perceptions and reflections about the program. Methods included: focus groups and surveys with current students and alumni, classroom observation, participation in supervision sessions, and reviews of archival data. Sandra Paik (a member of the 1998-99 MA Policy Analysis and Evaluation Program) is the most recent addition to the evaluation team.
All evaluation students are required to participate in an evaluation during their tenure at Stanford. This is simply a case example in progress. We hope this brief insight into a program is useful and illuminates one aspect of our program.
