1. COURSE ANNOUNCEMENT 3/93 Education 351 Spring 1993 David Rogosa (rag@psych) Design and Analysis of Longitudinal Research This course is offered at a little less than alternate-year frequency. Research questions about growth, change, stability/sameness, arise in many areas of behavioral and social science. This course is intended to be part lecture and part workshop. Lecture presentations cover the statistical and psychometric issues in the design and analysis of longitudinal research. (Some years the lecture presentations and readings have been bulk of the course). SELECTED LECTURE TOPICS: Critique of Measurement of Change Literature Growth Curve Analysis Methods for Longitudinal Panel Data, including the good and the bad (e.g. HLM) of random coefficient models Failures/Follies of Path Analysis and Structural Equation Models (LISREL) Standard Methods for Experimental Longitudinal Designs (including some material on repeated measures) Failures of adjustment procedures for quasi-experimental longitudinal studies Assessment of Stability (sameness) Approaches and Designs for the study of Reciprocal Effects Observation Of Behavior The other component of the course is a "Workshop in Longitudinal Research" devoted to student and/or prof research projects. The importance of this component depends on class size, composition, and interests. Some possible projects that I, myself, have in mind for the course are: Creation of a portable, computerized, instructional module on basic results for the measurement of change literature (either as a Mathematica notebook or through a multimedia authoring ssystem) Review and critique of methods for assessing cognitive decline in longitudinal studies of dementia patients (for National Institute on Aging) The role of assessing stability of behavior in debates on the existence of personality constructs. Statistical problems of assessing student learning in school effects studies (e.g. why Coleman's analyses can't be taken seriously) --------------- Prequisites: instruction in statistical methods to at least Ed257/Psych252 level; interest in (or even better activity in) longitudinal research Organization: We will jointly choose an appropriate time for the weekly class meetings at the beginning of the quarter--mail rag@psych 2. SYLLABUS ED 351 Education 351 Spring 1993 David Rogosa (rag@psych) Design and Analysis of Longitudinal Research Class: Mondays, Cubberley Auditorium stage, 2:45-5:05 Rogosa: Ceras 402K, Wed 2:15-3 by appointment (i.e., send me an e-mail message beforehand) WEEK OF March 29 Registration Day April 5 First class Meeting. Introduction to longitudinal panel data. Research questions for longitudinal research. Basics of Measurement of Change literature. Discussion of student research interests. April 12 Regression Methods for Longitudinal Panel Data. School-effects studies. Path Analysis and LISREL. April 19 Random Coefficient Models and Growth Curve Analysis, including HLM. April 26 Group comparisons from Experimental Designs, analysis of repeated measures data structures. May 3 More complex group comparison issues. Individual differences in response to interventions. Issues in Compliance. Analysis of quasi-experimental group comparisons, including Regression Discontinuity designs. May 10 Assessment of Stability over time. Analysis of reciprocal effects. May 17 Non-quantitative measures. Studies of states and durations. Overview of survival analysis. Introduction to Behavioral Observations May 24 Analysis of durations: Methods for Survival Analysis Overview of time-series methods. May 31 Memorial Day Holiday. Dead Week. 3. ED351 Readings Packet April 9 1993 1. Rogosa, D. R. (1988). Myths about longitudinal research. In Methodological issues in aging research, K. W. Schaie, R. T. Campbell, W. M. Meredith, and S. C. Rawlings, Eds. New York: Springer Publishing Company, 171-209. 2. Rogosa, D. R., Brandt, D., & Zimowski, M. (1982). A growth curve approach to the measurement of change. Psychological Bulletin, 92, 726-748. 3. Rogosa, D. R., & Willett, J. B. (1985). Understanding correlates of change by modeling individual differences in growth. Psychometrika, 50, 203-228. 4. Werts, C. E. & Hilton, T. L. (1977) Intellectual status and intellectual growth, again. American Educational Research Journal, 14, 137-146. 5. Rogosa, D. R., & Willett, J. B. (1985). Satisfying a simplex structure is simpler than it should be. Journal of Educational Statistics, 10, 99-107. 6. Rogosa, D. R. (1992). Individual unit models versus structural equations: Growth curve examples. In Proceedings of the international workshop on statistical modeling and latent variables, Trento Italy, July 1991. K. Haagen, D. Bartholomew, and M. Diestler, Eds. Amsterdam: Elsevier North Holland. 7. Blomqvist, N. (1977) On the relation between change and initial value. Journal of the American Statistical Association, 72, 746-9. 8. Rogosa, D. R. Longitudinal Analysis of Student Achievement Data: Issues for Chapter 1 Evaluation. In Planning Papers for the National Longitudinal Study of Chapter 1, U.S. Department of Education, Office of Planning, Budget & Evaluation, 1989. 9. Brogan, D. R. & Kutner, M. H. (1980) Comparative analyses of pretest-postest research designs. American Statistician 34, 229-32. Plus responses. 10. Bock, R. D. (1979). Univariate and multivariate analysis of variance of time-structured data. In Longitudinal methodology in the study of behavior and development, J. R. Nesselroade and P. B. Baltes, Eds. New York: Academic Press, 199-232. 11. Rogosa, D. R. (1991). A longitudinal approach to ATI research: Models for individual growth and models for individual differences in response to intervention. In Improving inquiry in social science: A volume in honor of Lee J. Cronbach, R. E. Snow and D. E. Wiley, Eds. Hillsdale, New Jersey: Lawrence Erlbaum Associates, 221-248. 12. Rogosa, D. R., Floden, R. E., & Willett, J. B. (1984). Assessing the stability of teacher behavior. Journal of Educational Psychology, 76, 1000-1027. 13. Foulkes, M. A. & Davis C.E. (1981) An index of tracking for longitudinal data. Biometrics, 37, 439-446. 14. Rogosa, D. R. (1985). Analysis of reciprocal effects. In International Encyclopedia of Education, T. Husen and N. Postlethwaite, Eds. London: Pergamon Press, 4221-4225. 15. Rogosa, D. R. (1980). A critique of cross-lagged correlation. Psychological Bulletin, 88, 245-258. 16. Willett, J. B. & Singer, J. D. (1991) How long did it take? Using survival analysis in educational and psychological research. In L. M. Collins and J. L. Horn (Eds.) Best Methods for the Analysis of Change. Washington, D. C.: American Psychological Association. 310-28. 17. Assorted bibliographies on longitudinal research. From NIH-NIMH panel on longitudinal research (circa 1986). ================================== Other Rogosa stuff on Longitudinal Research Rogosa, D. R. (1979). Causal models in longitudinal research: Rationale, formulation, and interpretation. In Longitudinal methodology in the study of behavior and development, J. R. Nesselroade and P. B. Baltes, Eds. New York: Academic Press, 263-302. Rogosa, D. R. (1980). Time and time again: Some analysis problems in longitudinal research. In The analysis of educational productivity, volume II: Issues in microanalysis, C. E. Bidwell and D. M. Windham, Eds. Boston: Ballinger Press, 153-201. Rogosa, D. R. (1980). Comparisons of some procedures for analyzing longitudinal panel data. Journal of Economics and Business, 32, 136-151. Rogosa, D. R., & Willett, J. B. (1983). Demonstrating the reliability of the difference score in the measurement of change. Journal of Educational Measurement, 20, 335-343. Rogosa, D. R., & Willett, J. B. (1983). Comparing two indices of tracking. Biometrics, 39, 795-6. Rogosa, D. R. (1987). Casual models do not support scientific conclusions: A comment in support of Freedman. Journal of Educational Statistics, 12, 185-195. Rogosa, D. R., and Ghandour, G. A. (1991). Statistical models for behavioral observations (with discussion). Journal of Educational Statistics, 16, 157-252. Rogosa, D. R., and Ghandour, G. A. (1991). Reply to discussants: Statistical models for behavioral observations. Journal of Educational Statistics, 16, 281-294. 4. MIDTERM EDUCATION 351 May 5 1993 "Midterm" Problems I. MEASUREMENT OF CHANGE 1. Relation of change with a background variable (W) is often an important object of descriptive longitudinal research. From the timepath output (first page) for the Trento straight-line data, use the estimated rate of change for each person and the value of the background variable to compute a (rough) estimate of the coefficient for W in a linear prediction of the rate of change. Also obtain a standard error for this estimate. Compare this estimate with the mle and the bootstrap standard error given by the timepath oputput. Carry out the same estimation and comparison substituting observed initial status (time 1) for W. 2. Residual Change Scores were promoted by Cronbach and Furby (and the orthodoxy) as something worthwhile. For the Trento straight-line data use the first and last observations (Times 1 and 6) to construct residual change scores (e.g. Rogosa Brandt Zimowski eq 17). Compute the square of the correlation between the growth parameter "theta" given in the trline.par and the estimated resid change score as a measure of the reliability (cf Eqs 19,20 Rogosa Brandt Zimowski). Compare that with the corresponding measure using the difference score: square of the correlation between "theta" and the difference score. Compare also with the estimated reliability of the rate of change from timepath where all 3 data points are used. 3. Throughout the 70's and onward there was a series of Psychological Bulletin articles on the following PARADOX [sic, their term]. In a two group longitudinal experiment (like the Brogan-Kutner example), the fact that the most precise comparison between the two experimental groups (i.e. greatest power) occured when the difference score within each of the groups had *zero* reliability. Construct an artificial data example where the within-group reliability of the difference score is 0, but an accurate comparison of the groups can be made. Comment on the "sense" of the paradox. II. COMPARING EXPERIMENTAL, OR NON-EXPERIMENTAL, GROUPS 4. Bock Vocabulary Data. We used this 4-occasion data to illustrate a single-group (no between subject factor) repeated measures analysis. There is also in the data set a gender indicator which would allow comparisons of males and females growth in vocabulary scores. Carry out a repeated measures anova with gender as the between subject factor and occasions (4-levels) as the within-subjects factor. I give below Bock's anova table for you to check against. What conclusions would you make about gender differences here? We could also look at the mean growth curves for males and females more directly by regression of vocab scores on Grade (8, 9, 10, 11). Try to plot the within-gender mean growth curves; carry out any statistical comparison that seems reasonable. ------------- Bock MSMBR Table 7.2-5 Source df SS Gender 1 .85 Subj within gender 62 873 Occasions 3 194.18 GenderxOcc 3 2.79 OccxSubj within Gender 186 152.17 ----------------------------------------------------------------------- 5. The smsg data set has been used from Ed191 onward in various ways. A description of the data is given preceding the data listing in file smsg.dat in the Ed351 directory. For this nonexperimental study, carry out the repeated measures anova, and compare this with other ways we have used to compare these two groups, either using or not using the pre-test information. Also try a standardized change score comparison. end 5. FINAL Education 351 D Rogosa May 31 1993 FINAL PROBLEMS 1. Stability over Time of an individual. a. We have below the counts of occurences of a type of event in second column and the time interval of observation (in hours) in the third column for each of eight occasions. Using the results given in Appendix A of the Stability of Teacher Behavior paper carry out a test of the homogeneity hypothesis. Also obtain an estimate of the mean rate and a variance component estimate for the amount of heterogeneity in the rate of the event. 1 13 0.75 2 20 1.25 3 18 0.75 4 22 1.25 5 19 0.50 6 17 1.50 7 15 0.50 8 24 1.50 b. Create your own Bernoulli Trial data and analyze. We have 5 occasions of observation. Draw the underlying proportion of 'success'--pi-- from a Beta distribution with parameters 4, 2. (E.g., see minitab manual or Handbook for "RANDOM" and subcommands.) So on each occasion there is a value for pi; use that value of pi (a number between 0 and 1) to generate the number of successes from 20 trials (c.f. Binomial subcommand). As in part a, use the results given in Appendix A of the Stability of Teacher Behavior paper carry out a test of the homogeneity hypothesis. Also obtain a variance component estimate for the amount of heterogeneity. 2. Stability of individual differences. Here are time 1 (c1) and time 2 (c2) scores for 6 individuals. Assess the stability of individual differences by computing the Foulkes-Davis gamma index for straight-line fits (i.e. connect the points). Compare your value of gamma with a time1-time2 product moment correlation (a typical measure of stability in behavioral science). What's up? ROW C1 C2 1 1 2.0 2 2 3.0 3 3 4.0 4 4 5.0 5 5 6.0 6 15 3.1 3. Become a longitudinal researcher. Laura Carstensen's presentation of her nursing home study brought up questions (i) about assessing individual and group change over time in measures such as Depression and Control (i.e. measurement of change issues) and (ii) about durations (i.e. differential survival in community and nursing home settings.) A combined data set containing both aspects is attached below with descriptions. A. Investigate group and individual differences in change in either the Depression or Control measures (descriptions can be found I believe in the handout for that presentation). (For individual change you may want to refer to the data analysis section of the Chapter 1 Evaluation paper.) B. Investigate group differences in survival. The outcome variable is number of days lived after onset of the study. The most elaborate analysis would be to use age as a covariate in a Cox regression; it would be great to just try to compare survival curves for the two groups. *********************************************** Here are the survival/mortality data along with the depression and control ratings of the BPRS. Please note that the BPRS was only done on the 1st, 3rd and 4th interviews, NOT the 2nd. The community residents are the 100 series and the nursing home residents are the 200 series. The righmost column is age at entry which I (drr) appended to the file from the main data set. The age measure could be used as a background variable in the measurement of change analyses (or survival analysis). ============================================================================== DATA LIST FILE= SUBJECT SURVIVAL DEP1 DEP2 DEP3 CONT1 CONT2 CONT3 age SUBJECT=SUBJECT NUMBER SURVIVAL=DAYS SUBJECT LIVED DURING FOLLOW-UP (MAXIMUM DAYS=550) DEP1=BPRS RATING FOR DEPRESSION ON 1ST INTERVIEW DEP2=BPRS RATING FOR DEPRESSION ON 3RD INTERVIEW DEP3=BPRS RATING FOR DEPRESSION ON 4TH INTERVIEW CONT1=BPRS RATING FOR CONTROL ON 1ST INTERVIEW CONT2=BPRS RATING FOR CONTROL ON 3RD INTERVIEW CONT3=BPRS RATING FOR CONTROL ON 4TH INTERVIEW 999 =MISSING DATA ============================================================================== 101 550 2.50 1.50 1.00 1.00 1.00 2.00 71 102 550 3.00 4.50 3.50 2.00 4.00 1.00 90 103 550 3.50 5.75 5.50 3.50 2.00 3.50 83 104 508 3.00 5.50 999 2.00 3.00 999 73 105 550 3.25 4.00 1.00 1.00 4.25 1.00 86 106 550 1.00 1.00 1.50 1.00 1.00 1.00 89 107 550 999 2.00 1.00 999 2.00 1.50 73 108 550 3.00 2.00 3.50 1.50 1.00 4.00 87 109 550 2.00 1.00 1.00 1.00 1.00 1.00 75 110 550 1.50 3.00 5.00 6.00 5.50 4.50 83 111 550 1.00 1.00 1.00 1.00 1.00 1.00 86 112 409 4.50 6.50 999 4.50 6.50 999 82 113 550 1.00 1.00 2.25 1.50 1.00 1.00 88 114 550 1.50 3.00 2.00 1.50 1.00 1.50 88 115 550 2.50 4.00 3.50 2.50 3.25 4.25 79 116 550 1.00 1.50 2.00 2.00 1.00 1.75 77 117 550 2.50 1.00 1.00 1.00 1.00 1.00 999 118 550 3.50 3.50 2.00 3.00 2.50 1.00 66 120 550 2.00 4.50 1.50 1.00 1.00 1.00 94 121 550 3.00 1.50 1.50 1.00 1.00 1.00 75 122 550 1.50 2.00 4.50 1.50 1.00 1.00 81 201 550 1.00 1.00 1.50 1.00 6.00 1.50 86 202 550 3.00 5.25 3.00 2.00 6.25 1.00 81 203 151 3.50 999 999 5.00 999 999 76 204 550 5.50 5.50 6.50 2.50 4.50 6.00 89 205 234 999 999 999 999 999 999 96 206 550 1.50 1.00 5.25 1.00 1.00 6.00 85 207 206 2.00 999 999 5.50 999 999 89 209 550 3.75 1.50 3.00 4.50 3.00 4.50 77 210 550 3.50 1.00 1.00 2.00 3.50 1.50 83 211 550 2.25 1.50 1.50 1.00 1.00 1.00 86 212 251 4.50 999 999 6.50 999 999 88 213 550 1.00 2.00 1.00 1.00 1.50 1.00 86 214 353 2.00 3.00 999 1.50 1.00 999 90 215 362 2.00 2.50 999 1.50 1.50 999 82 216 078 4.50 999 999 4.50 999 999 88 217 412 1.00 999 999 1.00 999 999 83 218 550 1.00 999 999 2.00 999 999 81 219 139 1.00 999 999 1.00 999 999 86 220 550 3.00 999 999 2.00 999 999 89 221 550 2.50 999 999 2.00 999 999 84 6. COURSE DIRECTORY FILES Course Directory Ed351 usr/class/ed351 elaine40.Stanford.EDU% cd class elaine40.Stanford.EDU% cd ed351 elaine40.Stanford.EDU% ls ALT.OUT brogkut.lis kpt11cox.log ramus.dat trentoln.out BOOTREPS.DAT brogkut.rd kpt11cox.lst ramus.out trentotp.scr announce elaine.out kpt11cox.sas rat.dat trexpon.eta bock.rd elaine.run kpt11lrg.log rat.out trexpon.y bockall.out final kpt11lrg.lst ratdat.hlm trline.eta bockall.run gg.m kpt11lrg.sas readings.list trline.par bockaov.dat kpt11.dat midterm shoes.lis trline.the bocktp.dat kpt11.log ncfm8.dat smsg.dat trline.y bocktrnd.lis kpt11.lst ncfm8.out timepath wperf.dat brogkut.dat kpt11.sas outline.351 tp.m wperf.out