SOLUTIONS TO QUESTIONS IN SCHOOLCN.IND We have a design with three factors: teaching methods, schools, and teachers. Teachers are nested within schools, whereas schools and teachers are each crossed with method. NWK calls this a partially nested or crossed-nested design. a) Means for method crossed with school: MTB > read '/usr/class/ed257/95ep3p2.dat' c1-c5 90 ROWS READ ROW C1 C2 C3 C4 C5 1 0 1 1 1 1 2 1 1 1 1 2 3 0 1 1 1 3 4 1 1 1 2 1 . . . MTB > name c1 'outcome' c2 'method' c3 'school' c4 'teacher' c5 'student' MTB > table c2 c3; SUBC> means c1. ROWS: method COLUMNS: school 1 2 ALL 1 0.44444 0.22222 0.33333 2 0.88889 0.55556 0.72222 3 0.55556 0.44444 0.50000 4 0.55556 0.33333 0.44444 5 0.22222 0.11111 0.16667 ALL 0.53333 0.33333 0.43333 CELL CONTENTS -- outcome:MEAN Profile plot: outcome - B - - 0.75+ - - - - 2 B 0.50+ - A C - - D - 0.25+ - E A - - E ----+---------+---------+---------+---------+---------+-- 1.00 1.20 1.40 1.60 1.80 2.00 school It looks like there are main effects for both method (some methods are consistently higher than others) and school (school 2 has lower outcome scores than school 1 for all 5 methods) but not much of an interaction; lines are somewhat nonparallel but it isn't very pronounced. b) Means for teacher within school MTB > table c4(c3); #note parens not really needed SUBC> means c1. ROWS: teacher COLUMNS: school 1 2 ALL 1 0.53333 0.33333 0.43333 2 0.46667 0.46667 0.46667 3 0.60000 0.20000 0.40000 ALL 0.53333 0.33333 0.43333 Note that since teachers are nested within school, teacher 1 in school 1 is different from teacher 1 in school 2. Therefore the row marginal means are meaningless. Teacher effects do not appear large. Mean scores range from .47 to .6 in school 1 and from .2 to .47 in school 2, which is probably not a big range given only 3 observations per class. c) Statistical model and ANOVA table There is similarity between the nested structure of this problem and repeated measures examples such as the homework problem using Winer dial data. If you set up the data in a table, you will see that the two designs are quite similar. One difference is that in this problem is the presence of more than one replication per cell. Consequently, the confounding that occurs in the repeated measures model (between beta_rho(ki(j)) and random error) is not a problem here. the ANOVA tables below show, however, that the results do not change dramatically when this interaction term is left out. Statistical model (see NWK p.1150) Y(ijkm)=mu + alpha(i) + beta(j) + gamma(k(i)) + alpha_beta(ij) + beta_gamma(jk(i)) + epsilon(m(ijk)) where Y(ijkm) is the outcome score for the mth student at level i of school, level j of method, and level k of teacher within school mu is the grand mean alpha(i) is the random effect of school at level i beta(j) is the fixed effect of method at level j gamma(k(i)) is the random effect of teacher k nested within school i alpha_beta(ij) is the random effect of interaction between school and method at level i of school and level j of method beta_gamma(jk(i)) is the random effect of interaction between method and teacher (nested within school) at level j of method and level k of teacher nested within school i epsilon(m(ijk)) is a random component of error for student m at level i of school, level j of method, and level k of teacher within school i=1,2 j=1,2,3,4,5 k=1,2,3 m=1,2,3 Constructing the ANOVA table: Once you have the model written down, you can write your anova statement so that it includes each component in the model. The statement below asks for main effects for method and school and for method*school interaction (using the shortcut notation method|school), main effect for teacher within school, and interaction effect for method*teacher(school). Be sure to declare both teacher and school as random factors, and use the restrict subcommand because this is a mixed model. The means subcommand is just a demonstration of an alternative way to get the means asked for in parts a and b (and then some), and the ems subcommand will come in handy in part d. The anova statement for our mixed-model repeated measures example works except that it leaves out the last interaction term, which we can estimate since there is more than one observation per cell. (note: The ANOVA tables presented below show that the results do not change too much if you leave out the beta_rho(ki(j)) interaction term. the tests for school & method main effects are exactly the same; the test for school*method interaction changes because the error term is different, and the test for teacher(school) changes because MSW changes slightly. See the ems tables for a comparison of the error terms used.) MTB > anova outcome=method|school teacher(school) method*teacher(school); SUBC> random school teacher; SUBC> restrict; SUBC> ems; SUBC> means method|school teacher(school). Factor Type Levels Values method fixed 5 1 2 3 4 5 school random 2 1 2 teacher(school) random 3 1 2 3 Analysis of Variance for outcome Source DF SS MS F P method 4 3.0444 0.7611 19.57 0.007 school 1 0.9000 0.9000 5.40 0.081 method*school 4 0.1556 0.0389 0.31 0.866 teacher(school) 4 0.6667 0.1667 0.65 0.628 method*teacher(school) 16 2.0000 0.1250 0.49 0.943 Error 60 15.3333 0.2556 Total 89 22.1000 Source Variance Error Expected Mean Square component term (using restricted model) 1 method 3 (6) + 3(5) + 9(3) + 18Q[1] 2 school 0.01630 4 (6) + 15(4) + 45(2) 3 method*school -0.00957 5 (6) + 3(5) + 9(3) 4 teacher(school) -0.00593 6 (6) + 15(4) 5 method*teacher(school) -0.04352 6 (6) + 3(5) 6 Error 0.25556 (6) MEANS method N outcome 1 18 0.33333 2 18 0.72222 3 18 0.50000 4 18 0.44444 5 18 0.16667 school N outcome 1 45 0.53333 2 45 0.33333 method school N outcome 1 1 9 0.44444 1 2 9 0.22222 2 1 9 0.88889 2 2 9 0.55556 3 1 9 0.55556 3 2 9 0.44444 4 1 9 0.55556 4 2 9 0.33333 5 1 9 0.22222 5 2 9 0.11111 school teacher N outcome 1 1 15 0.53333 1 2 15 0.46667 1 3 15 0.60000 2 1 15 0.33333 2 2 15 0.46667 2 3 15 0.20000 For comparison, here's the ANOVA table that omits the method*teacher(school) interaction term. Note especially the change in dfw, SSW and MSW due to lumping the method*teacher(school) interaction with random error, and the differences in the error terms used for test statistics (e.g., in ems table). MTB > anova outcome=method|school teacher(school); SUBC> random school teacher; SUBC> restrict; SUBC> ems. Factor Type Levels Values method fixed 5 1 2 3 4 5 school random 2 1 2 teacher(school) random 3 1 2 3 Analysis of Variance for outcome Source DF SS MS F P method 4 3.0444 0.7611 19.57 0.007 school 1 0.9000 0.9000 5.40 0.081 method*school 4 0.1556 0.0389 0.17 0.953 teacher(school) 4 0.6667 0.1667 0.73 0.574 Error 76 17.3333 0.2281 Total 89 22.1000 Source Variance Error Expected Mean Square component term (using restricted model) 1 method 3 (5) + 9(3) + 18Q[1] 2 school 0.01630 4 (5) + 15(4) + 45(2) 3 method*school -0.02102 5 (5) + 9(3) 4 teacher(school) -0.00409 5 (5) + 15(4) 5 Error 0.22807 (5) d) Statistical tests Here's where the ems subcommand is useful. The anova command automatically calculates the F statistics for you, and that through arithmetic will indicate the proper test statistic denominator. The ems table identifies such under "Error term." Also NWK provides an expected mean squares table for a similar design on Tab 28.11, but that design has two fixed factors, whereas ours has only one. This changes the test statistic for factor B, method: rather than comparing MSB to MSW, as NWK suggests, we compare it to MSAB (consistent with Minitab ems table). Since we're doing 5 tests, if we want familywise Type 1 error rate of .05, a quick Bonferroni approximation says we should use .01 for each test. For school (factor A) main effect Ho: sigma-sq(alpha)=0 vs. Ha: sigma-sq(alpha)>0 Test statistic: MSA/MSC(A) = .9000/.1667 = 5.40 Critical value: F(.99,1,4) = 21.1977 Do not reject Ho. For method (factor B) main effect Ho: beta(j)=0 for all j vs. Ha: not all beta(j)=0 Test statistic: MSB/MSAB = .7611/.0389 = 19.57 Critical value: F(.99,4,4) = 15.9774 Reject Ho. There is a significant effect for method. For school*method (AB) interaction Ho: sigma-sq(alpha_beta)=0 vs. Ha: sigma-sq(alpha_beta)>0 Test statistic: MSAB/MSBC(A) = .0389/.1250 = .31 Critical value: F(.99,4,16) = 4.7726 Do not reject Ho. For teacher within school (C(A)) main effect Ho: sigma-sq(gamma(alpha))=0 vs. Ha: sigma-sq(gamma(alpha))>0 Test statistic: MSC(A)/MSW = .1667/.2556 = .65 Critical value: F(.99,4,60) = 3.6490 Do not reject Ho. For method*teacher within school (BC(A)) interaction Ho: sigma-sq(beta_gamma(alpha))=0 vs. Ha: sigma-sq(beta_gamma(alpha))>0 Test statistic: MSBC(A)/MSW = .1250/.2556 = .49 Critical value: F(.99,16,60) = 2.3148 Do not reject Ho. These tests indicate that the only significant effect, using familywise Type I error rate .05, is the main effect for method (the p-values tell us the same thing).