SHOE SHOPPING and the RELIABILITY COEFFICIENT
Dedicated to Al Bundy
A man who cares as much about good measurement as he does about his own children.
what reliability coefficient would you assign to this man?
Try this on 1. A population of male and female shoe-shoppers who have true shoe sizes between size 5 and size 15 (e.g. the small sizes are female feet translated to the male shoe-size scale).

2. Mr. Bundy measures each shopper's shoe size as either too large or too small with equal probability.

  • On a good day Mr. Bundy misses the correct shoe size by one-half size too big or one-half size too small.
  • On other days Mr. Bundy misses the correct shoe size by a full size too big or a full size too small.

In each case the shoe size measurement error has mean 0 (overall and at each level of shoe size) and is uncorrelated with actual shoe size. (Standardized testing analogs might be to the full battery and abbreviated battery versions of testing.)

3. The accuracy of shoe fitting on the good day is poor (as most wearers would notice a half-size misfitting), and on the other days the accuracy is totally unacceptable (as a full-size misfitting would presumably be unwearable).

4. The reliability coefficient for Al Bundy on the good day is .973 (better than any standardized test, even though accuracy is poor). The reliability coefficient for Al Bundy making errors of a full shoe size is .902 (comparable to many standardized tests, even though accuracy is unacceptable).
Technical notes
The distribution of actual sizes has support (5,15) with values at integer and half-integer sizes, forming a discrete rectangular distribution. To create this discrete distribution, start with a rectangular distribution on integers 0 to 20, denoted by U[0,20]. Then the distribution of actual sizes is (10 + U[0,20])/2. Note: variance of U[0,n] is n(n+2)/12.
The reliability coefficient is computed as either:
the square of the correlation between true size and observed size;
the ratio of the variance of true sizes (9.167) and the variance of observed sizes (error variances are .25 for half-size errors and
1.0 for full-size errors).
David Rogosa Stanford University September 1999