EVALUATION OF MODELS

Different models can be applied to given set of data to develop alternative scales. These models and their associated scales sometimes lead to different substantive conclusions. Two different models might produce scales that are not linearly related. One model might suggest that data do not even possess ordinal properties, whereas another might indicate they clearly form an interval scale. 

A criteria to choose the model is as follow:

1.                  The intuitive appeal of a scaling model provides one criterion for “reasonable. ” Although the data of science must be public, a scientist’s intuition plays an indisputable role in the gathering and analysis of data. Looked at in one way, a measurement model is nothing more than an explicitly defined hunch that particular operations on data will be useful. In particular, we suggest that psychologists lean toward measurement models that are most analogous to the measurement of simple physical attribute, e.g. length.

2.                  Another aspect of “reasonable” is that one should exploit what is already known about similar data. For example, power functions are well known to describe relations between physical and perceived intensity. On the negative side, some models assume that individual test item responses are highly reliable; yet, a wealth of evidence shows that such responses usually are highly unreliable.

3.                  Preliminary analyses often provide cues about the usefulness of a scale. If the scale values for objects or persons are markedly affected by slight procedural differences, the scale will probably not work well in practice. There are, for example numerous ways in which subjects can judge weight. If two similar appearing approaches yield very different intervals of judged weight, either or both methods are suspect. Conversely, different models that yield similar results provide converging operations (Garner, Hake & Erikson, 1956) that mutually strengthen the confidence one may have about any given method. “Triangulation” is anther common term used to describe this.

4.                  Another important type of evidence is the magnitude of measurement error in using a particular scale. A scale that yields a great deal of measurement error cannot possible be useful.