| HOME | ARCHIVE | SEARCH | TABLE OF CONTENTS |
|---|
| ||||||||||||||||||||||||||||||||
RESEARCH ARTICLE |
Psychology Department, The University of Virginia.
Address correspondence to John R. Nesselroade, Psychology Department, 102 Gilmer Hall, The University of Virginia, P.O. Box 400400, Charlottesville, VA 22904-4400. E-mail: jrn8z{at}virginia.edu
 |
Abstract |
|---|
 TOP Abstract MethodsResultsDiscussionReferences |
|---|
SCIENTIFIC interest in short-term change and fluctuation in behavior has a long history (e.g., Cattell, 1957
; Fiske & Rice, 1955; Thouless, 1936; Woodrow, 1932) and has increased substantially over the past two decades. In addition to numerous publications on substantive aspects of the topic (e.g., Butler, Hokanson, & Flynn, 1994; Eizenman, Nesselroade, Featherman, & Rowe, 1997; Hertzog, Dixon, & Hultsch, 1992; Hultsch & MacDonald, in press), treatments of pertinent methodological issues are also appearing with rapidity (e.g., Boker & Nesselroade, 2002; Browne & Nesselroade, in press; Hamaker, Dolan, & Molenaar, 2003; McArdle, 1982; McArdle & Hamagami, 2001; Molenaar, 1985; Moskowitz & Hershberger, 2002; Nesselroade & Molenaar, 1999; West & Hepworth, 1991). Evidence concerning the salience of intraindividual variability to the study of behavior and behavior development and change is becoming a compelling reminder that the prevailing emphasis on one of the seemingly most fundamental concepts in traditional differential psychology—stability of level of attributes across time—represents an oversimplification that can hinder the search for powerful and general lawful relationships (Nesselroade, 2002).
Just how important does information on intraindividual variability seem to be in the current state of behavioral inquiry? Behavioral scientists tend to answer that question in terms of magnitude of variation in one form or another (e.g., standard deviation units or effect size). Nearly 40 years ago, Bereiter (1963) reminded us that, in examining the intraindividual concept of "quotidian variability," Woodrow (1932) reported several instances in which between-occasions variability was considerably larger than between-persons variability on the same attributes. Bereiter (1963) suggested that the evidence of disparity between the magnitudes of the two kinds of differences—between persons and within persons—was an index of the relative inefficiency of individual-differences analysis as a substitute for studying intraindividual variability. When intraindividual variability in a given attribute is small, the interindividual differences in that attribute supply the bulk of the useful information, from a prediction standpoint; when intraindividual variability is large, however, they may not. Indeed, in the latter case, scores from only one occasion can yield highly misleading interindividual-differences information.
From the perspective of classical test theory, short term, intraindividual variability is a nuisance, albeit, in some cases, an "attractive" nuisance. Illustrative is Gulliksen's (1950) comment that the problem with estimating the reliability of a test from immediate test–retest correlations is that the estimates are too high because there "is no possibility for the variation due to normal daily variability to lower the correlation between forms" (p. 197). Opposing such negative sentiments are the more positive findings that short-term intraindividual variability is a valid indicator of substantively important events. Siegler (1994), for example, identified increased intraindividual variability in cognitive performance to be a "leading indicator" of impending cognitive change in children. Eizenman and associates (1997) described how individual differences in the magnitude of week-to-week intraindividual variability in perceived control were predictive of mortality a few years later. Indeed, as was pointed out by Rowe and Kahn (1987), elevated intraindividual variability on various attributes can be considered a risk factor for mortality in the elderly population.
More systematic investigation of the nature of intraindividual variability and change in a wide array of attributes is both compelling and timely. Here, we examine two distinct aspects of intraindividual, or within-person, variability in perceptual-motor performance. The first aspect on which we focus is methodological and is relevant to evaluating the representativeness of single-occasion assessment. The second aspect is theoretical and relates to whether there are age differences in moment-to-moment, or day-to-day, intraindividual variability and, if so, what are their salient features.
The Representativeness of a Single-Occasion Measurement
There are two principal ways to construe intraindividual variability. They are not mutually exclusive. The first is to regard it as essentially "noise" that, if necessary, can and should be dealt with in order to enhance any measured "signal." The second is to regard intraindividual variability as "signal" in its own right and to attempt to measure and explicate it. As was pointed out by a reviewer, one needs to distinguish between the nature of the intraindividual variability and the nature of individual differences in that intraindividual variability. The former could be "noise" in some basic sense (e.g., degraded neural capacity) even while the latter (individual differences in variability caused by differential degradation in neural capacity) could be predictive of other attributes and, therefore, definitely not "noise."
It is generally assumed that behavioral measurements reflect the following: (a) the construct of interest (e.g., ability or trait); (b) other irrelevant constructs; (c) short-term fluctuations that are due to shifts in arousal, motivation, and self-perception; and (d) otherwise unaccounted for errors of measurement. Because of the latter two kinds of influences especially, we can expect to find variability in repeated measurements taken on a given individual, and his or her scores on two different occasions may not be particularly close. If so, then which of the individual measurements is the "proper" one with which to characterize the individual? Obviously, neither may be. Trying to characterize the individual with a single score (except at a specific moment) may simply not be appropriate, however inconvenient this may be. Rather, it may be far more appropriate to use parameters of that intraindividual variability (e.g., the mean, the variance, amplitude, or latency) to characterize the individual and, by implication, differences among individuals.
If intraindividual variability is small relative to interindividual differences, it may be of only theoretical interest. However, if the former variation is large relative to the latter, and not simply error variance, it demands explanation and, as already noted, a mere examination of interindividual differences in level is not sufficient to apprehend the nature of the attribute in question.
As key as we believe it to be, however, intraindividual variability magnitude is difficult to interpret in absolute terms, so it is most usefully contrasted and compared with reference points such as other types of variability. Specifically, the magnitude of within-subject variability can be evaluated by expressing it relative to between-subject variability, or relative to the variation associated with an "interpretable" individual-difference variable such as age. At this historical point, intraindividual variability is unlikely to be important if the session-to-session variability is only a small fraction of the person-to-person variability or is equivalent to only a short span of normal aging. However, the implications for assessment are substantial if the estimates of within-person variation are large relative to between-person or across-age variation.
Age Differences in Intraindividual Variability
The second reason for an interest in intraindividual variability is theoretical, and it relates to whether there are age differences in moment-to-moment, or day-to-day, consistency. If so, what is the nature and significance of those age differences? For instance, is there a counterpart in older individuals of the findings reported by Siegler (1994) concerning increased intraindividual variability in performance indicating impending cognitive change in children? Is it possibly an early sign of decline when the level of cognitive performance fluctuates substantially in elderly persons?
Here is a more concrete illustration of the idea: Using the term "senior moments" to refer to temporary cognitive failures suggests an association between aging and cognitive fluctuation, but there has been little substantial research on this topic. An increase with age in intraindividual variability might be expected if fluctuating levels of performance are an early sign of cognitive decline. It is also possible that, for some variables, higher amounts of intraindividual variability in elderly persons are positive, rather than negative, outcomes. For example, higher variability might signify greater adaptability, less rigidity, or more creativity.
The literature reveals some attempts to rigorously evaluate the magnitude of intraindividual variability relative to interindividual differences (e.g., Kraemer & Korner, 1976), but this is not a highly visible activity of past decades. Here, we assess the magnitude of both intraindividual variability and age-related differences in performance.
|
METHODS |
|---|
|
TOPAbstractMethodsResultsDiscussionReferences |
|---|
, Salthouse and Ferrer-Caja (2003), and Salthouse and Miles (2002). Key sample characteristics are presented in Table 1.
|
Performance Tasks
Two perceptual-motor tasks were administered to the participants—a tracking task and a connections task. In the tracking task, a target (ball) moves randomly across the computer screen and the participant attempts to keep a cursor on the target by controlling a track ball with the preferred hand. Because the target and cursor positions are updated every 20 ms, continuous attention is required to perform well. The initial 20-s practice trial (not analyzed) is followed by five 70-s trials in each of the three sessions. Both tracking error and tracking lag are examined in the tracking task. The error is the root mean square error (RMSE) between target position and cursor position across the middle 60 s of the trial, and the lag is the average delay in responding to the target (see Salthouse & Miles, 2002, for further description of these variables).
In the connections task (Salthouse et al., 2000), stimuli consisting of pages of 49 circles, with each circle containing either a number or a letter, are presented to the participant. The task is to draw lines to connect the circles as rapidly as possible according to numerical order, alphabetical order, alternating numerical and alphabetical, or alternating alphabetical and numerical order. Two pages of each condition were presented at each session, with 20 s allowed on each page. The score is the number of correct connections per page. For all subsequent analyses, the two same sequence conditions (i.e., numerical and alphabetical) were treated together; so were the two alternating sequence conditions (i.e., numerical and alphabetical; alphabetical and numerical).
Procedures
Participants were administered the same tasks on three separate sessions within a two-week period. Most of the sessions for a given individual were at approximately the same time of day. As already noted, the tracking task was performed for a practice trial and five additional trials at each session. There were four trials in the same condition and four trials in the alternating condition of the connections task on each session. Thus, the data permitted the examination of intraindividual variability both within and between sessions.
|
RESULTS |
|---|
|
TOPAbstractMethodsResultsDiscussionReferences |
|---|
|
These measures of within-person variability are standard deviations, but each is a score associated with an individual. Such scores were computed for four different measures: (a) tracking error; (b) tracking lag; (c) time per item to connect items in the same numeric or alphabetic sequence; and (d) time per item to connect items in alternating sequence. We elected to use standard deviations instead of variances because they are in the same metric as the original data and the means. We did not detrend the data or make other adjustments because the systematic changes in mean levels of performance across sessions were very small compared with the total between-session variability.
Characteristics of Intraindividual Variability
WPWS and WPBS variability scores were evaluated against two references. The first is the between-person variability (the standard deviation of the mean scores of individuals in the sample). The second reference for within-person variability scores is the slope of the cross-sectional age relation. If a variable is significantly related to age, the within-person standard deviation can be divided by the age slope to estimate the number of years of cross-sectional age difference corresponding to average within-person variability. The means and standard deviations across people (usual descriptive statistics) of the means and standard deviations for each session, and among the three sessions, are presented in Table 2. Also presented in Table 2 are the between-session correlations (test–retest stability coefficients) of the session means and standard deviations. These intersession correlations are noticeably higher for the tracking measures compared with the connections measures.
|
|
All four measures of average within-session intraindividual variability are positively correlated with age, as shown in row 8 of Table 3. The WPWS intraindividual variability scores (means of the three within-session standard deviations) correlate consistently higher with age than the WPBS intraindividual variability scores. The reason may be as simple as the generally robust quality of means, but it may also reflect the tendency of the construction of the former scores to enhance stable individual differences. From a regression standpoint, these relationships represent modest to moderate performance variability–age slopes as shown in row 9, which in turn translate into relatively modest annual cross-sectional age differences in the metric of between-persons standard deviations as shown in row 10. The cross-sectional age trends are close to what are found with cognitive variables, which tend to range from –.02 to –.04 SD/year, or approximately 1 SD over 30–50 years (see, e.g., Salthouse & Ferrer-Caja, 2003; Salthouse et al., 2000).
The information in row 9 provides the basis for the second comparison reference described earlier—the slope of the cross-sectional age relation that can be used to estimate the number of years of cross-sectional age differences that correspond to average within-person variability. These values are presented in rows 11 and 12 of Table 3. Their implication is that the fluctuation for an individual from one occasion to the next is roughly equivalent to the variation apparent across people covering a span of 12 to 27 years.
We also computed a series of correlations between intraindividual variability indicators and Sex, Education, Gc (a composite measure of crystallized intelligence consisting of Wechsler Adult Intelligence Scale [WAIS] III vocabulary, see Wechsler, 1997a, and Woodcock–Johnson picture vocabulary, see Woodcock & Johnson, 1990, variables), Gf1 (a composite measure of fluid intelligence consisting of WAIS III block design, see Wechsler, 1997a, and Woodcock–Johnson analysis–synthesis, see Woodcock & Johnson, 1990, variables), Gf2 (a second fluid ability composite made up of Raven's progressive matrices, see Raven, 1962, letter series, see Noll & Horn, 1998, spatial relations, see Bennett, Seashore, & Wesman, 1997, and paper folding, see Ekstrom, French, Harman, & Derman, 1976, variables), and Mem. Mem is a separate measure of specific memory based on story memory (Wechsler, 1997b), word list recall (Wechsler, 1997b), and paired associates recall (locally developed) variables (see Salthouse et al., 2002, for additional details). The results, summarized in Table 4, are generally quite consistent, showing intraindividual variability measures positively associated with age, unrelated to sex, and negatively correlated with education and nearly all of the cognitive variables (i.e., higher ability tends to be associated with less intraindividual variability).
|
|
DISCUSSION |
|---|
|
TOPAbstractMethodsResultsDiscussionReferences |
|---|
One of the more compelling findings is that within-person variability is 37% to 53% of the between-person variability when both are expressed in standard deviation units. This substantial range reinforces the idea that any single occasion of measurement might not accurately represent the "typical" performance of an individual. Indeed, it calls into question the very notion of "typical." It challenges the value of even a "working" notion of the classical test theory conception of true score.
There is a pitfall to be avoided in considering the lack of representativeness of single occasion of measurement scores. It can be seen, for example, in the distinction between reliability of measurement and actual lability or variability in the psychological or behavioral quality being measured. An attribute that is highly labile from one occasion to another, such as state anxiety, can still be measured very reliably on any given occasion, so one needs to be careful regarding the reference implicit in such terms as "lack of precision." A given score today may be quite precise but not representative of one's score tomorrow. Thus, if we assume adequate reliability of the measurement operations themselves, the problem arises when a single occasion of measurement does not adequately represent the potential range of scores that characterizes an individual's repertoire.
As was noted earlier, if the within-person score range is very small relative to the between-person score range, intraindividual variability can be ignored without significant consequences. If the converse is true, however, intraindividual variability cannot be ignored and lack of representativeness can be a genuine threat to valid conclusions regarding substantive phenomena. This has implications for both cross-sectional and longitudinal research designs. One of the more serious implications of the lack of representativeness of single-occasion measurements for students of age effects is illustrated by the fact that the average within-person standard deviation corresponds to a cross-sectional age difference of 12 to 27 years, which renders estimates less accurate than they would otherwise be. Moreover, average within-person standard deviations correspond to greater cross-sectional age differences with increased age. Thus, there will be even less precision in the estimates of age effects for older participants.
In longitudinal research, the existence of substantial intraindividual variability can hamper the disentanglement of the effects of aging, effects of prior experience, short-term fluctuation, and measurement error. Yaffe, Browner, Cauley, Launer, and Harris (1999) studied over 8,000 women (average age 70 years) who were retested after 4 to 6 years on the Digit Symbol and Trail Making Part B tests. The average difference was -.145 SD on the Trail Making test, and -.163 SD on the Digit Symbol test. The estimates of intraindividual variability in the current study (cf. rows 6 and 7 in Table 3) are two to three times those values, thus making it very difficult to detect change at the level of the individual, or to detect correlations among changes in different variables.
Clearly, ignoring short-term variability can lead to confusing true longitudinal change with within-person variability. A design implication that can be drawn from this is the need to more systematically incorporate measurement bursts at every occasion in longitudinal assessment in order to distinguish short-term fluctuation from other influences (Nesselroade, 1991). The measurement burst design would also allow an individual's change to be calibrated relative to his or her own variability instead of between-person variability (cf. Salthouse, Kausler, & Saults, 1986).
The theoretical concern around which this article was constructed relates to the nature of age differences in moment-to-moment, or day-to-day, intraindividual variability. To what extent do these data support the suggestion by Rowe and Kahn (1987) that increased intraindividual variability should be considered a risk factor for mortality in the elderly population? This, too, is a more complicated issue than it may first appear. Age was associated with increased variability, both within and between sessions, and increased intraindividual variability was negatively associated with performance on a battery of cognitive measures. Although the present study does not address the matter directly, the relationship between magnitude of intraindividual variability and mortality may simply represent the normative cross-sectional relationship between age and mortality rather than something having to do directly with increased intraindividual variability.
Finally, an obvious limitation of the current study is that only perceptual-motor variables were studied. However, these findings add to the substantial list of variables that have exhibited notable amounts of intraindividual variability in other studies (e.g., Eizenman et al., 1997; Hertzog et al., 1992; Hultsch, Hertzog, Dixon, & Small, 1998; Li, Aggen, Nesselroade, & Baltes, 2001; Rabbitt, Osman, Moore, & Stollery, 2001; Shammi, Bosman, & Stuss, 1998). As the list of variable domains evincing substantial intraindividual variability grows, it becomes more desirable to move beyond the descriptive, documentation phase toward the building of predictive relationships and their explanations by means of theory.
|
Acknowledgments |
|---|
|
Footnotes |
|---|
Received for publication August 25, 2003. Accepted for publication November 11, 2003.
|
References |
|---|
|
TOPAbstractMethodsResultsDiscussionReferences |
|---|
This article has been cited by other articles:
 |
|
 |
|
  A. A. M. Bielak, T. F. Hughes, B. J. Small, and R. A. Dixon It's Never Too Late to Engage in Lifestyle Activities: Significant Concurrent but not Change Relationships Between Lifestyle Activities and Cognitive Speed J. Gerontol. B. Psychol. Sci. Soc. Sci., November 1, 2007; 62(6): P331 - P339. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
T. A. Salthouse, J. R. Nesselroade, and D. E. Berish Short-term variability in cognitive performance and the calibration of longitudinal change. J. Gerontol. B. Psychol. Sci. Soc. Sci., May 1, 2006; 61(3): P144 - P151. [Abstract] [Full Text] [PDF]
|
|
| ||||||||||||||||||||||||||||||||
| HOME | ARCHIVE | SEARCH | TABLE OF CONTENTS |
|---|
