Introduction
The world is full of many variables and
relationships among the variables.
When people make judgments and decisions, they do so often on the basis
of what they believe about the relationships among variables, especially when
predicting of one variable on the basis of a set of other variables (e.g.,
Anderson, 1981; Norman, 1974). For
example, with price and quality, we may know that as quality goes up, so will
price. From this, we may infer
that as price goes up, so does quality, even though it is not necessarily
true. This inference is one of bi-directionality;
and it is often true for both correlational and deterministic relationships if
there are no mitigating forces.
A second type of inference is transitivity. Given a set of three variables, A, B,
and C, if B increases with A and C increases with B, then we might infer that C
increases with A. Again, this is
not necessarily true and one can think of many counterexamples; but in many
situations transitivity is appropriate.
Most studies on the relationships of variables and
inferences about them have been in context of multivariate relationships such
as BrunswikÕs lens model (Brunswik, 1955) and HammondÕs social judgment model
(Hammond, Stewart, Brehmer, Steinmann, 1975). In these examples, one variable is deemed the criterion
variable and the rest are referred to as cues or predictor variables. But in many situations, there are only
sets of variables and imperfect knowledge about their inter-relationships. We may use this partial knowledge
concerning the inter-relationship between some variables to infer the
relationship between other variables.
In a sense, we find ourselves in a task of intuitive statistics dealing
with causal reasoning and structural equation modeling and path analysis
(Campbell & Stanley, 1966; Byrne, 1994).
In other literatures on reasoning, such as
syllogistic logic (e.g; Revlin & Mayer, 1978; Wilkins, 1928), there is a
prescriptive truth. If all A are B
and all B are C, then it is true that all A are C. But in the relationships among three variables, there is no
such prescriptive truth.
Inferences may depend on the weight of linguistic tendencies such as with
atmosphere theory (Woodworth & Sells, 1935; Chapman & Chapman, 1959) or with the ability to generate
instances (e.g., Revlis, 1975). Moreover, with sets of the variables, the
instances will depend on the context and its implication on the structure of
the system. One would expect
different systems of inference to apply for sets of variables in mathematical
systems (e.g., A = B + C), physical systems (e.g., A = velocity, B = mass, C =
force), economic systems (e.g., A = net profit, B = advertising cost, C =
market penetration), ecological systems (e.g., A = population of Species 1, B =
population of Species 2, C = population of Species 3), and social and
personality systems (e.g., A = self esteem, B = personal income, C = number of
friends).
In this pilot study, we initiate a line of research
on inference about pairwise relationships in systems of variables. We present a new task and experimental
design in which we present two relationships and then ask the participant to
infer a third relationship. Each
task is of the form:
As A increases, B
increases.
As C decreases, B
increases.
Then as B increases,
what happens to A?
- increases
- decreases
- no change
Each letter (A, B, C) is
replaced in the problem by the name of some chemical.
Experiment
Participants.
Twenty-four undergraduates participated in the study
as partial fulfillment of a course requirement. Seven were female and 16 were male (one did not indicate
gender on the demographics questionnaire ). Their ages ranged from 17 to 24 with a mean of 19.75. When asked to give a self rating of use
of overall use of computers (1=no experience, 10=very experienced) the mean was
7.13 (s.d. = 1.68). When asked to
give a self rating on the World Wide Web on the same scale, the mean was 7.39
(s.d. = 1.37).
Task Materials
Seventy-two judgment problems were generated by
varying the relationship between the first two variables and second two
variables and permuting the order of the variables. The problems are listed in Table 1. Sets of chemical names were randomly
selected from a pool and substituted for the letters A, B, and C for each
participant such that no two names in one problem began with the same
letter. The order of the 72
problems was supposed to have been randomized for each of the participant; but
due to an error in the database, they were randomized once, and presented in
the same order for all of the participants.
All problems were presented in a browser window and
displayed on a 15 inch flat panel monitor. The task was administered on Apple Macintosh computers
running MacOS X 10.2 and the Safari web browser. Responses were indicated by clicking on one of three radio
buttons for the inferred relationship.
Figure 1 shows a screen shot of one of the problems. After the participant clicked on a
relationship and clicked on the ÒContinueÓ button, the browser paged to the
next problem.
Figure 1.
Screen shot of the one of the 72 relationship inference problems.
Procedure
The participants agreed to the conditions of the
informed consent and their questions were answered. The participants filled in a pre-test questionnaire for age,
gender, and self-report of computer knowledge and use of the World Wide
Web. The judgment task was then
described to the participants.
They were told that they were to help a geologist make inferences about
the relationships among chemicals in rocks. They would be told two relationships and then had to infer
the third. When they were finished
they were asked to take a test of spatial visualization ability called the VZ2
(Ekstrom, French, & Harmon, 1976).
This took six minutes and was administered in the browser window. After the test, they were debriefed and
asked if they had any questions.
Results
Due to response omissions, half of the 22 participants
had between one and five missing data points. The missing responses comprised 1.5% of all of the
data. Consequently, the data were
analyzed two different ways.
First, all 22 participants were included, and missing data points were
treated as "no change" responses. This introduces a slight bias in favor of "no
change," but does not introduce bias for or against "increases"
or "decreases." Second,
only the 11 subjects with no missing data points were considered. This sample had no bias, but less
power. Both samples were tested
using a repeated-measures ANOVA.
The factors of the ANOVA were the "+-0" condition, the
"ABC" condition, and the interaction. The "+-0" condition indicated the value of the
first two relationships in the question.
The "ABC" condition indicated the positions of the element names
in the second and third parts of the question. The effects for both conditions and the interaction for both
samples were significant,
(p < .05). The "+-0"
effect size was larger than the ÒABCÓ effect.
Although the missing data responses were counted as
"no change" responses for the repeated-measures ANOVA, it is easier
to compare the samples by simply excluding participants with the missing
data. For the following
comparisons, the 22-subject sample simply had the missing data points
removed. For the sake of
simplicity, all statistics quoted below and the values in Table 1 are from the
unbiased 11-subject sample. Where
the 22-subject sample differs more than a few points, this will be noted. All percentages are rounded to the
nearest 1%.
Let us first look at the "+-0" factor. Each condition will be indicated with
two signs, the first indicating the value of the first relationship, and the
second, the value of the second relationship. (The response was the participantÕs guess as to the value of
the third relationship.) For
instance, +- is the condition where the first relationship is
"increases," and the second, "decreases." A "no change" is indicated by
a 0.
If the first two relationships had the same non-zero
value, the responses were overwhelmingly "increases." The "++" condition had 97%
"increases" responses, and the "--" condition had 81%
"increases" responses.
If the first two relationships had different non-zero
value, the responses were overwhelmingly "decreases." The "+-" condition had 93%
"decreases" responses, and the "-+" condition had 83%
"decreases" responses.
If one of the first two relationships was
"doesn't change," the majority of responses were "doesn't
change," with the remaining responses split, but always favoring the same
value as the non-zero relationship. Condition "+0" had 83%
"doesn't change," "-0" had 75%, "0+" had 83%, and
"0-" had 76%. The
largest imbalance was for "0+", which had 11% "increases,"
and 6% "decreases." The
largest imbalance favoring decreases was "-0", 14% compared to 11%
increases.
Finally, when the first two relationships were both
"doesn't change," the majority of responses were "doesn't
change," but only 58% (just 53% in the full sample.) A large minority of responses (36%)
were "increases." This was
by far the condition with the weakest dominant answer as shown in Figure 2.
The differences among the "ABC" conditions
were not as large, but were also significant. These conditions are indicated by two pairs of two
letters. They represent the order
of the chemical names in the second and third parts of the question. The first two chemicals are always
"A" and "B."
The question asks about the value of the relationship between the last
pair of chemicals. If the question
first states a relationship between "A" and "B," then
between "A" and "C," and then asks the value of the
relationship between "B" and "C," this would be indicated
as "AC BC." For each
response, the most extreme conditions are listed.
Figure 2.
Proportions of participants inferring each type of relationship for the
ÒDoesnÕt change, DoesnÕt changeÓ problem.
In the "CB AC" condition, 39% of responses
were "increases" as shown in Figure 3. This was even higher, 43%, in the full sample. For the "BC AC" and "CA
CB" conditions, the percentage of "increases" responses was
24%. These were also somewhat
higher in the full sample.
Figure 3.
Proportions of participants inferring each type of relationship for the
ÒCD ACÓ problem.
In the "CA BC" condition, 33% of the
responses were "decreases."
The "AC BC" and "CB CA" conditions had only 22%
"decreases" responses.
In the "BC AC" condition, just under 50% of
the responses were "doesnÕt change." The "CB AC" condition had 34% "doesnÕt change"
responses.
Finally, the average VZ2 score for the participants
was 11.33 (s.d. = 4.16). We had
planned a median split on the sample of 22 to generate a Hi VZ2 Group and a Lo
VZ2 Group; however, due to a
coding error, it was impossible to identify which scores went with which
participants.
Discussion
The results are encouraging in that clear patterns
were evident in the data indicating that participants were not merely
responding randomly but were making consistent inferences on the basis of some
principles or strategies. Figure 4 shows a diagram of the predominate patterns
of inference.
Figure 4. Diagrams of the predominate patterns of
inference for the six relational structures. Red arrows show the inferred relationship.
Clearly, the most dominant pattern was for positive
relationships (e.g., As A increases, B increases; and as B increases, C
increases.) to lead to a positive
inference (e.g., As A increases, C increases). Moreover, the order could be changed and participants still
made the same positive inference suggesting that they were inferring
bi-directional and transitive relationships.
The second clear pattern was for two negative
relationships to result in a positive relationship between the third pair. The inference is that if both are in
the same negative direction, the two variables must be moving in the same
(positive) direction with each other.
Mixed relationships on the other hand result in the
inference of a negative relationship.
This is in line with the inference above. If one does one way and the other in the opposite direction,
then the two variables must be negatively related.
When there is a ÒdoesnÕt changeÓ for one of the
relationships, it nullifies the effect of the other (whether positive or
negative) and results in the inference of no relationship between the two
variables.
Finally, if both relationships are ÒdoesnÕt change,Ó
there is a split with the majority inferring that there is no relationship, but
with a substantial minority inferring a positive relationship between the two
variables. It is not clear at this
point, what the basis is for this inference.
Limitations
Several problems in the design of this pilot study
became evident. The first two
problems had to do so with technical software issues. First, the order of the 72 items should have been randomized
for each participant to eliminate order effects. Second, the design of the software allowed the participants
to omit or skip responses. This
resulted in a loss of data and resulting problems with the statistical
analysis. These two problems will
be corrected in subsequent experiments.
The third problem was that participants were forced to
answer with only three options: increase, decrease, and no change. They were not allowed to say that they
did not know. This may have forced
them to guess in situations where they were not able to make an inference. On the other hand, if they are allowed
to respond, ÒdonÕt knowÓ, they may over use this option. An experiment is needed to test what
will happen with the presence or absence of the ÒdonÕt knowÓ option.
Fourth, only one context or scenario was tested in
this study; namely, the chemical composition of rocks. Subsequent experiments should vary the
type of system of variables to see if inferences change.
Finally, in this study we used the VZ2 test of spatial
visualization ability. In
subsequent research, it might be preferable to use a test of formal reasoning
such as a test of syllogistic reasoning.
Conclusion
If relationships between variables are inferred with
some consistency from other relationships among variables in a set, these
inferences can be used to make judgments and predictions. In many judgment
situations, the most important thing is to know the direction of the
relationship between the cue and the criterion (e.g., Dawes). These directions may come from
inferences using information about other relationships in the set of
variables.
This study is an initial attempt to understand these
inferences and serves as the starting point for the development of a theory of
functional processing.
Acknowledgments
Thanks to Robert Rowsome and Melissa Tortoriello. This work as funded in part from a
grant from the U.S. Census Bureau, Statistical Research Division, Grant
50YABC166008.
References
Anderson,
N. H. (1981). Foundations of information integration theory. New York, NY: Academic Press.
Brunswik,
E. (1955). Representative design
and probabilistic theory in a functional psychology. Psychological Review,
62, 193-217.
Byrne,
B. (1994). Structural equation
modeling with EQS and EQS/Windows:
Basic concepts, applications, and programming. Thousand
Oaks, CA: Sage.
Campbell,
D. T., & Stanley, J. C. (1966). Experimental and quasi-experimental
designs for research. Boston, MA:
Houghton Mifflin.
Chapman,
L. J. & Chapman, J. P. (1950). Atmosphere effect reexamined. Journal of
Experimental Psychology, 58, 220-226.
Ekstrom,
R. B., French, J. W., & Harmon, H. H. (1976). Manual for kit of
factor-referenced cognitive tests. Princeton,
NJ: Educational Testing Service.
Hammond,
K. R., Stewart, T. R., Brehmer, B., Steinmann, D. O. (1975). Social judgment theory. In Kaplan, M.
F. & Schwartz, S. (eds.) Human judgment and decision processes (pp. 271-312). New York, NY: Academic Press.
Norman, K. L. (1974). Rule learning in a stimulus
integration task. Journal of
Experimental Psychology, 103, 941-952.
Revlin, R., & Mayer, R.(ed). (1978). Human Reasoning. Wahington, DC: Winston/Wiley.
Revlis, R. (1975). Two models of syllogistic reasoning: Feature selection and conversion. Journal
of Verbal Learning and Verbal Behavior, 14, 180-195.
Wilkins, M. C. (1928). The effect of changed material
on ability to do formal syllogistic reasoning. Archives of Psychology, No. 102.
Woodworth, R. S. & Sells, S. B. (1935). An
atmosphere effect in formal syllogistic reasoning. Journal of Experimental
Psychology, 18, 451-460.
.