Linear Models and Analysis of Variance:

CONCEPTS, MODELS, AND APPLICATIONS

 

Volume II

 

 

 

 

 

First Edition

 

 

 

 

 

 

 

 

David W. Stockburger

 

Southwest Missouri State University

 

 

 

 

 

 

@Copyright 1993


 

                                           TABLE OF CONTENTS

 

Title                                                                                            Page

EXPERIMENTAL DESIGNS.............................................................................. 132

Notation.................................................................................................... 133

Kinds of Factors........................................................................................ 134

Treatment...................................................................................... 134

Group Factors............................................................................... 135

Trials Factors................................................................................ 135

Blocking........................................................................................ 135

Unit Factors.................................................................................. 137

Error Factors................................................................................. 137

Fixed and Random Factors........................................................................ 137

Fixed Factors................................................................................ 137

Random Factors............................................................................ 138

Relationships Between Factors................................................................... 139

Crossed........................................................................................ 139

Nested.......................................................................................... 140

An Example Design................................................................................... 142

A Second Example Design......................................................................... 144

A Third Example Design............................................................................ 146

Determining the Number of Subjects and Measures per Subject................. 148

Setting up the Data Matrix......................................................................... 148

A Note of Caution..................................................................................... 149

 

One Between Group ANOVA.............................................................................. 150

Why Multiple Comparisons Using t-tests is NOT the Analysis of Choice..... 150

The Bottom Line - Results and Interpretation of ANOVA.......................... 151

HYPOTHESIS TESTING THEORY UNDERLYING ANOVA.............. 153

The Sampling Distribution Reviewed.............................................. 153

Two Ways of Estimating the Population Parameter σX˛................... 154

The F-ratio and F-distribution.................................................................... 157

Non-significant and Significant F-ratios....................................................... 159

Similarity of ANOVA and t-test................................................................. 162

EXAMPLE OF A NON-SIGNIFICANT ONE-WAY ANOVA.............. 164

EXAMPLE OF A SIGNIFICANT ONE-WAY ANOVA........................ 164

USING MANOVA.................................................................................. 164

The Data....................................................................................... 165

Example Output............................................................................. 166

Dot Notation............................................................................................. 168

POST-HOC Tests of Significance.............................................................. 170

Example SPSS Program using MANOVA................................................. 173

Interpretation of  Output............................................................................ 174

Graphs of Means....................................................................................... 175

The ANOVA Summary Table................................................................... 176

Main Effects.................................................................................. 177

Simple Main Effects....................................................................... 178

Interaction Effects.......................................................................... 178

Example Data Sets, Means, and Summary Tables...................................... 180

No Significant Effects..................................................................... 180

Main Effect of A............................................................................ 181

Main Effect of B............................................................................ 182

AB Interaction............................................................................... 183

Main Effects of A and B................................................................ 184

Main effect of A, AB Interaction ................................................... 185

Main Effect of B, AB Interaction.................................................... 186

Main Effects of A and B, AB Interaction........................................ 187

No Significant Effects..................................................................... 188

Dot Notation Revisited.............................................................................. 189

 

Nested Two Factor Between Groups Designs B(A)............................................... 192

The Design................................................................................................ 192

The Data................................................................................................... 192

SPSS commands....................................................................................... 193

The Analysis.............................................................................................. 193

The Table of Means....................................................................... 194

Graphs.......................................................................................... 194

The ANOVA Table....................................................................... 195

Interpretation of Output............................................................................. 195

Similarities to the A X B Analysis............................................................... 195

 

Contrasts, Special and Otherwise........................................................................... 197

Definition................................................................................................... 197

Sets of Contrasts....................................................................................... 198

Orthogonal Contrasts..................................................................... 198

Non-orthogonal Contrasts............................................................. 199

Sets of Orthogonal Contrasts......................................................... 199

Finding Sets of Orthogonal  Contrasts............................................ 199

The Data................................................................................................... 201

SPSS commands....................................................................................... 202

The Analysis.............................................................................................. 202

The Table of Means....................................................................... 202

The ANOVA table........................................................................ 203

Interpretation of Output............................................................................. 203

Constants.................................................................................................. 203

Contrasts, Designs, and Effects.................................................................. 205

Non-Orthogonal Contrasts........................................................................ 208

Smaller than Total Sum of Squares................................................. 208

Larger than Total Sum of Squares.................................................. 209

Standard Types of Orthogonal Contrasts.................................................... 209

DIFFERENCE.............................................................................. 210

SIMPLE....................................................................................... 210

POLYNOMIAL........................................................................... 210

Conclusion................................................................................................ 213

 

ANOVA and Multiple Regression.......................................................................... 214

ONE FACTOR ANOVA......................................................................... 214

ANOVA and Multiple Regression.................................................. 214

Example........................................................................................ 215

Example Using Contrasts............................................................... 216

Dummy Coding............................................................................. 216

ANOVA, Revisited....................................................................... 218

TWO FACTOR ANOVA........................................................................ 219

Example........................................................................................ 220

Example Using Contrasts............................................................... 220

Regression Analysis using Dummy Coding...................................... 221

Conclusion................................................................................................ 223

 

Unequal Cell Frequencies...................................................................................... 225

Equal Cell Frequency - Independence of Effects......................................... 225

Unequal Cell Frequency - Dependent Effects............................................. 226

Solutions for Dealing with Dependent Effects.............................................. 229

UNEQUAL CELL SIZES FROM A MULTIPLE REGRESSION VIEWPOINT        231

REGRESSION ANALYSIS OF UNEQUAL N ANOVA........................ 234

RECOMMENDATIONS......................................................................... 237

 

Subjects Crossed With Treatments  S X A............................................................. 238

The Design................................................................................................ 238

The Data................................................................................................... 239

SPSS commands....................................................................................... 239

The Correlation Matrix.................................................................. 240

The Table of Means....................................................................... 240

Graphs.......................................................................................... 241

The ANOVA Table....................................................................... 241

Interpretation of Output................................................................. 242

Additional Assumptions for Univariate S X A Designs................................ 244

SS, MS, and Expected Mean Squares (EMS)................................ 245

 

Subjects Crossed With Two Treatments -S X A X B............................................. 249

The Design................................................................................................ 249

The Data................................................................................................... 249

SPSS commands....................................................................................... 250

The Correlation Matrix.................................................................. 251

The Table of Means....................................................................... 251

Graphs.......................................................................................... 253

The ANOVA Table....................................................................... 253

Interpretation of Output................................................................. 253

Additional Assumptions for Univariate S X A X B Designs......................... 258

SS, MS, and Expected Mean Squares (EMS)................................ 258

 

Mixed Designs - S ( A ) X B................................................................................. 261

The Design................................................................................................ 261

The Data................................................................................................... 261

SPSS commands....................................................................................... 262

The Table of Means....................................................................... 263

Graphs.......................................................................................... 264

Interpretation of Output................................................................. 264

Expected Mean Squares (EMS).................................................... 267

 

Three Factor ANOVA.......................................................................................... 268

Effects....................................................................................................... 268

Main Effects.................................................................................. 269

Two-Way Interactions................................................................... 269

Three-Way Interaction................................................................... 271

Additional Examples.................................................................................. 272

All Effects Significant..................................................................... 272

 Example 3 - B, AC, and BC......................................................... 274

Two More Examples..................................................................... 275

Expected Mean Squares............................................................................ 276

Tests of Significance.................................................................................. 279

Error Terms................................................................................... 279

SPSS Output................................................................................. 279

Examples................................................................................................... 279

S ( A X B X C)............................................................................. 279

S ( A X B ) X C............................................................................ 280

S ( A ) X B X C............................................................................ 280

 

BIBLIOGRAPHY................................................................................................. 287

 

INDEX................................................................................................................. 289

 

 





                                                             Chapter

                                                                8

 

 

 

            EXPERIMENTAL DESIGNS

 

Experimental design refers to the manner in which the experiment was set up. Experimental design includes the way the treatments were administered to subjects, how subjects were grouped for analysis, how the treatments and grouping were combined. 

 

In ANOVA there is a single dependent variable or score.  In Psychology the dependent measure is usually some measure of behavior.  If more than one measure of behavior is taken, multivariate analysis of variance, or MANOVA, may be the appropriate analysis.  Because the ANOVA model breaks the score into component parts, or effects, which sum the total score, the one must assume the interval property of measurement for this variable.  Since in real life the interval property is never really met, one must be satisfied that at least an approximation of an interval scale exists for the dependent variable.  To the extent that this assumption is unwarranted, the ANOVA hypothesis testing procedure will not work.

 

In ANOVA there is at least one independent variable or factor.  There are different kinds of factors;  treatment, trial, blocking, and group.  Each will be discussed in the following section.  All factors, however, have some finite number of different levels.  Each level is the same in either some quality or quantity.   The only restriction on the number of levels is that there are fewer levels than scores, although in practice one seldom sees more than ten levels in a factor unless the data set is very large.  It is not necessary that the independent variables or factors be measured on an interval scale.  If the factors are measured on an (approximate) interval scale, then some flexibility in analysis is gained.  The continued popularity of ANOVA can partially be explained by the lack of the necessity of the interval assumption for the factors.

 


Notation     

 

Every writer of an introductory, intermediate, or advanced statistics text has his or her own pet notational system.  I have taught using a number of different systems and have unabashedly borrowed the one to be described below from Lee (1975).  In my opinion it is the easiest for students to grasp.

 

The dependent variable or score will be symbolized by the letter X.  Subscripts (usually multiple) will be tagged on this letter to differentiate the different scores.  For example, to designate a single score from a group of scores a single subscript would be necessary and the symbol Xs could be used.  In this case X1 would indicate the first subject, X2 the second, X3 the third, and so forth.

 

When it is desired to indicate a single score belonging to a given combination of factors, multiple subscripts must be used.  For example, Xabs would describe a given score for a combination of a and b.  Thus, X236 would describe the sixth score when a=2 and b=3.  Another example, X413, would describe the third score when a=4 and b=1.

 

Bolded capital letters will be used to symbolize factors.  Example factors are A, B, C, ..., Z.  Some factor names are reserved for special factors.  For example, S will always refer to  the subject factor, E will always be the error factor, and G will be the group factor.

 

Small letters with a numerical subscript are used to indicate specific levels of a factor.  For example c1 will indicate the first level of factor C, while cc will indicate a specific level of factor C, but the level is unspecified.  The number of levels of a factor are given by the unbolded capital letter of that factor.  For example there are 1, 2, ..., C  levels of factor C.

 


In an example experiment, let X, the score, be the dollar amount after playing WindowsTM Solitaire for an hour.  In this experiment the independent variable (factor) is the amount of practice, called factor A.  Let nine subjects each participate in one of four (A=4) levels of training.  The first level, a1, consists on no practice, a2 = one hour of practice, a3 = five hours of practice, and a4 = twenty hours of practice.  A given score (dollar amount) would be symbolized by Xas, where X35 would be the fifth subject in the group that received five hours of practice.

 

Kinds of Factors

 

Treatment

 

Treatments will be defined as quantitatively or qualitatively different levels of experience.  For example, in an experiment on the effects of caffeine, the treatment levels might be exposure to different amounts of caffeine, from none to .0375 milligrams.  In a very simple experiment there are two levels of treatment, none, called the control condition, and some, called the experimental condition. 

 

Treatment factors are usually the main focus of the experiment.  A treatment factor is characterized by the following two attributes (Lee, 1975):

 

1.         An investigator could assign any of his experimental subjects to any one of the levels of the factor.

 

2.         The different levels of the factor consist of explicitly distinguishable stimuli or situations in the environment of the subject.

 

In the solitaire example, practice time would be a treatment factor if the experimenter controlled the amount of time that the subject practiced.  If subject's came to the experiment having already practiced a given amount, then the experimenter could not arbitrarily or randomly assign that subject to a given practice level.  In that case the factor would no longer be considered a treatment factor.

 


In an experiment where subjects are run in groups, it sometimes is valuable to treat each group as a separate level of a factor.  There might be, for example, an obnoxious subject who affects the scores of all other subjects in that group.  In this case the second attribute would not hold and the factor would be called a group factor.

 

 

Group Factors

 

As described above, a group factor is one in which the subjects are arbitrarily assigned to a given group which differs from other groups only in that different subjects are assigned to it.  If each group had some type of distinguishing feature, other than the subjects assigned to it, then it would no longer be considered as a group factor.  If a group factor exists in an experimental design, it will be symbolized by G.

 

Trials Factors

 

If each subject is scored more than once under the same condition and the separate scores are included in the analysis, then a trials factor exists.  If the different scores for a subject are found under different levels of a treatment, then the factor would be called a treatment factor rather than a trials factor.  Trials factors will be denoted by T.

 

Trials factors are useful in examining practice or fatigue effects.  Any change in scores over time may be attributed to having previously experienced similar conditions.

 

Blocking

 


If subjects are grouped according to some pre-existing  subject similarity, then that grouping is called a blocking factor.  The experimenter has no choice but to assign the subject to one or the other of the levels of a blocking factor.  For example, gender (sex) is often used as a blocking factor.  A subject enters the experiment as either a male or female and the experimenter may not arbitrarily (randomly) assign that individual to one gender or the other.

 

Because the experimenter has no control over the assignment of subjects to a blocking factor, causal inference is made much more difficult.  For example, if in the solitaire experiment, the practice factor was based on a pre-existing condition, then any differences between the groups may be due either to practice or to the fact that some subjects liked to play solitaire, were better at the game and thus practiced more.  Since the subjects are self-selected, it is not possible to attribute the differences between groups to practice, enjoyment of the game, natural skill in playing the game, or some other reason.  It is possible, however, to say that the groups differed.

 

Even though causal inference is not possible, blocking factor can be useful.  A factor which  accounts for differences in the scores adds power to the experiment.  That is, a blocking factor which explains some of the differences between scores may make it more likely to find treatment effects.  For example,  if males and females performed significantly different in the solitaire experiment, it might be useful to include sex as a blocking factor because differences due to gender would be included in the error variance otherwise.

 

In other cases blocking factors are interesting in their own right.  It may be interesting to know that freshmen, sophomores, juniors, and seniors differ in attitude toward university authority, even though causal inferences may not be made.

 

In some cases the pre-existing condition is quantitative, as in an IQ score or weigh.  In these cases it is possible to use a median split where the scores above the median are placed in one group and the scores below the median are placed in another.  Variations of this procedure divide the scores into three, four, or more approximately equal sized groups.  Such procedures are not recommended as there are better ways of handling such data (Edwards, 1985).


 

Unit Factors

 

The unit factor is the entity from which a score is taken.  In experimental psychology, the unit factor is usually a subject (human or animal), although classrooms, dormitories, or other units may serve the same function.  In this text, the unit factor will be designated as S, with the understanding that it might be some other type of unit than subject.

 

Error Factors

 

The error factor, designated as E, is not a factor in the sense of the previous factors and is not included in the experimental design.  It is necessary for future theoretical development.

 

Fixed and Random Factors

 

Each factor in the design must be classified as either a fixed or random factor.  This is necessary in order to find the correct error term for each effect.  The MANOVA program in SPSS does not require that the user designate the type for each factor.  If the user is willing to accept the program defaults, which are correct in most cases, no problem is encountered.  There are situations, however, where the program defaults are incorrect and additional coding is necessary to do the correct hypothesis tests.

 

Fixed Factors

 

A factor is fixed if  (Lee, 1975)

 

1.         The results of the factor generalize only to the levels that were included in the experimental design.  The experimenter may wish to generalize to other levels not included in the factor, but it is done at his or her own peril.

 


2.         Any procedure is allowable to select the levels of the factor.

 

3.         If the experiment were replicated, the same levels of that factor would be included in the new experiment.

 

Random Factors

 

A factor is random if

 

1.         The results of the factor generalize to both levels that were included in the factor and levels which were not.  The experimenter wishes to generalize to a larger population of possible factor levels.

 

2.         The levels of the factor used in the experiment were selected by a random procedure.

 

3.         If the experiment were replicated, different levels of that factor would be included in the new experiment.

 

In many cases an exact determination of whether a factor is fixed or random is not possible.  In general, the subjects (S) and groups (G) factors will always be a random factor and all other factors will be considered fixed.  The default designation of MANOVA will set the subjects factor as random and all other factors as fixed.

 

Some reflection on the assumption of a random selection of subjects may cause the experimenter to question whether it is in fact a random factor.  Suppose, as often happens, subjects volunteered to participate in the experiment.  In this case the assumptions underlying the ANOVA are violated, but the procedure is used anyway.  Seldom, if ever, will all the assumptions necessary to do an ANOVA be completely satisfied.  The experimenter must examine how badly the assumptions were violated and then make a decision as to whether or not the ANOVA is useful.


In general, when in doubt as to whether a factor is fixed or random, consider it fixed.  One should never have so much doubt, however, as to consider the subjects factor as a fixed factor.

 

Relationships Between Factors

 

The following two relationships between factors describe a large number of useful designs.  Not all possible experimental designs fit neatly into categories described by the following two relationships, but most do.

 

Crossed

 

When two factors are crossed, each level of each factor appears with each level of the other factor.  A crossing relationship is indicated by an "X". 

 

For example, consider two factors, A and B, were A is gender (a1 = Females, a2 = Males) and B is practice (b1 = none, b2 = one hour, b3 = five hours, and b4 = twenty hours).  If gender was crossed with practice, A X B, then both males and females would participate in all four levels of practice.  There would be eight groups of subjects including:  ab11, females who had no practice, ab12, females who had one hour of practice, and so forth to ab24, males who practiced twenty hours.  An additional factor may be added to the design, say handedness (C), where c1 = right handed and c2 = left handed.  If the design of the experiment was A X B X C, then there would be sixteen groups, including abc231, left-handed males who practiced five hours.

 


If subjects (S) are crossed with treatments (A), S X A, each subject sees each level of the treatment conditions.  In a very simple experiment such as the effects of caffeine on alertness (A), each subject would be exposed to both a caffeine condition (a1) and a no caffeine condition (a2).  For example, using the members of a statistics class as subjects, the experiment might be conducted as follows.  On the first day of the experiment the class is divided in half with one half of the class getting coffee with caffeine and the other half getting coffee without caffeine.  A measure of alertness is taken for each individual, such as the number of yawns during the class period.  On the second day the conditions are reversed, that is, the individuals who received coffee with caffeine are now given coffee without and vice-versa. 

 

The distinguishing feature of crossing subjects with treatments is that each subject will have more than one score.  This feature is sometimes used in referring to this class of designs as repeated measures designs.  The effect also occurs within each subject, thus these designs are sometimes referred to as within subjects designs. 

 

Crossing subjects with treatments has two advantages.  One, they generally require fewer subjects, because each subject is used a number of times in the experiment.  Two, they are more likely to result in a significant effect, given the effects are real.  This is because the effects of individual differences between subjects is partitioned out of the error term.

 

Crossing subjects with treatments also has disadvantages.  One, the experimenter must be concerned about carry-over effects.  For example, individuals not used to caffeine may still feel the effects of caffeine on the second day, when they did not receive the drug.  Two, the first measurements taken may influence the second.  For example, if the measurement of interest was score on a statistics test, taking the test once may influence performance the second time the test is taken.  Three, the assumptions necessary when more than two treatment levels are employed in a crossing subjects with treatments  may be restrictive. 

 

When a factor is a blocking factor, it is not possible to cross that factor with subjects.  It is difficult to find subjects for a S X A design where A is gender.  I generally will take points off if a student attempts such a design.

 

Nested

 


Factor B is said to be nested within factor if each meaningful level of factor B occurs in conjunction with only one level of A.  This relationship is symbolized a B(A), and is read as "B nested within A".  Note that B(A) is considerably different from A(B).  In the latter, each meaningful level of A would occur in one and only one level of B.   These types of  designs are also designated as hierarchical designs in some textbooks.

 

A B(A) design occurs, for example, when the first three levels of factor B (b1 ,b3, and b3) appear only under level a1 of factor A and the next three levels of B  (b4 ,b5, and b6)  appear only under level a2 of factor A.  Depending upon the labelling scheme, b4 ,b5, and b6 may also be called b1 ,b3, and b3, respectively.  It is understood by the design designation that the b1 occurring under a1 is different from the b1 occurring under a2.

 

Nested or hierarchical designs can appear because many aspects of society are organized  hierarchically.  For example within the university, classes (sections) are nested within courses, courses are nested within departments, departments within colleges, and colleges within the university.. 

 

In experimental research it is also possible to nest treatment conditions within other treatment conditions.  For example, suppose a researcher was interested in the effect of diet on health in hamsters.  One factor (A) might be a high cholesterol (a1) or low cholesterol (a2) diet.  A second factor (B) might be type of food, peanut butter (b1), cheese (b2), red meat (b3), chicken (b4), fish (b5), or vegetables(b6).  Because type of food may be categorized as being either high or low in cholesterol, a B(A) experimental design would result.  Chicken, fish, and vegetables would be relabelled as b1 ,b3, and b3, respectively, but it would be clear from the experimental design specification that peanut butter and chicken, cheese and fish, and red meat and vegetables, were qualitatively different, even though they all share the same label.

 


While any factor may possibly be nested within any other factor, the critical nesting relationship is with respect to subjects.  If S is nested within some combination of other factors, then each subjects appear under one, and only one, combination of factors within which they are nested.  These effects are often called the Between Subjects effects.   If S is crossed with come combination of other factors, then each subject see all combinations of factors with which they are crossed.  These effects are referred to as Within Subjects effects.

 

As mentioned earlier subjects are necessarily nested within blocking factors.  Subjects are necessarily nested within the effects of gender and current religious preference, for example.

 

Treatment factors, however, may be nested or crossed with subjects.  The effect of caffeine on alertness could be studied by dividing the subjects into two groups, with one receiving a beverage with caffeine and one group not.  This design would nest subjects with caffeine and be specified as S(A), or simply A, as the S is often dropped when the design is completely between subjects. 

 

If subjects appeared under both caffeine conditions, receiving caffeine on one day and no caffeine on the other, then subjects would be crossed with caffeine.  The design would be specified as S X A.  In this case the S would remain in the design.

 

An Example Design

 

A psychologist (McGuire, 1993) was interested in studying adults' memory for medical information presented by a videotape.  She included one-hundred and four participants in which sixty-seven  ranged in age from 18 to 44 years and thirty seven ranged in age from 60 to 82 years.  Participants were randomly assigned to one of two conditions, either an organized presentation condition or an unorganized  presentation condition.  Following observation of the videotape, each participant completed an initial recall sequence consisting of free-recall and probed recall retrieval tasks.   A probed recall is like a multiple-choice test and a free-recall is like an essay test.  Following a one-week interval, participants completed the recall sequence again.

 


This experimental design provides four factors in addition to subjects (S).  The age factor (A) has two level a1=young and a2=old and would necessarily be a blocking factor.  The type of videotape factor (B) would be a treatment factor and would consist of two levels b1=organized and b2=unorganized.  The recall method factor (C) would be a form of trials factor and would have two levels c1=free-recall and c2=probed recall.  The forth factor (D) would be another trials factor where d1=immediate and d2=one week delay.

 

Each level of B appears with each level of A, thus A is crossed with B.  Since each subject appears in one and only one combination of A and B, subjects are nested within A X B.  That is, each subject is either young or old and sees either an organized or unorganized videotape.  The design notation thus far would be S ( A X B ).

 

Each type of recall (C) was done by each subject at both immediate and delayed intervals (D).  Thus subjects would be crossed with recall method and interval.  The complete design specification would be S ( A X B ) X C X D.   In words this design would be subjects nested within A and B and crossed with C and D.

 

In preparation for entering the data into a data file, the design could be viewed in a different perspective.  Listing each subject as a row and each measure as a column, the design would appear as follows:


 

     Immediate   Week Later

Age      Videotape        Subject Free     Probed                Free       Probed

 

S1

Organized         S2

Young              ...

Unorganized     S1

...

S1

Organized         S2                    

Old                              ...

Unorganized     S1

...

 

In this design, two variables would be needed.  One  to classify each subjects as either young or old, and one to document which type of videotape the subject saw.  In addition to the classification variables, each subject would require four variables to record the two types of measures taken at the two different times.

 

A score taken from the design presented above could be represented as Xabscd.  For example, the immediate probed test score taken from the third subject in the old group who viewed an organized videotape would be X21312.

 

A Second Example Design

 


The Lombard effect is a phenomenon in which a speaker or singer involuntarily raises his or her vocal intensity in the presence of high levels of sound.  In a study of the Lombard effect in choral singing (modified from Tonkinson, 1990), twenty-seven subjects, some experienced choral singers and some not,  were asked to sing the national anthem along with a choir heard through headphones.  The performances were recorded and vocal intensity readings from three selected places in the song were obtained from a graphic level recorder chart.   Each subject sang the song four times:  with a none, or a soft, medium, or loud choir accompaniment.  After some brief instructions to resist increasing vocal intensity as the choir increased, each subject again sang the national anthem four times with the four different accompaniments.  The order of accompaniments was counterbalanced over subjects.

 

In this design, there would be four factors in addition to subjects.  Subjects would be nested within experience level (A), with a1=inexperienced and a2=experienced choral singers.   This factor would be a blocking factor.   Subjects would be crossed with instructions (B), where b1=no instructions and b2=resist Lombard effect.  In addition, subjects would be crossed with accompaniment (C) and place in song  (D).  The accompaniment factor would include four levels c1=soft, c2=medium, c3=loud, and c4=none.  This factor would be considered a treatment factor.  The place in song factor could be considered a trial factor and would have three levels.

 

The experimental design could be written as S ( A ) X B X C X D.  In words, subjects were nested within experience level and crossed with instructions, accompaniment, and place in song.  In this design, one variable would be needed for the classification of each subject and twenty-four variables would be needed for each subject, one for each combination of instructions, accompaniment, and place in song.  The design could be written:

 


               No Instructions                   Resist Lombard Effect

        Soft  Medium  Loud   None   Soft  Medium  Loud   None  

Exp  S  1 2 3  1 2 3  1 2 3  1 2 3  1 2 3  1 2 3  1 2 3  1 2 3

 1   1

 1   2

  ...

 2   1

 2   2

  ...

 

A Third Example Design

 

From the Springfield News-Leader, March 1, 1993:

 

Images of beauty such at those shown by Sports Illustrated's annual swimsuit issue, are harmful to the self-esteem of all women and contribute to the number of eating disorder cases in the U. S., says a St. Louis professor who researches women's health issues.

 

In a recent study at Washington University, two groups of women - one with bulimia and one without - watched videotapes of SI models in swimsuits.

 

Afterwards, both groups reported a more negative self-image than they did before watching the tape, describing themselves as "feeling fat and flabby" and "feeling a great need to diet."

 

The experiment described above has a number of inadequacies, the lack of control conditions being the most obvious.  The original authors, unnamed in the article, may have designed a much better experiment than is described in the popular press.  In any case, this experiment will now be expanded to illustrate a complex experimental design.


The dependent measure, apparently a rating of "feeling fat and flabby" and "feeling a great need to diet", will be retained.  In addition, two neutral questions will be added, say "feeling anxious" and "feeling good about the environment."  These four statements will be rated by all subjects, thus subjects will be crossed with ratings.  The first two statements deal with body image and diet and the last two do not, thus they will form a factor in the design (called D).  Since the statements within each of body image factor share no similarity across levels of D, these statements (A) are nested within D. For example, the rating of "feeling a great need to diet" and "feeling good about the environment" share no qualitative relationship.  At this point the design may be specified as S X A(D).

 

Suppose the researcher runs the subjects in groups of six to conserve time and effort, thus creating a groups (G) factor.   In addition to the two groups, with bulimia and without (B), suppose the subjects viewed one of the following videotapes (V):  SI models, Rosanne Barr, or a show about the seals of the great northwest.  Assuming that all the subjects in each level of group either had bulimia or did not, then the design could be specified as

 

S(G(B X V)).

 

The factor B is crossed with V because each level of B appears with each level of V.  That is, subjects with and without bulimia viewed all three videotapes.  Because each group viewed only a single videotape and was composed of subjects either with bulimia or without, the groups factor is nested within the cross of B and V.  Because subjects appeared in only one group, subjects are nested within groups.

 

Combining the between subjects effects, S(G(B X V)), and the within subjects effects, A(D), yields the complete design specification

 

S(G(B X V)) X A(D).

 

 


 

Determining the Number of Subjects and Measures per Subject

 

It is important to be able to determine the number of subjects and the number of measures per subject for practical reasons, namely, is the experiment feasible?  After listening to a student propose an experiment and a little figuring, I remarked "according to my calculations, you should be able to complete the experiment sometime near the middle of the next century."  If an experimenter is limited in the time a subject is available, then the number of measures per subject is another important consideration.

 

To determine the number of subjects, multiply the number of levels of the between subjects factor together.  In the previous example, S = 6 because the subjects were run in groups of six.  Let G=4, or there be four groups of six each of combinations of bulimia and videotape.  Since there were two levels of bulimia, B=2, and three levels of videotape, V=3.  Since S(G(B X V)), then the total number of subjects needed would be S * G * B * V or 6*4*2*3 or 144.  Since half of the subjects must have bulimia, the question of whether or not 72 subjects with bulimia are available must be asked before the experiment proceeds.

 

To find the number of measures per subject, multiply the number of levels of the within subjects factors together.  In the previous example A(D), where A=2 and D=2, there would be A * D or 2 * 2 or 4 measures per subject.

 

Setting up the Data Matrix

 

             Columns

         1   2   ...   C

      1

Rows  2

      .

      R

 


A few rules simplify setting up the data matrix.  First, each subject appears on a single row of the data matrix.  Second, each measure or combination of within subjects factors appears in a column of data.  Third, each subject must be identified as to the combination of between subjects factors which he or she appears.

 

 1 1 1 3 5 4 3

 1 1 1 2 5 5 3

 1 1 1 5 5 5 4

 1 1 1 3 2 1 3

 1 1 1 2 5 3 1

 1 1 1 3 5 4 3

 2 1 1 5 4 5 5

 ...

 4 2 3 3 5 5 4

In the previous example, since there would be 144 subjects in the experiment, there would be 144 rows of data.  Each subject would be identified as to the level of  G, B, and V to which she belonged.  For example, a subject who appeared under g3 of b1 and v4 would be labelled as 3 1 4.   Since there are four measures per subject, these would appear as columns in addition to the identifiers.  An example data matrix might appear as follows.  In this example, the level of G is in the first column, B in the second, and V in the third.  The four combinations of within subjects factors appear next as ad11 ad12 ad12 ad22.

 

A Note of Caution

 

It is fairly easy to design complex experiments.  Running the experiments and interpreting the results are a different matter.  Many complex experiments are never completed because of such difficulties.  This is from personal experience.


                                                             Chapter

                                                                9

 

 

 

                One Between Group ANOVA

 

Why Multiple Comparisons Using t-tests is NOT the Analysis of Choice

 

  Group   Therapy Method      _     SX        SX˛

    1       Reality         20.53   3.45   11.9025

    2       Behavior        16.32   2.98    8.8804

    3       Psychoanalysis  10.39   5.89   35.7604

    4       Gestalt         24.65   7.56   57.1536

    5       Control         10.56   5.75   33.0625

 

Suppose a researcher has performed a study of the effectiveness of various methods of individual therapy.  The methods used were:  Reality Therapy, Behavior Therapy, Psychoanalysis, Gestalt Therapy, and, of course, a control group.  Twenty patients were randomly assigned to each group.  At the conclusion of the study, changes in self-concept were found for each patient.  The purpose of the study was to determine if one method was more or less effective than the other methods.

 

At the conclusion of the experiment the researcher organizes the collected data in the following manner:

 

The researcher wishes to compare the means of the groups with each other to decide about the effectiveness of the therapy. 

 


One method of performing this analysis is by doing all possible t-tests, called multiple t-tests.  That is, Reality Therapy is first compared with Behavior Therapy, then Psychoanalysis, then Gestalt Therapy, then the Control Group.  Behavior Therapy is then individually compared with the last three groups, and so on.  Using this procedure there would be ten different t-tests performed.  Therein lies the difficulty with multiple t-tests.

 

First, because the number of t-tests increases geometrically as a function of the number of groups, analysis becomes cognitively difficult somewhere in the neighborhood of seven different tests.  An analysis of variance organizes and directs the analysis, allowing easier interpretation of results.

 

Secondly, by doing a greater number of analyses the probability of committing at least one type I error somewhere in the analysis greatly increases.  The probability of committing at least one type I error in an analysis is called the experiment-wise error rate. The researcher may desire to perform a fewer number of hypothesis tests in order to reduce the experiment-wise error rate.  The ANOVA procedure performs this function.

 

The Bottom Line - Results and Interpretation of ANOVA

 

Results of an ANOVA are usually presented in an ANOVA table.  This table contains  columns labelled "Source", "SS or Sum of Squares", "df - for degrees of freedom", "MS - for mean square", "F or F-ratio", and "p, prob,  probability, sig., or sig. of F".  The only columns that are critical for interpretation are the first and the last, the others are used mainly for intermediate computational purposes.

 

    Source       SS    df      MS      F    sig of F

  BETWEEN    5212.960   4   1303.240   4.354   .0108

  WITHIN     5986.400  20    299.320               

  TOTAL     11199.360  24

 

        An example of an ANOVA table appears below:

 


The row labelled "BETWEEN" under "Source", having a probability value associated with it, is the only one of any any great importance at this time.  The other rows are used mainly for computational purposes.  The researcher then would most probably first look at the value ".0108" located under the "sig of F" column. 

 

Of all the information presented in the ANOVA table, the major interest of the researcher will most likely be focused on the value located in the "sig of F." column.  If the number (or numbers) found in this column is (are) less than the critical value (α) set by the experimenter, then the effect is said to be significant.  Since this value is usually set at .05, any value less than this will result in significant effects, while any value greater than this value will result in nonsignificant effects.

 

If the effects are found to be significant using the above procedure, it implies that the means differ more than would be expected by chance alone.  In terms of the above experiment, it would mean that the treatments were not equally effective.  This table does not tell the researcher anything about what the effects were, just that there most likely were real effects.

 

If the effects are found to be nonsignificant, then the differences between the means are not great enough to allow the researcher to say that they are different.  In that case no further interpretation is attempted.

 

When the effects are significant, the means must then be examined in order to determine the nature of the effects.  There are procedures called "post-hoc tests" to assist the researcher in this task, but often the analysis is fairly evident simply by looking at the size of the various means.  For example, in the preceding analysis Gestalt and Reality Therapy were the most effective in terms of mean improvement.

 


In the case of significant effects, a graphical presentation of the means can sometimes assist in analysis.  For example, in the preceding analysis, the graph of mean values would appear as follows:

 

 

HYPOTHESIS TESTING THEORY UNDERLYING ANOVA

 

       The Sampling Distribution Reviewed

 

In order to explain why the above procedure may be used to simultaneously analyze a number of means, the following presents the theory on ANOVA in relation to the hypothesis testing approach discussed in earlier chapters.

 

First, a review of the sampling distribution is necessary.  If you have difficulty with this summary, please go back and read the more detailed chapter on the sampling distribution.

 

A sample is a finite number (N) of scores.  Sample statistics are numbers which describe the sample.  Example statistics are the mean (_), mode (Mo), median (Md), and standard deviation (sX). 

 

Probability models exist in a theoretical world where complete information is unavailable.  As such, they can never be known except in the mind of the mathematical statistician.  If an infinite number of infinitely precise scores were taken, the resulting distribution would be a probability model of the population.  Population models are characterized by parameters.  Two common parameters are µX and σX.

 


Sample statistics are used as estimators of the corresponding parameters in the population model.  For example, the mean and standard deviation of the sample are used as estimates of the corresponding population parameters µX and σX.

 

The sampling distribution is a distribution of a sample statistic.  It is a model of a distribution of scores, like the population distribution, except that the scores are not raw scores, but statistics.  It is a thought experiment; "what would the world be like if a person repeatedly took samples of size N from the population distribution and computed a particular statistic each time?"  The resulting distribution of statistics is called the sampling distribution of that statistic. 

 

        The sampling distribution of the mean is a special case of a sampling distribution.  It is a distribution of sample means, described with the parameters µ_  and σ_.  These parameters are closely related to the parameters of the population distribution, the relationship being described by the CENTRAL LIMIT THEOREM.  The CENTRAL LIMIT THEOREM essentially states that the mean of the sampling distribution of the mean (µ_) equals the mean of the population (µX) and that the standard error of the mean (σ_) equals the standard deviation of the population (σX) divided by the square root of N.  These relationships may be summarized as follows:

 

Two Ways of Estimating the Population Parameter σX˛

 


When the data have been collected from more than one sample, there exists two independent methods of estimating the population parameter σX˛, called respectively the between and the within method.  The collected data are usually first described with sample statistics as demonstrated in the following example:

 

 Group   Therapy Method      _      SX        SX˛

 

   1       Reality         20.53   3.45   11.9025

   2       Behavior        16.32   2.98    8.8804

   3       Psychoanalysis  10.39   5.89   35.7604

   4       Gestalt         24.65   7.56   57.1536

   5       Control         10.56   5.75   33.0625

 

           Mean            16.49          29.3519

           Variance        38.83         387.8340

 

 

            THE WITHIN METHOD

 

Since each of the sample variances may be considered an independent estimate of the parameter σX˛, finding the mean of the variances provides a method of combining the separate estimates of σX˛ into a single value.  The resulting statistic is called the MEAN SQUARES WITHIN, often represented by MSW.  It is called the within method because it computes the estimate by combining the variances within each sample.  In the above example, the Mean Squares Within would be equal to 29.3519.

 

THE BETWEEN METHOD

 

The parameter σX˛ may also be estimated by comparing the means of the different samples, but the logic is slightly less straightforward and employs both the concept of the sampling distribution and the Central Limit Theorem. 

 


Sampling Distribution             Actual Data

    

                       _                            _

                       _                            _

                       _                            _

                       .                              _

                       .                              _

                       .                             

 

Mean             µ_                          __

Variance       σ_˛                         s_˛

First, the standard error of the mean squared (σ_˛) is the population variance of a distribution of sample means.  In real life in the situation where there is more than one sample, the variance of the sample means may be used as an estimate of the standard error of the mean squared (σ_˛).  This is analogous to the situation where the variance of the sample (sX˛) is used as an estimate of σ_˛.  The relationship is demonstrated below:

 

In this case the Sampling Distribution consists of an infinite number of means and the real life data consists of A (in this case 5) means.  The computed statistic s_˛ is thus an estimate of the theoretical parameter σ_˛.

 

        The relationship expressed in the Central Limit Theorem may now be used to obtain an estimate of σ˛.

 

 Thus the variance of the population may be found by multiplying the standard error of the mean squared (σ_˛) by N, the size of each sample.

 


Since the variance of the means, s_˛, is an estimate of the standard error of the mean squared, σ_˛, the variance of the population, σX˛, may be estimated by multiplying the size of each sample, N, by the variance of the means.  This value is called the Mean Squares Between and is often symbolized by MSB.  The computational procedure for MSB is presented below:

 

MSB = N * s_˛

 

 

MSB = N * s_˛

MSB = 20 * 38.83

MSB = 776.60

The expressed value is called the Mean Squares Between because it uses the variance between the samples, that is the sample means, to compute the estimate.  Using the above procedure on the example data yields:

 

At this point it has been established that there are two methods of estimating σX˛, Mean Squares Within and Mean Squares Between.  It could also be demonstrated that these estimates are independent.  Because of this independence, when both are computed using the same data, in almost all cases different values will result.  For example, in the presented data MSW=29.3519 while MSB=776.60.  This difference provides the theoretical background for the F-ratio and ANOVA. 

 

 The F-ratio and F-distribution

 

        A new statistic, called the F-ratio is computed by dividing the MSB by MSW.  This is illustrated below:

 

 Fobs =  MSB / MSW 

       

 

      

 

Using the example data described earlier the computed F-ratio becomes:


 

              Fobs =  MSB / MSW

              Fobs =  776.60 / 29.3519

              Fobs =  26.4582

 

The F-ratio can be thought of as a measure of how different the means are relative to the variability within each sample.  The larger this value, the greater the likelihood that the differences between the means are due to something other than chance alone, namely real effects.  The size of the F-ratio necessary to make a decision about the reality of effects is the next topic of discussion.

 

If the difference between the means means is due only to chance, that is, there are no real effects, then the expected value of the F-ratio would be one (1.00).  This is true because both the numerator and the denominator of the F-ratio are estimates of the same parameter, σX˛.  Seldom will the F-ratio be exactly equal to 1.00, however, because the numerator and the denominator are estimates rather than exact, known values.  Therefore, when there are no effects the F-ratio will sometimes be greater than one, and other times less than one.

 

To review, the basic procedure used in hypothesis testing is that a model is created in which the experiment is repeated an infinite number of times when there are no effects.  A sampling distribution of a statistic is used as the model of what the world would look like if there were no effects.  The results of the experiment, a statistic, is compared with what would be expected given the model of no effects was true.  If the computed statistic is unlikely given the model, then the model is rejected along with the hypothesis that there were no effects. 

 

In an ANOVA the F-ratio is the statistic used to test the hypothesis that the effects are real, in other words, that the means are significantly different from one another.  Before the details of the hypothesis test may be presented, the sampling distribution of the F-ratio must be discussed.

 


If  the experiment were repeated an infinite number of times, each time computing the F-ratio, and there were no effects, the resulting distribution could be described by the F-distribution.  The F-distribution is a theoretical probability distribution characterized by two parameters, df1 and df2, both of which affect the shape of the distribution.  Since the F-ratio must always be positive, the F-distribution is non-symmetrical, skewed in the positive direction.

 

Two examples of an F-distribution are presented below; the first with df1=1 and df2=5, and the second with df1=10 and df2=25. 

 

 The F-distribution has a special relationship to the t-distribution described earlier.  When df1=1, the F-distribution is equal to the t-distribution squared (F=t˛).  Thus the t-test and the ANOVA will always return the same decision when there are two groups.  That is, the t-test is a special case of ANOVA. 

 

 Non-significant and Significant F-ratios

 

        Theoretically, when there are no real effects, the F-distribution is an accurate model of the distribution of F-ratios.  The F-distribution will have the parameters df1=a-1 (where a-1 is the number of different groups minus one) and df2=a(N-1), where a is the number of groups and N is the number in each group. In this case an assumption is made that sample size is equal for each group. For example, if five groups of five subjects each were run in an experiment and there were no effects, then the F-ratios would be distributed with df1=k-1=5-1=4 and df2=k(n-1)=5(5-1)=5*4=20.  A visual representation of the preceding appears as follows:


 

The F-ratio in the above which cuts off various proportions of the distributions may be computed for different values of  α.  These F-ratios are called Fcrit values.  In the above example the Fcrit value for  α=.25 is 1.46, for  α=.10 results in a value of 2.25, for α=.05 the value is 2.87, and for α=.01 the value is 4.43.  These values are illustrated in the figure below:

 

When there are real effects, that is, the means of the groups are different due to something other than chance, then the F-distribution no longer describes the distribution of F-ratios.  In almost all cases the observed F-ratio will be larger than would be expected when there were no effects.  The rationale for this situation is presented below.

 

First, an assumption is made that any effects are an additive transformation of the score.  That is, the scores for each group can be modelled as a constant ( aa - the effect) plus error (eae).  The scores appear as follows:

 

                       Xae = aa + eae

 

where X is the score, aa is the treatment effect, and eae is the error.  The eea, or error, is different for each subject, while aa is constant within a given group.

 


As described in the chapter on transformations, an additive transformation changes the mean, but not the standard deviation or the variance.  Because the variance of each group is not changed by the nature of the effects, the Mean Square Within, as the mean of the variances, is not affected.  The Mean Square Between, as N time the variance of the means, will in most cases become larger because the variance of the means will most likely become larger. 

 

Imagine three individuals taking a test.  An instructor first finds the variance of the three score.  He or she then adds five points to one random individual and subtracts five from another random individual.  In most cases the variance of the three test score will increase, although it is possible that the variance could decrease if the points were added to the individual with the

     No effects              Real Effects

Group  Mean  Variance   Group    Mean    Variance

  1      µ      σ˛        1       µ + a1     σ˛

  2      µ      σ˛        2       µ + a2     σ˛

  3      µ      σ˛        3       µ + a3     σ˛

  4      µ      σ˛        4       µ + a4     σ˛

  5      µ      σ˛        5       µ + a5     σ˛

Mean     µ       σ˛                  µ       σ˛

Variance σ˛/N                     >σ˛/N

lowest score and subtracted from the individual with the highest score.  If the constant added and subtracted was 30 rather than 5, then the variance would almost certainly be increased.  Thus, the greater the size of the constant, the greater the likelihood of a larger increase in the variance.

 

        With respect to the sampling distribution, the model differs depending upon whether or not there are effects.  The difference is presented below:

 


 Since the MSB usually increases and MSW remains the same, the F-ratio (F=MSB/MSW) will most likely increase.  Thus, if there are real effects, then the F-ratio obtained from the experiment will most likely be larger than the critical level from the F-distribution.  The greater the size of the effects, the larger the obtained F-ratio is likely to become.

 

Thus, when there are no effects, the obtained F-ratio will be distributed as an F-distribution which may be specified.  If effects exist, then the obtained F-ratio will most likely become larger.  By comparing the obtained F-ratio with that predicted by the model of no effects, an hypothesis test may be performed to decide on the reality of effects.  If the obtained F-ratio is greater than the critical F-ratio, then the decision will be that the effects are real.  If not, then no decision about the reality of effects can be made.

 

Similarity of ANOVA and t-test

 

When the number of groups (A) equals two (2), an ANOVA and t-test will give similar results, with tCRIT˛=FCRIT and tOBS˛=FOBS.  This equality is demonstrated in the example below:

 


Given the following numbers for two groups:                                             

 

Mean    Variance

     Group 1 - 12 23 14 21 19 23 26 11 16    18.33     28.50

     Group 2 - 10 17 20 14 23 11 14 15 19    15.89     18.11

 

Computing the t-test

 

     s_1-_2 = Ö (s1˛ + s2˛)/ 9 = Ö (28.50 + 18.11)/9 =  Ö 5.18 = 2.28

     tOBS = ( _1-_2 ) / s_1-_2 = 18.33 - 15.89 / 2.28 = 1.07

     t(df=16) = 2.12        for α=.05 and two-tailed test

 

Computing the ANOVA

 

     MSBETWEEN = N * s_˛ = 9 * 2.9768 = 26.7912

     MSWITHIN  = Mean of the Variances = ( 28.50 + 18.11 ) / 2 = 23.305

     FOBS = MSBETWEEN/MSWITHIN = 1.1495

     F(1,16) = 4.41         for α=.05 - two-tailed test is assumed

 

Comparing the results

 

     tOBS˛ = 1.1449      FOBS  = 1.1449

     t(16)˛  = 4.49        F(1,16) = 4.49

    

The differences between the predicted and observed results can be attributed to rounding error (close enough for government work). 

 

Because the t-test is a special case of the ANOVA and will always yield similar results, most researchers perform the ANOVA because the technique is much more powerful in complex experimental designs.

 

 

 


EXAMPLE OF A NON-SIGNIFICANT ONE-WAY ANOVA

                                               MEAN VARIANCE

     7 7 5 4 2 7 5 4 1 7 5 6 7 6 3 5 2 5 1 4   4.65   4.03

     6 9 3 6 9 4 9 8 9 3 4 4 7 2 2 7 7 7 9 3   5.90   6.52

     5 5 2 5 6 2 3 3 6 8 2 1 1 2 5 7 9 6 5 7   4.50   5.63

     4 1 4 8 9 5 2 8 6 8 2 9 6 6 7 8 4 3 1 4   5.25   6.93

     3 6 1 2 3 5 8 4 1 5 4 5 6 9 4 2 4 8 9 3   4.60   6.04 

 

Computing the ANOVA

 

    MSBETWEEN = N * s_˛ = 20 * .351 = 7.015

    MSWITHIN  = Mean of the Variances = 5.83

    FOBS = MSBETWEEN/MSWITHIN = 1.20

    F(4,95) = 2.53  for α=.05 - non-directional test is assumed

 

 

        Given the following data for five groups, perform an ANOVA:

 

Since the FCRIT is greater than the FOBS, the means are not significantly different and no effects are said to be discovered.

 

EXAMPLE OF A SIGNIFICANT ONE-WAY ANOVA

 

Given the following data for five groups, perform an ANOVA.  Note that the numbers are similar to the previous example except that one has been subtracted from all scores in Group 3 and one has been added to all scores in Group 4.

 

In this case the FOBS is greater than FCRIT, thus the means are significantly different and we decide that the effects are real.

 

1 23

1 31

1 25

1 29

1 30

1 28

1 31

1 31

1 33

2 32

2 28

2 36

2 34

2 41

2 35

2 32

2 28

2 31

USING MANOVA

 


                                                                                         MEAN VARIANCE

     7 7 5 4  2 7 5 4 1 7 5  6 7 6 3 5 2 5 1 4   4.65   4.03

     6 9 3 6  9 4 9 8 9 3 4  4 7 2 2 7 7 7 9 3   5.90   6.52

     4 4 1 4  5 1 2 2 5 7 1  0 0 1 4 6 8 5 4 6   3.50   5.63

     5 2 5 9 10 6 3 9 7 9 3 10 7 7 8 9 5 4 2 4   6.25   6.93

     3 6 1 2  3 5 8 4 1 5 4  5 6 9 4 2 4 8 9 3   4.60   6.04 

 

Computing the ANOVA

 

     MSBETWEEN = N * s_˛ = 20 * 1.226 = 24.515

     MSWITHIN  = Mean of the Variances = 5.83

     FOBS = MSBETWEEN/MSWITHIN = 4.20

     F(4,95) = 2.53    for α=.05 - two-tailed test is assumed

 

While an single factor between groups ANOVA may be done using the MEANS command in SPSS, the MANOVA command is a general purpose command which allows the statistician to do almost any type of multifactor univariate or multivariate ANOVA.

 

The Data

 

The data is entered into a data file containing two columns.  One column contains the level of the factor to which the observation belongs and the second the score for the dependent variable.  A third column containing the observation number, in the example a number from one to nine, is optional.  As in all SPSS data files, the number of rows in the data file corresponds to the number of subjects and each variable is lined up neatly in each row.  In the example data file presented to the right, there are two groups of nine each.  The level of the independent variable is given in the first column of the data file, a space is entered, and the dependent variable is entered in columns 3 and 4.

 


RUN NAME   EXAMPLE FOR ANOVA BOOK - DESIGN A.

DATA LIST  FILE='DESIGNA DATA  A' /1 A 1 X 3‑4.

VALUE LABELS

   A 1 'BLUE BOOK' 2 'COMPUTER'

LIST.

MANOVA X by A(1,2)

   /PRINT CELLINFO(MEANS)

   /DESIGN.

 

The RUN NAME command of the example program gives a general description of the purpose of the program.  The second command reads in the data file.  Note that the group factor is called "A" and the dependent variable is called "X".  The value label command then describes the different levels of the group variable.  The LIST command gives a description of the data as the computer understands it.

 

The MANOVA command is followed by the name of the dependent variable, here X, and the keyword BY .  The factor name "A" is then entered, followed by the the beginning and ending levels of that factor.  In this case there were only two levels, defined by a beginning value of "1", and an ending value of "2".  The second line on the command is preceded by a slash "/'" and then the subcommand PRINT=CELLINFO(MEANS).  This command will print the means of the respective groups.  The last subcommand, "/DESIGN" , is optional at this point, but not including it will generate a WARNING.  Nothing is altered with the WARNING, but it is not neat.

 

 

Example Output

 

The output produced by the example MANOVA command is presented on the next page.


The default error term in MANOVA has been changed from WITHIN CELLS to WITHIN+RESIDUAL.  Note that these are the same for all full factorial designs.

 

* * * * * * A n a l y s i s   o f   V a r i a n c e * * * * *

 

     18 cases accepted.

      0 cases rejected because of out‑of‑range factor values.

      0 cases rejected because of missing data.

      2 non‑empty cells.

      1 design will be processed.

 

 

 Cell Means and Standard Deviations

 Variable .. X

      FACTOR        CODE        Mean  Std. Dev.          N

 

  A                   1         29.000      3.202          9

  A                   2         33.000      4.093          9

 For entire sample              31.000      4.116         18

 

* * * * * * A n a l y s i s   o f   V a r i a n c e ‑‑ design  

 Tests of Significance for X using UNIQUE sums of squares

 Source of Variation        SS    DF      MS       F  Sig of F

 

 WITHIN CELLS             216.00   16   13.50

 A                         72.00    1   72.00   5.33    .035

 

 (Model)                   72.00    1   72.00   5.33      .035

 (Total)                  288.00   17   16.94

 

 R‑Squared =           .250

 Adjusted R‑Squared =  .203

Note that the F-ratio was significant with a value of .03.  The means for the two  groups were 29 and 33 respectively.


Dot Notation

 

In order to simplify the notational system involving the summation sign, a notational system call dot notation has been introduced. Dot notation places a period in place of subscript to mean summation.  For example:

The symbol X. means that the variable X has been summed over whatever counter variable was used as a subscript.  In a like manner, if a bar is placed over a variable, it refers to a mean over the dotted counter variable(s).  For example:

 

where _. means the same thing as _. (The dot notation does become a bit tricky where real periods are involved.)

 

The real advantage is apparent when two or more subscripts are used.  For example

 

and

 


or if a=1 then

 

 

 

Using the same notational system, means may be given.  For example:

and

or if a=1 then

The difference between _a. and _1. is that the second is a special case of the first when a=1.

 

For example, if A=3, B=4 and X11=5, X12=8, X13=6, X14=9, X21=7, X22=10, X23=5, X24=9, X31=6, X32=4, X33=7, X34=3, then


X.. = 79,

_.. = 79/12 = 6.5833,

X1. = 28,

_1. = 28/4 = 7.

 

POST-HOC Tests of Significance

 

If the results of the ANOVA are significant, it indicates that there are real effects between the means of the groups.  The nature of the effects are not specified by the ANOVA.  For example, an effect could be significant because the mean of group three was larger than the means of the rest of the groups.  In another case, the means of groups one, four and five might be significantly smaller than the means of groups two, three, and six.  Often the pattern of results can be determined by a close examination of the means.  In other instances, the reason for the significant differences is not apparent.  To assist the statistician in interpreting effects in significant ANOVAs, post-hoc tests of significance were developed.

 

A post-hoc (after the fact) test of significance is employed only if the results of the overall significance test are significant.  A post-hoc test is basically a multiple t-test procedure with some attempt to control for the increase in the experiment wide error rate when doing multiple significance tests.  A number of different procedures are available to perform post-hoc tests differing in the means of control of the increase in error rates .  The different procedures include Duncan’s Multiple Range test, the Newman-Keuls procedure, and a procedure developed by Sheffe’.  The interested reader is referred to Winer (1971) or Hays (1981) for a thorough discussion of these methods.

 


My personal feeling is that post-hoc tests are not all that useful.  Most often, the reason for the significant results is obvious from a close observation of the means of the groups.  Better procedures are available using pre-planned contrasts to test patterns of results.  The use of pre-planned contrasts requires that the statistician have the type of comparisons in mind before doing the analysis.  This is the difference between data-driven (post-hoc) and theory-driven (pre-planned) analysis.  If a choice is possible, the recommendation is for theory-driven analysis.

 


                                                             Chapter

                                                              10

 

 

 

TWO Between Groups ANOVA  (A x B)

 

The Design

 

In this design there are two independent factors, A and B, crossed with each other.  That is, each level of A appears in combination with each level of B.  Subjects are nested within the combined levels of A and B such that the full design would be written as S ( A X B  ).  Because the Subjects (S) term is confounded with the error term, it is dropped from the description of the design.

 

 A B     X

 

 1 1    23

 1 1    32

 1 1    25

 1 2    29

 1 2    30

 1 2    34

 1 3    31

 1 3    36

 1 3    33

 2 1    32

 2 1    26

 2 1    26

 2 2    34

 2 2    41

 2 2    35

 2 3    24

 2 3    27

 2 3    31

Suppose a statistics teacher gave an essay final to his class.  He randomly divides the classes in half such that half the class writes the final with a blue-book and half with notebook computers.  In addition the students are partitioned into three groups, no typing ability, some typing ability, and highly skilled at typing.  Answers written in blue-books will be transcribed to word processors and scoring will be done blindly.  Not with a blindfold, but the instructor will not know the method or skill level of the student when scoring the final.  The dependent measure will be the score on the essay part of the final exam.

 


The first factor  (A) will be called Method and will have two levels, a1=blue-book and a2 =  computer.  The second factor (B) will be designated as Ability and will have three levels:  b1=none, b2=some, and b3=lots.  Because each level of A appears with each level of B, A is said to be crossed with B (AXB).  Since different subjects will appear in each combination of A and B, subjects are nested within AXB.  Each subject will be measured a single time.  Any effects discovered will necessarily be between subjects or groups and hence the designation "between groups" designs.

 

The Data

 

The data file for the A X B design is similar to the data file for design A with the addition of the second descriptive variable, B, for each subject.  In the case of the example data, the A factor has two levels while the B factor has three.  The X variable is the score on the final exam.  The example data file appears in the text box on the right.

 

Example SPSS Program using MANOVA

 

The SPSS commands necessary to do the analysis for an A X B design are given in the text box below.

RUN NAME EXAMPLES FOR ANOVA BOOK ‑ A X B DESIGNS

DATA LIST FILE='AXB DATA A' / A 3 B 6 X 15‑19.

VARIABLE LABELS

     A 'METHOD OF WRITING EXAM'

     B 'KEYBOARD EXPERIENCE'

     X 'SCORE ON FINAL EXAM'.

VALUE LABELS

     A 1 'BLUE‑BOOK' 2 'COMPUTER'/

     B 1 'NONE' 2 'SOME' 3 'LOTS'.

LIST.

MANOVA X BY A (1,2) B (1,3)

     /PRINT CELLINFO(MEANS)

     /DESIGN.

 


Note that the DATA LIST command must read in a variable to code each factor.  In this case the variables were named A and B to correspond with the factor names, although in most real-life situations more descriptive names will be used.  The addition of the optional VALUE LABELS command will label the output from the PRINT CELLINFO(MEANS) command, making the output easier to interpret.

 

The MANOVA command is followed by the name of the dependent variable, in this case X and then the variable names of the factors.  As in the previous MANOVA commands, the factor names are each followed by the beginning and ending levels enclosed in parentheses.  In this case the A factor has two levels beginning at level 1 and the B factor has three.  The PRINT CELLINFO(MEANS) command is optional, but usually included because the means are the central focus of the analysis.  The DESIGN command is optional, but excluding it will generate a warning when the program is run, so it is usually included for the sake of neatness.

 

All of the analyses on the following pages were generated from the SPSS program presented above.  The program will not be included as part of the interpretation of the output.

 

Interpretation of  Output

 

  Cell Means and Standard Deviations

  Variable .. X                SCORE ON FINAL EXAM

       FACTOR       CODE        Mean  Std. Dev.          N   95 percent Conf. Interval

 

   A           BLUE‑BOO

    B           NONE           26.717      4.822          3     14.739     38.695

    B           SOME           30.923      2.418          3     24.916     36.930

    B           LOTS           33.277      2.157          3     27.919     38.634

   A           COMPUTER

    B           NONE           28.057      3.285          3     19.896     36.217

    B           SOME           36.627      3.413          3     28.148     45.106

    B           LOTS           27.080      3.673          3     17.955     36.205

  For entire sample            30.447      4.675         18     28.122     32.771


The interpretation of the output from the MANOVA command will focus on two parts:  the table of means and the ANOVA summary table.  The table of means is the primary focus of the analysis while the summary table directs attention to the interesting or statistically significant portions of the table of means.

 

A table of means generated using the example data and the PRINT CELLINFO(MEANS) subcommand in MANOVA is presented below:

 

 

Often the means are organized and presented in a slightly different manner than the form of the output from the MANOVA command.  The table of means may be rearranged and presented as follows:

 

 

    b1       b2   b3

┌────────┬────────┬────────┐

  a1   26.72   30.92   33.28 30.31

     ├────────┼────────┼────────┤ 

  a2   28.06   36.62   27.08 30.58

    └────────┴────────┴────────┘  

   27.39    33.78   30.18   30.447

 

The means inside the boxes are called cell means, the means in the margins are called marginal means, and the number on the bottom right-hand corner is called the grand mean.  An analysis of  these means reveals that there is very little difference between the marginal means for the different levels of A across the levels of B (30.31 vs. 30.58).  The marginal means of B over levels of A are different (27.39 vs. 33.78 vs. 30.18) with the mean for b2 being the highest.  The cell means show an increasing pattern for levels of B at a1 (26.72 vs. 30.92 vs. 33.28) and a different pattern for levels of B at a2 (28.06 vs. 36.62 vs. 27.08). 

 

Graphs of Means

 


 

Graphs of means are often used to present information in a manner which is easier to comprehend than the tables of means.  One factor is selected for presentation as the X-axis and its levels are marked on that axis.  Separate lines are drawn the height of the mean for each level of the second factor.  In the following graph, the B, or keyboard experience, factor was selected for the X-axis and the A, or method, factor was selected for the different lines.  Presenting the information in an opposite fashion would be equally correct, although some graphs are more easily understood than others, depending upon the values for the means and the number of levels of each factor.  The second possible graph is presented below.  It is recommended that if there is any doubt that both versions of the graphs be attempted and the one which best illustrates the data be selected for inclusion into the statistical report.  It is hopefully obvious that the graph with B on the X-axis is easier to understand than the one with A on the X-axis.

 

Because the interpretation of the graph of the interaction depends upon the results of the analysis, the ANOVA summary table will now be presented.  Following this, the graph of the interaction will be reanalyzed.

 

The ANOVA Summary Table

 


The results of the A X B ANOVA are presented in the ANOVA summary table by MANOVA.  An example of this table is presented below:

 

  Tests of Significance for X using UNIQUE sums of squares

  Source             SS    DF      MS          F  Sig of F

 

WITHIN CELLS      139.36   12     11.61

  A                  .36    1       .36       .03      .863

  B               123.08    2     61.54      5.30      .022

 A BY B           108.73    2     54.36      4.68      .031

 

The items of primary interest in this table are the effects listed under the "Source" column and the values under the "Sig of F" column.  As in the previous hypothesis test, if the value of "Sig of F" is less than the value of α as set by the experimenter, then that effect is significant.  If α=.05, then the B main effect and the A BY B interaction would be significant in this table.

 

Main Effects

 

 Main effects are differences in means over levels of one factor collapsed over levels of the other factor.  This is actually much easier than it sounds.  For example, the main effect of A is simply the difference between the means of final exam score for the two levels of Method, ignoring or collapsing over experience.  As seen in the second method of presenting a table of means, the main effect of A is whether the two marginal means associated with the A factor are different.  In the example case these means were 30.31 and 30.58 and the differences between these means was not statistically significant.

 

As can be seen from the summary table, the main effect of B is significant.  This effect refers to the differences between the three marginal means associated with factor B.  In this case the values for these means were 27.39, 33.78, and 30.18 and the differences between them may be attributed to a real effect.

 


Simple Main Effects

 

A simple main effect is a main effect of one factor at a given level of a second factor.  In the example data it would be possible to talk about the simple main effect of B at a1.  That effect would be the difference between the three cell means at level a1 (26.72, 30.92, and 33.28).  One could also talk about the simple main effect of A at b3 (33.28 and 27.08).  Simple main effects are not directly tested in design A X B, however they are necessary to understand an interaction.

 

Interaction Effects

 

An interaction effect is a change in the simple main effect of one variable over levels of the second.  An AB or A BY B interaction is a change in the simple main effect of B over levels of A or the change in the simple main effect of A over levels of B.   In either case the cell means cannot be modelled simply by knowing the size of the main effects.  An additional set of parameters must be used to explain the differences between the cell means.  These parameters are collectively called an interaction.

 

The change in the simple main effect of one variable over levels of the other is most easily seen in the graph of the interaction.  If the lines describing the simple main effects are not parallel, then a possibility of an interaction exists.  As can be seen from the graph of the example data, the possibility of a significant interaction exists because the lines are not parallel.  The presence of an interaction was confirmed by the significant interaction in the summary table. 

 


The following graph overlays the main effect of B on the graph of the interaction.  Two things can be observed from this presentation.  The first is that the main effect of B is possibly significant, because the means are different heights.  Second, the interaction is possibly significant because the simple main effects of B at a1 and a2 are different from the main effect of B.

 

One method of understanding how main effects and interactions work is to observe a wide variety of data and data analysis.  With three effects, A, B, and AB, which may or may not be significant there are eight possible combinations of effects.  All eight are presented on the following pages. 

 


Example Data Sets, Means, and Summary Tables

 

No Significant Effects

 

  A B   X

 

 1 1  22

 1 1  24

 1 1  25

 1 2  26

 1 2  29

 1 2  23

 1 3  21

 1 3  25

 1 3  22

 2 1  21

 2 1  26

 2 1  25

 2 2  24

 2 2  20

 2 2  24

 2 3  23

 2 3  26

 2 3  20

 

 

 

 

 

 

 

 Cell Means and Standard Deviations

  Variable .. X                SCORE ON FINAL EXAM

       FACTOR           CODE                  Mean  Std. Dev.          N   95 percent Conf. Interval

 

   A               BLUE‑BOO

    B               NONE                    23.667      1.528          3     19.872     27.461

    B               SOME                    26.000      3.000          3     18.548     33.452

    B               LOTS                    22.667      2.082          3     17.495     27.838

   A               COMPUTER

    B               NONE                    24.000      2.646          3     17.428     30.572

    B               SOME                    22.667      2.309          3     16.930     28.404

    B               LOTS                    23.000      3.000          3     15.548     30.452

  For entire sample                         23.667      2.401         18     22.473     24.861

 

 

  Tests of Significance for X using UNIQUE sums of squares

  Source of Variation          SS      DF        MS         F  Sig of F

 

  WITHIN CELLS              74.00      12      6.17

  A                          3.56       1      3.56       .58      .462

  B                          7.00       2      3.50       .57      .581

  A BY B                    13.44       2      6.72      1.09      .367


Main Effect of A

 

A B   X

 

 1 1  32

 1 1  34

 1 1  35

 1 2  36

 1 2  39

 1 2  33

 1 3  31

 1 3  35

 1 3  32

 2 1  21

 2 1  26

 2 1  25

 2 2  24

 2 2  20

 2 2  24

 2 3  23

 2 3  26

 2 3  20

 

 

 

 

 

 

 

Cell Means and Standard Deviations

  Variable .. X                SCORE ON FINAL EXAM

       FACTOR           CODE                  Mean  Std. Dev.          N   95 percent Conf. Interval

 

   A               BLUE‑BOO

    B               NONE                    33.667      1.528          3     29.872     37.461

    B               SOME                    36.000      3.000          3     28.548     43.452

    B               LOTS                    32.667      2.082          3     27.495     37.838

   A               COMPUTER

    B               NONE                    24.000      2.646          3     17.428     30.572

    B               SOME                    22.667      2.309          3     16.930     28.404

    B               LOTS                    23.000      3.000          3     15.548     30.452

  For entire sample                         28.667      6.078         18     25.644     31.689

 

  Tests of Significance for X using UNIQUE sums of squares

  Source of Variation          SS      DF        MS         F  Sig of F

 

  WITHIN CELLS              74.00      12      6.17

  A                        533.56       1    533.56     86.52      .000

  B                          7.00       2      3.50       .57      .581

  A BY B                    13.44       2      6.72      1.09      .367


Main Effect of B

 

 A B   X

 

 1 1  42

 1 1  44

 1 1  45

 1 2  36

 1 2  39

 1 2  33

 1 3  21

 1 3  25

 1 3  22

 2 1  41

 2 1  46

 2 1  45

 2 2  34

 2 2  30

 2 2  34

 2 3  23

 2 3  26

 2 3  20

 

 

 

 

 

 

 

  Cell Means and Standard Deviations

  Variable .. X                SCORE ON FINAL EXAM

       FACTOR           CODE                  Mean  Std. Dev.          N   95 percent Conf. Interval

 

   A               BLUE‑BOO

    B               NONE                    43.667      1.528          3     39.872     47.461

    B               SOME                    36.000      3.000          3     28.548     43.452

    B               LOTS                    22.667      2.082          3     17.495     27.838

   A               COMPUTER

    B               NONE                    44.000      2.646          3     37.428     50.572

    B               SOME                    32.667      2.309          3     26.930     38.404

    B               LOTS                    23.000      3.000          3     15.548     30.452

  For entire sample                         33.667      9.133         18     29.125     38.208

 

  Tests of Significance for X using UNIQUE sums of squares

  Source of Variation          SS      DF        MS         F  Sig of F

 

  WITHIN CELLS              74.00      12      6.17

  A                          3.56       1      3.56       .58      .462

  B                       1327.00       2    663.50    107.59      .000

  A BY B                    13.44       2      6.72      1.09      .367


AB Interaction

 

 A B   X

 

 1 1  42

 1 1  44

 1 1  45

 1 2  36

 1 2  39

 1 2  33

 1 3  21

 1 3  25

 1 3  22

 2 1  21

 2 1  26

 2 1  25

 2 2  34

 2 2  30

 2 2  34

 2 3  43

 2 3  46

 2 3  40

 

 

 

 

 

 

 

  Cell Means and Standard Deviations

  Variable .. X                SCORE ON FINAL EXAM

       FACTOR           CODE                  Mean  Std. Dev.          N   95 percent Conf. Interval

 

   A               BLUE‑BOO

    B               NONE                    43.667      1.528          3     39.872     47.461

    B               SOME                    36.000      3.000          3     28.548     43.452

    B               LOTS                    22.667      2.082          3     17.495     27.838

   A               COMPUTER

    B               NONE                    24.000      2.646          3     17.428     30.572

    B               SOME                    32.667      2.309          3     26.930     38.404

    B               LOTS                    43.000      3.000          3     35.548     50.452

  For entire sample                         33.667      8.738         18     29.321     38.012

 

  Tests of Significance for X using UNIQUE sums of squares

  Source of Variation          SS      DF        MS         F  Sig of F

 

  WITHIN CELLS              74.00      12      6.17

  A                          3.56       1      3.56       .58      .462

  B                          7.00       2      3.50       .57      .581

  A BY B                  1213.44       2    606.72     98.39      .000


Main Effects of A and B

 

 A B   X

 

 1 1  52

 1 1  54

 1 1  55

 1 2  46

 1 2  49

 1 2  43

 1 3  31

 1 3  35

 1 3  32

 2 1  41

 2 1  46

 2 1  45

 2 2  34

 2 2  30

 2 2  34

 2 3  23

 2 3  26

 2 3  20

 

 

 

 

 

 

 

  Cell Means and Standard Deviations

  Variable .. X                SCORE ON FINAL EXAM

       FACTOR           CODE                  Mean  Std. Dev.          N   95 percent Conf. Interval

 

   A               BLUE‑BOO

    B               NONE                    53.667      1.528          3     49.872     57.461

    B               SOME                    46.000      3.000          3     38.548     53.452

    B               LOTS                    32.667      2.082          3     27.495     37.838

   A               COMPUTER

    B               NONE                    44.000      2.646          3     37.428     50.572

    B               SOME                    32.667      2.309          3     26.930     38.404

    B               LOTS                    23.000      3.000          3     15.548     30.452

  For entire sample                         38.667     10.705         18     33.343     43.990

 

  Tests of Significance for X using UNIQUE sums of squares

  Source of Variation          SS      DF        MS         F  Sig of F

 

  WITHIN CELLS              74.00      12      6.17

  A                        533.56       1    533.56     86.52      .000

  B                       1327.00       2    663.50    107.59      .000

  A BY B                    13.44       2      6.72      1.09      .367


Main effect of A, AB Interaction

 

 A B   X

 

 1 1  52

 1 1  54

 1 1  55

 1 2  46

 1 2  49

 1 2  43

 1 3  31

 1 3  35

 1 3  32

 2 1  21

 2 1  26

 2 1  25

 2 2  34

 2 2  30

 2 2  34

 2 3  43

 2 3  46

 2 3  40

 

 

 

 

 

 

 

  Cell Means and Standard Deviations

  Variable .. X                SCORE ON FINAL EXAM

       FACTOR           CODE                  Mean  Std. Dev.          N   95 percent Conf. Interval

 

   A               BLUE‑BOO

    B               NONE                    53.667      1.528          3     49.872     57.461

    B               SOME                    46.000      3.000          3     38.548     53.452

    B               LOTS                    32.667      2.082          3     27.495     37.838

   A               COMPUTER

    B               NONE                    24.000      2.646          3     17.428     30.572

    B               SOME                    32.667      2.309          3     26.930     38.404

    B               LOTS                    43.000      3.000          3     35.548     50.452

  For entire sample                         38.667     10.370         18     33.510     43.823

 

  Tests of Significance for X using UNIQUE sums of squares

  Source of Variation          SS      DF        MS         F  Sig of F

 

  WITHIN CELLS              74.00      12      6.17

  A                        533.56       1    533.56     86.52      .000

  B                          7.00       2      3.50       .57      .581

  A BY B                  1213.44       2    606.72     98.39      .000


Main Effect of B, AB Interaction

 

 A B   X

 

 1 1  32

 1 1  34

 1 1  35

 1 2  46

 1 2  49

 1 2  43

 1 3  21

 1 3  25

 1 3  22

 2 1  31

 2 1  36

 2 1  35

 2 2  34

 2 2  30

 2 2  34

 2 3  33

 2 3  36

 2 3  30

 

 

 

 

 

 

 

  Cell Means and Standard Deviations

  Variable .. X                SCORE ON FINAL EXAM

       FACTOR           CODE                  Mean  Std. Dev.          N   95 percent Conf. Interval

 

   A               BLUE‑BOO

    B               NONE                    33.667      1.528          3     29.872     37.461

    B               SOME                    46.000      3.000          3     38.548     53.452

    B               LOTS                    22.667      2.082          3     17.495     27.838

   A               COMPUTER

    B               NONE                    34.000      2.646          3     27.428     40.572

    B               SOME                    32.667      2.309          3     26.930     38.404

    B               LOTS                    33.000      3.000          3     25.548     40.452

  For entire sample                         33.667      7.268         18     30.052     37.281

 

  Tests of Significance for X using UNIQUE sums of squares

  Source of Variation          SS      DF        MS         F  Sig of F

 

  WITHIN CELLS              74.00      12      6.17

  A                          3.56       1      3.56       .58      .462

  B                        397.00       2    198.50     32.19      .000

  A BY B                   423.44       2    211.72     34.33      .000


Main Effects of A and B, AB Interaction

 

 A B   X

 

 1 1  22

 1 1  24

 1 1  25

 1 2  36

 1 2  39

 1 2  33

 1 3  41

 1 3  45

 1 3  42

 2 1  41

 2 1  46

 2 1  45

 2 2  44

 2 2  40

 2 2  44

 2 3  43

 2 3  46

 2 3  40

 

 

 

 

 

 

 

  Cell Means and Standard Deviations

  Variable .. X                SCORE ON FINAL EXAM

       FACTOR           CODE                  Mean  Std. Dev.          N   95 percent Conf. Interval

 

   A               BLUE‑BOO

    B               NONE                    23.667      1.528          3     19.872     27.461

    B               SOME                    36.000      3.000          3     28.548     43.452

    B               LOTS                    42.667      2.082          3     37.495     47.838

   A               COMPUTER

    B               NONE                    44.000      2.646          3     37.428     50.572

    B               SOME                    42.667      2.309          3     36.930     48.404

    B               LOTS                    43.000      3.000          3     35.548     50.452

  For entire sample                         38.667      7.700         18     34.837     42.496

 

  Tests of Significance for X using UNIQUE sums of squares

  Source of Variation          SS      DF        MS         F  Sig of F

 

  WITHIN CELLS              74.00      12      6.17

  A                        373.56       1    373.56     60.58      .000

  B                        247.00       2    123.50     20.03      .000

  A BY B                   313.44       2    156.72     25.41      .000


No Significant Effects

 

 A B   X

 

 1 1  32

 1 1  24

 1 1  15

 1 2  46

 1 2  39

 1 2  23

 1 3  31

 1 3  45

 1 3  52

 2 1  31

 2 1  46

 2 1  55

 2 2  34

 2 2  40

 2 2  54

 2 3  33

 2 3  46

 2 3  50

 

 

 

 

 

 

  Cell Means and Standard Deviations

  Variable .. X                SCORE ON FINAL EXAM

       FACTOR           CODE                  Mean  Std. Dev.          N   95 percent Conf. Interval

 

   A               BLUE‑BOO

    B               NONE                    23.667      8.505          3      2.539     44.794

    B               SOME                    36.000     11.790          3      6.712     65.288

    B               LOTS                    42.667     10.693          3     16.104     69.229

   A               COMPUTER

    B               NONE                    44.000     12.124          3     13.881     74.119

    B               SOME                    42.667     10.263          3     17.171     68.162

    B               LOTS                    43.000      8.888          3     20.920     65.080

  For entire sample                         38.667     11.499         18     32.948     44.385

 

  Tests of Significance for X using UNIQUE sums of squares

  Source of Variation          SS      DF        MS         F  Sig of F

 

  WITHIN CELLS            1314.00      12    109.50

  A                        373.56       1    373.56      3.41      .090

  B                        247.00       2    123.50      1.13      .356

  A BY B                   313.44       2    156.72      1.43      .277

 

 

Note that the means and graphs of the last two example data sets were identical.  The ANOVA table, however, provided a quite different analysis of each data set.  The data in this final set was constructed such that there was a large standard deviation within each cell.  In this case the marginal and cell means were not different enough to warrant rejecting the hypothesis of no effects, thus no significant effects were observed.


Dot Notation Revisited

 

The reader may recall from the previous chapter that placing dots instead of subscripts is a shorthand notation for summation.  For example

 

When two subscripts are involved the notation becomes somewhat more complicated (and powerful).  For example

 

and

 

or if a=1 then

 

When three subscripts are involved as is necessary in an A X B design the notation involves even more summation signs.  For example

 


and, for example

 

where one sums over the subscript containing the dot.

 

Using the dot notation with means rather than sums is a relatively simple extension of the dot notation. The mean is found by dividing the sum by the number of scores which were included in the sum.  For example, the grand mean can be found as follows:

 

and the cell means

 

 

 

All this is most easily understood in the context of a data table.

 


    b1         b2             b3  

 

       X111=10 X11.=33  X121=22 X12.=72  X131=14 X13.=45    X1..=150

  a1  X112=11 _11.=11  X122=24 _12.=24  X132=15 _13.=15    _1..=16.67

       X113=12         X123=26         X133=16             

 

       X211=20 X21.=63  X221=21 X22.=66  X231=18 X23.=57    X2..=186

  a2  X212=21 _21.=21  X222=22 _22.=22  X232=19 _23.=19    _2..=20.67

       X213=22         X223=23         X233=20             

 

 X.1..=96          X.2..=138    X.3..=102        X...=336

 _.1..=16          _.2..=23     _.3..=17         _...=18.67


                              Chapter

                                                              11

 

 

 

Nested Two Factor Between Groups Designs B(A)

 

The Design

 

Factor B is said to be nested within factor A when a given level of B appears under a single level of A.  This occurs, for example, when the first three levels of factor B (b1 ,b3, and b3) appear only under level a1 of factor A and the next three levels of B  (b4 ,b5, and b6)  appear only under level a2 of factor A.  These types of  designs are also designated as hierarchical designs in some textbooks.

 

  A  B  X

 

  1  1  1

  1  1  2

  1  1  3

  1  2  5

  1  2  6

  1  2  7

  1  3  9

  1  3 10

  1  3 11

  2  1  1

  2  1  2

  2  1  3

  2  2  1

  2  2  2

  2  2  3

  2  3  1

  2  3  2

  2  3  3

 

Nested or hierarchical designs can appear because many aspects of society are organized  hierarchically.  For example within the university, classes (sections) are nested within courses, courses are nested within instructors, and instructors are nested within departments. 

 

In experimental research it is also possible to nest treatment conditions within other treatment conditions.  For example in studying the addictive potential of drugs, drug type could be nested within drug function, i. e.  Librium, Valium, and Xanax are drugs used to treat anxiety, Prozac is used to treat depression, and Halcion sold as a sleeping pill.  Here each drug appears in one and only one level of drug function.

 

The Data

 

The data are organized similarly to the A x B design.  Note that the data in the following table is identical to that in Table 6.1.

 


 

SPSS commands

 

The SPSS commands to do  this analysis are identical to those necessary to do the A x B analysis with one exception; the DESIGN subcommand must completely specify the design.  In this case the design can be specified with the effects A and B WITHIN A, where A corresponds to the main effect of A, and  B WITHIN A, the nested main effect of B.  The following text box contains the SPSS program to run the analysis.

 

The Analysis

 

  RUN NAME PSYCHOLOGY 460 ‑ ASSIGNMENT 4 ‑ DAVID W. STOCKBURGER

  DATA LIST FILE='ANOVA2 DATA A'/A B X X2 X3 X4 1‑12

  LIST.

  MANOVA X BY A(1,2) B(1,3)

      /PRINT=CELLINFO(MEANS)

      /DESIGN A B WITHIN A.

 

The analysis of the B(A) design is similar to the A x B design.  The Tables of Means will be identical, but the graphs drawn from the means will be done differently than the A x B design.  The ANOVA table will contain only two effects and interpretation will be discussed.


Cell Means and Standard Deviations

FACTOR CODE  Mean  Std.Dev. N

   A   1

    B   1   2.000  1.000    3

    B   2   6.000  1.000    3

    B   3  10.000  1.000    3

   A   2

    B   1   2.000  1.000    3

    B   2   2.000  1.000    3

    B   3   2.000  1.000    3

 

The Table of Means

 

The Table of Means produced in this analysis is identical to that produced in the A x B design.

 

Graphs

 

 

Because the qualitative or quantitative meaning of levels of a at each level of b is different, the graph of the means must be modified.  The nested main effects of b are presented side-by-side rather than on top of one another.  This is necessary because b1 is different depending on whether it appears under a1 or a2.


The ANOVA Table

 

The ANOVA table produced by design B(A) is presented below.

 

Source of Variation    SS   DF    MS     F  Sig of F

 

WITHIN CELLS        12.00   12   1.00

A                   72.00   1   72.00   72.00  .000

B WITHIN A          96.00   4    4.00   24.00  .000

 

 

Interpretation of Output

 

The interpretation of the ANOVA table is straightforward.  The WITHIN CELLS and A main effect SS's, DF's, and MS's are identical to the analysis done in design A x B and are interpreted similarly.

 

 The B WITHIN A term is called a nested main effect and is the sum of the two simple B main effects.  If this term is significant, then the graph of the simple main effects should be drawn (Figure 8.1).  What significance means is that the points WITHIN each line are not the same height (value).  That is, the means for b1, b2, and b3 within levels a1 and a2 are different.  In the example graph the simple main effect of B under a1 would be significant while the simple main effect of B under a2 would not.  Because the B WITHIN A effect is the sum of both simple effects, the combined effect was found to be significant in this case.

 

Similarities to the A X B Analysis

 


As noted earlier, the data files for the A x B and B(A) designs are identical.  Likewise, the MANOVA commands are similar except that the DESIGN subcommand on the B(A) design must be specified because it is not a completely crossed design.  The table of means of the B(A) design will be identical to the A x B design, the difference being that they will be plotted differently.  The ANOVA source table will slightly differ for the two types of designs.  Both are presented below to allow them to be contrasted.

 

Source Table for the B(A) Design

 

Source of Variation    SS   DF    MS     F  Sig of F

 

WITHIN CELLS        12.00   12   1.00

A                   72.00   1   72.00   72.00  .000

B WITHIN A          96.00   4    4.00   24.00  .000

 

Source Table for the A x B Design

 

Source of Variation    SS   DF    MS     F  Sig of F

 

 WITHIN CELLS       12.00   12   1.00

 A                   72.00   1  72.00    72.00   .000

 B                   48.00   2  24.00    24.00   .000

 A BY B              48.00   2  24.00    24.00   .000

Note that the WITHIN CELLS and A effects are identical in both analyses.  Note also that both the SS and DF for B and A BY B in the A x B design together add up to the SS and DF for B WITHIN A for the B(A) design.  What is happening here is that the B main effect and the A x B interaction in the A x B design is collapsed into the nested main effect of B in the B(A) design.


                                                             Chapter

                                                              12

 

 

 

        Contrasts, Special and Otherwise

 

Understanding how contrasts may be employed in the analysis of experiments gives the researcher considerably flexibility in the specification of  "effects". 

In addition, the study of contrasts leads to even greater flexibility if multiple regression models are employed.  Using contrasts, the researcher can test specific “theory-driven” comparisons between groups.

 

Definition

 

A contrast is a set of numbers.  When a set of means are being contrasted, a contrast for the set of means will contain the same number of numbers as the number of means.  For example, if six means are being contrasted, a contrast for the set of means would contain six numbers.  The order of the numbers in the contrast corresponds to the order of the means.  Examination of the signs of the numbers in the contrast describes which means are being contrasted.

 

For example, the following contrast

 

1          1          1          -1         -1         -1

 

would compare the first three means with the last three means.

 

A contrast of the form

 

2          2          -1         -1         -1         -1

 


would compare the first two means with the last four means.  The manner in which the means are compared is determined by similar number.  In the above example, the first two groups share the same number (2), while the last four groups share the number minus one (-1).  All groups sharing the same positive number are compared with all groups sharing the same negative number.  If a contrast contains zeros, then those groups with zeros are not included in the contrast.  For example:

 

0          0          3          -1         -1         -1

 

Would compare the third mean with the last three means.  The first and second means would not be included in the calculation or interpretation of the contrast.

 

Sets of Contrasts

 

A set of contrasts is simply a number of contrasts considered simultaneously.  For example, the three contrasts presented above could be combined into a set of contrasts as follows:

 

contrast 1                     2          2          -1         -1         -1            -1

contrast 2                     0          0          3         -1         -1            -1

contrast 3                     1          1          1         -1         -1            -1

 

Orthogonal Contrasts

 

An orthogonal contrast is two contrasts such that when corresponding numbers in the contrasts are multiplied together and then the products are summed, the sum of the products is zero.  For example, in the preceding example, contrasts 1 and 2 are orthogonal as can be seen in the following:

 

contrast 1                        2          2          -1         -1         -1            -1

contrast 2                     x 0       x 0       x  3      x -1      x -1         x -1

products                       = 0       = 0       = -3     =  1      =   1        =   1     

 

The sum of the products, 0 + 0 + -3 + 1 + 1 + 1 , equals zero, thus contrast 1 and contrast 2 are orthogonal contrasts.


Non-orthogonal Contrasts

 

Non-orthogonal contrasts are two contrasts such that when corresponding numbers in the contrasts are multiplied together and then the products are summed, the sum of the products is not zero.  For example, in the preceding example, contrasts 2 and 3 are non-orthogonal as can be seen in the following:

 

contrast 3                        1          1           1         -1         -1            -1

contrast 2                     x 0       x 0       x  3      x -1      x -1         x -1

products                       = 0       = 0       =  3      =  1      =   1        =   1     

 

The sum of the products, 0 + 0 + 3 + 1 + 1 + 1 , equals six, thus contrast 2 and contrast 3 are non-orthogonal contrasts.  In a similar manner contrasts 1 and 3 are non-orthogonal.

 

Sets of Orthogonal Contrasts

 

The guiding principle is that the number of possible orthogonal contrasts will always equal the number of means being contrasted.  If six means are being contrasted, there will be no more than six contrasts which will be orthogonal to one another.  If six means are being contrasted and five orthogonal contrasts have already been found, then there exists a contrast which is orthogonal to the first five contrasts.  That contrast is not always easy to find.

 

Finding Sets of Orthogonal  Contrasts

 

In ANOVA the first contrast, which will be referred to as contrast 0, is a set of 1's.  This contrast will be seen to be equivalent to the m term in the score model.  For a contrast comparing six means, the first  contrast would be:

 

contrast 0                     1          1          1          1          1             1

 


In order for any following contrasts to be orthogonal with contrast 0, it can be seen that the sum of the numbers in the contrast must equal zero.  Both  contrasts 1 and 2 described earlier fit this criterion, and since they are orthogonal to one another, they will be included in the current set of orthogonal contrasts.

 

contrast 1                     2          2          -1         -1         -1            -1

contrast 2                     0          0          3          -1         -1            -1

 

In finding a fourth contrast which is orthogonal to the first three, look for patterns of means in the preceding contrasts which have the same number.  In both contrasts 1 and 2, the first and second means have the same number, 2 in contrast 1 and 0 in contrast 2.  Working with this subset, finding numbers which sum to zero, and setting all other numbers to zero , the following contrast is found:

 

contrast 3                     1          -1         0          0          0             0

 

The student should verify that this contrast is indeed orthogonal to all three preceding contrasts.

 

Using the same logic, the fifth contrast will involve the fourth, fifth, and sixth means.  There are any number of possibilities, but the following will be used:

 

contrast 4                     0          0          0          -2         1             1

 

The final contrast will compare the fifth mean and the sixth mean.  Any two numbers which sum to zero (i. e.  2.34 and -2.34) could be used without changing the analysis or interpretation, but 1 and -1 make the contrasts parsimonious.

 

contrast 5                     0          0          0          0          1             -1

 


Putting the six contrasts together as a set results in the following:

 

contrast 0                     1            1        1         1         1            1

contrast 1                     2            2        -1         -1         -1            -1

contrast 2                     0            0          3        -1         -1            -1

contrast 3                     1          -1           0        0         0            0

contrast 4                     0            0          0        -2         1            1

contrast 5                     0            0          0        0         1            -1

 

G  X

 

  1  1

  1