UNIVERSITY OF NORTH CAROLINA
Department of Statistics
Chapel Hill, N. C.
ON THE lVIONOTONICITY PROPEH'ry OF THE THREE MAIN TESTS FOR
MULTIVARIATE AFfALYSIS OF VARIANCE
by
J. N. Srivastava
Ie
December 1962
Contract No. A}' 49(638)-213
This paper presents a unified proof of the monotonicity
property of each of the three main tests used in multiresponse analysis of variance, and also some other interesting side-results.
ThiS research was supported by the Air Force Office of Scientific
Research.
Institute of Statistics
Mimeo Series No. 342
ON THE MONOTONICITY PROPEffi'Y OF THE 'I'HREE MAIN TESTS FOR
MULTIVARIATE ANALYSIS OF VARIANCE
l
by
===1.
~
J. N. Srivastava
University of North Carolhla
-- -- -- - - - -- -- - - - -- = =
=====~ =
~
======= ==
-
=
This paper presents a unified and reJatively simple proof of the mono-
tonicity property of the three well known tests for the multivariate analysis of
variance (MANOVA) viz, (i) Rotelling - Lawley trace criterion (ii) Wilkfs likeIihood ratio criterion and (iii) Royls largest root criterion.
the property has
2.
alI'<:~ady
been proved by Roy and Mikhail
For the last one,
£)] .
To provide a background, we shall start with the multivariate linear
model, present the three criteria in explicit form and discuss the general nature
of the power function.
To this end we shall state several results, for the proof
of which the reader is referred to f~7.
The usual multivariate linear model is:
E (Y)
::::
nXll
(1)
Var (Y)::::
where l:
A
!1
nxm mxp
I
n
ts an unknown p.d. matrix,
(g) l:
®
denotes Kronecker or direct product of
matrices, and
,
..
i~(l)
f
,
:
Ji
'y
, -en)
c.
is a matrix of obse:cvations, in which the j-th column
e
lxp
~
y.
corresponds to the
-J
j-th variate, and the i-th row to the i-th experimental unit.
(1),
I(r)
and l(r )
'
~his research
"JaS
are assumed to be independent if
r
As indicated in
~ ri
•
Further each
supported by the Air Force Office of Scientific Research.
2
vector X(r)
tribution.
(r
= 1,
2, ••• , n)
In fact
(2) Prob (y) = const. exp
where
is assumed to have a multivariate normal dis-
d Y
£-
TI. .'
stand.s for
~ tr
2::-
1
(yt - III Al
d y.. •
~,J
~J
Let Al(~xr) be a basis of A, where
nxY.'
where
(y - A llt7d Y •
)
man
nxr rxr
T is triangular, and
r
= Rank
nxr
L is orthono:cmal.
trary orthogonal completion of
L1 •
(A), and let
mo:n-r
Also let
Ll(nin:r) be an arbi-
Consider the (testable) null hypothesis
c
(4 )
sxm
where
C (atxm)
1
C, where s!
is the matrix obtained by taking any independent
= Raru~ C. Now in the model (1), the
of II
induces a partitioning
pal~itioning
rows of
S I
(3) of
A
as below
r
I Til
rxp
=
II
mxp
This is turn induces, through
(4),
a partitioning of
( 6)
Make the fGctorization
where
r
'r,-·L
= r
N
11
s'xr rxr
s'xs's'xr
is triangLilar and N is orthonormal, and let
gonal completion of
N.
C
1
N1 ( -'~-s t , 1~)
..
Consider first the transformation from
given by
W
2
n~rxp
=
L
1
Y
be an ortho-
Y to Wl , W2
3
One can interpret this by observing (as can be mathematically proved) that
relates to the set of best linear unbiased estimates of estimable linear
tions) and W
2
(8)
Z
==
1
s'xp
relates to the error.
N
W
s ?xr rxp
It can be shoinl that Zl
Next make the decomposition:
Z2 ==
Nl
M t :h'P X;:'s t Xl'
1
~lnc-
Wl
rxp
relates to the set of linear functions which are being
tested under H ' and Z2 corresponds to the rest of the estimable linear functions.
o
Indeed it can be easily seen
pothesis
H
o
(9)
e
is
-r2';
-'
Y'S Y where
H
Y' SH Y ==
and the
S11.111
(10)
that the sum of products matrix due to the hy-
Z?
1 Zl
,
of products matrix due to error is
Y'
SE
Y == H!
2
Since the three tests of
Vl
2
lf~OVA
are in terms of the roots of
2
(Y' SE y)-l
i.e.) of (Zi Zl)(H W )-1, we restrict ourselves to
2
Their joint distribution is given by
Zl
(y t SH y)
and W
2
0
where
The null hypothesis
(13)
H :
o
H
o
reduces to
==
'rhe following results ivill now be needed.
For its proof, one may see
£~7.
4
Theorem 2.1.
with
c
l
Let
~
c
2
~
c
.••
(j -
j
?
1, 2, ••• , p)
cpo
be the roots of
(Zi Zl)(W~ W )-1 ,
2
Then the joint distribution of these roots involves
as parameters just the following quantities:
(14)
where y
(
Q
> y
1-
>...
2 -
> yare the p urJmown latent roots of the matrix
P
gl )-1
I:
•
t
l
Theorem. 2.2<
There exists a matrix VI (p x
of the elements of I:
W , and a matrix
and of
2
functions of the elements of I:
(15)
e
ch
L(Zi
T
the distribution of VI
_ 12 (n-r+s t
(2rc)
(V2 )ii
2
and of
_7
c j (j = 1, 2, ••• , p)
so that
where
l
2 V, 2 f
Zl)(VJ
)
r-
.I~.
whose elements are functions
V2(pxs')
whose elements are
Zl' such that
Vif~)
2
= ch £(V 2 V )(V I
are the p roots of
and V
2
e"p
J'
'n,~r)
(V2
,
V~)(VI
V])-l also.
Also
is given by
-12
t
2) +l=. I:1y.l
tr(VIVi + V2V
represents the (i,i) element of
1
t
'2
- 2 2: (V).:Y.
. I ' l l J.
l::::
_7
V2 , and t;::: Rank (Qi Ql) •
(It is clear that the truth of Theorem 2.2 implies the proof of Theorem 2.1).
We now come to the three tests for the hypothesis
(i)
(16)
Hotelling-Lawley trace criterion, say
P
2:
j=l
c.
J
~l
where
::: 0
t
S
:P
•
These are
5
(ii)
W:i.lks I likelihood ratio criterion:
(17)
(Hi)
Roy1 s lsrgest root criterion:
(18)
The distribution of
does not involve as an unkno'W!l parameter just
but all the roots
Yl'
Y'':JJ'.
Co
Similarly the distribution of
P
does not involve just
separately.
0' \( p •
(1 + y,) or Y ' but all the roots
1
j=l
J
The problem before us is to prove that the power of each of these
three criteria is a monotonically increasing function of each of the
meterG
p
para-
Y1 ' Y2 ,···,Y p •
Write
VII
.OQ • • • •
I
I
V
p1
Let
*
~
....•. v
I
pSIJ
•••••••••• VI
-l
•••••••••• V I
.J
l,n- ;r
vls'l
v1 =
pxn-T
be any of the three criteria, and
~I
11
VI
pl
p,n-r
the corresponding power.
acceptance region is of the form
where
~
is a constant, and consequently we have by
(16):
Then the
6
P
l1-r
.E v ,2
.. +
"
i::l
L>
+
s'
p
.E
2
t
P
Pr'
I)'
ex~~-
P
"
1
2
n-'1'
L>
d\T2
pSi
.2
v:.
.E
i=l j==l
IJ.
1.J
_7 d v1
v.. +.E 'Y. - 2.E v.. !'Y i
1=1 j=l lJ i=l 1
i=l 11.
.E
' -2\n-:r.+s
::, j (2n)
w ,:::
j::;:l
"
~
+
.E
i=l j=2
lJ
2
vij
+
,
I'There we define
== 0,
p?:
i > t.
Now vlrite
,
=
(21)
so that
!y";.
~l
is the first column of V •
Then the last integral can be written
2
~)
dV
1l
where
¢o
dV
2
21
represents the part of the integrand (say at (20»
the big curly brackets, and where
R
1W
obtained by taking any fixed value of
which is not inside
represents the section of the region
V
l
proof of the monotonicity with respect to
and V
21
~
0
For any given
W,
the
will be completed, if we can
7
fY~
show that as
increases the quantity in the big curly brackets decreases,
for any fixed value of VI
then follow that
and V •
21
From the symmetry of the proof it will
for
is an increasing function of
~~
i :; 1, 2, ••• p.
3. Some simple general results on monotonicity •
.Lemma 3.1.
Let
... , xp
decreasing function of
("
i)
e
is the
R
jHx
(23)
l
x.•
Then for all
l
, x '
2
. .,
i
=J"
~(xl
i:; 1.
-II... C
r
J
shifted by a distance
1
~(xl" x2 '
.
... , xp
) d x
t Qi )
We have
+ Ql' x2 ' ... , xp ) dxl dx 2 ••• dxp
_il..
dx
P
since
Lemma
is a decreasing function of
9
1
3~2.
In the above, srrppose further that
(24)
where
o •• ,
=
+1
or
,
x )
-1,
p
i ::;: 1, 2, .,. p •
R in the
R, and suppose (for any
:
.-" L'
Take for example
Let
p
regioll-0~
x ) d x >
p
~
JL
Proof:
(separatelY) in a rectangle
.. , x.
p-dimensional space of
along the axis
be a positive, real-valued monotonically
• ••" x )
p
Q.
l
(> 0)
8
Suppose also thatJl- ( C,. R)
is such that
..... ,
for
-1 < c.
~
J Hx
(26)
l
< 1,
i ~ 1, 2,
, x2 '
.. , xP ) dx
p.
"'J
"
----lL-
>J
.-lL.
~
Proof:
1jr(xl , x2 ' ••• ,
X
\~(xl' x2 '
.... ,
i,
x ) dx
P
(g. )
~
Consider fo::: example the case
J
Then again, for any
= 1-
i
We have
p ) dx
_,'1'1(9 )
1
=
J ~J
x , x .,
2
3
H
X
0'
p
f
1jr (xl' x2 ' ... ,
Xl I_~
€
..fl.-I(G J
Xp )
dx l }
L7
dx
Now take a fixed value of
Xl
such that
X
€
x2'~' "0'
i
idx
dx
1 )~
2' • •
p
xp
for which there exists an
From (25)1 it follows that the range of possible
R ~ 0, and
l
depends on the chosen value of x , X ' ••• , x •
2 3
p
the last integral can be vlritten as
values of
Xl
in
~L_
.-f2..
dx2 • •• dxP
is an interval (-R , R ), where
1
1
Hence
9
l
I
cUe
p
R + Q
l
l
=
J { I V(x
1,
X2'
-Rl + Q
1
However we can write
R + (;:1
l
(27)
,J
W(xl , x 2' ••• , xp )
~Rl
=
l
+ 91
R
l
'e
dx
J
t(~)
dx.1
r
-
l1r (x) clx
,j
-R l
R + Q
l
l
-R-.1 + Q1
'-,-'
.1
+
J ~(~)
Rl
-R1
But
-R
dx
1
1
+ 9
J ~(~)
-
dX l
1
CL"C I
- Rl
R
l
J'
=
$(x) dx
,
using (24)
R ~ ~\
l
This last expression is negative for all real
a decreasing function of
Xl
for
Xl
~
O.
gl' since
v
x2"
r
x , •• o,X
3
:P
)
)
I
\.
J
-Rl
}
~ (.?E) dx l
7
I
dx
,I
,
2
x , ••• ,
2
X
p
)
Hence the expression (27) is less
than
R
l
~(xl'
... CL"C:P
is
10
which completes the proof.
4.
The monotonicity property of the trace and the likelihood ratio criteria.
Theorem 4.1.
The power of the trace criterion is a monotonically increasing
function of
IYtl
for each
i
= 1,
2, ••• , p.
He have
Proof~
=
Let
(v V,)-l
(28)
1
"t-There V (pxp)
1.
=
triangUlar~
is
V'
V
,
Then
V'
Hence the region
Since V Vi
l
'1'2 _<
,I,
can be written
~
is clearly positive definite, the region (29) is the interior of
an ellipsoid (centered at the origin) in the p-dimensional space of v
v
lp
•
Also then there exists an orthogonal matrix p*
P* i
4It
where
D
l
(
V1 Vi~ )-1 P *
=
Dl
such that
'
is the diagonal matrix of the roots of
(V Vi)-1
l
l1
, v
12
, ••• ,
11
Now in (22)
consider the integral
,
( 0)
and make the transformation
v
==
-1
to get
(1)
J
where
*
Pi1
lemma 3.2,
.*
P u
,
form the first column of P* ,and u == (u , ••• ,up ) '
1
'Vle
take
·e
u' D u
==
1-
< canst.
~
* ~,
Pi1
==
i == 1, 2, ••• p, and
const. exp
1
f
-"2
P
I:
1
is a decreasing function of
_7 ,
The pO';"er of the likelihood ratio test
~lnction
of
~
for each
i == 1, 2, ••• , P •
From the mechanics of the proof of the last theorem, it is clear that it
is enough to show that the region
the origin.
e
i
This completes the proof.
is a monotonically increasing
Proof:
2
U
From this it follows that (30)
then the conditions of the lemma are satisfied.
Theorem 4.2.
If now in
Rl $2 in (22) is an ellipsoid with center at
Now
*2
==
Det • .LIp + (V2V'2) (V1V])-1
==
jvlvi
+
V2V'2. / / !vlVi)
_7
12
Hence (32) reduces to
But
*
clearly VI
is nonsingular p.d. alnost eveIJrwhere.
=
Hence
r( .::1 "::1 VI*-1 + I p ) V*l _7
I
det _
*.
V !.
l'
Let
~
be the nonzero root of
IIp + ~l ~i V~-l
,
*-1
root of !l VI
I
~l}
~l
I
*-1
Xl VI } then
= (~+l)( 0+1) ••• (0+1) =
which is a scalar.
C)
+ 1.
But
w
is then also the
Thus
=
so that (32) becomes
(33 )
1
Clearly VI
*
depends just on VI
and V21 and is
is an ellipsoid with center at the origin.
p.d.
Hence by (33)
Rl~
This completes the proof.
5. Same interesting sidelights on the proof of the monotonicity of the
largest root criterion.
2
13
As stated earlier, a proof of this property has already been given by Roy
and Mikhail.
However, if we attempt to carry out the proof along the lines used
for the other two tests we pass through certain results which appear to be
interesting on their mm.
We shall state these and present the proof only in
case the result is new or the proof is of use otherwise.
Lemma 5, J. :
If
c
VI )-1 then the equation
( V2V"2 )(V
. 11'
is any latent root of
::::
(34)
0
yields a homogeneous second degree surface in !l ' provided VI and V21 are
held fixed.
The equation (34) is equivalent to
Proof:
or
(35)
I.!l xi + v~c \ ::::
In case
V*lc
0
,say
is nonsingular, this is equivalent to (as in the last theorem):
, *-1
1 + .Y.l VIc .!l
(36)
,
.X-
v'
··-1 '. -VIC
fl
Xl
::::
0
,
::::
1
,
or
which completes the proof.
Lemma
5.2~
(Random Separation of the largest root).
There exists a real ntwber
Al
* which depends on VI
and V21 but not on
such that
(37)
It is well known that both equalities can hold only at a set of measure zero.
14
Proof.
c > c , which is true a.e.
l
2
Assume
Let El and
(V V )(V VV- l ::: f, say.
characteristic vectors of
2
b~
'''J
2
l
f b.
-J
-
c.,
~2
be the normalised
Then it is well known that
j ::: I, 2 •
J
Let
,
V' )-1 - V' V
( V1
1-
d.
b~
:::
-'J
,
V
'-'J
V is triangular, and
'Where
Then
C
j
:::
d' v
v'
"'j -1 --I
d
,
d~ V
V ' 1 <1.
-"J 2J- 2
.J
+
-j
Define
,
A,j
and suppose that the largest root of
:;<..
:::
(1?,
sup
b
;:
A'I
*,
j
21
V~l
V V21 V~l VI
V V
E1
V V
21
V
V'
Then
E)
b* , say .
21 VI M1
Hence
(:58)
'7
VI)
* < E 1*' V (v···1 v'
A,1
-1 + '21 '21'
*1
E1
f
For aay' fixed V !1
,
:::
"
°~ ~i ' Q~ 5
:::
:;:
0,
But
~'
~l E1 +
are real nurubers.
1.
such that
choose a vector ~
~
gl' G
2
b,*
-.L
b* < c
1
--1 .'•..
~' V .v,
- .l.
where
Vi
~
Q2
:::
E2
1,
antI
,
Notice that this can always be done, and that
15
g;' rg;
;::
~2)
(Q1 Ei + Q2
r;l1-1
b'
r
;::
c
2
Q
1
+
c
:=
c Q2
1 l
+
(1 -
~1 + Q2
2
~1 + Q2 E2)
(Q1
2
;::
1
r
b'
r
-2
E2
2
Q
2
~~)
c
2
Hence
(40)
c
< gl
2--
However
21 V'
< g;'
~
V V21 V
< /1.1*
Thus
and the lemma is proved.
Corollary.
Consider the roots
V2l • As !1 varies, let
c ,
1
Sli(~)
c.~
C
2
' ••• , c
jJ., if
if any
Sll(lJ.)
=
from the fact
Lemma
Sll(~)
and
lJ.
exi.sts, then none of
Sli(lJ.) (1 ~ 2)
for dny fixed value of VI
(i:= 1, 2, ••• , p) be the surface
in the p-dimensional space of 2:1' where
of
p
exists, then
jJ.
is a constant.
S12(lJ.),
Sll(~)
.u,
Then for any value
SlP(jJ.)
does not exist.
exists or not according as
lJ. ~ /1.*
1
or
exist.
Similarly
A proof follows
~
< /1.1*
5.3. (a) Recall (36). Let Sl(lJ.) be the surface given by the equation
16
where
is a given real number.
~
(b)
Let
surface
~
Then
be a given real number.
Sll(f1) exists (Le.
if
For fixed
V
l
and V ' if the
21
*
~ ~ /1.1)' then its equation is given by (40).
The proof of this lemma is easy.
The proof of the monotonicity of ROY!s
test then follow's by showing that (41) is an ellipsoid or that (-V~I)
*
definite (if Il ~ /1.1)'
But this it true since
(-V* )
l~
::::
"(V'
V)-l _ V21 V'21
,...
r
:::: _ Il I p and
/1.*
1
is positive
is the largest root of
(V21 V:21 )(V
1
V) _71\ VI V)-l
,
(V 21 V )(V: V) •
21
The crucial role played by the separation property (of Lemma 5.2) is evident.
The author feels that this and other related properties may be useful in other
techniques also in normal multivariate analysis.
6.
Ackncwledgement.
Thanks are due to Professor S. N. Roy for going
through this paper and for the research opportunity he gave me during 1961-62.
Lemma 5.2
was suggested by him, together with a partial proof.
Professor Roy
also informs me that some of the above results have been independently obtained
by G. S. Mudholkar (using a different approach) some time after I obtained them.
17
REFERENCES
i-~7
Anderson, R. W.,
Introduction to Multivariate
Analysis, John Wiley and
Sons, New Yorl~, 1956.
15.7
Roy,
s.
N., .~~~.ne Aspects of !'i:ultiva:date. Analysis, John i'liley and Sons,
Nevr York, 1957.
f)]
Roy, S. N. and Mikhail, W., "On the monotonic character of the power functions of tyro multivariate t.ests", Annals of Mathematical Statistics,
Vol. ,32, 1961.