STABILITY OF STOCHASTIC JUMP PARAMETER SEMIMARKOV LINEAR SYSTEMS OF DIFFERENCE EQUATIONS
Efraim Shmerling
Department of Computer Science and Mathematics,
Ariel University Center of Samaria, Ariel, Israel
Abstract. The asymptotic stability of stochastic Itỏ-type jump parameter semiMarkov systems of linear difference equations is examined. A system of matrix
equations is presented for which the existence of a positive definite solution of the
system implies the asymptotic stability of the stochastic semi-Markov system. Finally,
an illustrative example is given.
1. INTRODUCTION
Linear systems of differential and difference equations whose coefficient matrices
depend on finite-valued Markov processes have been studied intensively during the
last decades (see [3] and [4] ) and have been shown to have broad applicability. We
started to study more general semi-Markov systems in [1,2].
Here, we consider the following stochastic semi-Markov system of difference
equations
X (t )  A( (t )) X (t )dt  B( (t )) X (t )w(t ), t  0,1,2,.... ,
(1)
where w(t) , t= 0,1,2,… is a discrete analogue of the standard Wiener process ,
ξ(t), t= 0,1,2,… is a finite-valued discrete-time semi-Markov process with n possible
states 1 , 2 ,...,  n that jumps from state  k to state  s at times tj, j =0,1,2, . . . , with t0
= 0. Note that ‘jumps’ to the same state are possible, in which case k = s. The random
sequence { ξ(tj, j =0,1,2, . . .} is a stationary Markov chain with known transition
probabilities
 ks : P{ (t j 1 )   s |  (t j )   k } ; k, s = 1, 2, . . . , n.
The duration of time during which the process belongs to state  k before it jumps
to state  s , k, s = 1, 2, . . . , n, is given by a discrete integer-valued random variable
Tks whose distribution function Fks(t)= P{Tks ≤ t} and corresponding probability
density function pks(t) exist and are known. The intensities qks of the jumps from state
 k to state  s are given by
qks(t) = πksp ks(t),
k, s = 1, 2, . . . , n, t = 1, 2, . . .
and the probability density of the elapsed time Tk between two jump times tj and
tj+1,
provided that the process jumps to state  k at time tj, is given by
n
q k (t )   q ks (t ), t = 1, 2, . . .
s 1
If Fk(t) denotes the probability distribution function of Tk and ψk(t) denotes the
probability of the event that no jump takes place during the interval (tj, tj + t),
provided that the process jumps to state  k at time tj, then
7-150
 k (t )  1  Fk (t ) 

q
 t 1
k
( ), k  1, n ,, t = 1, 2, . . .
In Section 2 of this paper, we state as an assertion a sufficient condition for
asymptotic stability of the zero solution of system (1). The assertion is analogous
to a theorem proved for stochastic semi-Markov systems of differential equations (the
proof of the theorem is given in [1]). We assume that the assertion for system (1) can
be proved in a similar way. Then, in Section 3, we give an example which illustrates
how this assertion can be used to examine asymptotic stability.
2. FORMULATION OF ASSERTION
Mean square asymptotic stability of the zero solution of the system of stochastic
difference equations(1) follows from the convergence of the series

H k   k (t )k* (t )Wk , k  1, n
t 0
and the existence of a positive definite solution of the system of matrix equations

n
S k  Lk   q ks (t )k* (t ) S s , k  1, n
(2)
t  0 s 1
where
S k , k  1, n
are matrix variables, Lk , k  1, n
are some positive
matrices, and Wk , k  1, n are some positive definite symmetric matrices. The linear
transformations k* (t ), k  1, n, t  1,2,... are defined by the equalities
Wk (t )  k* (t )Wk (0), k  1, n, t  1,2,... , where Wk (t ), k  1, n, t  1,2,..., are the
solutions of the matrix difference equations
Wk (t )  AkT Wk (t )  Wk (t ) Ak  BkT Wk (t ) Bk , k  1, n, t  0,1,2,... (3)
3. EXAMPLE
Consider the stochastic difference equation
x(t )  a(t ,  (t )) x(t )dt  b( (t )) x(t )w(t ), t  0,1,2,.... ,
where w(t) designates a discrete analogue of the standard Wiener process, ξ(t) is
a semi-Markov process which takes two values, 1 and  2 , and is defined by the
intensities
q11 (t )  q 22 (t )  0, t  1,2,...
0, t  (0, T ), T  0
q 21 (t )  q12 (t )  
t T
 (1   ) , t  T
We use the notations: a( k ) : ak , b( k ) : bk , k  1,2.
We have k (t ) 

(1   )


T
 t 1T , k  1,2, t  T
t 1
 (t )  (1  2a1  b12 ) t , 2* (t )  (1  2a2  b22 ) t , t  0,1,2,...
Obviously , the series h1 and h2 converge iff , a1 , b1 , a2 , b2 satisfy the
*
1
inequalities  (1  2a1  b12 )  1 and  (1  2a2  b22 )  1.
7-151
System (2) takes the form :


t T
s

l

s
 (1  2a1  b12 ) t
2  (1   )
 1 1
t T


s 2  l 2  s1  (1   )t T  (1  2a 2  b22 ) t

t T
and can be rewritten as
2

T  (1  2a1  b1 )
s

l

s
(
1


)

1 1
2
1   (1  2a1  b12 )


2
s  l  s (1   )T  (1  2a 2  b2 )
2
1
 2
1   (1  2a 2  b22 )

(4)
(5)
The condition s1 , s2  0 is satisfied, provided that the inequality
(1  2a1  b12 )(1  2a 2  b22 )
(1   ) 2 (T ) 2 2
 1 (6)
[1   (1  2a1  b12 )][1   (1  2a1  b12 )]
holds. The inequality (6) is a sufficient condition for the mean square asymptotic
stability of the zero solution.
REFERENCES
[1] E. Shmerling, K.Hochberg
Stability of Stochastic Jump Parameter Semi-Markov Linear Systems of
Differential Equations, to appear in: Stochastics: An International Journal of
Probability and Stochastic Processes.
[2] E. Shmerling and K.J. Hochberg, Solution of Jump Parameter Systems of
Differential and Difference Equations with Semi-Markov Coefficients, J. Appl.
Probab. 40 (2003), pp. 442–454.
[3] M. Mariton, Jump Linear Systems in Automatic Control, Marcel Dekker, New
York, 1990.
[4] D.D. Sworder and J.E. Boyd, Estimation Problems in Hybrid Systems,
Cambridge University Press,New York, 1999.
7-152