Solutions I - Markovian modeling and Bayesian learning
1. The complete proof is given in the lecture material (excerpt from Koski’s HMM book).
2. a) For example, a reducible DMTC is de…ned by the following transition probability matrix,
where rows 1-4 correspond to fA; C; G; T g, respectively:
0
1
1=2 0
0 1=2
B 1=4 1=4 1=4 1=4 C
C
P =B
@ 0 3=4 1=4 0 A ;
1=3 0
0 2=3
since C = f1; 4g is a closed set. The network graphs are straightforward to obtain and are not
shown in these solutions.
b) For example, de…ned otherwise similarly to the above, the following P corresponds to the
two communicating classes C1 = f1; 4g; C2 = f2; 3g:
1
0
1=2 0
0 1=2
B 0 1=5 4=5 0 C
C
P =B
@ 0 3=4 1=4 0 A :
1=3 0
0 2=3
c) For example, the following P has the sought
0
1=3 1=6
B 1=4 1=4
P =B
@ 1=10 1=5
1=9 1=9
properties:
1
1=6 1=3
1=4 1=4 C
C:
1=5 1=2 A
1=9 2=3
To obtain the stationary distribution, solve the following set of equations:
= P;
i.e. we have
1
1
1
1
+ 2 + 3
+ (1
3
4
10
1
1
1
1 + 2 + 3 + (1
6
4
5
1
1
1
1 + 2 + 3 + (1
6
4
5
1
1
1
1 + 2 + 3 + (1
3
4
2
which yields the solution:
1
=
108
641 ;
2
=
1
2
1
2
1
2
1
2
100
641 ;
3
=
1
1
9
1
3)
9
1
3)
9
2
3)
3
3)
100
641 ;
4
=
1
=
2
=
3
=
(1
=
333
641 .
1
2
3 );
3. Here is a Matlab code for the task and the convergence is illustrated by a …gure below the
code.
%Simulate a MC de…ned in 2-c and show the empirical stationary distribution.
P=[1/3c1/6 1/6 1/3;1/4 1/4 1/4 1/4;1/10 1/5 1/5 1/2;1/9 1/9 1/9 2/3];
n=10000;%Length of simulated chain
MC_sim=zeros(n,1);%Simulated realization of the MC
MC_sim(1)=1;%Initialized with a …xed value
for t=2:n
MC_sim(t)=…nd(mnrnd(1,P(MC_sim(t-1),:)));
%Draws a multinomially distributed random variable corresponding to a
%row of P, where ’…nd’de…nes the index of the sole non-zero value
%in the 1 x 4 vector outputted by mnrnd
end
tabulate(MC_sim)%Show the empirical stationary distribution and compare with the analytical
solution
true_distribution=[108/641,100/641,100/641,333/641]
%This shows how the empirical frequencies converge
MC_conv=zeros(4,n-9);%Only t>=10 will be considered to avoid problems with zero frequencies
for t=10:n
freqs_upto_t=tabulate(MC_sim(1:t));%Checks the output up to t
MC_conv(:,t-9)=freqs_upto_t(:,2)/t;%Picks the frequencies and normalizes them
end
%Plots the changes in frequencies against t, starting from t=10
plot(10:n,MC_conv(:,1:n-9))
0.55
0.5
0.45
0.4
0.35
0.3
0.25
0.2
0.15
0.1
0.05
0
2000
4000
6000
8000
10000
s
2
4. The counting process Nt will take values in the set f0; 1; :::g and it is a Markov chain as the
Markov property is satis…ed for all t :
p(Nt = nt jNt
1
= nt
1 ; :::; N1
= n1 ; N0 = n0 ) = p(Nt = nt jNt
1
= nt
1 );
where nt is the state of the process at time t and
p(Nt
=
nt
p(Nt
=
nt
1 jNt 1
1
= nt
+ 1jNt
1
1)
=1
p;
1)
= p:
= nt
The process is stationary because the two transition probabilities from any state are the same
irrespectively of the time index. The waiting time distribution for the transition from a state nt to
nt + 1 is the Geometric(p) distribution:
p(T = k) = (1
p)k p; k = 0; 1; 2; :::;
where k corresponds to the length of an uninterrupted sequence of copies without an error.
5. The 1st order Markov chain is obtained by enlargening the state space to the cartesian
product fX X g, with X = fA; C; G; T g. The transition probability will thus be a 16 16 matrix,
where every row has only 4 non-zero elements according to the example shown below.
Xn
1 nXn
AA
AC
AG
AT
CA
CC
CG
CT
GA
GC
GG
GT
TA
TC
TG
TT
AA
p j
0
0
0
p j
0
0
0
p j
0
0
0
p j
0
0
0
AC
p j
0
0
0
p j
0
0
0
p j
0
0
0
p j
0
0
0
AG
p j
0
0
0
p j
0
0
0
p j
0
0
0
p j
0
0
0
AT
p j
0
0
0
p j
0
0
0
p j
0
0
0
p j
0
0
0
CA
0
p j
0
0
0
p j
0
0
0
p j
0
0
0
p j
0
0
CC
0
p j
0
0
0
p j
0
0
0
p j
0
0
0
p j
0
0
CG
0
p j
0
0
0
p j
0
0
0
p j
0
0
0
p j
0
0
CT
0
p j
0
0
0
p j
0
0
0
p j
0
0
0
p j
0
0
3
GA
0
0
p j
0
0
0
p j
0
0
0
p j
0
0
0
p j
0
GC
0
0
p j
0
0
0
p j
0
0
0
p j
0
0
0
p j
0
GG
0
0
p j
0
0
0
p j
0
0
0
p j
0
0
0
p j
0
GT
0
0
p j
0
0
0
p j
0
0
0
p j
0
0
0
p j
0
TA
0
0
0
p j
0
0
0
p j
0
0
0
p j
0
0
0
p j
TC
0
0
0
p j
0
0
0
p j
0
0
0
p j
0
0
0
p j
TG
0
0
0
p j
0
0
0
p j
0
0
0
p j
0
0
0
p j
TT
0
0
0
p j
0
0
0
p j
0
0
0
p j
0
0
0
p j