2004 International Conference on Signal Processing & Communications (SPCOM)
DATA ASSOCIATION FOR MULTI TARGET - MULTI MODEL PARTICLE FILTERING:
IMPLICIT ASSIGNMENT TO WEIGHTED ASSIGNMENT
Mukesh A. Znveri Udny B. Desni S.N. Merchant
SPA" Lab, Electrical Engineering Dept., IIT Bombay - 400076.
Email address : [mazaveri,ubdesai,merchant] @ee.iitb.ac.in
ABSTRACT
In multiple target tracking the data association, i.e. observation
to track assignment, and the model selection to track arbitrary trajectory play an important role for success of any tracking algorithm. In this paper we propose various methods for data association in the presence of multiple targets and dense clutter along with
the tracking algorithm using multiple model based particle filtering. Particle filtering allows one to use non-lineadnon-Gaussian
state space model for target tracking. Data association problem is
solved using (a) an implicit observation, (h) a centroid of observations (c) Markov random field (MRF) for observation to track
assignment.
1. INTRODUCTION
The standard Kalman filter assumes a linear model for target dynamics. Morcover, the process and observation noises are modelled as Gaussian distributed [I]. In real world applications, thesc
assumptions do not hold. For nonlinear case, one typically uscs
thc extended Kalman filter (EKF) [I]. EKF requires Hcssian and
Jacohian matrices to he evaluated and it may lead to divergence.
Particle filtering is being investigated extensively due to iw important feature of target tracking based on nonlinear and non-Gaussian
model [2, 3, 4, 5 , 61. Basic philosophy behind-the tracking using Bayesian approach is to propagate and update the probabilistic
density funclion (pdo of the state to be trackcd. In particlc filter a
pdf of the state is represented by a set of random samples, called
particles [7, 8, 91. Each particle is assigned a weight, known as
importance weight. As the number of particlc is very large, it cepresents true pdf of the state. Using these particles and weights, it
is possible to cstimate any moment of pdf of il state vector. Comparison of particle fiIter with other nonlinear filters can be found
in [IO, 1I].
Particle filtering has been extended to multiple target tracking and different methods have been proposed for this problem
[12, 13, 14, 151. In [I31 particles are sampled from the pdf representing the combined state of all the targets. In multiple target scenario the number of state parameters varies from target to target.
Morcover the computational complexity of this method increases
exponentially as number of observations increase, and number of
targets to be tracked increase. Estimating the joint probability distribution of the state of all targets makes the problem intractable in
practice.
Particle filter, of course, needs the knowledge of the model
to track a target: but more importantly it needs to know the time
instant when trajectory switches from one mode1 to another model.
Now, if the target movement is random, then the trajectory formed
0-7803-8674-4/04/$20.00
02004 IEE€
by the target is arbitrary and there is no apriori knowledge about
which model to use at a given time and when to switch. In such a
situation, panicle filter suffers from the degeneracy problem, and
the pdf of the state coIlapses. To track an arbitrary trajectory, it is
incumbent to use a multiple model based approach.
In this paper we propose the multiple model based particle
filtering algorithm which overcomes the above problems. In a proposed method different models for target dynamics are incorporated along with particle filtering, which automates the model selection process for tracking an arbitrary trajectory. The proposed
method is able to track multiple trajectories in the presence of
dense clutter, and does not require the apriori knowledge of the
time when the trajectory switches form one model to another. It
is important to note that the proposed approach does not require
any apriori information about the exact models which targets may
follow for particle filtering. We performed the simulations where
trajectories are generated using E-spline function and tracked successfully using the proposed mcthod.
Another problem with multiple trajectories tracking wing particle filter is the data association, i.e. observation to track fusion.
Various methods €or data association are described i n literature
[ 1, 161. For data association three methods arc used. In first case an
implicit observation to track assignment is performed using neighbor neighbor (NN) method for data association, which is fast and
easy to implement. In second method the uncertainty about the
origin of an observalion is overcome by using a centroid of observations to evaluate weights for particles 3s well as to calculate
likelihood of a model. In third method MRF based method has
been utilized. It allows us to exploit the neighborhood concept for
data association, i.e. the association of an observation innuences
an association of its neighbor Observation.
2. PROBLEM FORMULATION
I n this section, the problem is described in multimodel framework
to track both maneuvering and non-maneuvering targets. Let, J j
and rP denote the observation process and the state process respectively. Y t is a set of all observation set for time t 2 I, where t is
currcnt time. Yt and at represent thc realization of observation
process and state process. At time t , a vector of observations Y t
is received
Yt =
( y t ( l ) ,. . . , Y t ( N e ) )
where No represents the number of observations received. Similarly,
*t
= (@t(l),. . . , @ t ( N t ) )
Here, N t is the total number of targets at time instant t and @*(s)
(1 5 s 5 N t ) represents the combined state vector for target 3.
27
Authorized licensed use limited to: INDIAN INSTITUTE OF TECHNOLOGY BOMBAY. Downloaded on December 3, 2008 at 00:46 from IEEE Xplore. Restrictions apply.
$;"(s) is the state vector of target .s due to model m at time t ,
where 1 5 m 5 h.f and h.1 is the total number of models used to
track a particular target.
For each model the probability density function is approximated by a set of samples, called particles. Each particle is assigned a weight, known as importance weight. For every model,
particle weights [17, 7, S] are evaluated at each time instant independently. If the lrajectory does not follow any model at a
given time instant its probability density function may collapse,
or all importance weights may have negligible value for respective particles. At this time instant, particles are initialized using
mix state vector given by the interacting multiple model (IMM)
filtering method and hence, it is possible to follow an arbitrary trajectory. IMM filtering mixes the state vector from different models
using model probabilities. Mix state vector takes care of the likelihood of a model for a given trajectory. Inclusion of IMM based
approach allows us to track an arbitrary trajectory with different
models.
3. DATA ASSOCIATION FOR PARTICLE FILTERING
It is assumed that one observation originates from one target or
clutter, which leads to following constraint on assignment probabilities,
NI
s--1
3.3. Method-111: Markov random field based approach
This method also uses a centroid of observations similar to Method11, but it uses the MRF approach for characterizing the concept of
neighborhood. The association of an observation influences the
association of its neighbor observation. Here too, the model probability is catculated using a centroid of observations. This centroid
of observations is calculated using the assignment weights and assignment probabilities.
To assign observations to targets an association process, defined by 2, is formed. It is used to represent the true but unknown
origin of observations. Zi is a realization of an association For
IMM, we rcpresent Zt as combined (logically OR operation) realization of 2 and is defined as,
Zt
In the multiple model approach, it needs to assign an observation
to a track for evaluating model probability for each model. For
data association we considered three different methods.
3.1. Method-I: Implicit observation assignment
An implicit observation to track is assigned using the nearest neigh-
bor (NN)method. The model probability is calculated using an
observation assigned by the NN method. The reason behind using
the NN method is that it is easy to implement and computationally
efficient. NN method is based on minimum statistical distance.
For our proposed algorithm, innovation error is used as a statistical distance, NN method is implemented using Munkres' optimal
data assignmen1 algorithm [lS].
e Zt.1
+
Zt,2
+ .. +
'
Zt,Al
where z ~is the
, association
~
matrix at time instant t for model m.
Each zt,mis (Nt 1) x ( N o 1) matrix; the ( s , j ) th element of
the association matrix is z L , , , , ( s , j =
) 1 it' observation y t ( j ) falls
within a validation gate formed using thc predicred position from
model m for the sth target, otherwise it i s set to zero. The idea
behind using M R F based modeling for data association is that if
the observations y t ( i ) and yt($ are within the validation region
of targel s, then the association of observation yt(i) to target s
must influence the association of observation yt(j) where i # j.
For MRF hascd data association, the association is modeled as
+
+
1
~ ( Z t l G t=
) 2 exp [-V(Ztl*t)l
(3)
where V(Ztl@t)is a potential function. In our case it is written
using the first and second order cliques,
3.2. Method-11: Centroid based data association
In the presence of multiple targets and clutter, there is uncertainty
about an origin of an observation. To avoid the uncertainty about
an observation origin, thc ccntroid of observations is used for data
association. The model probability is calculated using this centroid of observations. For observation to track association, PMHT
based approach is used [19]. To overcome the uncertainty about
the observation origin, an assignment process K is used and K' is
a set of all its realization for time t 2 1. its realization at time t is
denoted by,
Kt = (kt(1) ,..., kt(N0))
wh$reVl(iiPt) is avectorwitheachelementequal toIogp(zt(i) =
for s = 1,. . . , N L and V z ( 0 , ) is a matrix of dimension
Nt x Nt with ( s , ~ L
element
)
cqual to l o g p ( z t ( i ) = e , , z t ( j ) =
e,l@t). Here zt,* represents the ith column of the combined association matrix z t . For simplicity, the normalizing term Z is
neglected, It can be shown that using this model the assignment
weight, i ? ) ( s , i ) , is given by 1201
kt ( j ) = s indicates that target s produces observation j at time t .
The elements of assignment vector K t are assumed to be independent of each other. The observation to track assignment probability
n at time 1 is given by,
and the assignment probability is given by
,(PI
t
Here, rt(s) indicates the probability that an observation originates
from target s, and is independent of the observation, namely,
7rt(S)
= p ( k * ( j ) = s),
v j = 1,.. . , N o
(1)
WEC
1
*
I
(.,4=
exp ( - v 1 z t ( i ) = esl&P)])
E
,::
(6)
exp ( - ~ [ z t ( i )= e,
I&,~)I)
Here p;"(s) represents the model probability of a model m used
for tracking with IMM filtering for target s. Here, e, is a vector
28
Authorized licensed use limited to: INDIAN INSTITUTE OF TECHNOLOGY BOMBAY. Downloaded on December 3, 2008 at 00:46 from IEEE Xplore. Restrictions apply.
of length Nt with zero value at all index i (z # s) and 1 at index s. n?)(,~,
i) is modeled as MRF-Gibbs distribution. These
assignment weights and assignment probabilities are evaluated iteratively till the predefined error convergence criterion is satisfied.
p indicates an iteration number. It shows that assignment probability for an observation associated to a target s is not the same for
all'observations but it varies from observation to observation and
depends on its neighborhood association. For simulation, we used
an approximation to (6) as in [20], namely
KjP)(S,i)
25
p [ z t ( i )= e * p ( S , j ) , j f 9 . L ( i ) t & I P ) ]
I
exp( - V [ . t ( i ) = e , lz:%,m€7fzL(')
E Nn
~ (.-eo
,+?I)
- W Z ~ ( * ) = e l l I ~ $ ~ ) ( ~ . J ) , J Ev*+~! p~ ',~~) )
vztzt(,)represents the neighbor for z t ( i ) . For target s, it is defined
as a sct of all columns of the association matrix Zt, z t ( j ) , ( j# i
and 1 5 j 5 m t )which has z t ( s , j ) = 1. With this problem formulation, the proposed algorithm, multiple model based tracking
using particle filter is described in the following scction.
to 47 and (iii) 12C'rcper second from frame 58 to 68. Then target
has acceleration of (0.02,O.OZ) in X-Y.The true trajectory plot is
shown in Figure 2.
TOdetermine consistency of the filter, the normalized state estimation error squared (NEES) is evaluated [22]. In our simulations, for each state parameter two sided Normal test statistic is
performed with a = 0.01. With normal test statistic, the upper and
lower critical values are (2.58, -2.58) for two sided test. For comparison among various algorithms NEES for each state parameter
is shown in Figure 3. T h i s confidence interval is marked with black
line in the Figure. Similarly, the Root Mean Square (RMS) error
for position, velocity and acceleration is compared and plotted in
Figure 4. From Monte-Carlo simulations it is found that with all
the three methods, NEES errors in position, velocity and acceleration are within the confidence interval for the trajectories depicted
in Figure 2. In Figure 3, at some points NEES error crosses the
confidcnce interval. The reason bcing that the third trajectory is
of "MIX' type, Le., the trajectory is generated using constant velocity and coordinated turn model but it is tracked using constant
acceleration and Singer model.
4. MULTIPLE MODEL BASE PARTICLE FILTERING
6. CONCLUSION
The algorithm consists of three steps: (a) observation to track assignment (data association) (b) update particle weight using assigned observation, and (c) propagate particle. These three steps
are repeated at each time instant when the current set of observations is available. The algorithmic flow chart of the proposed
algorithm is depicted in a Figure 1, anddescribed by the following
steps. Detail mathematical steps for tracking algorithm are not described hew due to space limitation. These steps can be found in
[19,211. It is important to note that the proposcd method does not
need to have any apriori information about thc exact dynamic models that targets may follow at a given time instant. This approach
is different compared to conveniional panicle filtering. The convcntional particle filteIs nceds the exact knowledge of thc dynamic
modcl at every time instant to track a target.
5. SIMULATION RESULTS
The Monte-Carlo simulations have heen performed with different
set of trajectory sets to evaluate the performance of the proposed
tracking aIgorithm. All the trajectory parameters are specified using pixel as units. Fifty simulations are performed for a given set
of trajectories. The process noise covariance and observation noise
covarinace are set to 0.2 and 2.0 respectively for trajcctory generation. The number of clutter is assumed to Poisson distributed. The
size of clutter window is chosen 10 x 10 s o u n d the actual observed target position. The average number of clutter falls in side
the clutter window is set to 1.
For one of the trajectory set the details are depicted as follows.
The trajectory set consists of three trajectories, namely. (a) First is
a constant acceleration trajectory with initial position, velocity and
acceleration are set to (70,701, (20,3) and (0.5,0.5). The trdjectory
exists for 22 frames. (b) Second trajectory is generated using constant velocity model. The trajectory exist for 30 frames and it is
generated using constant velocity model. The initial X-Y position
and vetocity are set to (70,200) and (20,-3). (c) The third trajectory
is of "MIX" type. It exists for 70 frames. The initial position and
velocity are set to (30,30) and (10,l). The target travels with constant velocity from frame 1 to 15. It takes three turns (i) 15""" per
second from frame 16 to 27 (ii) -1SCi" per second from frame 36
From Monte-Carlo simulations it is found that interacting multiple model based particle filtering using three different methods
for data association, namely, Method-I: using implicit observation assignment based on nearest neighbor method, Method-I1 and
Method-HI: using assignment weights and assignment probabilities, perform equaIly well. The advantage of Method41 and Method
III is that both these methods avoid an uncertainty about the origin
of an observation. Moreover, Method411 exploits neighborhood
property using MKF for data association.
From simulation results it is concluded that i n absence of any
apriori knowledge about transition time instant from one model to
anothcr model, using the proposed method it is possible to track
multiple arbitrary target trajectories using known nonlinear/nonGaussian state space models. This is also true i n the case where
thcre is no apriori information about the exact dynamic models
which targets may follow at a givcn time. In a case of degeneracy
re-initialization of the particles using the mix state allows us to
track random movement of the t q c t . Our proposed method is
able to track multiple arbitrary target trajectories in the presence
of dense clutter. Only two filters, namely, CA and SMM filters
were used in IMM mode to track random movement of targets.
7. REFERENCES
Y. Sar-shalom and T. E. Fortmann, Tracking and Data Association, Academic Press, 1989.
N.J. Gordon, D.J. Salmond and A.F.M. Smith, "Novel
approach to nonlinearhon-Gaussian Bayesian stale cstimation," IEEProceedings-F, vol. 140, no. 2, pp, 107-1 13, April
1993.
Neil Gordon, "A Hybrid Bootstrap Filter for Target Tracking in Clutter," IEEE Trans. on Aerospace and Etecrronics
Sysrems, vol. 33, no. 1, pp. 353-358, Jan. 1997.
Rickard Karlsson and Niclas Bergman, "Auxiliary Particle
Filters for Tracking a Maneuvering Target," Proc. 0139th
IEEE con$ on Decision and Control, pp, 389L-3895, Dec.
2000.
29
Authorized licensed use limited to: INDIAN INSTITUTE OF TECHNOLOGY BOMBAY. Downloaded on December 3, 2008 at 00:46 from IEEE Xplore. Restrictions apply.
Step 1 :
and foreach model
sample partk1-s~
t o Step 6 at
each time instan
Evaluate wrnbined
Step 2:
For each rnrger, and
foreach u r d e -
Step 5:
Method-I1 I l I I
For each target, nnd
for en& incdel ck cx
nermcy.
if yes, then
initialize pnrticks with
arsignmcnt using
probability
Predict particles
Obtain d e l state
L
pcedicrion
Updrae particle
weight using
assignment probability
Perforin I<e-ainpliog
Evnlunte combined
Fig. 1. Flow chart for the proposed method
Fig. 2. True trajectory plot for Monte-Carlo simulation
[51 Neil Gordon and Simon Maskell. “Efficient Particle Filters
[6]
[7]
[SI
for Joint Tracking and Classification,” in SPIE - Signal and
Data Processing of Small Targets, April 2002, vol. 4728.
David J. Bullock and John S. Zelek, “Real-Time Tracking
for Visual Interface Applications in Cluttered and Occluding Situations,” Pmc. of 15th Intemaiionul Con5 on Ksion
Interjiwe, 2002.
Arnnud Doucet et. al., “On Sequential Monte Carlo Sampling Methods for Bayesian Filtering,” Technical Report
CUED/F--INFENG/TR310, 1998.
Rudolph van der Menve, Amaud Doucet, et. al., “THE
UNSCENTED PARTICLE FILTER,” www,ece.ogi.edu/ rvdmerwe Technical Report CUED/F-INFENG TR 380, Aug.
2000.
Paul Fearnhead, “MCMC, Sufficient Statistics and Particle
Filters,” www.stars.ox.ac.uk/~ead/MCMCP~ps.gz,
July
[I21
[I31
[I41
-
[9]
2001.
[IO] A. Farina, B. Ristic and D.Benvenuti, “Tracking a Ballistic Target: Comparison of Several Nonlinear Filters,” IEEE
Transactions on Aerospace and Electronics Systems, vol. 38,
no. 3, pp. 854-867, July 2002.
[ I I ] X. Lin, T. Kirubarajan, Y. Bar-Shalom and Simon Maskell,
“Comparison of EKF, Pseudomeasurement and Particle
[I51
[I61
[17]
Filters for a Rearing-only Target Tracking Problem,’’
hltp://www-sigproc.eng.cum.ac.uk/ sm224/comparison.pdJ,
April 2002.
D. Avitzour, “Stochastic simulation Bayesian approach to
multitarget tracking:’ IEE Proc. - Radar; Sonar Nuvig., vol.
142, no, 2, pp. 4144, April 1995.
C. Hue, J-P. Le Cadre and P. PErez, “Tracking Multiple Objects with Particle Filtering,” IEEE Transactions on
Aerospace and EIecrronics Sysiems, vol. 38, no. 3, pp. 791812, July 2002.
Simon Maskell, Malcom Rollason and Neil Gordon, “Efficient Pmticle Filtering for Multiple Targct Tracking with
Application to Tracking in Structured Images,” SPIE - Signal and Data Processing of Small Targets, vol. 4728, April
2002.
Cody Kwok, Dieter Fox and Marina Meila, “Real-time Particle Filters,” Advances in Neural Information Processing
Sysfem (NIPS), 2002.
Samuel S . Blackman and Robert Popoli, Design andilnalysis ofMOdent Tracking Systems, Artech House, 1999.
Amaud Doucet, Nando de Freitas and Neil Gordon, Sequential Monte Curlo Melhods in Practice, Springer-Verlag, New
York, 2W 1.
30
Authorized licensed use limited to: INDIAN INSTITUTE OF TECHNOLOGY BOMBAY. Downloaded on December 3, 2008 at 00:46 from IEEE Xplore. Restrictions apply.
Target No.3 : N o t m a l i d Estimation Squared Error in state parameters
f---l
5
‘A
c
2%.
a
.-
P o
-
d
e
r)
2
i
X
D
x
-2
t
-5
?U
40
60
I
20
Frame
60
40
Frame
PF-IMM-NN
PF-IMM-PMW
PF-I PAM-MRF
95-36 confaewe
e
Q
E
2
Q
r
-2
20
40
Frame
40
20
60
60”
20
Frame.
40
Frame
68
Fig. 3. NEES plot for trajectory 3 (Monte-Carlo simulation)
PF-IMM-N
N
PF-IMM-RMHT
+ PF-IMM-MRF
II*
-
I
10
40
30
50
60,
I
5,
10
2Q
30
40
50
60
40
50
6Q
Frame
Fig. 4. RMS emor plot for trajectory 3 (Monte-Carlo simulation)
Samuel S. Blackman, Multiple-TargetTrucking with Radar
Applirarions, Artech House, Inc., Boston, 1986.
Mukesh A. Zaveri. Uday B. Desai, S.N. Merchant, “Automated Model Selection based Tracking of Multiple Targets
using Particle Filtering,” in Proceedings of IEEE TEMCON 2003, Banglore, India, Oct 2003, vol. 2. pp. 831-835.
Mukesh A. Zaveri, Uday B. Desai and S.N. Merchant, “lntcracting Multiple Model Based Tracking of Multiple Point Targets using Expectation Maximization Algorithm in Infrared
image sequence,” in Proceedings of S H E (VCIP 2003),
Lugano, Switzerland, July 2003, vol. 5 150, pp. 303-314.
[211 Mukesh A. Zaveri, Uday B. Desai, S.N. Merchant, “Automated Model Selection Based Algorithm for Tracking
Multiple Nonlinear Trajectories,” in Proceedings of SPIE
(E12004),California, USA, Jan 2004, vol. 5298.
[=I Yaakov Bar-shamlom, X.-Rong Li and Thiagalingam
Kirubarajan, Estimation with Applicurions To Tracking and
Navigatiun, John Wiley & Sons, Inc.. New Youk,2001.
31
Authorized licensed use limited to: INDIAN INSTITUTE OF TECHNOLOGY BOMBAY. Downloaded on December 3, 2008 at 00:46 from IEEE Xplore. Restrictions apply.