Institut für Informatik
Intelligente Autonome Systeme
Technische Universität München
Learning Organizational Principles
in Human Environments
Martin J. Schuster
27.04.2011
Institut für Informatik
Intelligente Autonome Systeme
Technische Universität München
Outline
• Motivation: Object Allocation Problem
• Organizational Principles in Kitchen Environments
• Datasets
• Learning Organizational Principles
– Features
– Classifiers
– Feature Importance Measures
• Evaluation
• System Integration
• Extensions to Probabilistic Modelling Methods
– Degrees of Truth and Soft Evidence
– Naive Bayesian Model
– Markov Logic Networks
• Conclusions and Outlook
Martin J. Schuster
27.04.2011
Learning Organizational Principles in Human Environments
2
Institut für Informatik
Intelligente Autonome Systeme
Technische Universität München
Motivation: Object Allocation Problem
• Pick and place tasks important for assistive robots
in human environments
• High level tasks gain importance
⇒ Where to pick objects up and where to place them?
• Scenario: Assistive robot in human kitchen environment
– Return home from shopping with full shopping basket
– Robot should “put things away”
– ⇒ Infer locations where best place each object
• ⇒ Learn and apply organizational principles
– For a particular environment instance (user specific)
– Governed by notion of similarities
– Formulate as classification task
Martin J. Schuster
27.04.2011
Learning Organizational Principles in Human Environments
3
Institut für Informatik
Intelligente Autonome Systeme
Technische Universität München
Organizational Principles in Kitchens
• Analysis of photographs, blogs, videos, etc.
• Prevalent organizational principles:
– Class in taxonomy, e.g. distinguish between food and non-food
– Physical Constraints, e.g. size to fit into container
– Purpose, e.g. coffee, coffee filters, sugar together
• Additional Principles
– Packaging, Safety
• Translate criteria into similarities between pairs of objects
Martin J. Schuster
27.04.2011
Learning Organizational Principles in Human Environments
4
Institut für Informatik
Intelligente Autonome Systeme
Technische Universität München
Datasets
Mockup Kitchen Datasets
• Annotated location of each object
• 10 datasets: 6/12 locations, 66 object classes, 152 objects each
Martin J. Schuster
27.04.2011
Learning Organizational Principles in Human Environments
5
Institut für Informatik
Intelligente Autonome Systeme
Technische Universität München
Datasets
Real Kitchen Datasets
• 2 datasets: 19/15 locations, 166/87 classes, 408/149 objects each
Martin J. Schuster
27.04.2011
Learning Organizational Principles in Human Environments
6
Institut für Informatik
Intelligente Autonome Systeme
Technische Universität München
Learning Organizational Principles
Features
• WUP similartiy:
Root
Thing
wupSim(C1 , C2 ) =
depth(LCS(C1 , C2 ))
1
(depth(C
1 ) + depth(C2 ))
2
SpatialThing
4
LCS
2
DrinkingVessel
4
• Purpose, meal relevance
– several binary features
Cup
Glass
• Size ∈ {s, m, l}
• Shape ∈ {box, cylindric
C1
CoffeeCup
C2
SodaGlass
flat, bag, other}
Martin J. Schuster
27.04.2011
Learning Organizational Principles in Human Environments
7
Institut für Informatik
Intelligente Autonome Systeme
Technische Universität München
Learning Organizational Principles
Features
• Aggregate WUP similarities
maxWup(O, L) =
max wupSim(class(O), class(O 0 ))
O 0 ∈L
avgWup(O, L) =
X wupSim(class(O), class(O 0 ))
|L|
0
O ∈L
• Plot distances (mds)
Mij := 1 − wupSim(Ci , Cj )
Martin J. Schuster
27.04.2011
Learning Organizational Principles in Human Environments
8
Institut für Informatik
Intelligente Autonome Systeme
Technische Universität München
Learning Organizational Principles
Classifiers
• Maximum WUP similarity
• Decision Trees
• Boosted Decision Trees
• Support Vector Machines
• Naive Bayesian Classifier
– NB discrete (Clustering)
– NB continuous (Gaussian)
– NB soft (Soft Evidence)
• Markov Logic Networks
⇒ weight learning computationally too expensive
for any reasonable complex models
Martin J. Schuster
27.04.2011
Learning Organizational Principles in Human Environments
9
Institut für Informatik
Intelligente Autonome Systeme
Technische Universität München
Learning Organizational Principles
Feature Importance Measures
• Feature importance (inverse normalized entropy):
P
f ∈dom(F ) PD (F = f | L) log(PD (F = f | L))
D
IF (L) := 1−
log(|dom(F )|)
• Discriminative power (Hellinger distance):
s
Xp
HFD (L1 , L2 ) =
1−
PD (F = f | L1 )PD (F = f | L2 )
f ∈dom(F )
D
H (F ) :=
−1 X
|LD |
2
X
HFD (Li , Lj )
Li ∈LD Lj ∈LD ,i<j
Martin J. Schuster
27.04.2011
Learning Organizational Principles in Human Environments
10
Institut für Informatik
Intelligente Autonome Systeme
Technische Universität München
Evaluation
• Crossvalidation: train & test for each object class in each dataset
• Maximum WUP similarity classifier:
average correct classification rate of:
– 88% for mockup kitchens dataset
– 72% for real kitchens dataset
• Decision Trees: average correct classification rate of:
– 86% / 88% for mockup kitchens dataset
with WUP similarity / all features
– 79% / 74% for real kitchens dataset
with WUP similarity / all features
• ⇒ WUP similarity alone highly discriminative
• standard classification methods perform sufficiently well
• in real kitchens: subset of objects placed arbitrarily (noise)
⇒ 100% may not be possible to achieve in practice
Martin J. Schuster
27.04.2011
all results
Learning Organizational Principles in Human Environments
11
Evaluation
Feature Importance
P
IFD (L)
f ∈dom(F )
:= 1−
PD (F = f | L) log(PD (F = f | L))
log(|dom(F )|)
1.0
0.8
0.6
0.4
avgWup
maxWup
mealRelevance
purpose
shape
size
0.2
0.0
1
2
3
4
5
6
7
8
9
10
11
12
Evaluation
Discriminative Power
HFD (L1 , L2 )
=
s
1−
Xp
PD (F = f | L1 )PD (F = f | L2 )
f ∈dom(F )
D
H (F )
:=
|L |−1 X
D
2
L ∈L
i
X
HFD (Li , Lj )
D Lj ∈LD ,i<j
1.0
0.8
0.6
0.4
0.2
0.0
avgWup
maxWup mealRelevance purpose
shape
size
Institut für Informatik
Intelligente Autonome Systeme
Technische Universität München
System Integration
Germandeli
Website
Germandeli
Ontology
KnowRob: Prolog Knowledge Base
Object Locations
High-level Task
Ask & Tell Interface
Robot Perception
Kitchen Ontology
Best Object Location Inference
Similarity Computation
Classification
Martin J. Schuster
27.04.2011
Learning Organizational Principles in Human Environments
14
Institut für Informatik
Intelligente Autonome Systeme
Technische Universität München
System Integration
KnowRob
?− h i g h l i g h t b e s t l o c a t i o n d t r e e (
o r g p r i n c i p l e s d e m o : ’ C o f f e e F i l t e r 1 ’ , Canvas ) .
B e s t l o c a t i o n : knowrob : Drawer7
O b j e c t s a t l o c a t i o n knowrob : Drawer7 :
WUP s i m i l a r i t y : o b j e c t ( c l a s s )
0.87500: o r g p r i n c i p l e s d e m o : CoffeGround1
( germandeli : Dallmayr Classic Ground Coffee 250g )
0.75000: orgprinciples demo : EspressoBeans1
( germandeli : illy Espresso Whole Beans 88 oz )
0 . 7 0 5 8 8 : o r g p r i n c i p l e s d e m o : Sugar1
( germandeli : Nordzucker Brauner Teezucker 500g )
0 . 6 6 6 6 7 : o r g p r i n c i p l e s d e m o : Tea2
( germandeli : Teekanne Rotbusch Tee Vanille 20 Bags )
Martin J. Schuster
27.04.2011
Learning Organizational Principles in Human Environments
15
Institut für Informatik
Intelligente Autonome Systeme
Technische Universität München
System Integration
Context: High-level task
1. Empty shopping basket on table ⇒ separate the objects
2. Perceive and segment objects
3. Match objects with object classes in knowledge base
4. General knowledge about kitchens ⇒ handle certain types of objects
5. Our new algorithm ⇒ infer best place for each of the
(remaining) kitchen objects
6. For each object:
6.1
6.2
6.3
6.4
Pick up object
Move to inferred location, open container if necessary
Search for free space inside location, if fails ⇒ infer a new location
Place object inside location, close container if necessary
Martin J. Schuster
27.04.2011
Learning Organizational Principles in Human Environments
16
Institut für Informatik
Intelligente Autonome Systeme
Technische Universität München
Extensions to Probabilistic Modelling Methods
• Probabilistic generative methods
– Probability distribution over classes
– Insight into organizational structure through model parameters
– Allow construction of more complex models
• Naive Bayesian Model
• Markov Logic Networks
• Extensions to handle soft evidence
– model continuous features (WUP similarities)
Martin J. Schuster
27.04.2011
Learning Organizational Principles in Human Environments
17
Institut für Informatik
Intelligente Autonome Systeme
Technische Universität München
Extensions to Probabilistic Modelling Methods
Degrees of Truth and Soft Evidence
• For a proposition R: distinguish between
– Probability P(R) → probabilistic framework
– Degree of truth T (R) → fuzzy logic framework
– Degree of belief: expected value of degrees of truth
→ tendency to act as if R
• No (available) implementation out there
• Our approach: approximate degrees of truth as probabilities
⇒ handle them in (extended) probabilistic frameworks
⇒ soft evidence: binary features, true with a certain probability
Martin J. Schuster
27.04.2011
Learning Organizational Principles in Human Environments
18
Institut für Informatik
Intelligente Autonome Systeme
Technische Universität München
Extensions to Probabilistic Modelling Methods
Naive Bayesian Model:
C
X1
• Independence assumption:
P(X = x | C = c) =
N
Y
X2
...
XN
P(Xi = xi | C = c)
i=1
• Classification (MAP inference):
arg max P(X = x | C = c) · P(C = c)
c∈C
Martin J. Schuster
27.04.2011
Learning Organizational Principles in Human Environments
19
Extensions to Probabilistic Modelling Methods
Naive Bayesian Model
Institut für Informatik
Intelligente Autonome Systeme
Technische Universität München
Extensions to Probabilistic Modelling Methods
Naive Bayesian Model: Soft Evidence
• Inference:
P(C = c | e) = P(C = c | Xh = xh , e)
X P(Xh = xh , Xs = xs | C = c) · P(C = c)
· P(Xs = xs | e)
=
P(Xh = xh , Xs = xs )
Q
xs ∈
Xi ∈Xs
dom(Xi )
compute P(Xs = xs | e) implicitly
Q (e.g. Gibbs sampling) or
approximate P(Xs = xs | e) ≈ Xi ∈Xs P(Xi = xi | ei ) assuming the
independence of the pieces of soft evidence
• Learning: use soft counts:
P
P(Xi = xi | C = c) =
Martin J. Schuster
27.04.2011
e∈Tc
P(Xi = xi | ei )
|Tc |
Learning Organizational Principles in Human Environments
21
Institut für Informatik
Intelligente Autonome Systeme
Technische Universität München
Extensions to Probabilistic Modelling Methods
Markov Logic Networks
• First order logic (FOL) + soft constraints
⇒ Markov Logic Network (MLN)
• Each FOL formula Fi is assigned a weight wi
• Probability distribution over all possible worlds:
P
X
exp
1
i wi ni (x)
P
wi ni (x) = P
P(X = x) = exp
0
Z
x 0 ∈X exp
i wi ni (x )
i
ni (x): the number of true groundings of Fi in world x.
• Template for Markov Networks (graphical model):
instantiation for a particular set of constants C
Martin J. Schuster
27.04.2011
Learning Organizational Principles in Human Environments
22
Institut für Informatik
Intelligente Autonome Systeme
Technische Universität München
Extensions to Probabilistic Modelling Methods
Markov Logic Networks: Parameter Learning
• Maximize probability P(X = x | w ) of world x given by the training
data: ⇒ Maximize log-likelihood:
L(X = x) =
X
wi ni (x) − log(Z )
i
• Optimize model parameters w using L-BFGS
⇒ compute arg maxw L(X = x | w )
• Gradient:
δ
L(X = x)
δwi
=
ni (x) −
X
=
ni (x) −
X
x 0 ∈X
Martin J. Schuster
27.04.2011
ni (x 0 ) · P(X = x 0 )
x 0 ∈X
P
exp ( k wk nk (x 0 ))
P
ni (x ) · P
00
x 00 ∈X exp (
k wk nk (x ))
0
Learning Organizational Principles in Human Environments
23
Institut für Informatik
Intelligente Autonome Systeme
Technische Universität München
Extensions to Probabilistic Modelling Methods
Markov Logic Networks: Soft Evidence Weight Learning
• Hard evidence: defines single world x with P(X = x) = 1
• Soft evidence: defines probability distribution P(X = x | e) over
possible worlds x ∈ X
• We developed four methods to handle soft evidence weight learning
Martin J. Schuster
27.04.2011
Learning Organizational Principles in Human Environments
24
Institut für Informatik
Intelligente Autonome Systeme
Technische Universität München
Extensions to Probabilistic Modelling Methods
Markov Logic Networks: Soft Evidence Weight Learning
Log-Likelihood with Weighting of Formulas (LL-ISE):
• Approximate distribution of worlds with “soft world”
⇒ Formulas true to certain degree that corresponds to probabilities
given by the soft evidence
• Assumes independence of the pieces of the soft evidence (ISE)
• Log-likelihood:
X
LLL-ISE (X = x) =
wi ñi (x) − log(Z )
i
• Gradient:
P
X
exp ( k wk nk (x 0 ))
δ
0
P
LLL-ISE (X = x) = ñi (x) −
ni (x ) · P
00
δwi
x 00 ∈X exp (
k wk nk (x ))
0
x ∈X
• Computation of Z computationally intractable (2|X | possible worlds)
Martin J. Schuster
27.04.2011
Learning Organizational Principles in Human Environments
25
Institut für Informatik
Intelligente Autonome Systeme
Technische Universität München
Extensions to Probabilistic Modelling Methods
Markov Logic Networks: Soft Evidence Weight Learning
Pseudo-Log-Likelihood with Weighting of Formulas (PLL-ISE):
• Approximate the log-likelihood: product of conditional likelihoods of
all variables Xk in world x, each given the values of its direct
neighbors (Markov blanket)
• Use soft-counts (similar to LL-ISE)
• Pseudo log-likelihood:
LPLL-ISE (X = x)
=
log
N
Y
P(Xk = xk | MBx (Xk ))
k=1
• Fast, but rough approximation
⇒ in many cases: no reasonable results
Martin J. Schuster
27.04.2011
Learning Organizational Principles in Human Environments
26
Institut für Informatik
Intelligente Autonome Systeme
Technische Universität München
Extensions to Probabilistic Modelling Methods
Markov Logic Networks: Soft Evidence Weight Learning
LL with Sampling and Weighting of Formulas (SLL-ISE):
• Use soft counts for evidence “soft world”
• Sample normalization Z̃
– Run MCSAT, take Markov chain states as samples
exp( k wk nk (s))
– Sample s drawn with probability ∝ P(X = s) ≈
Z̃
⇒ remove duplicate samples
⇒ high probability to draw samples with high contribution to Z̃
P
• Log-Likelihood:
!

LSLL-ISE (X = x) ≈
X
i
• Gradient:
wi ñi (x) − log 
X
exp
s∈S
Martin J. Schuster
27.04.2011
wk nk (s) 
k
s∈S̄
X
δ
LSLL-ISE (X = x) ≈ ñi (x) −
δwi
X
ni (s)
|S|
Learning Organizational Principles in Human Environments
27
Institut für Informatik
Intelligente Autonome Systeme
Technische Universität München
Extensions to Probabilistic Modelling Methods
Markov Logic Networks: Soft Evidence Weight Learning
LL with Double Sampling and Weighting of Worlds (DSLL-WW):
• Draw samples from distribution of worlds defined by soft evidence
– Maximize weighted average of probabilities of sampled worlds
⇒ drop assumption of independent soft independence
– Samples in Se drawn with ≈ P(X = s | e)
• Sample Z̃ as in SLL-ISE
• Log-Likelihood:
LDSLL-WW (e) = log
!!
X
1 X
·
exp
wk nk (s)
− log(Z̃ )
|Se |
s∈Se
k
• Gradient:
δ
LDSLL-WW (e)
δwi
Martin J. Schuster
27.04.2011
=
X ni (s) s∈Se
|Se |
−
X ni (s) s∈S̄
|S̄|
Learning Organizational Principles in Human Environments
28
Institut für Informatik
Intelligente Autonome Systeme
Technische Universität München
Conclusions and Outlook
• Organizational principles are manifestations of clusterings
governed by similarity
• WUP similarity highly informative
• Standard classification methods (e.g. decision trees) adequate
• We provide open source implementation within KnowRob
• MLNs: Developed new weight learning algorithms for soft evidence
– Computationally still too expensive for practical use here
– Further reduction of complexity necessary
• Outlook:
– More features (spatial relations, . . . )
– Other human environments
– Integration in robotic system to solve complex everyday tasks
Martin J. Schuster
27.04.2011
Learning Organizational Principles in Human Environments
29
Institut für Informatik
Intelligente Autonome Systeme
Technische Universität München
Thank you for your attention!
Martin J. Schuster
27.04.2011
Learning Organizational Principles in Human Environments
30
Institut für Informatik
Intelligente Autonome Systeme
Technische Universität München
Crossvalidation Results
Mockup Kitchens Dataset
max. avgWup
max. maxWup
DecisionTrees
Boosted DecisionTrees
SVM
NB Discrete
NB Continuous
NB Soft
avgWup and maxWup
mean
std
77.45%
20.85%
87.52%
17.91%
86.61%
12.46%
87.68%
13.95%
77.46%
24.11%
78.37%
15.64%
69.97%
28.63%
42.16%
39.38%
all features
mean
std
—
—
—
—
88.12% 14.16%
89.50% 9.92%
89.49% 17.68%
85.69% 15.56%
82.61% 17.75%
82.32% 18.73%
back to evaluation
Martin J. Schuster
27.04.2011
Learning Organizational Principles in Human Environments
31
Institut für Informatik
Intelligente Autonome Systeme
Technische Universität München
Crossvalidation Results
Real Kitchens Dataset
max. avgWup
max. maxWup
avgWup and maxWup
Dr 1
Dr 2
mean
48.19% 70.24%
59.22%
72.29% 71.43%
71.86%
DTrees
Boosted DTrees
SVM
84.94%
84.94%
57.23%
73.81%
71.43%
69.05%
79.37%
78.18%
63.14%
79.52%
80.72%
73.49%
69.05%
69.05%
76.19%
74.28%
74.89%
74.84%
NB Discrete
NB Continuous
NB Soft
50.00%
41.57%
13.86%
50.00%
58.33%
50.00%
50.09%
49.95%
31.93%
57.83%
60.24%
65.66%
64.29%
63.10%
59.52%
61.06%
61.67%
62.59%
Dr 1
—
—
all features
Dr 2
mean
—
—
—
—
back to evaluation
Martin J. Schuster
27.04.2011
Learning Organizational Principles in Human Environments
32
Institut für Informatik
Intelligente Autonome Systeme
Technische Universität München
Extensions to Probabilistic Modelling Methods
Markov Logic Networks: Soft Evidence Weight Learning
Comparison:
Independent soft evidence
Soft evidence world
Approximative
Sampling-based
Runtime
Applicable for complex models
Martin J. Schuster
27.04.2011
LL-ISE
X
X
–
PLL-ISE
X
X
X
++
SLL-ISE
X
X
X
X
+
(X)
DSLL-WW
X
X
(X)
Learning Organizational Principles in Human Environments
33