Modeling changing objects

This page discusses methods for reasoning about the effects actions have on the objects they manipulate. Especially in cooking tasks, the induced changes can be significant: Objects can be created, destroyed, split in pieces, combined, etc. By combining the representation of actions with projection methods and linking the actions to the manipulated objects, KnowRob can reason about what happened to objects, predict what will happen, or plan such that certain things happen.

The techniques described below have been published in Tenorth et al, IROS 2012.

Projection is performed by querying for the postActors, i.e. everything that is a kind of output or post-condition of an action:

rdf_triple(knowrob:postActors, object_change:'put-mix-on-pan1', Post).

The projection rules are implemented as computables for the postActors relation. First, they compute direct effects of an action (e.g. the pancake mix being on top of the pancake maker).

These direct effects can trigger indirect effects, i.e. processes whose pre-conditions become true by the effects of an action. Therefore, the projection rule calls the generic process. predicate after the projection took place to check whether any processes became active. An example of such a process is the baking process that happens if dough is put somewhere where a thermal connection to something hot exists.

After projection, the inputs and effects of the action can be read with

rdf_has(object_change:'put-mix-on-pan1', Rel, Post), rdfs_subproperty_of(Rel, knowrob:preActors).
rdf_has(object_change:'put-mix-on-pan1', Rel, Post), rdfs_subproperty_of(Rel, knowrob:postActors).

The transformedInto relation is a transitive relation that covers all object transformations, including destruction, creation, and transformation of objects. After projection, the transformations can be queried with:

?- rdf_triple(object_change:transformedInto, From, To).
From = 'http://ias.cs.tum.edu/kb/object-change.owl#pancake-dough1',
To = 'http://ias.cs.tum.edu/kb/knowrob.owl#Baked1' .

As an example task we consider making pancakes; a static test-sequence of actions can be found in pancake-making.owl in the package knowrob_actions.

?- use_module(library(comp_spatial)).
?- use_module(library(object_change)).

# initially, the egg is computed to be on the table
?- rdf_triple(knowrob:'on-Physical', A, pancake:table1).
A = 'http://ias.cs.tum.edu/kb/pancake-making.owl#egg1' .

?- rdf_triple(knowrob:postActors, pancake:turnon1, Post).
pancakemaker1 switched on 

?- rdf_triple(knowrob:postActors, pancake:cracking1, Post).
egg1 -> EggShell0
egg1 -> EggYolk-Food1

?- rdf_triple(knowrob:postActors, pancake:mixing1, Post).
EggYolk-Food1 added to -> Dough2
milk1 added to -> Dough2
pancakemix1 added to -> Dough2
 
?- rdf_triple(knowrob:postActors, pancake:pour1, Post).
Dough2 on top of pancakemaker1
Dough2 -> Baked4

?- rdf_triple(knowrob:postActors, pancake:put1, Post).
Baked4 on top of plate1
 
# the egg is now gone and therefore not on the table any more
?- rdf_triple(knowrob:'on-Physical', A, pancake:table1).
false.
 
# what has been transformed into Baked4?
?- rdf_triple(object_change:transformedInto, A, knowrob:'Baked4').
A = 'http://ias.cs.tum.edu/kb/knowrob.owl#Dough2' ;
A = 'http://ias.cs.tum.edu/kb/knowrob.owl#EggYolk-Food1' ;
A = 'http://ias.cs.tum.edu/kb/pancake-making.owl#egg1' ;
A = 'http://ias.cs.tum.edu/kb/pancake-making.owl#milk1' ;
A = 'http://ias.cs.tum.edu/kb/pancake-making.owl#pancakemix1' ;
false.

# where did the egg go?
?- rdf_triple(object_change:transformedInto, pancake:egg1, Res).
Res = 'http://ias.cs.tum.edu/kb/knowrob.owl#Baked4' ;
Res = 'http://ias.cs.tum.edu/kb/knowrob.owl#Dough2' ;
Res = 'http://ias.cs.tum.edu/kb/knowrob.owl#EggShell0' ;
Res = 'http://ias.cs.tum.edu/kb/knowrob.owl#EggYolk-Food1' ;
false.

rosrun rosprolog rosprolog comp_spatial
?- use_module(library(object_change)).
?- object_change:project_and_debug('http://ias.cs.tum.edu/kb/pancake-making.owl#MakingPancakes', 
                                    OrigActionSeq, DebuggedActionSeq).
 
egg1 -> EggShell1
egg1 -> EggYolk-Food2
milk1 added to -> Dough4
flour1 added to -> Dough4
Dough4 added to -> Dough6
EggYolk-Food2 added to -> Dough6
Dough4 on top of pancakemaker1

OrigActionSeq = ['CrackingAnEgg',
                 'MixFlourAndMilk', 
                 'MixEggAndDough',
                 'PourDoughOntoPancakeMaker', 
                 'FlippingAPancake'],

DebuggedActionSeq = ['CrackingAnEgg',
                     'MixFlourAndMilk', 
                     'MixEggAndDough', 
                     'PourDoughOntoPancakeMaker', 
                     'TurningOnHeatingDevice', 
                     'BakingFood',
                     'FlippingAPancake'].