AffAction

Affordances and actions for robot behavior generation.

Concepts

Conceptually, this project connects a semantic text-level action language with the low-level motor control of a robot (or its simulation).

We represent a scene as a composition of several objects, each of them comprising several affordances. For instance, a table can have the "Supportable" affordance to indicate that something can be put on its support surface. Similarly, objects can have the Affodance to be "Graspable", where this can be detailed down to "Powergraspable" for cylindrical objects etc. Affordance come typically with a coordinate frame that describes the geometry of its interaction. For instance, the "Powergraspable" affordance is described by a frame that has the z-axis pointing into the invariant rotation direction.

The scene also contains a set of Manipulators that are composed of several "Capabilities". A "Capability" can for instance be a "Powergrasp", a "Palmgrasp", a "FingerpushCapability" and more. At runtime, an action is checking all possible combinations of Capabilities and Affordances, and computes the feasible pairs.

Here is an example: The action "get apple" leads to:

• matching all manipulator's (e.g. left and right hand) capabilities (e.g. Powergraps, Palmgrasp) against all object affordances (e.g. how it is graspable). The resulting combinations are sorted according to a heuristic (usually distance in wrench space). In this computation step, the robot's particular embodiment (reachability, joint ranges ...) is not considered.
• The action is simulated forward with a prediction class, and is checked against violations of the robot's limits and against collisiins. The first feasible solution is sent for execution. In this computation step, the robot's embodiment is considered.

Actions

The "get" action

get(<object-to-pick-up> <which-hand (optional)> <"from" parent-object (optional>)

Grasps an object and lifts it a little bit up. If no manipulator (which-hand) is given, the action will determine the best possible manipulator. Internally, the action matches the manipulator's capabilities with the object's affordances. These combinations are supprted:

Affordance	Capability
PowerGraspable	PowergraspCapability
PincerGraspable	PincergraspCapability
BallGraspable	PincergraspCapability
PalmGraspable	PalmgraspCapability

The set of matches is evaluated, and the best feasible one is determned. If the "from" attribute is used, the object-to-pick-up is determined as being a child of the object-to-pick-up. This allows to give a spatial constraint in case there are several objects with the same name. For instance:

get lemon from cutboard   // Gets a lemon that is a child of the cutboard. The hand is auto-selected.
get apple_1 hand_right    // Gets the apple_1 with the right hand.

The "put" action

put(