Adding a new action

ADAM provides a limited variety of actions to use in your curricula. You may need for your experiments actions not already included in ADAM. Adding actions is not too difficult. However, be aware that not all actions can be adequately represented in ADAM by default. For example, we can’t represent “speak” as an action, since we can’t describe in ADAM either the mouth movements or the speech sounds. Furthermore, not all verbs correspond to actions in ADAM; for example, the verb “to be” cannot reasonably be represented as an action.

With these constraints in mind, in this tutorial we’re going to add an action for carrying an object. Specifically, the action will cover an agent carrying an already-held object from one place to another. For example, carrying a box to a table.

Adding an action

The general outline for creating an action is simple. First, the same as for objects and properties, we describe the action to ADAM as an ontology type together with some perceptual realization. Second, we create lexicon entries for the action for each language we want to use with the action. We’ll cover describing the action first.

Defining an ontology node for the action is simple. We want to define an ontology node for the carry action in adam.ontology.phase1_ontology:

CARRY = OntologyNode("carry")
subtype(CARRY, ACTION)

Note

If our action involved a transfer of possession, as in “Mom gave a ball to Dad,” we would mark that as a property of the ontology node: CARRY = OntologyNode("carry", [TRANSFER_OF_POSSESSION]). Since there is no second person involved in the carry action we want to use, we don’t need to mark it like this.)

Next, we’ll create an action description that tells ADAM how the action should be perceived. To do this, we’ll need to use ActionDescriptionVariables (which work similarly to TemplateObjectVariables) and an ActionDescriptionFrame inside an ActionDescription.

An ActionDescriptionVariable describes a participant in some action and the property constraints (or lack thereof) on that participant. ActionDescriptionVariables are also used to define region and part-of constraints. For example, we’d use these to set up the constraint that the carrying agent has a hand (or other manipulator; hereafter “hand”), and the relation that the “hand” is in contact with the object while the object is being carried.

To describe the carry action, we’ll need four variables: The person or animal doing the carrying; the thing they carry; the body part they use to carry it; and the place they put the thing. We’ll call these the agent, theme, manipulator, and goal:

_CARRY_AGENT = ActionDescriptionVariable(
    THING, properties=[ANIMATE], debug_handle="carry_agent"
)
_CARRY_THEME = ActionDescriptionVariable(THING, debug_handle="carry_theme")
_CARRY_GOAL = ActionDescriptionVariable(THING, debug_handle="carry_goal")
_CARRY_MANIPULATOR = ActionDescriptionVariable(
    THING, properties=[CAN_MANIPULATE_OBJECTS], debug_handle="carry_manipulator"
)

Next, we’ll define our ActionDescriptionFrame. The ActionDescriptionFrame collects together the participants in the action and their roles. Each participant has a semantic (or thematic) role in the action which describes how the participant takes part in the action. These roles are agent, patient, theme, and goal. These roles serve to identify participants in an action description in a language-independent way. The language generators can then use this role information in various ways when generating descriptions of a situation involving the action. Note that like ontology types, these roles are only for creating curricula and are not visible to learners.

(For a more complete description of these semantic roles, see “Summary of semantic roles and grammatical relations” (Thomas Payne, 2007).

In our frame, we will mark the person or animal doing the carrying as the agent, the thing they carry as the theme, and the place they put it as the goal:

ActionDescriptionFrame({AGENT: _CARRY_AGENT, THEME: _CARRY_THEME, GOAL: _CARRY_GOAL}),

Our overall ActionDescription will make use of both our frame and our variables. Our ActionDescriptionFrame describes the linguistically relevant information, while we use the ActionDescriptionVariables as part of relations and spatial paths to describe the physical aspects of the action: What happens? How do objects move during the action? (We’ll use SpatialPaths to describe this.) What relationships hold before, during and after the action? What relationships hold over the whole course of the action for it to even make sense? What properties do the participants have as a result of their role in the action? (These are used to mark, for example, the goal as stationary.)

Putting it all together, our action description is going to look like this:

_CARRY_AGENT = ActionDescriptionVariable(
    THING, properties=[ANIMATE], debug_handle="carry_agent"
)
_CARRY_THEME = ActionDescriptionVariable(THING, debug_handle="carry_theme")
_CARRY_GOAL = ActionDescriptionVariable(THING, debug_handle="carry_goal")
_CARRY_MANIPULATOR = ActionDescriptionVariable(
    THING, properties=[CAN_MANIPULATE_OBJECTS], debug_handle="carry_manipulator"
)

_CONTACTING_MANIPULATOR = Region(
    reference_object=_CARRY_MANIPULATOR, distance=EXTERIOR_BUT_IN_CONTACT
)

_CARRY_ACTION_DESCRIPTION = ActionDescription(
    frame=ActionDescriptionFrame({AGENT: _CARRY_AGENT, THEME: _CARRY_THEME, GOAL: _CARRY_GOAL}),
    during=DuringAction(
        objects_to_paths=[
           (
                _CARRY_AGENT,
                SpatialPath(
                    operator=TO,
                    reference_source_object=Region(_CARRY_GOAL, distance=DISTAL),
                    reference_destination_object=_CARRY_GOAL,
                ),
           ),
           (
                _CARRY_THEME,
                SpatialPath(
                    operator=TO,
                    reference_source_object=_CONTACTING_MANIPULATOR,
                    reference_destination_object=_CARRY_GOAL,
                ),
            )
        ]
    ),
    enduring_conditions=[
        Relation(SMALLER_THAN, _CARRY_THEME, _CARRY_AGENT),
    ],
    preconditions=[
        Relation(IN_REGION, _CARRY_THEME, _CONTACTING_MANIPULATOR),
        # THEME is not already located in GOAL
        Relation(IN_REGION, _CARRY_THEME, _CARRY_GOAL, negated=True),
    ],
    postconditions=[
        Relation(IN_REGION, _CARRY_THEME, _CONTACTING_MANIPULATOR, negated=True),
        Relation(IN_REGION, _CARRY_THEME, _CARRY_GOAL),
    ],
    asserted_properties=[
        (_CARRY_AGENT, VOLITIONALLY_INVOLVED),
        (_CARRY_AGENT, CAUSES_CHANGE),
        (_CARRY_AGENT, MOVES),
        (_CARRY_THEME, UNDERGOES_CHANGE),
        (_CARRY_GOAL, STATIONARY),
    ],
)

Note

These relationships (or relations) work just like those in situation templates, and we can describe them the same way: Using the relation DSL functions (like “on”, or “near”) together with itertools.chain. In this case, though, we have few and simple enough relations that we can just describe them directly.

Finally, we’ll tell ADAM to associate this description with our ontology node for the carry action (CARRY) so that we can use it when creating curricula. To do this, we’ll add our description to _ACTIONS_TO_DESCRIPTIONS (ADAM’s list of such associations, defined in adam.ontology.phase1_ontology) after the entry for PUT:

_ACTIONS_TO_DESCRIPTIONS = [
    (PUT, _PUT_ACTION_DESCRIPTION),
    (CARRY, _CARRY_ACTION_DESCRIPTION),
    (GIVE, _GIVE_ACTION_DESCRIPTION),
    ...
]

That takes care of describing the action. Now, as the final step, we’ll add carrying to our lexicon. We’ll add a lexicon entry to GAILA_PHASE_1_ENGLISH_LEXICON after FLY:

GAILA_PHASE_1_ENGLISH_LEXICON = OntologyLexicon(
    ontology=GAILA_PHASE_1_ONTOLOGY,
    ontology_node_to_word=(
        ...
        (FLY, LexiconEntry("fly", VERB, verb_form_sg3_prs="flies")),
        (CARRY, LexiconEntry("carry", VERB, verb_form_sg3_prs="carries")),
        (RED, LexiconEntry("red", ADJECTIVE)),
        ...
    ),
)

(Note that sg3_prs stands for “singular third-person present (form).”)

We should now be able to go create situations using this carry action.

Conclusion

In this tutorial you saw how to define a concrete action. The general process should be similar whatever action you want to add, as long as it can be represented in ADAM.

For more examples of actions and their descriptions, refer to adam.ontology.phase1_ontology.