Date created: | Last Updated:

Category: Project

Description: Going from natural language instructions to fully specified executable plans for household robots, various reasoning steps are required. We propose and implement a processing pipeline to tackle these steps and use the ontological Socio-physical Model of Activities (SOMA) as a common interface between its components. One major advantage of employing an overarching ontological framework is that we can store its asserted facts alongside the semantics of instructions, contextual knowledge, and annotated activity models in one central knowledge base. This allows for a unified and efficient knowledge retrieval across all pipeline components, providing flexibility and reasoning capabilities as we combine symbolic knowledge with annotated sub-symbolic models. Using our pipeline, a robot faced with a natural language task description is able to simulate the task and thus, demonstrate its understanding.