Skip to main content

Glossary: Key Terminology

This glossary defines key terms used throughout Module 4: Vision-Language-Action (VLA).

Vision-Language-Action (VLA)​

Definition: A unified framework that combines vision, language processing, and action execution to enable natural language interaction with robots.

Context: VLA systems enable robots to understand spoken or written commands, perceive their environment, and execute physical actions based on language instructions.

Related Terms: Cognitive planning, voice-to-action, natural language processing

Example: A VLA system allows a user to say "Pick up the red cup" and the robot understands the command, identifies the cup, and picks it up.

Voice-to-Action​

Definition: The capability that enables robots to convert spoken commands into robot actions through speech recognition and cognitive planning.

Context: Voice-to-action systems use speech recognition (like OpenAI Whisper) to transcribe spoken commands, which are then processed by cognitive planning systems to generate robot actions.

Related Terms: Speech recognition, cognitive planning, natural language processing

Example: When a user says "Clean the room," a voice-to-action system transcribes the speech and generates a sequence of robot actions to accomplish the task.

Cognitive Planning​

Definition: The process by which Large Language Models (LLMs) translate natural language commands into sequences of executable robot actions.

Context: Cognitive planning decomposes high-level natural language instructions into structured action plans that robots can execute through ROS 2 actions.

Related Terms: LLM, natural language processing, action sequence, ROS 2

Example: Cognitive planning translates "Clean the room" into a sequence of navigation, perception, and manipulation actions.

Natural Language Intent​

Definition: The semantic meaning and goal extracted from a voice command or text instruction.

Context: Natural language intent represents what the user wants the robot to accomplish, including the goal, required capabilities, constraints, and context.

Related Terms: Intent understanding, goal decomposition, cognitive planning

Example: The natural language intent of "Pick up the red cup" includes the goal (pick up), the target (red cup), and implicit constraints (use appropriate grasp).

Action Sequence​

Definition: An ordered list of ROS 2 actions that implement a cognitive plan.

Context: Action sequences represent the executable robot behaviors generated by cognitive planning, including individual actions, action parameters, dependencies, and execution order.

Related Terms: ROS 2 actions, cognitive plan, action execution

Example: An action sequence for "Pick up the cup" might include: navigate to table, detect objects, identify cup, plan grasp, execute grasp.

VLA Pipeline​

Definition: The complete flow from voice input to physical action in VLA systems.

Context: The VLA pipeline includes voice capture, speech recognition, text transcription, cognitive planning, action generation, perception, navigation, and manipulation stages.

Related Terms: Voice-to-action, cognitive planning, action execution

Example: The VLA pipeline flows: Voice command → Whisper transcription → Cognitive planning → ROS 2 actions → Robot execution.

Cognitive Plan​

Definition: A structured sequence of robot actions generated by an LLM from natural language input.

Context: Cognitive plans bridge natural language intent to executable robot behaviors, containing high-level goals, decomposed sub-tasks, action sequences, and execution parameters.

Related Terms: Cognitive planning, action sequence, natural language intent

Example: A cognitive plan for "Clean the room" includes sub-tasks like navigate, identify objects, pick up objects, and place objects, each with associated ROS 2 actions.

Voice Command​

Definition: A spoken instruction given to a humanoid robot.

Context: Voice commands contain audio waveform data, transcribed text, semantic meaning, and intent, enabling natural language human-robot interaction.

Related Terms: Voice-to-action, speech recognition, natural language intent

Example: "Pick up the red cup from the table" is a voice command that initiates the VLA pipeline.

Action Sequence​

Definition: An ordered list of ROS 2 actions that implement a cognitive plan.

Context: Action sequences contain individual actions (navigation, manipulation, perception), action parameters, dependencies between actions, and execution order.

Related Terms: ROS 2 actions, cognitive plan, action execution

Example: An action sequence might include: NavigateToPose, DetectObjects, PickPlace actions in a specific order.

Capstone Project Scenario​

Definition: A complete autonomous behavior demonstration integrating all VLA components.

Context: Capstone project scenarios demonstrate practical application of VLA concepts, including voice command input, planning phase, navigation phase, object identification phase, and manipulation phase.

Related Terms: VLA pipeline, autonomous behavior, integration

Example: The capstone project demonstrates a complete scenario where a robot receives a voice command, plans a path, navigates obstacles, identifies objects, and manipulates them.