Glossary: Key Terminology
This glossary defines key terms used throughout Module 4: Vision-Language-Action (VLA).
Vision-Language-Action (VLA)
Definition: A unified framework that combines vision, language processing, and action execution to enable natural language interaction with robots.
Context: VLA systems enable robots to understand spoken or written commands, perceive their environment, and execute physical actions based on language instructions.
Related Terms: Cognitive planning, voice-to-action, natural language processing
Example: A VLA system allows a user to say "Pick up the red cup" and the robot understands the command, identifies the cup, and picks it up.
Voice-to-Action
Definition: The capability that enables robots to convert spoken commands into robot actions through speech recognition and cognitive planning.
Context: Voice-to-action systems use speech recognition (like OpenAI Whisper) to transcribe spoken commands, which are then processed by cognitive planning systems to generate robot actions.
Related Terms: Speech recognition, cognitive planning, natural language processing
Example: When a user says "Clean the room," a voice-to-action system transcribes the speech and generates a sequence of robot actions to accomplish the task.
Cognitive Planning
Definition: The process by which Large Language Models (LLMs) translate natural language commands into sequences of executable robot actions.
Context: Cognitive planning decomposes high-level natural language instructions into structured action plans that robots can execute through ROS 2 actions.
Related Terms: LLM, natural language processing, action sequence, ROS 2
Example: Cognitive planning translates "Clean the room" into a sequence of navigation, perception, and manipulation actions.
Natural Language Intent
Definition: The semantic meaning and goal extracted from a voice command or text instruction.
Context: Natural language intent represents what the user wants the robot to accomplish, including the goal, required capabilities, constraints, and context.
Related Terms: Intent understanding, goal decomposition, cognitive planning
Example: The natural language intent of "Pick up the red cup" includes the goal (pick up), the target (red cup), and implicit constraints (use appropriate grasp).
Action Sequence
Definition: An ordered list of ROS 2 actions that implement a cognitive plan.
Context: Action sequences represent the executable robot behaviors generated by cognitive planning, including individual actions, action parameters, dependencies, and execution order.
Related Terms: ROS 2 actions, cognitive plan, action execution
Example: An action sequence for "Pick up the cup" might include: navigate to table, detect objects, identify cup, plan grasp, execute grasp.
VLA Pipeline
Definition: The complete flow from voice input to physical action in VLA systems.
Context: The VLA pipeline includes voice capture, speech recognition, text transcription, cognitive planning, action generation, perception, navigation, and manipulation stages.
Related Terms: Voice-to-action, cognitive planning, action execution
Example: The VLA pipeline flows: Voice command → Whisper transcription → Cognitive planning → ROS 2 actions → Robot execution.
Cognitive Plan
Definition: A structured sequence of robot actions generated by an LLM from natural language input.
Context: Cognitive plans bridge natural language intent to executable robot behaviors, containing high-level goals, decomposed sub-tasks, action sequences, and execution parameters.
Related Terms: Cognitive planning, action sequence, natural language intent
Example: A cognitive plan for "Clean the room" includes sub-tasks like navigate, identify objects, pick up objects, and place objects, each with associated ROS 2 actions.
Voice Command
Definition: A spoken instruction given to a humanoid robot.
Context: Voice commands contain audio waveform data, transcribed text, semantic meaning, and intent, enabling natural language human-robot interaction.
Related Terms: Voice-to-action, speech recognition, natural language intent
Example: "Pick up the red cup from the table" is a voice command that initiates the VLA pipeline.
Action Sequence
Definition: An ordered list of ROS 2 actions that implement a cognitive plan.
Context: Action sequences contain individual actions (navigation, manipulation, perception), action parameters, dependencies between actions, and execution order.
Related Terms: ROS 2 actions, cognitive plan, action execution
Example: An action sequence might include: NavigateToPose, DetectObjects, PickPlace actions in a specific order.
Capstone Project Scenario
Definition: A complete autonomous behavior demonstration integrating all VLA components.
Context: Capstone project scenarios demonstrate practical application of VLA concepts, including voice command input, planning phase, navigation phase, object identification phase, and manipulation phase.
Related Terms: VLA pipeline, autonomous behavior, integration
Example: The capstone project demonstrates a complete scenario where a robot receives a voice command, plans a path, navigates obstacles, identifies objects, and manipulates them.