What is Physical AI?

The analogy

Reading a book is static — the book doesn't change based on what you understood. Having a conversation is dynamic — what you say changes what the other person says next. Physical AI systems are always in a conversation with the physical world.

From pixels to numbers

cap.read()  # returns numpy array, shape (480, 640, 3)
# 480 rows × 640 columns × 3 (B, G, R values)
# 640 × 480 × 3 colour-channel values

The compression pipeline

Camera frame (480×640×3 ints)
  → MediaPipe model
  → 21 landmark (x, y, z) coordinates
  → dot-product angle formula
  → 3–4 joint angles in degrees
  → numpy array shape (4,) float32  ← the state vector

MediaPipe & landmarks

LIVE_STREAM mode — why async

State vectors

State vectors in this lab

Gymnasium & MuJoCo

obs, info = env.reset()
obs, reward, done, trunc, info = env.step(action)
env.close()

The episode loop

Reinforcement Learning

The core loop

1. Agent observes current state ("pole tilted 5° right, moving at 0.3 rad/s")
2. Agent picks an action ("push cart right")
3. Environment transitions to new state (cart moves, pole responds)
4. Agent receives reward (+1 because pole still upright)
5. Repeat millions of times

RL vs supervised learning

Markov Decision Processes

The Markov property — why it matters

For CartPole: knowing current (position, velocity, angle, angular_velocity) is sufficient to predict the next state. No history needed. This is what makes the problem mathematically tractable.

PPO — Proximal Policy Optimisation

The reward curve

Closing the loop

The exact loop you build in Module 4

Your hand moves slightly
  → camera captures new frame
  → MediaPipe detects landmark 8 at (x=0.6, y=0.4)
  → scale_landmark_to_action → action = [0.2, -0.2]
  → env.step([0.2, -0.2]) → Reacher arm moves
  → env.render() → new frame sent to browser
  → you see the arm respond
  → your hand adjusts
  → the person sees the result and adjusts their hand

The update rate and end-to-end latency depend on the laptop, camera, rendering path and browser. Treat any frame-rate or latency figure as an estimate unless it has been measured on the setup being used. Here, the person closes the feedback loop by watching the simulated arm and adjusting their hand.

Foundation Models & VLMs

VLA — Vision-Language-Action models

The sim-to-real gap

Three sources

• Physics mismatch — simulated friction, contacts, and inertia are approximations. Real motors have backlash, thermal drift.
• Sensor noise — real cameras have blur, distortion, lighting variation. Simulated cameras are noiseless.
• Appearance gap — MuJoCo renders look nothing like reality. A policy trained on sim visuals won't recognise real scenes.

How to bridge the gap

The pipeline. Everything builds toward this. 15 min

We're not building from scratch. We're wiring together pre-trained pieces. The glue code is what you're writing today.

modules/00_kickoff/overview.ipynb   # kernel: Physical AI Lab — run all cells

Print your first state vector

import numpy as np
print(np.array([145.2, 163.8, 171.4, 158.0], dtype=np.float32))
# [145.2 163.8 171.4 158.0]
# That printed line is a state vector.

Try it yourself first. Then refer back to the pipeline diagram (slide 2) and name which module handles each stage.

From webcam image to useful measurements. 30 min

A webcam frame contains many pixel values. MediaPipe’s hand-landmark model returns 21 landmarks; selected landmarks can then be used to calculate joint angles. The angle formula compresses that to 4 joint angles. That's your state vector.

python modules/01_perception/01_webcam_basics.py
# Prints frame shape: (480, 640, 3)

python modules/01_perception/02_hand_tracking.py
# 21 green dots appear. Hold your hand up.

python modules/01_perception/03_joint_angles.py
# Prints [145.2, 163.8, 171.4] once per second
# "That is a state vector."

21 landmarks. Normalised to [0, 1].

The dot-product angle

The code skeleton — MediaPipe async pattern

import queue
import mediapipe as mp
from mediapipe.tasks import python as mp_python
from mediapipe.tasks.python import vision as mp_vision

result_queue = queue.Queue(maxsize=1)  # holds the latest result only

def on_result(result, image, timestamp):
    # Called by MediaPipe's background thread when a frame is processed
    # We put it in the queue — the main loop picks it up
    if result_queue.full():
        result_queue.get_nowait()   # drop the stale result
    result_queue.put_nowait(result)

# Build the landmarker once — reused for every frame
options = mp_vision.HandLandmarkerOptions(
    base_options=mp_python.BaseOptions(model_asset_path="assets/hand_landmarker.task"),
    running_mode=mp_vision.RunningMode.LIVE_STREAM,
    result_callback=on_result          # ← our function gets called here
)
landmarker = mp_vision.HandLandmarker.create_from_options(options)

cap = cv2.VideoCapture(0)             # open webcam
while True:
    ret, frame = cap.read()           # BGR frame from camera
    mp_image = mp.Image(image_format=mp.ImageFormat.SRGB,
                        data=cv2.cvtColor(frame, cv2.COLOR_BGR2RGB))
    landmarker.detect_async(mp_image, timestamp_ms)  # non-blocking — returns immediately

    # Main loop continues here — doesn't wait for MediaPipe
    if not result_queue.empty():
        result = result_queue.get_nowait()
        lm = result.hand_landmarks[0]         # list of 21 landmarks
        x = lm[8].x                          # index fingertip x, in [0, 1]
        # → compute angles, build state vector, etc.

We never write the physics. 20 min

CartPole: 4 observations, 2 discrete actions. Reacher: 11 observations, 2 continuous torques. Those torques are what your hand controls in 20 minutes.

python modules/02_simulation/01_gym_intro.py
# Observation space: Box(-4.8, 4.8, (4,), float32)
# Action space: Discrete(2)

python modules/02_simulation/02_mujoco_reacher.py
# Observation space: Box(-inf, inf, (11,), float64)
# Action space: Box(-1.0, 1.0, (2,), float32)

11 observations. You'll control 2 of them.

The episode loop in code

import gymnasium as gym
import numpy as np

env = gym.make("CartPole-v1")       # create the environment
obs, info = env.reset()             # reset to start state, get first observation
# obs is a numpy array: [cart_pos, cart_vel, pole_angle, pole_vel]
print(obs.shape)                    # → (4,)

for step in range(200):             # run for up to 200 timesteps
    action = env.action_space.sample()  # random action: 0 (left) or 1 (right)
    obs, reward, terminated, truncated, info = env.step(action)
    # obs    : new state after the action
    # reward : +1.0 if the pole is still upright
    # terminated : True if pole fell or cart went out of bounds
    # truncated  : True if we hit the max episode length (500 steps for CartPole)

    if terminated or truncated:
        obs, info = env.reset()     # start a new episode

env.close()

It starts random. Watch it figure it out. 30 min

python modules/03_rl/02_train_cartpole.py
# ~2–3 min. Keep matplotlib window visible.
# Walk the MDP cycle diagram while it trains.
# STOP TALKING when the curve climbs — let everyone watch.

State → Policy → Environment → Reward → repeat

PPO training — what the code actually does

import gymnasium as gym
from stable_baselines3 import PPO
from stable_baselines3.common.callbacks import BaseCallback

env = gym.make("CartPole-v1")

# Create the PPO agent
# "MlpPolicy" = a Multi-Layer Perceptron network (2 hidden layers of 64 neurons)
# The network takes the 4-element CartPole state and outputs action probabilities
model = PPO(
    "MlpPolicy",
    env,
    verbose=1,           # print training progress every n_steps
    n_steps=2048,        # collect this many timesteps before each update
    batch_size=64,       # mini-batch size for gradient updates
    gamma=0.99,          # discount factor
)

# .learn() runs the entire training loop:
#   1. Run policy for n_steps, collect (obs, action, reward, next_obs) tuples
#   2. Compute advantage (GAE) for each action
#   3. Update network weights using the clipped objective
#   4. Repeat until total_timesteps reached
model.learn(total_timesteps=50_000)

model.save("models/ppo-CartPole-trained")  # save for later

From perception to action, in real time. 40 min

python modules/04_perception_to_action/01_hand_to_reacher.py
# Two windows: camera + Reacher sim.
# Move your hand slowly.
# x position → joint 0 torque. y position → joint 1 torque.

Landmark x → robot torque

The human-in-the-loop feedback path

import queue, cv2, gymnasium as gym
# (MediaPipe setup identical to M1 — same async pattern)

result_queue = queue.Queue(maxsize=1)

def on_result(result, image, timestamp):
    if result_queue.full(): result_queue.get_nowait()
    result_queue.put_nowait(result)

# ── MediaPipe setup (same 4 lines as M1 — see "M1 — How it works") ───────────
options = mp_vision.HandLandmarkerOptions(
    base_options=mp_python.BaseOptions(model_asset_path="assets/hand_landmarker.task"),
    running_mode=mp_vision.RunningMode.LIVE_STREAM,
    result_callback=on_result)
landmarker = mp_vision.HandLandmarker.create_from_options(options)  # async
env = gym.make("Reacher-v5", render_mode="rgb_array")
obs, _ = env.reset()
cap = cv2.VideoCapture(0)

# ── The closed loop ─────────────────────────────────────────
while True:
    ret, frame = cap.read()
    landmarker.detect_async(mp_image_from(frame), timestamp_ms)  # non-blocking

    if not result_queue.empty():
        result = result_queue.get_nowait()
        lm = result.hand_landmarks[0]

        # The one line that closes the loop:
        action = [2 * lm[8].x - 1,   # landmark 8 x → joint 0 torque
                  2 * lm[8].y - 1]   # landmark 8 y → joint 1 torque

        obs, reward, terminated, truncated, _ = env.step(action)
        # env.step() advances the simulation by one timestep
        # The robot arm moves. env.render() will show the new frame.

    if terminated or truncated:
        obs, _ = env.reset()

    # Display both windows, handle quit key...

A separate image-to-JSON model demo. 25 min

python modules/05_foundation_models/01_gemini_vision.py
# Captures one frame, sends "Describe this scene." ~2 sec.

python modules/05_foundation_models/02_gemini_robot_brain.py
# Returns a JSON-style dict with an action, observation, reason and next step

Camera image → Gemini → structured JSON response

Writing your own prompt — the template

# This mirrors the current demo's action vocabulary and response fields.
# It is a teaching prompt, not a dependable assistive-robot policy.
SYSTEM_PROMPT = (
    "You are ARIA, an assistive robot in a workspace. "
    "Observe the image and respond ONLY in JSON with: "
    '{"action":"APPROACH|WAIT|ALERT|RETREAT","confidence":0.0,'
    '"observation":"...","reason":"...","next_step":"..."}'
)

# Exercise: change one instruction or action definition at a time.
# Compare the returned JSON fields and discuss where the model may be wrong.
# Do not treat the confidence field as calibrated certainty or use this
# prompt to make real decisions about a person's wellbeing.

Established tools, small examples. 15 min

Every policy you trained today lives in simulation.

No extra hardware needed

With a Raspberry Pi and two SG90 servos

Hardware cost is an estimate and depends on what you already own; check the bill of materials before buying. Projects 7–8 are starting points for experimenting with sensors and actuators on real hardware.