Workshop Documentation
Everything you need to go from a fresh machine to running the workshop hub — usually under 15 minutes.
Written by Jim SeelanThe short version: double-click setup.bat and wait. The script handles everything and opens the browser when it's done. The steps below are just there in case you want to know what's happening.
Clone it with Git, or download the ZIP from GitHub and unzip it somewhere sensible.
git clone https://github.com/learnermaker/physical-ai-lab.git
cd physical-ai-labsetup.batNo admin rights needed. A terminal window will open and work through the setup automatically. Total time is usually 10–15 minutes on a decent connection.
Here's what it does:
| Stage | What happens |
|---|---|
| [1/9] | Looks for Python 3.12 — downloads and installs it quietly if not found |
| [2/9] | Creates a .venv folder inside the repo (isolated Python environment) |
| [3/9] | Activates the environment |
| [4/9] | Installs PyTorch CPU-only (the big one — about 200 MB) |
| [5/9] | Installs everything else from requirements.txt |
| [6/9] | Registers the kernel so VS Code notebooks can use this environment |
| [7/9] | Downloads the MediaPipe hand landmark model (about 8 MB) |
| [8/9] | Runs a quick check — prints pass/warn/fail for each package |
| [9/9] | Opens the hub server and launches the browser |
Step 8 prints something like this:
[PASS] numpy
[PASS] mujoco
[PASS] gymnasium
...
[WARN] Camera unavailable -- fallback video will be used
...
Setup complete: 13/14 checks passed[WARN] is fine — the workshop has built-in fallbacks for webcam and OpenGL. [FAIL] means something needs fixing; the script prints the exact pip install command to run.
Module 5 can call Google Gemini live to analyse camera frames. You don't need a key — it falls back to pre-recorded responses — but live results are noticeably more interesting.
Get a free key at aistudio.google.com (sign in → Get API key), then:
# In the repo folder — Windows
copy .env.example .env
# Open .env in Notepad and add your key:
# GEMINI_API_KEY=AIzaSy...your_keyBefore the workshop, run this to see what a trained RL agent actually looks like. It's a good warm-up for Module 3, and people tend to find it surprisingly impressive.
python modules/02_simulation/03_pretrained_agent.pyA MuJoCo window opens with a cheetah robot running. No GPU required.
setup.bat again. It detects what's already done and skips those steps.
If you'd rather do it yourself — or if setup.bat is giving you grief — here are the individual steps. You'll need Python 3.12 installed first.
git clone <repo-url>
cd physical-ai-workshopThis keeps the workshop packages separate from everything else on your machine.
# Create
py -3.12 -m venv .venv
# Activate (Windows)
.venv\Scripts\activate
# Your prompt should now show (.venv).venv\Scripts\activate again before continuing.
python -m pip install --upgrade pipThis needs to go in before the rest so that packages like Stable-Baselines3 link against the right version. It's about 200 MB so give it a minute.
pip install torch==2.14.0+cpu --index-url https://download.pytorch.org/whl/cpu
# Quick check:
python -c "import torch; print(torch.__version__)"
# Should print: 2.14.0+cpupip install -r requirements.txt
# Installs: numpy, mujoco, gymnasium, stable-baselines3,
# opencv-python, mediapipe, google-genai, fastapi, and more.
# Around 500 MB. Takes 3–10 minutes.python -m ipykernel install --user --name physical-ai --display-name "Physical AI Workshop"This should already be in assets/ if you cloned the repo. If it's missing:
python -c "
import urllib.request, pathlib
pathlib.Path('assets').mkdir(exist_ok=True)
urllib.request.urlretrieve(
'https://storage.googleapis.com/mediapipe-models/hand_landmarker/hand_landmarker/float16/latest/hand_landmarker.task',
'assets/hand_landmarker.task'
)
print('done')
"python verify_install.pyAll 14 checks should be [PASS] or [WARN]. A warning just means a fallback is active — the workshop still works.
copy .env.example .env
# Edit .env and set:
# GEMINI_API_KEY=AIzaSy...your_key
# Get a free key at https://aistudio.google.com/python api/server.pyOpen http://localhost:8000 in Chrome or Edge.
.venv\Scripts\activate
python api/server.pyIf you've been through the setup already, you just need two lines to get going:
# Option A — terminal
.venv\Scripts\activate
python api/server.py
# Option B — double-click setup.bat again.
# It'll see the .venv exists and skip straight to launching the server.Then open http://localhost:8000 in Chrome or Edge and you're in.
| Module | What you actually do | Server? |
|---|---|---|
| 0 — Kickoff | Overview notebook, pipeline diagram | — |
| 1 — Perception | Live hand tracking in the browser — no server needed | No |
| 2 — Simulation | Reacher-v5 environment streaming live from the server | Yes |
| 3 — RL | Train a PPO agent, watch the reward chart update in real time | Yes |
| 4 — Perception → Action | Move your hand, watch the Reacher arm follow | Yes |
| 5 — Foundation Models | Camera frame goes to Gemini, robot action comes back | Yes |
| 6 — Wrap-up | Concept map, sim-to-real, what's next | — |
Every module has an exercise.py. Open it in VS Code next to the browser window. Each one has a # TODO START / END block — usually 3–5 lines of code, completable in 10–15 minutes. The # SOLUTION block at the bottom can be uncommented if you get stuck.
modules/00_kickoff/exercise.py
modules/01_perception/exercise.py
modules/02_simulation/exercise.py
modules/03_rl/exercise.py
modules/04_perception_to_action/exercise.py
modules/05_foundation_models/exercise.pyIn VS Code: File → Open Folder → select the repo. Then open a notebook and choose the Physical AI Workshop kernel from the picker in the top-right corner.
modules/00_kickoff/overview.ipynb
modules/03_rl/01_mdp_concepts.ipynb
modules/06_wrapup/concepts_map.ipynbThere's a setup.sh that mirrors the Windows script. It needs Python 3.12 already installed:
bash setup.shIf Python 3.12 isn't on your machine yet:
# Homebrew first, if you don't have it
/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"
brew install python@3.12
python3.12 --version # should print 3.12.xsudo add-apt-repository ppa:deadsnakes/ppa
sudo apt update
sudo apt install python3.12 python3.12-venv python3.12-dev
python3.12 --versionSame as the Manual Setup tab, with two differences:
# Create and activate the venv
python3.12 -m venv .venv
source .venv/bin/activate # ← not .venv\Scripts\activate
# Everything else is identical
pip install torch==2.14.0+cpu --index-url https://download.pytorch.org/whl/cpu
pip install -r requirements.txt
python verify_install.py
python api/server.py| Topic | Notes |
|---|---|
| MuJoCo | Pip wheels exist for macOS (arm64 + x86_64) and Linux x86-64. Installs without issues. |
| PyTorch on Apple Silicon | The CPU wheel works fine. For faster Module 3 training you can skip the --index-url flag and get the native MPS build instead — entirely optional. |
| OpenGL on Linux | If you're on a headless server, make_env automatically falls back from human rendering to rgb_array to None. Nothing to configure. |
| MediaPipe | Wheels exist for macOS arm64/x86_64 and Linux x86_64. Linux aarch64 is not covered. |
| Webcam on Linux | Add yourself to the video group if Chrome can't access the camera: sudo usermod -aG video $USER, then log out and back in. |
| Browser | Chrome or Edge required on all platforms. SharedArrayBuffer (needed by MediaPipe WASM) isn't available in Firefox. |
pip install as normal — pip picks the right one automatically.
Most issues fall into one of these categories. If none of these help, check instructor/COMMON_ISSUES.md in the repo — it has 13 failures documented with exact fix commands.
Right-click setup.bat → Open (instead of double-clicking) to keep the window visible. Or run it from a terminal:
cd C:\path\to\physical-ai-workshop
.\setup.batPython 3.12 couldn't be found or downloaded. Check your connection, then install it manually from python.org — tick Add Python to PATH during setup — then re-run setup.bat.
The CPU wheel is about 200 MB. If pip times out, run it manually in a terminal with the venv active:
.venv\Scripts\activate
pip install torch==2.14.0+cpu --index-url https://download.pytorch.org/whl/cpu[FAIL] mujoco in the verify outputARM64 Windows (Surface Pro X, Snapdragon): MuJoCo doesn't have an ARM64 Windows wheel. Run everything inside WSL2:
# PowerShell, as Administrator
wsl --install
# Restart, open Ubuntu, then run: bash setup.shRegular x86-64 Windows: Usually a missing C++ runtime. Install it:
winget install Microsoft.VCRedist.2015+.x64You need Chrome or Edge. Firefox and Safari don't support SharedArrayBuffer, which MediaPipe's WASM inference requires. Switch browsers and try again.
# Find who's using it
netstat -ano | findstr :8000
# Kill it (replace 12345 with the PID in the last column)
taskkill /PID 12345 /F
python api/server.pyClick the kernel name in the top-right of any notebook → Python Environments… → pick Physical AI Workshop. If it's not there, re-register it:
.venv\Scripts\activate
python -m ipykernel install --user --name physical-ai --display-name "Physical AI Workshop"Check .env exists in the repo root and has a valid key. Module 5 falls back to cached responses automatically if the key is missing — the workshop works either way.
type .env # should show GEMINI_API_KEY=AIzaSy...