Installation on WSL (Windows Subsystem for Linux)

This guide covers installing MOSAIC on Windows using WSL 2 with an Ubuntu distribution. MOSAIC was designed for native Ubuntu, but WSL 2 provides a compatible Linux environment with some additional caveats.

For native Ubuntu, see Installation on Ubuntu (Native).

Note

WSL 2 is required (not WSL 1). WSL 2 runs a real Linux kernel, which is needed for native package builds and GPU passthrough.

Requirements

  • Windows 11 (or Windows 10 build 19044+)

  • WSL 2 with an Ubuntu distribution (22.04 or 24.04 recommended)

  • NVIDIA GPU driver installed on Windows (for CUDA support)

Setting Up WSL

If you don’t have WSL installed yet:

# From PowerShell (Admin)
wsl --install -d Ubuntu-22.04

After installation, open the Ubuntu terminal from the Start menu or run:

wsl -d Ubuntu-22.04

Warning

Always work inside a native WSL terminal. Do not use PowerShell or Git Bash to run Python or pip against the WSL filesystem – this uses Windows Python, not Linux Python, and will fail.

Method

Shell

Python Used

PowerShell (cd \\\\wsl.localhost\\...)

Windows

Windows Python (wrong)

Git Bash (cd //wsl.localhost/...)

MSYS2/MinGW

Windows Python (wrong)

WSL terminal (wsl -d Ubuntu-22.04)

Linux Bash

WSL Python (correct)

Install System Dependencies

Inside your WSL terminal:

sudo apt-get update
sudo apt-get install -y \
    python3.11 python3.11-venv python3.11-dev \
    build-essential \
    cmake \
    swig \
    flex \
    bison \
    libbz2-dev \
    libopenmpi-dev \
    libegl1 libgl1 libopengl0 libxkbcommon0 \
    libnss3 libnspr4 libasound2 \
    libxcomposite1 libxdamage1 libxkbfile1 \
    libxcb-cursor0 \
    fontconfig fonts-dejavu-extra fonts-liberation fonts-noto \
    stockfish \
    jq

Tip

If python3.11 is not available, add the deadsnakes PPA:

sudo add-apt-repository ppa:deadsnakes/ppa
sudo apt-get update
sudo apt-get install -y python3.11 python3.11-venv python3.11-dev

Setup

All commands below must be run inside the WSL terminal:

# Clone the repository
git clone https://github.com/Abdulhamid97Mousa/MOSAIC.git
cd MOSAIC

# Create virtual environment
python3.11 -m venv .venv
source .venv/bin/activate

# Upgrade pip
pip install --upgrade pip setuptools wheel

# Install core GUI (minimal)
pip install -e .

# Install with optional extras
pip install -e ".[minigrid,mosaic_multigrid,multigrid_ini,pettingzoo,mujoco,atari,vizdoom,crafter,nethack,overcooked,rware,cleanrl,xuance,chat]"

Configure Environment Variables

MOSAIC uses a .env file for configuration. Copy the example file and customize it for your setup:

# Copy the example file
cp .env.example .env

# Edit with your preferred editor
nano .env
# or: code .env, vim .env, etc.

Required API Keys

You must set your API keys in .env to use these features:

Feature

Variable

Get Key From

LLM Chat (cloud)

OPENROUTER_API_KEY

https://openrouter.ai/keys

Gated Models (Llama)

HF_TOKEN

https://huggingface.co/settings/tokens

Experiment Tracking

WANDB_API_KEY

https://wandb.ai/authorize

Edit the .env file and replace the placeholder values:

# Example: Set your OpenRouter API key
OPENROUTER_API_KEY=sk-or-v1-your_actual_key_here

# Example: Set your HuggingFace token
HF_TOKEN=hf_your_actual_token_here

# Example: Set your Weights & Biases API key
WANDB_API_KEY=your_actual_wandb_key_here
WANDB_ENTITY_NAME=your_wandb_username
WANDB_PROJECT_NAME=MOSAIC

WSL-Specific Settings

On WSL, you may need to adjust these settings for proper GUI and GPU support:

# Platform identifier
PLATFORM=ubuntu

# MuJoCo rendering backend (egl works best with WSLg)
MUJOCO_GL=egl

# Qt debugging (set to 1 if you have display issues)
QT_DEBUG_PLUGINS=0

# For WSLg (Windows 11), DISPLAY should be :0
# If using an external X server (VcXsrv, X410), uncomment and set:
# export DISPLAY=$(hostname).local:0

Important

If you experience GUI issues on WSL, see the Broken .venv symlinks (I/O errors) section below.

Install Local 3rd-Party Packages

pip install -e 3rd_party/environments/mosaic_multigrid/
pip install -e 3rd_party/environments/multigrid-ini/
pip install -e 3rd_party/environments/overcooked_ai/
pip install -e 3rd_party/environments/robotic-warehouse/

Install MOSAIC Native Workers

MOSAIC ships five native worker packages in 3rd_party/workers/mosaic/. These workers cover human, LLM, VLM, random-baseline, and passive-baseline paradigms:

# Human Worker (human-in-the-loop action selection via GUI)
pip install -e 3rd_party/workers/mosaic/human_worker/

# LLM Worker (multi-agent LLM coordination with Theory of Mind)
pip install -r requirements/mosaic_llm_worker.txt
pip install -e 3rd_party/workers/mosaic/llm_worker/

# VLM Worker (vision-language model agents with image observations)
pip install -r requirements/mosaic_vlm_worker.txt
pip install -e 3rd_party/workers/mosaic/vlm_worker/

# Random Worker (uniform random action baseline)
pip install -r requirements/mosaic_random_worker.txt
pip install -e 3rd_party/workers/mosaic/random_worker/

# Passive Worker (deterministic noop/still baseline)
pip install -r requirements/mosaic_passive_worker.txt
pip install -e 3rd_party/workers/mosaic/passive_worker/

Worker

Description

human_worker

Human-in-the-loop action selection via GUI clicks.

llm_worker

Multi-agent LLM coordination with Theory of Mind. Supports OpenRouter, OpenAI, Anthropic, and local vLLM.

vlm_worker

Vision-Language Model agents with image observations. Fork of LLM Worker with vision support.

random_worker

Uniform random action selection baseline. Deterministic seeding for reproducibility.

passive_worker

Deterministic do-nothing (noop/still) baseline agent.

Tip

The LLM and VLM workers require API keys. Set them in your .env file:

# For OpenRouter (recommended - access to multiple models)
OPENROUTER_API_KEY=sk-or-v1-your_key_here

See the Required API Keys section above.

Each worker also has a matching requirements file in requirements/ (e.g., requirements/mosaic_llm_worker.txt) for fine-grained dependency management.

WSL-Specific Issues

python3.11: command not found

Cause: Ubuntu 22.04 ships with Python 3.10 by default. Python 3.11 must be installed separately.

Fix:

sudo add-apt-repository ppa:deadsnakes/ppa
sudo apt-get update
sudo apt-get install -y python3.11 python3.11-venv python3.11-dev

GUI not appearing – wrong DISPLAY variable

Error:

qt.qpa.xcb: could not connect to display 172.26.192.1:0
This application failed to start because no Qt platform plugin could be initialized.

Cause: The DISPLAY environment variable points to an external X server (e.g. 172.26.192.1:0 for VcXsrv or X410) instead of WSLg’s built-in display (DISPLAY=:0). This commonly happens when a user previously configured an external X server in ~/.bashrc and later upgraded to a Windows 11 build that includes WSLg (built-in GUI support).

The MOSAIC settings output shows the current display configuration in the "display" section:

"display": {
  "DISPLAY": "172.26.192.1:0",
  "WAYLAND_DISPLAY": "wayland-0",
  "QT_QPA_PLATFORM": "xcb",
  "wslg_available": true,
  "x11_socket_exists": true
}

If wslg_available is true but DISPLAY points to a network address, you should switch to WSLg.

Fix: Comment out the old DISPLAY setting and use WSLg:

# Remove the hardcoded DISPLAY from ~/.bashrc
sed -i 's|export DISPLAY=.*|# &  # Commented out: use WSLg instead|' ~/.bashrc

# Set WSLg display for this session
export DISPLAY=:0

# Verify
echo $DISPLAY    # Should show :0

# Relaunch
bash ./run.sh

Tip

WSLg (Windows Subsystem for Linux GUI) is included in WSL 2 on Windows 11. It automatically sets DISPLAY=:0 and provides an X11 socket at /tmp/.X11-unix/X0. You do not need an external X server (VcXsrv, X410, MobaXterm) when WSLg is available.

If WSLg is not available (older Windows 10 builds), you can either:

  • Install and run an external X server on Windows, or

  • Use a virtual framebuffer for headless operation:

sudo apt-get install -y xvfb
xvfb-run python -m gym_gui

CUDA / GPU Not Detected

Cause: On WSL 2, CUDA uses the Windows NVIDIA driver – you should not install the Linux NVIDIA driver inside WSL.

Fix:

  1. Install the latest NVIDIA GPU driver on Windows (from nvidia.com/drivers).

  2. Verify inside WSL:

    nvidia-smi                   # Should show your GPU
python -c "import torch; print(torch.cuda.is_available())"

  3. If PyTorch doesn’t detect CUDA, reinstall with the correct CUDA version:

    pip install torch --index-url https://download.pytorch.org/whl/cu121

Filesystem Performance

Symptom: pip install or git operations are very slow.

Cause: Accessing files across the WSL/Windows boundary (/mnt/c/... or \\wsl.localhost\...) is significantly slower than working within the native WSL filesystem.

Fix: Keep the project on the Linux filesystem (~/projects/MOSAIC), not on a Windows-mounted drive (/mnt/c/Users/.../MOSAIC).

Common Build Errors

Build failures and dependency conflicts are shared across Ubuntu and WSL. See the Common Errors page for all known issues and fixes, including:

  • swig not found (Box2D)

  • nle build failure (NetHack)

  • smac/smacv2 protobuf version conflict with gRPC

  • mpi4py MPI compiler not found

  • Broken venv / wrong Python version

  • Proto stub regeneration with tools/generate_protos.sh

Verify Installation

# Inside WSL terminal with venv activated
./run.sh

# Or launch GUI only
python -m gym_gui

If successful, the console will log detected optional dependencies and system info:

bash ./run.sh
Checking for existing trainer processes...
Starting trainer daemon...
Waiting for trainer daemon...
Trainer daemon ready.
Launching MOSAIC...
[gym_gui] Loaded settings:
{
  "qt_api": "PyQt6",
  "log_level": "DEBUG",
  "default_control_mode": "human_only",
  "default_seed": 1,
  "allow_seed_reuse": false,
  "ui": {
    "chat_panel_collapsed": true
  },
  "vllm": {
    "max_servers": 4,
    "gpu_memory_utilization": 0.85
  },
  "system": {
    "cpu_model": "x86_64",
    "cpu_cores_physical": 8,
    "cpu_cores_logical": 16,
    "cpu_freq_max_mhz": 0,
    "ram_total_gb": 15.5,
    "ram_available_gb": 13.3,
    "ram_used_gb": 2.2,
    "ram_percent_used": 14.0
  },
  "cuda": {
    "available": true,
    "device_count": 1,
    "current_device": 0,
    "device_name": "NVIDIA GeForce RTX 3060 Laptop GPU",
    "memory_total_gb": 6.0,
    "memory_free_gb": 5.9,
    "memory_used_gb": 0.1
  },
  "display": {
    "DISPLAY": ":0",
    "WAYLAND_DISPLAY": "wayland-0",
    "XDG_RUNTIME_DIR": "/run/user/1000/",
    "QT_QPA_PLATFORM": "xcb",
    "wslg_available": true,
    "x11_socket_exists": true
  },
  "env": {
    "MUJOCO_GL": "egl",
    "QT_DEBUG_PLUGINS": "0",
    "PLATFORM": "ubuntu",
    "MPI4PY_RC_INITIALIZE": "0"
  },
  "optional_deps": {
    "chat": true,
    "minigrid": true,
    "babyai": true,
    "mosaic_multigrid": true,
    "multigrid_ini": true,
    "pettingzoo": true,
    "mujoco": false,
    "atari": true,
    "vizdoom": false,
    "crafter": true,
    "nethack": false,
    "overcooked_ai": false,
    "smac": false,
    "smacv2": false,
    "rware": false,
    "ray_worker": true,
    "cleanrl_worker": false,
    "xuance_worker": false
  },
  "protobuf": {
    "compiled": true,
    "location": "gym_gui/services/trainer/proto",
    "pb2_files_count": 1,
    "grpc_files_count": 1
  }
}
[gym_gui] Qt platform plugin: xcb