UXpert

Appearance

Getting Started ​

UXpert is an AI-powered UX friction detection tool that runs multiple behavioral agents to identify usability issues through cross-agent comparison.

What is UXpert? ​

UXpert uses Stagehand (a Playwright-based browser automation framework) to run AI agents with different behavioral constraints on your website. By comparing how a "scanner" agent (limited visibility) performs versus a "reader" agent (full visibility), UXpert detects friction points where quick-skimming users might struggle.

Key Concepts ​

  • Agent-Based Testing: Multiple AI agents navigate your site with the same goal but different constraints
  • DOM Filtering: Scanner agents only see headings, CTAs, and microcopy—simulating users who skim
  • Friction Detection: When Scanner fails but Reader succeeds, that's a friction signal
  • Trace Recording: Every step is logged with screenshots for debugging and analysis

Prerequisites ​

  • Bun 1.0 or higher (recommended) or Node.js 18+
  • Anthropic API key for Claude AI access

Installation ​

1. Clone the Repository ​

bash
git clone https://github.com/GCheeYang/UXpert.git
cd UXpert

2. Install Bun (if not already installed) ​

bash
curl -fsSL https://bun.sh/install | bash

3. Install Dependencies ​

bash
cd stagehand-poc
bun install

This installs:

  • @browserbasehq/stagehand - Playwright wrapper with accessibility tree snapshots
  • @anthropic-ai/sdk - Claude API client for planning

For the web UI frontend:

bash
cd frontend
bun install

4. Install Playwright Browsers ​

Stagehand uses Playwright under the hood. Install browser binaries:

bash
bunx playwright install chromium

5. Configure Environment ​

Create a .env file in the project root (one level above stagehand-poc/):

bash
cd ..
echo "ANTHROPIC_API_KEY=your_api_key_here" > .env

Running UXpert ​

CLI Usage ​

Run agents directly from the command line:

bash
cd stagehand-poc

# Default run (books.toscrape.com)
bun run run.js

# Custom URL and goal
bun run run.js --url "https://example.com" --goal "Find pricing information"

# Specific agents
bun run run.js --agents scanner,reader,explorer

Web UI (Development) ​

Start both the API server and frontend dev server:

bash
cd stagehand-poc
bun run dev

This runs:

  • API server at http://localhost:3000
  • Vite dev server at http://localhost:5173 (proxies to API)

Web UI (Production) ​

Build and serve the frontend:

bash
cd stagehand-poc
bun run build
bun run server

Access the UI at http://localhost:3000.

Example Output ​

=== UXpert Agent Runner ===
  URL:     https://books.toscrape.com/
  Goal:    Find a book under £15 and view its details
  Agents:  scanner, reader

--- SCANNER agent starting ---
  [step 1/15] https://books.toscrape.com/ — 45/120 elements visible
  [step 1] click → "Books" (link)
  [step 1] OK — now at https://books.toscrape.com/catalogue/category/books_1/index.html
  ...

============================================================
  COMPARISON
============================================================
  scanner    | goal_achieved      | 8 steps  | 45/120 elements visible
  reader     | goal_achieved      | 6 steps  | 120/120 elements visible

  Both agents succeeded — flow appears accessible to skimmers and readers alike.

Verification ​

CLI Verification ​

bash
cd stagehand-poc
bun run run.js --url "https://example.com" --goal "Find the main heading" --agents scanner

Expected output:

  • Agent starts and captures accessibility tree
  • Planner selects actions from filtered elements
  • Trace saved to traces/ directory

Web UI Verification ​

  1. Start the server: bun run dev
  2. Open http://localhost:3000
  3. Enter a test URL and goal
  4. Click "Run" and wait for agents to complete
  5. View traces in the timeline

Troubleshooting ​

Playwright Browser Issues ​

If you encounter browser launch errors:

bash
# Install system dependencies (Linux)
bunx playwright install-deps chromium

# Or reinstall browsers
bunx playwright install --force chromium

API Key Issues ​

Ensure your Anthropic API key is valid. The key should start with sk-ant-.

Check the key is being loaded:

bash
cd stagehand-poc
ANTHROPIC_API_KEY=your_key bun run run.js

Stagehand Timeout Issues ​

If actions timeout frequently:

  • Ensure you're not blocking the browser with a firewall
  • Try running with headless: false in run.js to debug visually
  • Check if the target site has aggressive bot detection

Port Conflicts ​

If port 3000 is in use, modify server.js:

javascript
const PORT = 3001; // Change to available port

Project Structure ​

UXpert/
├── .env                      # API keys (create this)
├── stagehand-poc/
│   ├── run.js                # CLI runner
│   ├── server.js             # API server (Bun)
│   ├── src/
│   │   ├── agent.js          # Agent loop
│   │   ├── planner.js        # Claude planning
│   │   ├── dom-filter.js     # DOM filtering
│   │   └── overlays.js       # Overlay dismissal
│   ├── traces/               # Output traces + screenshots
│   └── frontend/             # React UI
│       ├── src/
│       └── dist/             # Built assets
└── docs/                     # Documentation

Next Steps ​