Ctrlk
X/TwitterGithubWebsiteDiscord
For the complete documentation index, see llms.txt. This page is also available as Markdown.

E2E Testing

End-to-end testing with WDIO + Appium. CI and local setup.

Overview

Desktop E2E tests use WebDriverIO (WDIO) to drive the Tauri app through Appium:

Platform
Driver
Port
App format
Selectors

Linux / Appium Chromium

Appium Chromium

4723

Debug binary

CSS / DOM

macOS / Appium Chromium

Appium Chromium

4723

.app bundle

CSS / DOM

OpenHuman's desktop app currently uses the CEF runtime (tauri-runtime-cef). CI drives the Linux debug binary with Appium's Chromium driver; manual macOS and Windows E2E use the same Chromium-driver backend.

Quick start

Linux / Appium Chromium

# Install Appium and the Chromium driver (one-time)
npm install -g appium@3
appium driver install --source=npm appium-chromium-driver

# Build the E2E app
pnpm --filter openhuman-app test:e2e:build

# Run all flows
pnpm --filter openhuman-app test:e2e:all:flows

# Run a single spec
bash app/scripts/e2e-run-spec.sh test/e2e/specs/smoke.spec.ts smoke

On headless Linux, the harness runs under Xvfb for a virtual display.

macOS / Appium Chromium

Docker on macOS (Linux harness locally)

Run the same Linux-based harness from macOS using Docker.

Requires Docker Desktop or Colima. The repo is bind-mounted so builds persist between runs.

Architecture

Platform detection

app/test/e2e/helpers/platform.ts exports:

  • isTauriDriver(), legacy shim that now always returns true for the DOM-capable Chromium session

  • isMac2(), legacy shim that now always returns false

  • supportsExecuteScript(), true because the Chromium driver supports browser.execute() on every platform

Element helpers

app/test/e2e/helpers/element-helpers.ts provides a unified API:

Helper
Appium Chromium

waitForText(text)

XPath over DOM text content

waitForButton(text)

button / [role="button"] XPath

clickText(text)

Standard el.click()

clickNativeButton(text)

Standard el.click() on button

clickToggle()

[role="switch"] / input[type="checkbox"]

waitForWindowVisible()

Window handle check

waitForWebView()

document.readyState check

hasAppChrome()

Window handle check

dumpAccessibilityTree()

HTML page source

Stable test IDs

Prefer stable data-testid hooks for UI affordances that E2E specs click or poll. Use the taxonomy <surface>-<element>-<id?>, for example:

  • cron-jobs-panel, cron-refresh

  • cron-job-row-<jobId>, cron-job-toggle-<jobId>, cron-job-run-<jobId>, cron-job-view-runs-<jobId>, cron-job-remove-<jobId>

  • settings-nav-<routeId>

  • skill-row-<skillId>, skill-install-<skillId>, skill-uninstall-<skillId>

  • thread-row-<threadId>, new-thread-button, send-message-button

  • onboarding-next-button

Use waitForTestId(testId) and clickTestId(testId) from element-helpers.ts when a spec targets one of these hooks. Keep text selectors for user-visible copy assertions, not row/action discovery.

Deep link helpers

app/test/e2e/helpers/deep-link-helpers.ts handles auth deep links:

  • Appium Chromium: browser.execute(window.__simulateDeepLink(url)) on every platform

  • macOS fallback: macos: deepLink extension command, then open -a ...

For release candidates, also run one manual secondary-instance smoke on Linux or macOS when touching CEF preflight, single-instance, or deep-link startup code:

  1. Launch OpenHuman normally and leave it running.

  2. Trigger openhuman://auth?token=e2e-token&key=auth through the OS opener.

  3. Confirm the already-running window receives the callback and does not start a second full CEF instance.

  4. Confirm the secondary process exits cleanly without a CEF cache-lock error.

This catches the class of regressions where a secondary process exits during CEF cache preflight before Tauri's deep-link forwarding path is installed.

Writing cross-platform specs

  1. Use helpers from element-helpers.ts, never use raw XCUIElementType* selectors in specs

  2. Use clickNativeButton(text) instead of inline button-clicking code

  3. Use hasAppChrome() instead of checking for XCUIElementTypeMenuBar

  4. Use waitForWebView() instead of checking for XCUIElementTypeWebView

  5. For macOS-only tests, use process.platform guards or separate spec files

  6. Use navigateViaHash(route) for hash routes; it waits for the hash, document.readyState, and a mounted React root before returning. After onboarding, walkOnboarding() also waits for #/home plus a Home-page marker before specs navigate elsewhere.

Environment variables

Variable
Default
Description

APPIUM_PORT

4723

Appium server port

E2E_MOCK_PORT

18473

Mock backend server port

OPENHUMAN_WORKSPACE

(temp dir)

App workspace directory

OPENHUMAN_SERVICE_MOCK

0

Enable service mock mode

OPENHUMAN_E2E_MODE

unset

Enables destructive test-support RPCs; the E2E runner sets this to 1

OPENHUMAN_E2E_AUTH_BYPASS

unset

Enable JWT bypass auth

DEBUG_E2E_DEEPLINK

(verbose)

Set to 0 to silence deep link logs

E2E_FORCE_CARGO_CLEAN

unset

Force cargo clean before E2E build

CI workflows

Push / PR checks

The default pull-request gate is .github/workflows/ci-lite.yml (quick lane: quality checks + unit tests scoped to the changed files). E2E suites do not run on PRs to main โ€” the full E2E matrix (Rust mock-backend, Playwright web, desktop on Linux/macOS/Windows) runs in .github/workflows/ci-full.yml on PRs targeting the release branch and on every push to it.

macOS and Windows desktop E2E do not run on every PR. Use the manually dispatched E2E workflow (.github/workflows/e2e.yml) when cross-platform desktop signal is needed before promotion.

macOS / Appium Chromium

macOS/Appium Chromium is available for local runs and through the manually dispatched E2E workflow:

  1. Installs Appium + Chromium driver

  2. Builds the .app bundle

  3. Runs all E2E flows

Troubleshooting

Linux: "WebView not ready" timeout

For the default CEF runtime, this usually means a stale local runner is trying to drive a CEF-backed WebView through WebKitWebDriver. Current CI uses the Appium Chromium driver on Linux; use app/scripts/e2e-run-session.sh or the PR CI workflow for the supported Linux path.

Ensure DISPLAY is set and Xvfb is running:

Also ensure dbus is started (required by webkit2gtk):

Linux: Appium Chromium driver not found

macOS: Deep links not working in tauri dev

Deep links require a .app bundle. Use pnpm tauri build --debug --bundles app instead.

Docker: Build is slow on first run

The first Docker build compiles Rust and installs the E2E harness dependencies. Subsequent runs use cached layers. Cargo registry and git sources are cached via Docker volumes.

Spec: Notifications

File: app/test/e2e/specs/notifications.spec.ts

Tests notification RPC methods via the live core sidecar and the Notifications UI page:

  • notification_ingest, creates a new notification via core RPC

  • notification_list, verifies the ingested notification is returned

  • notification_mark_read, marks a notification as read

  • notification_stats, checks aggregate statistics shape

  • UI: Notifications page renders the integration notifications section ([data-testid="integration-notifications-section"])

  • UI: Notifications page shows the System Events section ([data-testid="system-events-section"])

Run:

Platform note: RPC tests (notification_ingest, notification_list, notification_mark_read, notification_stats) run through the unified Appium Chromium backend. UI assertions require browser.execute() support, which the current backend provides on every platform.

Agent-observable artifact flow

For a canonical, inspectable run that drops screenshots, page-source dumps, and mock request logs on disk:

Artifacts land in app/test/e2e/artifacts/<timestamp>-agent-review/. Full details + helper API: AGENT-OBSERVABILITY.md. Any failing test triggers wdio.conf.ts's afterTest hook, which writes failure-*.png + failure-*.source.xml into the same run dir.

Rust inference provider E2E

These tests (tests/inference_provider_e2e.rs) use wiremock to mock HTTP upstreams and require no live LLM API calls. They cover OpenAI-compat chat, Anthropic auth style, per-model temperature suppression, Ollama local provider, and the /v1 HTTP endpoint auth layer.

PreviousTesting StrategyNextRelease Policy

Last updated