Files
website-evaluator/spec/lighthouse.md
2026-04-17 23:52:40 +12:00

4.2 KiB

Lighthouse Spec (WE-L)

Overview

Runs Lighthouse audits against a URL in two presets (desktop and mobile) and extracts structured findings. Returns a normalised result object used by the report assembler. Also saves the full Lighthouse JSON to the raw directory for reference.

Responsibilities

  • Launch Chrome via chrome-launcher using Playwright's bundled Chromium
  • Run a desktop Lighthouse audit
  • Run a mobile Lighthouse audit
  • Extract scores, Core Web Vitals, and categorised audit findings
  • Save raw results to the raw directory
  • Return structured results for report assembly

Delegates to: Lighthouse, chrome-launcher, Playwright (for executable path)

Dependencies

Read pipeline.md for output directory layout.

Data Model

LighthouseResult (returned per preset)

{
  preset: 'desktop' | 'mobile',
  scores: {
    performance: 87,      // 0-100
    accessibility: 92,
    bestPractices: 100,
    seo: 90,
  },
  vitals: {
    lcp:  { value: '1.2 s',  status: 'good' },     // good | needs-improvement | poor
    tbt:  { value: '120 ms', status: 'good' },
    cls:  { value: '0.02',   status: 'good' },
    ttfb: { value: '380 ms', status: 'needs-improvement' },
    fcp:  { value: '0.8 s',  status: 'good' },
    si:   { value: '1.1 s',  status: 'good' },
  },
  findings: {
    critical: [ { id, title, description, impact } ],  // score === 0
    warnings: [ { id, title, description, impact } ],  // 0 < score < 0.9
  },
}

Vital status thresholds

Vital Good Needs improvement Poor
LCP ≤ 2.5s ≤ 4.0s > 4.0s
TBT ≤ 200ms ≤ 600ms > 600ms
CLS ≤ 0.1 ≤ 0.25 > 0.25
TTFB ≤ 800ms ≤ 1800ms > 1800s
FCP ≤ 1.8s ≤ 3.0s > 3.0s
SI ≤ 3.4s ≤ 5.8s > 5.8s

Requirements

WE-L-1: Use chromium.executablePath() from Playwright to locate the Chrome binary. Pass this path to chrome-launcher as chromePath. Why: The Docker base image includes Playwright's Chromium — we must use that executable rather than letting chrome-launcher search default system paths.

WE-L-2: Run two separate audits: one with formFactor: 'desktop' and one with formFactor: 'mobile'.

WE-L-3: Desktop screen emulation: { mobile: false, width: 1440, height: 900, deviceScaleFactor: 1, disabled: false }.

WE-L-4: Mobile screen emulation: { mobile: true, width: 375, height: 812, deviceScaleFactor: 2, disabled: false }.

WE-L-5: Run only the four standard categories: performance, accessibility, best-practices, seo. Omit pwa.

WE-L-6: Scores are extracted from lhr.categories[id].score * 100, rounded to the nearest integer.

WE-L-7: Core Web Vitals are extracted from these audit IDs:

  • LCP: largest-contentful-paint
  • TBT: total-blocking-time (TBT is the Lighthouse proxy for FID/INP)
  • CLS: cumulative-layout-shift
  • TTFB: server-response-time
  • FCP: first-contentful-paint
  • SI: speed-index

WE-L-8: A finding is critical if audit.score === 0 and the audit is not informative mode. A finding is a warning if 0 < audit.score < 0.9. Audits with score ≥ 0.9 or score null are omitted.

WE-L-9: Finding description is taken from audit.description (Markdown, may contain links). Finding impact is taken from audit.details.type === 'opportunity' savings estimate if present, otherwise omitted.

WE-L-10: Save the full lhr object as JSON to raw/lighthouse-{preset}.json.

WE-L-11: Kill the Chrome instance in a finally block even if the audit throws. Why: Orphaned Chrome processes inside Docker cause container bloat.

Scenarios

Scenario: Happy path

Given: Reachable URL, valid Chrome path When: runLighthouse(url, rawDir) is called Then: Returns [desktopResult, mobileResult]; both lighthouse-desktop.json and lighthouse-mobile.json exist in rawDir

Scenario: Unreachable URL

Given: URL returns connection refused When: Lighthouse audit runs Then: Error is thrown; Chrome is killed before the error propagates

Scenario: Score extraction

Given: lhr.categories.performance.score = 0.87 When: Score is extracted Then: scores.performance === 87