60 lines
1.9 KiB
Markdown
60 lines
1.9 KiB
Markdown
# Future Ideas — website-evaluator
|
|
|
|
## M2: Claude API Analysis Layer
|
|
|
|
**Problem:** The report is structured data — Claude still has to read it and form recommendations
|
|
manually each time.
|
|
|
|
**Idea:** Add an optional `--analyze` flag. After generating the report, call the Claude API
|
|
with the report + screenshots (vision) and append an AI-generated "Recommendations" section
|
|
with prioritised, actionable fixes.
|
|
|
|
**Open questions:**
|
|
- Should the recommendations overwrite a section or be a separate file?
|
|
- Which Claude model for analysis? Sonnet for cost, Opus for depth?
|
|
- How to handle the screenshot as base64 input to the API?
|
|
|
|
**Depends on:** M1 verified and stable.
|
|
|
|
---
|
|
|
|
## M3: Comparison Mode
|
|
|
|
**Problem:** It's hard to tell if improvements actually moved the needle without a before/after view.
|
|
|
|
**Idea:** Accept two URLs (or a URL + cached baseline) and produce a diff report highlighting
|
|
changes in scores, new/resolved issues, and visual diffs between screenshots.
|
|
|
|
**Open questions:**
|
|
- Store baselines as JSON in the output dir? Or accept two full report.md files?
|
|
- Visual diff: pixel-diff the screenshots or just note score changes?
|
|
|
|
**Depends on:** M1 stable. M2 optional.
|
|
|
|
---
|
|
|
|
## Batch Mode
|
|
|
|
**Problem:** Evaluating 10 pages of a site requires running the container 10 times.
|
|
|
|
**Idea:** Accept a newline-delimited list of URLs and produce one report per URL plus
|
|
a summary roll-up.
|
|
|
|
**Open questions:**
|
|
- Output structure: one folder per URL? Or a single multi-page report?
|
|
|
|
**Depends on:** M1 stable.
|
|
|
|
---
|
|
|
|
## CI Integration Helper
|
|
|
|
**Problem:** Running the evaluator manually is fine for one-offs but doesn't scale to
|
|
catching regressions in CI.
|
|
|
|
**Idea:** Produce a machine-readable `scores.json` alongside the Markdown report.
|
|
Add a `--fail-under` flag that exits non-zero if any Lighthouse category drops below
|
|
a threshold. This makes it usable as a CI gate.
|
|
|
|
**Depends on:** M1 stable.
|