QA tooling for city-scrapers
See exactly what your scraper produced.
Meetings Viewer turns raw Scrapy JSON into a browsable, filterable table. Review a spider's output in minutes instead of an afternoon — and stop debating whether a scraper is actually working.
- No database
- No API server
- Reads scrapy output straight from disk
Two modes, one interface
Identical routes, columns, and stats in both modes. Only the data source changes — it lives entirely in lib/scraper-data.ts.
Local
While developing a scraper
Your spider writes its output to data/scrapers/ and the viewer renders it. Re-run the crawl, refresh the page, see the new records.
Source
data/scrapers/*.json
Production
During PR review
A GitHub Actions workflow attached to the open PR publishes the run, and the viewer reads it. Review output before the scraper is merged.
Source
meetings-viewer-data
Everything you check during QA
The record table is built on MUI X Data Grid, so the whole run stays scannable no matter how many meetings a spider returns.
Search across records
Free-text search over every record in a run, so you can jump straight to the meeting you are checking.
Status & date filters
Narrow to passed, cancelled, or tentative records, and clamp results to a date range.
Duplicate highlighting
Duplicate groups are detected and colour-coded, which catches the classic lowercase -o append mistake instantly.
Sort by start time
Order records chronologically to spot bad date parsing, missing end times, and off-by-a-year bugs.
Column visibility
Show only the fields under review, then open the detail panel for the full record including links and source.
Stats that always render
Total, passed, cancelled, tentative, and duplicate counts are shown even when the count is zero.