MarketMuse

Build a closest honest personal substitute for MarketMuse in an empty repository. Use Python 3.12, FastAPI, SQLite, Playwright, and an HTMX interface; do not offer alternative stacks. The core loop is: crawl a user-owned site and user files into a content inventory, calculate transparent internal topic coverage, explain prioritized issues, and export a reproducible audit. Make the first run work locally with one documented command. Store all user data locally by default and make export straightforward. Put secrets in .env, ship .env.example, and never commit credentials. Require an explicit ownership or permission acknowledgement before a crawl starts. Respect robots.txt, rate limits, canonical URLs, nofollow, redirects, and a configurable URL cap. Collect status, title, description, headings, canonical, robots, links, images, structured data, and rendered text. Detect duplicates, orphan candidates, broken links, redirect chains, missing metadata, and indexability conflicts. Show every issue with affected URLs, evidence, severity, and a concrete remediation note. Export crawl data and issues to CSV plus a self-contained HTML report. Include clear empty, loading, success, and recoverable error states. Add input validation, safe filenames, and graceful handling of unavailable APIs. Write focused tests for the core transformation and one end-to-end happy path. Create a README with setup, architecture, permissions, data location, and backup steps. Do not add accounts, billing, telemetry, analytics, or a hosted control plane. Do not claim to reproduce proprietary data, network liquidity, regulated access, or frontier infrastructure. Deliberately leave out crawling sites without permission. Deliberately leave out web-scale backlink or keyword datasets. Deliberately leave out automated changes to production websites. Finish by running the tests and listing the exact commands used.