Content Quality Scorer (Tool)
Production lead-magnet web tool scoring any article URL across 14 content quality dimensions through a four-stage async pipeline. Multi-method evaluator system (LLM-as-judge, regex / structural parsing, embedding-based scoring) routed across multiple inference providers.
The Content Quality Scorer is a production lead-magnet web tool that scores any article URL across 14 content quality dimensions and returns a per-dimension report with highlights and actionable recommendations. Four-stage async pipeline behind a job queue.
The gap it closes
Off-the-shelf SEO tools score keyword density and meta tags but miss readability, hierarchy, schema, citation evidence, and CTA quality. A single LLM judge wrapped in a form drifts on mechanical questions (schema presence, broken links) and hallucinates on interpretation-heavy ones (CTA quality, list parallelism). A scoring tool that ships as a public-facing product needs to be fast enough for an end-user wait, defensible enough that the per-dimension scores reproduce, and structured enough that the report converts a curious visitor into a lead.
Pipeline architecture
Four-stage async pipeline
User submits { url, keyword }. The backend returns a job id immediately. The frontend polls the job status until it is done. Inside the job, the pipeline runs four stages.
- Capture. Page scraping with fallback, SERP fetching for the keyword, HTML parsing. A custom parser handles heading extraction, list parsing, image alt detection, anchor text patterns, and structured-data detection.
- Extract. LLM-based extraction of headings, lists, links, images, and schema markup into structured data.
- Evaluate. 14 dimension evaluators, each implementing its own scoring method.
- Score. Per-dimension aggregation, weighted final result, per-dimension highlights and recommendations for the report UI.
14 dimensions across three categories
Structure and Readability (9): content readability, CTA quality, hierarchy consistency, paragraph length, sentence length, descriptive headings, list type, list parallelism, bullet brevity. SEO and Discoverability (3): schema markup, image alt text, anchor text. Accuracy and Trust (2): sources cited with evidence, broken links.
Three evaluator strategies
Different dimensions need different evaluation methods.
- LLM-as-judge for interpretation-heavy dimensions (readability, CTA quality, descriptive headings, list parallelism).
- Regex and structural parsing for mechanical dimensions (schema markup presence, image alt text presence, broken links, anchor text patterns).
- Embedding-based scoring for dimensions that compare against SERP context.
Mixing methods avoids the failure mode where a single LLM judge rates everything and drifts on the dimensions it is not suited for.
Multi-provider inference layer
A shared client abstracts multiple inference providers behind one interface. Fast-inference endpoints handle dimensions where speed matters. Reasoning-quality endpoints handle dimensions where interpretation matters. Provider selection per evaluator is configured, not hardcoded into evaluator logic.
What shipped
Production lead-magnet web tool. Per-dimension scoring delivered across 14 dimensions with highlights and recommendations per report. Pipeline iterated to a stable production path that handles the end-to-end capture, extraction, evaluation, and score-generation flow.