How we measure LLM brand visibility

We curate a representative set of natural prompts that reflect how potential customers, journalists, analysts, and enthusiasts actually ask about a brand. Each query carries:

• Category: e.g. direct brand, performance, style, comparisons, skepticism.
• Priority weight: 1–5, reflecting business impact. Weight-5 queries count five times more in aggregate metrics.
• Query style: structured (research-style phrasing) or consumer_voice (natural language). LLMs often behave differently on the two, so we track both.

For Nike, we ship 10 categories spanning direct-brand awareness through comparison, innovation, sustainability, and skepticism-bait queries.

1. Each query is sent to Claude (or replayed from fixture in this demo).
2. URLs are extracted from the response and normalised to domains.
3. Domains are classified by rule into source_type (owned_brand, editorial, retailer, forum, wikipedia, review, social, government) and quality_tier.
4. A deterministic analysis engine runs regex-based checks for brand presence, mention rank, directness, competitor mentions, framing tags, and owned-source support.
5. A composite visibility score (0–100) is computed from the analysis output.
6. Typed recommendations are generated from the analysis using readable rules.

No LLM-as-judge. The only LLM call is the one that produced the response under analysis. Everything after that is readable TypeScript you can step through.

The score is the sum of five transparent components. Weights are defined in src/lib/config/index.ts and can be tuned per engagement.

WVI is the priority-weighted average of the latest Query Visibility Score across all active queries:

WVI = Σ(score × priorityWeight) / Σ(priorityWeight)

A weight-5 query counts five times more than a weight-1 query. This ensures executive metrics reflect business priority, not query count.

Framing tags capture the sentiment and narrative lens around how the brand is discussed. They are the most nuanced signal: positive framings reward the score, mixed framings are neutral or slightly negative, and negative framings reduce it.

See the full dictionary with score impacts in the Settings page.

Three scopes of recommendation are generated:

• Query-level, generated immediately after each run from deterministic rules.
• Cluster: pattern detection across the latest run per query (e.g. repeated absence, repeated negative framing in a category).
• Global: threshold-triggered across the whole query universe (e.g. overall absence rate > 40%).

Each recommendation carries an evidence payload linking back to the specific runs or queries that triggered it, never a black box.