Category Data Quality Checklist: How to Evaluate a Website Categorization Dataset
Every categorization provider says its data is large, accurate and fresh. This checklist shows how to test those claims yourself, on your own domains, before you commit to a licence that will sit inside your products for years. It is written for adtech, data, security and AI teams buying category data.
Illustrative scores, not a real evaluation.
The eight dimensions of category data quality
Quality is not one number. A dataset can be accurate on popular sites and weak on the long tail, or fresh but shallow. Score each dimension separately and weight them by your use case, so one strong number cannot hide a weak one.
Coverage
How many of the domains you actually see get a category.
Accuracy
How often the category is right, measured on a blind test set.
Freshness
How quickly new and changed domains are reflected.
Multi-label depth
Whether broad sites carry several ranked labels.
Parked and dead domains
Whether empty, parked and expired sites are labeled as such.
Taxonomy fit
Whether the categories match your use and your partners.
Delivery
Formats, update method, API and on-premise options.
Licence and support
What you may do with the data, and what the provider shares about quality.
1. Coverage: test on your own domains
Headline domain counts say little about your traffic, because the domains you meet are not a random slice of the web. What matters is the match rate on the domains you will actually look up, broken down so the weak spots show.
| Breakdown | How to test | What good looks like |
|---|---|---|
| Overall match rate | Normalise a real list of your domains and count matches | High and consistent across repeated lists |
| By popularity band | Split your list into popular, mid and long-tail domains | Near-complete on popular sites, solid on the long tail |
| By country | Group by country or country-code domain | No large gaps in the markets you serve |
| By traffic share | Weight matches by how often each domain appears | Matches cover almost all of your lookups by volume |
| Newly registered | Include domains first seen in the last weeks | New domains present within a day or two |
- Use logs, bid requests or CRM exports, not a list the provider chose
- Normalise to registrable domains before matching
- Count unique domains and traffic-weighted matches separately
- Check that counts do not include large numbers of empty or dead domains
More on how coverage and update cycles interact in coverage and freshness.
2. Accuracy: build a blind test set
Accuracy claims mean nothing without knowing what was measured. The reliable route is your own labeled test set, built before you look at the provider's answers and kept for future checks after you buy.
Stratify
Pick domains across popularity bands and the categories you care about, not just famous sites.
Label blind
Have reviewers label each site from its content without seeing any provider's answer.
Double-check
Use two reviewers per domain and settle disagreements with a third. Note sites that are honestly ambiguous.
Compare
Score the provider per category: main label correct, any label correct, and clearly wrong.
How big should the test set be?
The margin of error shrinks with more labeled domains. These are approximate 95% margins when measured accuracy is around 90%.
Metrics worth tracking
- Main-label accuracy per category
- Any-label agreement for multi-label sites
- Precision of sensitive categories such as Adult or Gambling
- Share of "unknown" or empty answers
Traps to avoid
- Testing only the top 1,000 sites, which every dataset gets right
- Letting reviewers see the provider label first
- Treating close sibling categories as hard errors
- Comparing providers on different domain lists
Single domains can be checked by hand with our domain checker; for bulk work see website category lookup.
3. Freshness: new, changed and re-checked domains
The web changes every day. New domains launch, old ones expire, and sites change subject. Freshness has three parts, and each needs its own question, because a provider can be strong on one and weak on another.
New domains
How soon after registration does a domain appear? Daily additions matter for security, open exchange buying and crawl filtering.
Re-checks after launch
Many new sites show a placeholder first and real content weeks later. Ask whether new domains are checked again.
Full refresh cycle
How often are all established domains reviewed? Monthly is a sound baseline; yearly is too slow.
- Test with domains registered in the past week
- Re-test the same list a month later to see what changed
- Ask how updates are delivered: full file, daily deltas or both
- Confirm that each row or file carries a date you can store
Our daily feed is described in newly registered domains list.
4. Multi-label depth: one site, several subjects
A news portal covers sport, finance and politics. A marketplace sells everything. Single-label data forces one answer and hides the rest, which hurts both targeting and safety.
| Question | Why it matters | What to look for |
|---|---|---|
| Can a domain carry several labels? | Broad sites need it to be described honestly | Several labels on large, multi-purpose sites |
| Are labels ranked? | You need a main subject for targeting and reporting | A clear order, with the main subject first |
| Do niche sites stay focused? | Too many labels on a small site is noise | One or two labels for focused sites |
| Are there deeper tiers? | Precise targeting needs more than tier 1 | Tier 2 and 3 where the site is specific enough |
In our data, most domains carry one main web filtering label, while large platforms carry many, ranked by how much of the site they describe. IAB categories add up to four tiers of depth.
5. Parked, dead and expired domains
Empty domains inflate counts and pollute results. A good dataset names them for what they are instead of guessing a subject from a holding page, and keeps that label current as domains expire and are registered again.
Why it matters
- Ad spend on parked pages reaches no real audience
- CRM records on parked domains often belong to closed companies
- Training data from holding pages adds duplicated template text
- Lists sold "by category" can be padded with empty sites
How to test
- Include known parked and for-sale domains in your test set
- Check whether they carry a parked label or a misleading subject
- Look at a slice of the long tail by hand for empty sites
- Ask how expired and re-registered domains are handled
Our full database labels over 596,000 domains as Parked / For-Sale. A related problem, sites built only for ad revenue, is covered in made-for-advertising sites.
6 to 8. Taxonomy, delivery and licence
The last three dimensions decide whether good data is usable in your stack and under your contracts. They are easy to skip during a technical test, and expensive to discover after signing.
| Dimension | Questions to ask | Why |
|---|---|---|
| Taxonomy fit | Which IAB versions are included? Is there a security-oriented set? Are IDs stable? | Partners may read IAB v2 or v3; filters need policy categories |
| Extra fields | Are personas, country and popularity included in the same row? | Saves joining several sources with different keys |
| Formats | CSV, JSON, Parquet? Export formats for filters? | Each system loads a different format best |
| Access | File, API or both? On-premise and air-gapped allowed? | Real-time and private uses need local data |
| Updates | Full refreshes, daily deltas, delivery method? | Determines how fresh your copy stays |
| Licence scope | Internal use, embedded in a product, group-wide? | Using data outside the licence is a real risk |
| Quality reporting | Is accuracy measured and shared with licence holders? | Shows the provider tracks quality over time |
Formats are explained in domain list download formats, and the trade-offs of file vs API in URL categorization API vs database.
Category-specific checks
Some categories carry more weight than others, depending on what you build. Give these extra attention in your test set, with more labeled domains than the average category.
| Category | Who cares most | What to check |
|---|---|---|
| Adult | Adtech, filters, AI data | Very few misses on known adult sites, and few false hits on health, lingerie or art sites |
| Gambling | Regulated advertisers, filters | Separation from video gaming and fantasy sports news |
| Malware / Phishing / Suspicious | Security teams | Fast inclusion of new scam domains; check freshness more than size |
| VPN / Proxy / Anonymizers | Filters, schools, enterprises | Coverage of small proxy sites, not only the best-known services |
| News/Media vs Blogs/Personal Sites | Publishers, brand safety | Clear line between professional news and personal blogs |
| AI / LLM Tools | Security, IT governance | Speed of adding new AI products, which launch weekly |
| Business and industry tiers | Sales and data teams | Deep enough tiers to tell software firms from consultancies |
A two-week evaluation plan
Most teams can finish a solid evaluation in two weeks of part-time work. Plan it before you speak to providers, so every candidate is tested the same way.
Prepare lists
Export a real domain list from logs or CRM, plus a smaller stratified set for labeling.
Label blind
Two reviewers label the test set from site content. A third settles disagreements.
Run lookups
Match the lists against each candidate dataset and record match rate and answers.
Score and decide
Fill the scorecard, review red flags and check licence terms with your legal team.
Ask for these in writing
- Update schedule for new domains and full refreshes
- Taxonomies and versions included, with ID lists
- Licence scope: internal, embedded, group-wide
- Delivery formats and how updates arrive
- How accuracy is measured and what licence holders receive
- Whether on-premise or air-gapped use is permitted
A scorecard you can copy
Weight each dimension by your use case. The weights below are a starting point for three common buyers; adjust them before testing, not after you see the results, so the scorecard stays fair.
| Dimension | Adtech weight | Security weight | AI data weight | Pass signal |
|---|---|---|---|---|
| Coverage | 20% | 20% | 25% | High traffic-weighted match rate on your own list |
| Accuracy | 20% | 25% | 20% | Strong main-label accuracy in your key categories |
| Freshness | 10% | 25% | 10% | New domains within days, monthly full refresh |
| Multi-label | 15% | 5% | 10% | Ranked labels on broad sites |
| Parked and dead | 10% | 5% | 15% | Parked domains clearly labeled |
| Taxonomy fit | 15% | 10% | 10% | Categories map to your use without heavy work |
| Delivery and licence | 10% | 10% | 10% | Formats, access and scope match your plans |
Red flags when evaluating category data
Counts without context
Huge totals with no breakdown by popularity, country or category, and no word on empty domains.
Accuracy without a method
A single accuracy figure with no description of what was tested or how often it is measured.
One label per domain
Broad platforms squeezed into a single category, which breaks safety rules and targeting.
No update dates
Rows or files with no date, so you cannot tell how old an answer is.
Vague licence terms
Unclear rules on embedding, sharing within a group or use in your own products.
Only famous examples
Demonstrations limited to the best-known sites, where every dataset looks good.
No word on parked domains
If a provider cannot say how parked and for-sale domains are labeled, assume they are mixed into ordinary categories.
Taxonomy without IDs
Category names with no stable IDs make joins fragile and every future update a manual job.
No local option
An API-only offer rules out real-time, private and air-gapped uses, whatever the quality of the data.
How our database answers the checklist
We describe outputs and commitments, not internal methods. Domains are categorized from site content, then reviewed and validated.
Coverage
Over 120 million categorized domains, with country and popularity fields to break coverage down.
Freshness
About 300,000 newly registered domains categorized daily, re-checked after a week and a month; all others refreshed monthly.
Multi-label
Several ranked web filtering labels where a site needs them, plus up to four IAB tiers.
Parked domains
A dedicated Parked / For-Sale category, plus a Malware / Phishing / Suspicious category.
Taxonomy
IAB v2 and v3 and 59 web filtering categories in one row, with personas, country and popularity.
Delivery and quality
CSV, JSON or Parquet, a lookup API included, on-premise use possible, and validation reports for licence holders.
For the full field list, see the database of websites by category, or browse the taxonomy.
Category data quality: frequently asked questions
How do I evaluate a website categorization dataset?
Test it on your own domains: match rate for coverage, a blind labeled set for accuracy, recent domains for freshness, and a review of taxonomy, formats and licence.
How many domains do I need to test accuracy?
About 100 labeled domains per key category gives a rough read, around plus or minus six points near 90% accuracy. Use more for high-stakes categories.
What is a good match rate for categorization data?
It depends on your traffic. Compare providers on the same list, by popularity band, and weighted by how often each domain appears.
Why do parked domains matter in category data?
They inflate counts but have no content. Good data labels them clearly so you can exclude them.
Should categorization data be multi-label?
Yes, for most uses. Ranked labels let you target on the main subject and apply safety rules to any label.
How fresh should a categorization database be?
Daily additions for new domains, re-checks after launch, and a full refresh at least monthly.
Is a bigger domain count always better?
No. A count padded with empty, parked or long-dead domains looks impressive but adds little. What matters is the match rate on your own traffic and how clearly empty domains are labeled.
How should I treat disagreements between reviewers?
Record them. If two careful reviewers disagree, the site is probably ambiguous, and a provider that picks either answer should not be marked wrong. Score those domains separately from clear-cut ones.
Can I compare two providers fairly?
Yes, if you use the same domain list, the same normalisation, the same blind labels and the same scoring rules for both. Map their categories to your own labels before scoring so taxonomy differences do not count as errors.
What should I re-check after buying?
Repeat the coverage and accuracy tests every few months on fresh lists. Quality can shift as the web changes, and a regular check shows whether updates keep pace with your traffic.
Category data built to pass the checklist
Over 120 million domains with ranked categories, IAB v2 and v3, personas, country and popularity. Daily new domains, monthly refreshes and validation reports for licence holders.
See plans and pricing Explore the databaseIAB is a trademark of the Interactive Advertising Bureau. URL Categorization Database is not affiliated with, sponsored or endorsed by IAB.