Database of Websites by Category
A categorized website database tells you what each domain on the web is about. This page explains what is inside ours: the fields, the three taxonomies, the file formats, how it stays current and how licences work.
It covers over 120 million domains and is used by adtech, marketing, sales intelligence, research and security teams. To see it in action, explore the database.
What a categorized website database is
At its simplest it is a very long table. Each row is a domain, and each row tells you the subject of the site plus facts that help you act on it.
One row per domain
Registrable domains such as a shop, a newspaper or a city council site. Subdomains of large platforms appear where they act as separate sites.
Topic labels
Each domain gets labels from standard taxonomies, so the same site can be read by an ad platform, a filter and an analyst without translation.
Context fields
Country, audience personas and popularity group tell you where a site matters, who reads it and how big it is.
Built for filtering
The point is to select. You ask for "health sites in Canada among the top million" and get a clean list in seconds.
Think of it as a map of the web by subject. Where a search engine answers "which pages match these words", a category database answers "which sites are about this topic, where, and how big are they".
The fields in each row
Every record has the same columns. The fill rate shows how many domains have a value for the field, across the whole database.
| Field | What it holds | Filled for |
|---|---|---|
Domain | The website, without protocol or path | All domains |
IAB v2 Tier 1 | Broad IAB subject, for example Travel or Automotive | Nearly all |
IAB v2 Tier 2 | Narrower subject inside Tier 1 | About 90% |
IAB v2 Tier 3 | Specific topic | About 51% |
IAB v2 Tier 4 | Most detailed topic, where the tree goes that deep | About 14% |
IAB v3 Tier 1 to 4 | The same four levels in version 3 of the taxonomy | Tier 2 about 96% |
Web filtering category | One or more of 59 categories built for allow and block decisions | Nearly all |
Personas | Audience types the site attracts, such as "Frequent Flyer" | About 97% |
Country | The country the site mainly serves, where it can be told | About 40% |
Global popularity group | Band such as top 1,000 or top 1 million worldwide | Most visited 11 million |
Country popularity group | Band within the site's own country | Most visited 4.6 million |
Three taxonomies in one file
Different teams speak different category languages. Instead of making you map between them, each domain carries all three. Our dual taxonomy categorization guide explains the idea in depth.
IAB Content Taxonomy v2
The version many ad servers, demand-side platforms and contextual tools still use. Four tiers, from broad subjects to specific topics.
- Widely supported in existing ad stacks
- Has a combined "News and Politics" branch
- Good for older integrations
IAB Content Taxonomy v3
The newer version with 703 categories. It reorganizes several branches and adds topics that the older version handled poorly.
- Separates News from Politics
- Clearer structure for attractions and events
- See IAB Content Taxonomy 3.1
59 web filtering categories
A flat set designed for allow and block decisions, from Shopping and News to Gambling, VPN and Parked domains.
- Simple to read for policy owners
- Includes risk categories IAB lacks
- Several categories per site where needed
| Question | IAB v2 | IAB v3 | Web filtering |
|---|---|---|---|
| Structure | Four tiers | Four tiers | Flat list |
| Best for | Existing ad stacks | New contextual projects | Networks, brand safety, policies |
| Risk topics | Partly, under sensitive branches | Partly, under Sensitive Topics | Yes, as their own categories |
| Infrastructure topics | Mixed into technology | Mixed into technology | CDN, hosting, URL shorteners, parked |
| Reference | IAB 3.0 vs 3.1 and the IAB categories list | Taxonomy page | |
Several categories per site
Real websites do not fit one box. A newspaper also runs travel and recipe pages, and a large platform touches dozens of subjects. The database records that.
Strict selection
Select sites where a category is the main subject when you need a pure list for outreach or research.
Broad selection
Select every site where the category applies when you need reach, or when you build exclusion lists.
Ranked labels
Categories are ranked by how much of the site they describe, so you can keep the top one or all of them.
Coverage by domain ending and country
The database is global. The largest share is .com, followed by country endings from Europe and the Americas, and newer endings such as .shop and .online.
| Domain ending | Domains (approx.) | Note |
|---|---|---|
| .com | 48.7 million | Almost half of all records |
| .de | 9.3 million | Largest country ending |
| .org | 3.7 million | Non-profits, projects, communities |
| .net | 3.1 million | Mixed use |
| .uk | 3.1 million | Including .co.uk and .org.uk |
| .fr, .ru, .nl, .br | 1.5 to 2.2 million each | Strong national web markets |
Ready views exist for markets such as United Kingdom domains and Brazil domains. The top-level domains list has counts for every ending.
Popularity groups
Most of the web gets very few visitors. Popularity groups, based on popularity among real users, let you keep the sites that matter for your use case.
| Global group | Domains in group | Typical use |
|---|---|---|
| Top 1,000 | 646 | Headline platforms, benchmarks |
| 1,001 to 10,000 | about 5,800 | Premium publisher lists |
| 10,001 to 100,000 | about 51,000 | Media planning, key accounts |
| 100,001 to 1 million | about 498,000 | Contextual targeting, filtering priorities |
| 1 million to 10 million | about 5.6 million | Long tail reach, lead generation |
| 10 million to 50 million | about 5.0 million | Coverage and completeness |
File formats and delivery
The database is delivered as files you keep and run inside your own systems. Nothing has to call out to us at query time.
CSV
Opens anywhere, loads into every database. Best for spreadsheets, ad platforms and quick checks of a subset.
JSON
Categories and personas as real arrays. Best for applications, APIs and document stores.
Parquet
Compressed and columnar. Best for data warehouses and lakes, where you read only the columns you need.
Subsets
Pick categories, countries, popularity groups and columns, and receive only that slice.
The full file is large, in the tens of gigabytes as uncompressed CSV, and much smaller as Parquet. Our guide to domain list download formats covers loading and sizes in detail.
Update cadence
A category database ages quickly if nobody maintains it. Sites change subject, domains expire and new ones launch every day.
Regular refreshes
Existing domains are categorized again in regular refreshes, so labels follow what the site is today.
New domains
Newly registered domains are categorized as they appear and checked again as their content grows. See the newly registered domains list.
Validation
Categories are assigned from site content, reviewed and validated in regular manual checks.
Popularity
Popularity groups are refreshed too, so rising sites move up and fading ones move down.
More detail on scope and timing is on our coverage and freshness page.
One record, read three ways
To make the fields concrete, here is an illustrative record for a fictional recipe site in Germany, and what three different teams take from it.
| Field | Illustrative value |
|---|---|
| Domain | example-kochen.de (fictional) |
| IAB v3 tiers | Food & Drink > Cooking > (no Tier 3) |
| IAB v2 tiers | Food & Drink > Cooking |
| Web filtering categories | Food & Drink, Blogs/Personal Sites |
| Personas | Food Blogger; Baking Enthusiast; Nutrition Enthusiast |
| Country | Germany |
| Popularity groups | Global: 1 million to 5 million. Germany: top 100,000 |
The media planner
Sees a German cooking site with a baking audience in the national top 100,000: a good fit for a kitchen appliance campaign in Germany.
The network admin
Sees Food & Drink and a personal blog, nothing risky: allowed on staff and guest networks without a second look.
The analyst
Counts it as one of many German food blogs when sizing the creator market for recipes and baking.
Joining the database to your own data
The value appears when the categories meet your records: bid logs, CRM accounts, DNS logs or survey panels. The join itself is simple if you prepare the keys.
1. Normalise hosts
Lowercase every host, drop "http://", paths and ports, and strip a leading "www." before matching.
2. Match the domain
Try the full host first. If there is no match, fall back to the registrable domain, so shop.brand.com finds brand.com.
3. Pick the columns
Bring in only what the job needs. Ad teams often want IAB tiers, security teams the web filtering field.
4. Measure the match rate
Track the share of your hosts and traffic that found a category. Weighted by traffic it is usually far higher than by unique hosts.
Licensing
Licences are about how the data is used, not only how much of it you take. These are the main options.
Internal use
Use the data inside your own company: analytics, targeting, security, research and sales work.
OEM or embedded
Use the data inside a product you sell, such as a contextual ad tool or a filtering service, without exposing the raw file.
Group-wide
One licence for a company group with several products or brands, so each team does not buy separately.
Not allowed
Resale or publication of the raw data, in whole or in large parts, is not part of any licence.
Volume tiers and terms are on the pricing page.
How to judge any categorized website database
Whether you buy from us or not, these are the questions that separate a useful database from a long list of labels.
- Coverage of what you see. Match a sample of your own traffic or prospect list against it. The share of matches matters more than the headline total.
- Depth where you need it. Check how often Tier 2 and Tier 3 are filled for your key subjects, not only for the web as a whole.
- Multi-category support. Ask whether sites can carry several categories and whether those are ranked.
- Risk categories. For brand safety or filtering, make sure gambling, adult, weapons and similar topics are explicit categories.
- Freshness. Ask how new domains are handled and how often existing ones are checked again.
- Context fields. Country and popularity often decide whether a list is usable. Check they are present and well filled.
- Formats and licence. Make sure the format fits your stack and the licence fits your use, especially for embedded products.
You can test single sites in the domain checker and read more about the wider field in our domain intelligence guide.
Who uses a database of websites by category
Adtech platforms
Contextual targeting and brand safety at bid time. See programmatic advertising.
Sales intelligence
Turning categories into account lists and enriching CRM records with industry labels.
Security teams
Category-based allow and block rules on networks, plus fast triage of unknown domains.
Researchers
Measuring the size and mix of the web by country, topic and popularity.
Data enrichment firms
Adding web category fields to company and audience data. See data enrichment services.
Machine learning teams
Labelled domains as training and evaluation data for their own models.
Typical subsets buyers start with
Few buyers need every row on day one. These starting points are common, and each can grow into a larger licence later.
| Starting subset | Approximate size | Why it works |
|---|---|---|
| Most visited domains worldwide, all categories | Top 1 to 20 million | The most visited sites carry almost all real traffic, so match rates are high from the start |
| One country, all categories | From under 100,000 to over 9 million | Fits local ad networks, national ISPs and country sales teams |
| A few categories, worldwide | From about 100,000 to several million | Fits single-topic campaigns and vertical lead generation |
| Risk categories only | A few million | Fits brand safety and exclusion lists |
| New domains as they appear | Daily additions | Fits security teams that care most about the newest sites |
Category-by-category counts are on websites by category, and the technical details of loading large files are in the formats guide linked above.
Questions about the database
What is a database of websites by category?
It is a file with one row per domain and one or more topic labels for each row, plus context fields such as country, audience personas and popularity. It lets you filter the web by subject without visiting sites one by one.
How many websites does it contain?
Over 120 million categorized domains, ranked by popularity among real users.
Which taxonomies are included?
Every domain carries IAB Content Taxonomy version 2 and version 3 categories in up to four tiers, and one or more of 59 web filtering categories.
How often is the database updated?
The database gets regular refreshes. Existing domains are categorized again and newly registered domains are added as they appear.
Which file formats are available?
CSV, JSON and Parquet. You can take the full file or a subset of columns, categories, countries and popularity groups.
Can I resell the data?
No. Licences cover internal use, use embedded inside your own product, or group-wide use across several products. Resale or publication of the raw data is not allowed.
Does the database include subdomains?
It is built around registrable domains. Subdomains are included where they work as separate sites with their own content, such as large blogging or hosting platforms.
Can the data run in a closed network?
Yes. The files are delivered to you and run on your own servers, so the data also works in networks without outside access.
Why is the country field empty for many domains?
Country is only set where a site clearly serves one country. Global sites and sites with no clear market stay empty rather than getting a guess.
Can I buy just one category?
Yes. Category lists are sold on their own. Browse them on websites by category.
License the category database
Tell us which taxonomies, countries and volumes you need, and how you plan to use the data. We will come back with the right licence.