Every hostname comes with its IAB category, the traffic role it plays, the company that owns it and its infrastructure provider. One file to license, join to your own data and use in your products.
A domain list tells you what a website is about. The hostnames under it tell you what each server does: one serves pages, another images, another sign-in, another ads. Logs, bid requests, network flows and DNS records all carry hostnames, not just domains.
This database gives each of those hostnames a category and a role, so you can analyze, enrich and filter data at the level it is actually recorded.
One row per hostname, with the root domain as a key that joins directly to our 120 million domain database.
The free sample has 1,000 hostnames with all 11 columns, in the same format as the full database. A few of them:
| SNI / hostname | Root domain | Category (IAB) | Role | Owner | Infrastructure provider |
|---|---|---|---|---|---|
| www.european.air.express.de | express.de | News | website | express.de | - |
| img.ma-shops.de | ma-shops.de | Hobbies & Interests > Antiquing and Antiques | cdn | ma-shops.de | - |
| api-ebanking.at.ing.be | ing.be | Business and Finance > Industries > Financial Industry | api | ing.be | - |
| mail.maillist.xdf.cn | xdf.cn | Education | xdf.cn | - | |
| analytics.native-instruments.com | native-instruments.com | Technology & Computing > Computing > Computer Software and Applications > Digital Audio | ads_tracking | native-instruments.com | - |
| contrataciones-qa-sso.telmex.com | telmex.com | Personal Finance > Home Utilities > Internet Service Providers | login | telmex.com | - |
| ns848.hostgator.com.br | hostgator.com.br | Technology & Computing > Computing > Internet | network | hostgator.com.br | - |
| monitor.laguida.it | laguida.it | News | telemetry | laguida.it | - |
| dl.apteka911.ua | apteka911.ua | Medical Health > Pharmaceutical Drugs | download_update | apteka911.ua | - |
| vpn.pami.org.ar | pami.org.ar | Business and Finance > Industries > Healthcare Industry | remote_access | pami.org.ar | - |
| push.ariva.de | ariva.de | Business and Finance > Industries > Financial Industry | push_chat | ariva.de | - |
| burk-io-blog.s3.wasabisys.com | wasabisys.com | Technology & Computing > Computing > Internet > Cloud Computing | cloud_hosting | unknown | Wasabi |
The role is what turns a list of hostnames into usable data: it separates what people visit from what runs in the background.
Why a category per domain is not enough once you look at real traffic.
When someone opens a single article on a news website, their device does not talk to one server. It contacts the page server, an image server, a server for scripts and styles, a sign-in service, one or more analytics and advertising servers, and often a video player hosted somewhere else. Each of these shows up as its own hostname in network logs, DNS records and encrypted traffic. A typical news page triggers connections to dozens of hostnames, and only one of them is the page the person actually chose to open.
Here is how the hostnames of one news website look in the database. The category stays with the publisher, while the role tells you what each server is doing:
| Hostname | Role | What it means for your data |
|---|---|---|
| www.examplenews.com | website | The page the reader opened. Count it as a visit. |
| img.examplenews.com | cdn | Pictures in the article. Part of the same visit, not a second one. |
| api.examplenews.com | api | The app and the page loading content in the background. |
| login.examplenews.com | login | Sign-in for subscribers. A signal of a registered user, not of reading. |
| metrics.examplenews.com | telemetry | Usage reporting. Exclude it from audience and time-spent figures. |
| mail.examplenews.com | Newsletter and mail traffic, separate from browsing. |
If you count every hostname as a visit, a single article read becomes six visits and the share of news in your traffic report is inflated by background calls. If you only use the root domain, you cannot tell reading from telemetry. With a role on every hostname, you keep what people chose to see and set aside what their devices did on their own.
Who is behind a hostname, even when it runs on someone else's cloud.
Many hostnames do not live under the company's own domain. A retailer's images may be served from a content delivery network, a startup's application may run on a cloud platform address, and a bank's marketing pages may sit on a hosting service. In these cases the root domain belongs to the cloud or network provider, not to the company whose service you are looking at.
The Owner column names the company domain behind the hostname, so traffic can be grouped by the business that runs the service rather than by the server address. The Infra Provider column names the hosting, cloud or content network the hostname runs on. Together they answer two different questions: which companies your users reach, and which infrastructure that traffic flows through.
The Non-production flag marks development, test and staging hostnames. Analytics teams usually remove them, because they are not used by the public. Security teams often keep them, because test systems that can be reached from the internet are worth knowing about.
Teams that hold large volumes of hostnames and need to know what they are.
Concrete uses, from network reports to security reviews.
The hostname is already in your logs. These are the fields to match.
Before matching, write the hostname in lower case and remove a trailing dot and any port number. Then match in three steps: first the exact hostname against the SNI column; if there is no exact match, the root domain against our 120 million domain database; and for anything left, Engine Offline, which is included. A join in SQL looks like this:
Matching is a plain table lookup on an exact key, with no calls to us and no model in the path, so it keeps up with the largest log volumes. Teams typically enrich billions of log lines a day this way, either in batch inside the data warehouse or in a stream as the logs arrive, and keep the result as extra columns next to the original record.
For the full file, a columnar format works best. The Parquet delivery loads straight into analytics engines and data warehouses, and slices by top-level domain, country, category or role keep the size down when you only need part of the data.
CSV or Parquet, one row per hostname. The full 2.7 billion rows, or slices by top-level domain, country, category or role, delivered by secure download link.
Weekly update files with new and changed hostnames, daily files on request, and a full snapshot every quarter. 300,000 newly registered domains are reviewed every day.
For hostnames that are not yet listed, the included engine gives a category from the root domain and assigns a role. It runs on your own servers and makes no calls to our systems.
New hostnames appear every day. The engines give them an answer straight away.
The database is updated every week, but your traffic will always contain hostnames that appeared after the last update: new campaigns, new subdomains, one-off server names. The engines classify those hostnames with the same rules used to build the database, so every hostname in your data gets a category and a role. Each engine includes everything of the previous one and adds one step.
The Server Name Indication is the hostname a device sends when it opens an encrypted connection. It is visible to networks even when the rest of the traffic is encrypted, which is why it appears in so many logs and data feeds.
The domain database categorizes 120 million websites. The hostname database lists the hostnames under those domains and adds a role, an owner and an infrastructure provider for each. Both share the root domain as a key, so they join directly.
Yes. You can license slices by top-level domain, country, category or role, instead of the full 2.7 billion rows.
Yes. Licenses are available for internal use and for use in products and services you sell to others.
Many hostnames are simply parts of the same website: the page server, the image server, the server for scripts. They belong to the same publisher, so they share its category, and their role shows what each one does. Hostnames that run a different service, such as a separate shop or community under the same domain, get their own category. The Method column shows which of the two applies to each row.
When a hostname runs on a cloud platform or content network address, the root domain belongs to the provider. The Owner column then names the company behind the service where it is known, and the Infra Provider column names the platform, so you can group traffic by company and by infrastructure separately.
No. It lists server names and what those servers do. It contains no users, no IP addresses of people, no browsing histories and no traffic records.
Many services create hostnames with numbers or random strings, for example for each customer or each server. They take the category of their root domain and a role, both in the database and through Engine Offline for new ones, so they are classified even when the exact name has never been seen before.
Parquet for the full 2.7 billion rows, because it loads quickly into analytics engines and data warehouses and keeps the files small. CSV is available for every delivery, and slices by top-level domain, country, category or role are the easiest way to start.
Weekly update files list the hostnames that were added or changed since the previous file, and daily files are available on request. Between two updates, Engine Offline classifies any new hostname as soon as it shows up in your traffic, so there is no gap while you wait for the next file.
By quote. The price depends on the number of hostnames, the slices you need, the update cadence and the license type.
Tell us which slices you need, how often you want updates and how you plan to use the data. We reply within one business day.