SNI/Hostname Database

A database with 2.7 billion SNIs/Hostnames

Every hostname comes with its IAB category, the traffic role it plays, the company that owns it and its infrastructure provider. One file to license, join to your own data and use in your products.

2.7BSNIs and hostnames
11columns per hostname
12traffic roles
Weeklyupdate files, daily on request

Hostnames are where the data is

A domain list tells you what a website is about. The hostnames under it tell you what each server does: one serves pages, another images, another sign-in, another ads. Logs, bid requests, network flows and DNS records all carry hostnames, not just domains.

This database gives each of those hostnames a category and a role, so you can analyze, enrich and filter data at the level it is actually recorded.

The columns

One row per hostname, with the root domain as a key that joins directly to our 120 million domain database.

SNI
The hostname, as it appears in traffic and logs.
Root Domain
The registered domain the hostname belongs to.
Category
The main IAB category of the service behind the hostname.
Multiple Categories
All matching IAB categories, each with a confidence score.
Method
Whether the category comes from the hostname itself or from its root domain.
Description
A short description of the service, where one is available.
Role
What the server does: website, content delivery, application interface, ads and tracking, sign-in, email and more.
Owner
The company domain the hostname belongs to.
Infra Provider
The hosting, cloud or content network, where the hostname runs on shared infrastructure.
Role Confidence
How certain the role is.
Non-production
Marks development, test and staging hostnames.

Rows from the sample

The free sample has 1,000 hostnames with all 11 columns, in the same format as the full database. A few of them:

SNI / hostnameRoot domainCategory (IAB)RoleOwnerInfrastructure provider
www.european.air.express.deexpress.deNewswebsiteexpress.de-
img.ma-shops.dema-shops.deHobbies & Interests > Antiquing and Antiquescdnma-shops.de-
api-ebanking.at.ing.being.beBusiness and Finance > Industries > Financial Industryapiing.be-
mail.maillist.xdf.cnxdf.cnEducationemailxdf.cn-
analytics.native-instruments.comnative-instruments.comTechnology & Computing > Computing > Computer Software and Applications > Digital Audioads_trackingnative-instruments.com-
contrataciones-qa-sso.telmex.comtelmex.comPersonal Finance > Home Utilities > Internet Service Providerslogintelmex.com-
ns848.hostgator.com.brhostgator.com.brTechnology & Computing > Computing > Internetnetworkhostgator.com.br-
monitor.laguida.itlaguida.itNewstelemetrylaguida.it-
dl.apteka911.uaapteka911.uaMedical Health > Pharmaceutical Drugsdownload_updateapteka911.ua-
vpn.pami.org.arpami.org.arBusiness and Finance > Industries > Healthcare Industryremote_accesspami.org.ar-
push.ariva.deariva.deBusiness and Finance > Industries > Financial Industrypush_chatariva.de-
burk-io-blog.s3.wasabisys.comwasabisys.comTechnology & Computing > Computing > Internet > Cloud Computingcloud_hostingunknownWasabi

Twelve traffic roles

The role is what turns a list of hostnames into usable data: it separates what people visit from what runs in the background.

Visited

  • website: pages people open
  • login: sign-in services
  • email: mail servers and webmail

Supporting

  • cdn: images, video, scripts
  • api: application interfaces
  • push_chat: messaging and push
  • download_update: downloads and updates

Background and infrastructure

  • ads_tracking: ads and analytics
  • telemetry: usage and crash reports
  • remote_access: VPN and remote desktop
  • network: name servers, gateways
  • cloud_hosting: shared storage and hosting

One website, many hostnames

Why a category per domain is not enough once you look at real traffic.

When someone opens a single article on a news website, their device does not talk to one server. It contacts the page server, an image server, a server for scripts and styles, a sign-in service, one or more analytics and advertising servers, and often a video player hosted somewhere else. Each of these shows up as its own hostname in network logs, DNS records and encrypted traffic. A typical news page triggers connections to dozens of hostnames, and only one of them is the page the person actually chose to open.

Here is how the hostnames of one news website look in the database. The category stays with the publisher, while the role tells you what each server is doing:

HostnameRoleWhat it means for your data
www.examplenews.comwebsiteThe page the reader opened. Count it as a visit.
img.examplenews.comcdnPictures in the article. Part of the same visit, not a second one.
api.examplenews.comapiThe app and the page loading content in the background.
login.examplenews.comloginSign-in for subscribers. A signal of a registered user, not of reading.
metrics.examplenews.comtelemetryUsage reporting. Exclude it from audience and time-spent figures.
mail.examplenews.comemailNewsletter and mail traffic, separate from browsing.

If you count every hostname as a visit, a single article read becomes six visits and the share of news in your traffic report is inflated by background calls. If you only use the root domain, you cannot tell reading from telemetry. With a role on every hostname, you keep what people chose to see and set aside what their devices did on their own.

Owner and infrastructure provider

Who is behind a hostname, even when it runs on someone else's cloud.

Many hostnames do not live under the company's own domain. A retailer's images may be served from a content delivery network, a startup's application may run on a cloud platform address, and a bank's marketing pages may sit on a hosting service. In these cases the root domain belongs to the cloud or network provider, not to the company whose service you are looking at.

The Owner column names the company domain behind the hostname, so traffic can be grouped by the business that runs the service rather than by the server address. The Infra Provider column names the hosting, cloud or content network the hostname runs on. Together they answer two different questions: which companies your users reach, and which infrastructure that traffic flows through.

The Non-production flag marks development, test and staging hostnames. Analytics teams usually remove them, because they are not used by the public. Security teams often keep them, because test systems that can be reached from the internet are worth knowing about.

Who licenses it

Teams that hold large volumes of hostnames and need to know what they are.

Telecom and internet providersCategorize subscriber traffic for usage analytics, network planning and parental control products.
Security and network vendorsShip hostname categories and roles inside firewalls, gateways and monitoring products.
Ad tech and measurementClassify the hostnames in bid requests and logs, and separate ad and tracking servers from content.
Data and analytics teamsEnrich clickstream, proxy and DNS logs with a category, a role and an owner for every hostname.
Market intelligenceMeasure which services and companies show up in traffic, grouped by owner rather than by hostname.
ResearchStudy how websites and apps are built across content networks, clouds and third-party services.

What teams do with it

Concrete uses, from network reports to security reviews.

Usage reports that match realityA telecom or internet provider measures which categories its subscribers use, counting only website, streaming and messaging roles, so background calls from phones and apps do not distort the picture.
Parental control that does not break appsA family filter blocks categories such as adult or gambling, while the content, interface and sign-in hostnames of allowed services keep working, because their roles are known.
Remote access and shadow servicesA security team lists every remote access, file sharing and cloud hosting hostname seen on the network, grouped by owner, to find tools that were never approved.
Ad and tracking calls in app trafficA measurement company separates ad and tracking hostnames from content in app traffic, and sees which companies collect data inside which apps.
Market share by companyAn analyst groups hostnames by owner to compare how much traffic goes to each streaming, retail or software company, including the servers they run on third-party clouds.
Enriching security alertsAn alert that names only a hostname gains a category, a role and an owner, so an analyst can see at once whether it points to a known service or to something unusual.

Joining it to your own data

The hostname is already in your logs. These are the fields to match.

Encrypted trafficThe server name a device sends when it opens an encrypted connection. Network probes, firewalls and gateways record it as the SNI field.
DNS logsThe name in each DNS query. Resolvers and DNS security products log it for every lookup.
Proxy and web logsThe host part of each requested address, or the Host header of the request.
Bid requests and app dataThe domain or page address of the impression, and the hostnames an app contacts.

Before matching, write the hostname in lower case and remove a trailing dot and any port number. Then match in three steps: first the exact hostname against the SNI column; if there is no exact match, the root domain against our 120 million domain database; and for anything left, Engine Offline, which is included. A join in SQL looks like this:

SELECT l.*, COALESCE(h.category, d.category) AS category, h.role, h.owner, h.infra_provider FROM traffic_log l LEFT JOIN sni_hostnames h ON h.sni = LOWER(l.hostname) LEFT JOIN domains d ON d.domain = l.root_domain;

Matching is a plain table lookup on an exact key, with no calls to us and no model in the path, so it keeps up with the largest log volumes. Teams typically enrich billions of log lines a day this way, either in batch inside the data warehouse or in a stream as the logs arrive, and keep the result as extra columns next to the original record.

For the full file, a columnar format works best. The Parquet delivery loads straight into analytics engines and data warehouses, and slices by top-level domain, country, category or role keep the size down when you only need part of the data.

Delivery, freshness and new hostnames

Files

CSV or Parquet, one row per hostname. The full 2.7 billion rows, or slices by top-level domain, country, category or role, delivered by secure download link.

Updates

Weekly update files with new and changed hostnames, daily files on request, and a full snapshot every quarter. 300,000 newly registered domains are reviewed every day.

Engine Offline, included

For hostnames that are not yet listed, the included engine gives a category from the root domain and assigns a role. It runs on your own servers and makes no calls to our systems.

Three engines for hostnames that are not yet listed

New hostnames appear every day. The engines give them an answer straight away.

The database is updated every week, but your traffic will always contain hostnames that appeared after the last update: new campaigns, new subdomains, one-off server names. The engines classify those hostnames with the same rules used to build the database, so every hostname in your data gets a category and a role. Each engine includes everything of the previous one and adds one step.

Engine OfflineIncluded with the database. A hostname that is not in the database gets the category of its root domain from the 120 million domain database and a traffic role. It runs entirely on your servers with no network access and classifies 100,000 hostnames in under a second on a 12-core server.
Engine Offline + Live Owner ChecksFor root domains that are not in the database, checks run from your own network and add a status (live, unresolvable, parked, seized) and the owner where it can be proven. Nothing is sent to us.
Engine Offline + Live Owner Checks + API ClassificationRegistered domains that are still uncategorized can be sent to our classification API, which categorizes them from their live website. Only the domain is sent, never hostnames, users or traffic data.

Questions

What is an SNI?

The Server Name Indication is the hostname a device sends when it opens an encrypted connection. It is visible to networks even when the rest of the traffic is encrypted, which is why it appears in so many logs and data feeds.

How does it relate to your domain database?

The domain database categorizes 120 million websites. The hostname database lists the hostnames under those domains and adds a role, an owner and an infrastructure provider for each. Both share the root domain as a key, so they join directly.

Can we license only part of it?

Yes. You can license slices by top-level domain, country, category or role, instead of the full 2.7 billion rows.

Can we use the data in a product we sell?

Yes. Licenses are available for internal use and for use in products and services you sell to others.

Why do some hostnames carry the category of their root domain?

Many hostnames are simply parts of the same website: the page server, the image server, the server for scripts. They belong to the same publisher, so they share its category, and their role shows what each one does. Hostnames that run a different service, such as a separate shop or community under the same domain, get their own category. The Method column shows which of the two applies to each row.

How are hostnames on shared clouds handled?

When a hostname runs on a cloud platform or content network address, the root domain belongs to the provider. The Owner column then names the company behind the service where it is known, and the Infra Provider column names the platform, so you can group traffic by company and by infrastructure separately.

Does the database contain personal data?

No. It lists server names and what those servers do. It contains no users, no IP addresses of people, no browsing histories and no traffic records.

What about random-looking or one-off hostnames?

Many services create hostnames with numbers or random strings, for example for each customer or each server. They take the category of their root domain and a role, both in the database and through Engine Offline for new ones, so they are classified even when the exact name has never been seen before.

Which format should we use for the full file?

Parquet for the full 2.7 billion rows, because it loads quickly into analytics engines and data warehouses and keeps the files small. CSV is available for every delivery, and slices by top-level domain, country, category or role are the easiest way to start.

How quickly do new hostnames appear in the data?

Weekly update files list the hostnames that were added or changed since the previous file, and daily files are available on request. Between two updates, Engine Offline classifies any new hostname as soon as it shows up in your traffic, so there is no gap while you wait for the next file.

How is it priced?

By quote. The price depends on the number of hostnames, the slices you need, the update cadence and the license type.

Get a quote for the SNI/Hostname Database

Tell us which slices you need, how often you want updates and how you plan to use the data. We reply within one business day.

Request a quoteDownload free sample