How we detect invalid traffic

This article explains the basics of invalid traffic and how we detect it.

Ronald Ng avatar
Written by Ronald Ng
Updated over a week ago

Types of invalid traffic

Spider AF detects two types of invalid traffic: general and sophisticated.

General invalid traffic (GIVT):

  • Crawlers

  • Data centers

  • Invalid user agents

Sophisticated invalid traffic (SIVT):

  • Bots

  • Fake user agents

  • Repeated clicks from the same user

  • Invalid IP addresses

  • Domain spoofing

Categories of general invalid traffic

Crawlers

What is it?

Crawlers are programs that automatically browse the internet. Some crawlers collect data to sell, and others may collect info about rivals' campaigns or pricing.

How do we detect this?

We detect this based on the IP, internet service provider, or user agent details.
Repeated traffic from the same IP address is common, so we can prevent this by blocking the address.

Data centers

What is it?

Data centers are facilities with racks of computers for large systems such as cloud services. These data centers may scan public websites to collect information. Data centers do not use the same IP addresses as normal users.

How do we detect this?

We can detect this using our own list of IP addresses. Repeated traffic from the same IP address is common, so we can prevent this by blocking the address.

Invalid user agents

What is it?

Apps that use the internet show user agent data to identify themselves. This data can show when a request for a page does not come from a person using a browser.

How do we detect this?

We can detect this based on the user agent data.

Categories of sophisticated invalid traffic

Bots

What is it?

Bots are remotely controlled computer programs that are often used for hacking and other malicious attacks.

How do we detect this?

We identify bots based on clues such as browser automation tools. Repeated traffic from the same IP address is common, so we can prevent this by blocking the address.

Fake user agents

What is it?

Bad actors may fake the user agent to make fraudulent traffic look like normal activity.

How do we detect this?

We can detect this by looking at details such as whether the user agent and platform data match.

Repeated clicks from the same user

What is it?

This refers to repeated access from the same IP address or browser. This can include competitors checking the content of your ads.

When the bidding price for ads is high, there is a high chance of fraudulent users clicking to deplete your budget and hide your ads.

How do we detect this?

We detect this by tracking repeated traffic from the same IP or browser.

Invalid IP addresses

What is it?

A user can hide their identity with a virtual private network (VPN) or proxy server. They may use these for fraudulent activity. Most users do not use these tools, so we consider this to be invalid traffic.

How do we detect this?

We detect this based on the IP address.

Domain spoofing

What is it?

Domain spoofing shows a fake source for incoming traffic. This fraudulent traffic often aims to deplete advertising spending.

How do we detect this?

We can detect this when the referrer data and page location do not match.

Did this answer your question?