Paying to Win

.title[
# Paying to Win
]
.subtitle[
## Exploring the Influence of the Meta Ad Delivery Algorithm
]
.author[
### Fabio Votta, Tom Dobber, Claes de Vreese, Natali Helberger
]
.date[
### 2023-04-27
]

---

---

### Political Microtargeting

> Targeting and tailoring of political campaign messages to specific groups based on (large-scale) data collection

+ Discourse on (online) political microtargeting is often focused on the "bad actor" story

+ Russian election interference campaign (IRA)
  
  + Cambridge Analytica
  
  + Social media company allowing all this to happen

+ The explicit assumption here is that the advertisers have strong control over who sees which ad

There is a distinct lack of research on the *automated* aspect of political microtargeting:

the **ad delivery algorithm**

---

# Ad Objectives, Optimizations, Auctions & Relevance

---

### Ad Objectives, Optimizations, Auctions & Relevance

There is more than just targeting criteria that decides who sees political ads:

+ **Ad objective** = what is the goal of the ad?

for example: generate web traffic, like Facebook page, buy products
    
--

+ **Optimization** = how should the goal be achieved?

key metric to be optimized on: number of exposed users, clicks, conversions
    
--

+ **Ad auctions** = an auction takes place that determines which ad by whom is shown: based on budget and *relevance*

+ **Relevance** = how relevant is the ad to the user

→ ***Implies big role of the ad delivery algorithm***

---

![](img/plantuml.png)

---

## Prior investigations into the role of the ad delivery algorithm

#### Algorithm Audit studies

---

### Pricing differences in the US 2020 election

+ Biden campaign paid more than **6x more** 
 + compared to Trump campaign when targeting older voters. [(The Markup 2020)](https://themarkup.org/election-2020/2020/10/29/facebook-political-ad-targeting-algorithm-prices-trump-biden)
 
 
<center>
<img src="img/older.png" width="60%" />
</center>
 

 
*However, this compares different advertisements*

---

### Prior Research (Ali et al., 2020,2021)

When targeting the same audience, at the same time, with the same budget:

+ Ad delivery is heavily skewed along gendered and racial stereotypes
  + even without the intent of the advertiser [(Ali et al., 2020)](https://dl.acm.org/doi/10.1145/3359301)

Regarding political ads [(Ali et al., 2021)](https://dl.acm.org/doi/pdf/10.1145/3437963.3441801):

+ **Skewed delivery**

+ Political ads more often delivered to ideologically congruent audience 
      + Bernie ads → higher % D; 
      + Trump ads → higher % R

+ **Increased cost**

+ Liberal ad to a liberal audience: *21 Dollar per 1000 users*; 
  + Conservative ad delivered to liberal audience: *40 Dollar per 1000 users*.

*However, this research uses researcher accounts*

---

## Research Question

### How does the Meta ad delivery algorithm influence the pricing & distribution of political ads in the Netherlands?

---

# Research Design

---

### Research Design

+ Algorithm audit study

+ Place the same ads targeting the same audiences (9 different ones)

+ Collaborate with Dutch parties to place political ads

+ Final collaboration with 3:

1. GroenLinks (Green party)
  2. VVD (centre-right party of PM Rutte)
  3. PvdA (social democrats)

+ Place ads before nationwide local elections on March 16th 2022
  + 1st to 7th February 2022

+ Spend 2 Euros a day on 45 ad copies

+ in total: 630 Euro per party
  
  
--
  
+ Pre-registered research design and hypotheses [(osf.io/xmc7g)](osf.io/xmc7g)

---

### Dependent Variables

+ Price per 1k users reached

+ this measure is an industry standard

+ Ad delivery share

+ Audience shares measured by % of ads delivered to specific group
  
  + "How many people were reached in Audience A vs. Audience B"
  
  + Pairwise setup which allows us to observe presence and absence of target audience
  
     + e.g. ~60% of audience was interested in politics vs. 40% was not

---

### Research Design

In order to test the algorithm we need to:

+ Place ads with ad goal: *Reach*

+ Ad optimization: *Unique users reached*

Optimizing on Reach (rather than Traffic): 
   
+ the algorithm has to work more on its internal assumptions

+ doesn't use user cues (i.e. clicks) as much to decide who is going to see the ad

---

### Within- and between-party comparisons

The study design incorporates two *levels of comparisons*

Observing differences of **pricing and delivery**:

1. *between parties*
  
    + i.e. we expect pricing and delivery to differ by party

1. *within a party*
  
    + i.e. we expect pricing and delivery to differ by audience

---

### Ad Relevance

We theorize two different levels of (predicted) relevance:

1. Relevant audience for party (i.e. source of ad)

+ Ads from an environmentalist party more likely to be relevant for audience interested in environmentalism.

<ol start="2">
 <li>Relevant audience for ad content (i.e. political message)</li>
</ol>

+ Political message likely to be relevant for people interested in politics

---

### Hypotheses

![](img/relevant_quote.png)

[(Meta Business Help Center, 2022)](https://www.facebook.com/business/help/430291176997542)

> **H1:** **The more relevant** an audience is for an ad, **the cheaper is the cost** for reaching 1000 users in that audience.

> **H2:** **The more relevant** an audience is for an ad, **the more are ads delivered** to that audience.

We expect that ads by party with a greater share of supporters are less expensive (H3a) and reach more people (H3b)

> **H3a:** Parties with a greater share of supporters pay less for reaching 1000 users.

> **H3b:** Parties with a greater share of supporters reach more people than smaller parties.

---

### Targeting criteria (Sub-hypotheses for H1 & H2)

We used 9 different (paired) targeting criteria for our advertisements

1. Political interests
2. Excluding political interest

]

]

---

### Targeting criteria (Sub-hypotheses for H1 & H2)

We used 9 different (paired) targeting criteria for our advertisements

1. Political interests
2. Excluding political interest

]

**Relevant audience for ad content**

> Targeting political ads to audiences who are **interested in politics**

> + H1a) *is less expensive*

> + H2a) *deliver more*

> than targeting audiences who are not interested in politics.

]

---

### Targeting criteria (Sub-hypotheses for H1 & H2)

We used 9 different (paired) targeting criteria for our advertisements

1. Political interests
2. Excluding political interest

<ol start="3">
 <li>Higher educated audience</li>
 <li>Lower educated audience</li>
</ol>

]

**Relevant audience for ad content**

> Targeting political ads to **higher-educated audiences**

> + H1b) *is less expensive*

> + H2b) *deliver more*

> than targeting lower-educated audiences.

]

---

### Targeting criteria (Sub-hypotheses for H1 & H2)

We used 9 different (paired) targeting criteria for our advertisements

1. Political interests
2. Excluding political interest

<ol start="3">
 <li>Higher educated audience</li>
 <li>Lower educated audience</li>
</ol>

]

**Relevant audience for party**

> Targeting political ads to **higher-educated audiences**

> + H1c) *is less expensive*

> + H2c) *deliver more*

for GroenLinks compared to the PvdA and VVD.

]

---

### Targeting criteria (Sub-hypotheses for H1 & H2)

We used 9 different (paired) targeting criteria for our advertisements

1. Political interests
2. Excluding political interest
 
 
<ol start="3">
 <li>Higher educated audience</li>
 <li>Lower educated audience</li>
</ol>

<ol start="5">
 <li>Environmental interests</li>
 <li>Excluding environmental interests</li>
</ol>

]

**Relevant audience for party**

> Targeting political ads to audiences **interested in environenmentalism**

> + H1d) *is less expensive*

> + H2d) *deliver more*

for GroenLinks compared to the PvdA and VVD.

]

---

### Targeting criteria (Sub-hypotheses for H1 & H2)

We used 9 different (paired) targeting criteria for our advertisements

1. Political interests
2. Excluding political interest
 
<ol start="3">
 <li>Higher educated audience</li>
 <li>Lower educated audience</li>
</ol>

<ol start="5">
 <li>Environmental interests</li>
 <li>Excluding environmental interests</li>
</ol>

<ol start="7">
 <li>Economic interests</li>
 <li>Excluding Economic interests</li>
</ol>

]

**Relevant audience for party**

> Targeting political ads to audiences **interested in the economy**

> + H1e) *is less expensive*

> + H2e) *deliver more*

for VVD compared to the PvdA and GroenLinks.

]

---

### Targeting criteria (Sub-hypotheses for H1 & H2)

We used 9 different (paired) targeting criteria for our advertisements

1. Political interests
2. Excluding political interest

<ol start="3">
 <li>Higher educated audience</li>
 <li>Lower educated audience</li>
</ol>

<ol start="5">
 <li>Environmental interests</li>
 <li>Excluding environmental interests</li>
</ol>

<ol start="7">
 <li>Economic interests</li>
 <li>Excluding Economic interests</li>
</ol>
 
<ol start="9">
 <li>No Targeting</li>
</ol>

]

---

### Relevance Coding

![](img/relevance_coding.png)

---

### Design Overview

+ Pre-test determined minimum number of ads per audience: *5*

+ 9 audiences * 5 ads = 45 ads per party (135 in total)

---

# Ad Creative and Setup

---

## Ad Creative and Setup

+ Infos about the election and how to vote from the *Rijksoverheid*

+ Official Get-Out-To-Vote campaign "Elke Stem Telt" for local elections

+ Selected *Five* Images from campaign that we would like to run

+ https://www.kiesraad.nl/verkiezingen/gemeenteraden

---

## How the ad looked like on Desktop

---

## How the ad looked like on Desktop

---

## Results

---

### Relevance X Price (A) / Ad delivery shares (B)

![](img/scatters.png)

---

### Regression Results (H1 & H2)

![](img/reg1.png)

---

### Regression Results (H1 & H2)

![](img/reg2.png)

---

### Between-party differences

`\(\rightarrow\)` we consistently find one party that pays less and reaches more people

---

#### Between-party differences (for entire ad campaign)

.font80[PvdA pays the least (**32-42 cents less** or: 9-11%) & reaches most people (**14-17k more** or: 8-10%)]

]

]

---

#### Between-party differences (per individual ad)

.font80[PvdA pays the least (**10-12 cents less** or: 8-10%) & reaches most people (~**1.1 - 1.3k more** per ad)]

```
## # A tibble: 15 × 5
## party reach share targeting relevance
## <chr> <dbl> <dbl> <chr> <dbl>
## 1 PvdA 13138 52.1 Higher Education 1
## 2 PvdA 12917 51.7 Higher Education 1
## 3 GroenLinks 11938 51.7 Higher Education 2
## 4 VVD 11528 51.6 Higher Education 1
## 5 VVD 11845 51.6 Higher Education 1
## 6 GroenLinks 11622 51.6 Higher Education 2
## 7 PvdA 12860 51.6 Higher Education 1
## 8 GroenLinks 11727 51.4 Higher Education 2
## 9 PvdA 12729 51.1 Higher Education 1
## 10 GroenLinks 11486 51.1 Higher Education 2
## 11 VVD 11388 51.0 Higher Education 1
## 12 PvdA 12632 50.9 Higher Education 1
## 13 GroenLinks 11509 50.9 Higher Education 2
## 14 VVD 11344 50.8 Higher Education 1
## 15 VVD 11260 50.6 Higher Education 1
```

]

]

---

#### Between-party differences (per target audience)

---

#### Between-party differences (per target audience)

---

### Within-party differences

---

### Within-party differences

Let's compare now:

Ad price and ad delivery share of

+ **higher-educated** vs. **lower-educated** audience

+ Audience **interested in the economy** vs. **not interested**

+ Audience **interested in politics** vs. **not interested**

+ Audience **interested in the environment** vs. **not interested**

---

### Within-party differences - Price per 1k

Ads **cost less for**:

+ *higher-educated* vs. *lower-educated audience*

Ad price **does not statistically differ for**:

+ Audience *interested in the economy* vs. *not interested*

+ Audience *interested in politics* vs. *not interested*

Ads **cost more for**:

+ Audience *interested in the environment* vs. *not interested*

]

![](img/diffs1.png)

]

---

### Within-party differences

Ad delivery shares

`\(\rightarrow\)` Results are (almost) the same

---

### Within-party differences - Ad delivery shares

Ads **deliver more to**:

+ *higher-educated* vs. *lower-educated audience*

Ad delivery **does not statistically differ for**:

+ Audience *interested in the economy* vs. *not interested* **except for VVD**

+ Audience *interested in politics* vs. *not interested*

Ads **deliver less to**:

+ Audience *interested in the environment* vs. *not interested*

]

![](img/diffs2.png)

]

---

## Summary

---

### Summary

Our findings do not always align with expectations:

+ In particular, the measure of ad relevance does not seem to drive pricing as much (although it does affect ad delivery share)

**However:**

> We find that Meta ad delivery algorithm prioritizes certain parties and audiences for political advertising

---

### Summary - between-party comparisons

+ Compared to GroenLinks and VVD:

+ PvdA pays less (**32-42 cents less** per 1k overall, ~-9-11%)
  + PvdA reaches more people (~1.1 - 1.3k more per ad, in total: 14-17k more)
  
--

+ People interested in environment
  
  + cheaper for GroenLinks compared to VVD (4 cent, or ~4.1%)
  + no difference in delivery
  
--

+ People interested in economy

+ no difference in pricing (but more expensive when VVD **excludes** audience)
  + VVD higher ad delivery share compared to GroenLinks (1.08% higher)

+ Higher-educated people

+ PvdA cheapest and most reach, no advantage for GroenLinks
  
  
---

### Summary - within-party comparisons

+ People interested politics (vs. not)
  + no difference in delivery and pricing
  
--
  
+ Higher-educated people (vs. lower-educated people) 
    
  + cheaper to reach (~-5%)
  + deliver more (~2.6% higher share)
  
--

+ People interested in environment (vs. not)
  
  + more expensive to reach (~+10%)
  + deliver less (~-5.11% lower share)
  
--

+ People interested in economy (vs. not)
  
  + no difference in pricing
  + delivers more for VVD (1.55% higher share)

---

### Age, gender and regional biases

Even after controlling for Meta user breakdown:

+ Men systematically receive more ads than women

+ 18-24 year olds systematically receive less ads

+ Certain Dutch provinces systematically receive less ads than others

---

### Implications

+ Meta (dis-)advantages certain parties

+  the findings presented in this paper show that political parties were not charged the same price for the same service
  
  + Unequal playing field

+ Some groups of people and regions are **systematically** less likely to receive political advertisements and more expensive to reach

+ isolating these groups from receiving election-related information
  
  + may deepen political, social and geographical inequalities

+ Little to no transparency by Meta about these systematic biases

+ difficult to research and make visible instances of unequal treatment and price discrimination
  + highlighting importance of access to data
  
--
  
+ Simply "banning" microtargeting would be inadequate

+ more power to the black box algorithm

---

## Thank you for your attention!

---

## Appendix

---

### Within-party differences

Reach and cost **over time**

Potential *market shock* on February 4th?

---

### Within-party differences per day - Reach and Cost

---

### Within-party differences

Reach and cost **over time** and **per party**

`\(\rightarrow\)` party differences remain constant despite *"shock"*

---

## Price differences per day

We observe:

*Consistent results*

+ "Market shock" **hits all parties equally**

+ Environment audience consistently *more expensive*

+ Higher educated audience consistently *less expensive*

*Inconsistent results*

+ Audiences interested in economy & politics are typically cheaper except on the day of the spike

]

---

### Price differences per day

---

### Bulk Discount?

]

]

---

## Skewed delivery

in terms of gender, age and region

---

## Differences in delivery by gender

.pull-left[
+ *Line at zero* shows empirical equilibrium of target audiences (i.e. the observed share of men and women in target audience)

+ *Deviation from zero* are algorithmic biases

+ above zero: prioritization
  
  + below zero: de-prioritization

+ Ads *deliver to more men* for every party

+ However: bias towards men seems smaller for GroenLinks
]

---

## Differences in delivery by age group

+ Ads *deliver less to young people*

+ aged 18-24

+ Consistent for each party

]

---

## Region differences

+ Ads deliver more to some regions

+ for example: Limburg, Friesland, Drenthe

+ Ads deliver less to other regions

+ Utrecht, North Holland, North Brabant

+ Consistent for each party

]

---

.font80[If we exclude economic interests/target environmental interests: VVD reaches less people and cheaper than GL]

]