App-Ads.txt Aggregated CSV Files with 42matters

The IAB app-ads.txt standard is a critical step for combating mobile, in-app advertising fraud and building trust in the advertising industry more broadly. 42matters is now offering a way to efficiently implement the standard at scale.

How to Scale the app-ads.txt Standard with 42matters

This year, the Interactive Advertising Bureau (IAB) released specifications for the app-ads.txt standard, an extension of the ads.txt specific for the mobile ecosystem. The IAB’s goal is to enable aggregators and other companies buying ad space to do so from legitimate and authorized suppliers. Needless to say, the move is an extraordinary step in the world of in-app advertising.

Indeed, IAB’s app-ads.txt provides a crucial mechanism for transparency where app owners are able to declare who is authorized to sell their ad inventory. However, collecting app-ads.txt data for all 6 million+ mobile apps available on Google Play and the Apple App Store is not an easy task. Consider, for example, that the  App Store is expected to host over five million apps by 2020 and that roughly 6,000 mobile apps are released through Google Play every day. The constant changes and releases of new apps demand a way to reliably and continuously keep a database of app-ads.txt data up-to-date.

Simply put, IAB’s app-ads.txt has the unfortunate side-effect of creating a lot more work for folks in the online advertising industry. In fact, we here at 42matters had a feeling this might be the case.

For this reason, we have been keeping track of the app-ads.txt standard since it was first announced by IAB last year. As such, we have aggregated app-ads.txt data for apps across both Google Play and the Apple App Store. This enables our customers  to find the authorized suppliers of any given app owner in a matter of seconds.

Additionally, we update this data on a weekly basis so that we can offer our customers the latest and greatest information at all times. By giving them access to our managed data service, they will have no need to implement their own crawler, nor to individually validate the data they acquire about each app. As such, we are able to save our customers both time and money.

How Does 42matters Do it?

The most popular app stores are supposed to support the newly introduced app-ads.txt standard by specifying an ad-hoc meta field in the HTML code of an app’s download-page (i.e. on the website version). However, when this is going to happen is still unclear. 

Once app stores begin to support this new standard more broadly, companies interested in app-ads.txt would need to crawl those pages for each app and app store combination, read the meta value, and download the app-ads.txt as linked in the app-page.

This is the “official” way to pursue app-ads.txt information. However, it might take time for app stores to implement it, if they implement it at all. 

Fortunately, at 42matters, we already collect this data. We do so by analyzing the website, email, and privacy URLs of more than 6 million mobile apps and follow the IAB specifications to identify the location of the app-ads.txt file for each one.

Additionally, we take data quality seriously. Thus we make sure to properly analyze and validate all URLs associated with an app. For example, we discard URLs that point to “base domains” that clearly do not belong to the app owner. As a matter of fact, many mobile apps use a Facebook Page (or Google Site page) as their official websites, but use the privacy URLs or email addresses with the correct company’s owned domains. This happens for the game Critical Ops, where, as of today, the website linked by the developer points to the Facebook Page of the game, but the privacy URL points to the official web domain of the owner of the app. In this case, the latter is the domain we use to crawl the app-ads.txt data.

For each app, and its cleansed domain, we schedule regular crawling of the app-ads.txt file. We then aggregate this data and release it as a convenient CSV file for our customers.

What is the Format of the Aggregated app-ads.txt by 42matters?

After crawling app-ads.txt for mobile apps, we aggregate this data in a single CSV file that includes the following information:

  • the date and time of the last crawling (i.e. update) 
  • the app’s domain
  • the information contained in app-ads.txt (i.e. AdDomainName, PubAccId, Relationship, CAId)
  • to which app (i.e. AppId) the app-ads.txt belongs 
  • the platform (i.e. app store) on which the app is distributed

The result is an easy-to-parse CSV file that can be quickly plugged into your existing data ingestion pipeline for similar data points (e.g. if you already onboard websites’ ads.txt from other aggregators).

Here a sample of the CSV file content for the The Weather Channel and Pandora apps:

DateTime,DomainName,AdDomainName,PubAccId,Relationship,CAId,AppId,AppStoreId
2019-03-10T15:57:00Z,weather.com,criteo.com,137482,DIRECT,NULL,com.weather.Weather,GooglePlay
2019-03-10T15:57:00Z,weather.com,Openx.com,537150861,DIRECT,NULL,com.weather.Weather,GooglePlay
2019-03-10T15:57:00Z,weather.com,criteo.com,137482,DIRECT,NULL,295646461,AppleAppStore
2019-03-11T15:58:00Z,pandora.com,indexexchange.com,188031,DIRECT,50b1c356f2c5c8fc,com.pandora.android,GooglePlay
2019-03-11T15:58:00Z,pandora.com,indexexchange.com,188031,DIRECT,50b1c356f2c5c8fc,284035177,AppleAppStore


How Can I Get Access to app-ads.txt?

By accessing our aggregated app-ads.txt data for mobile apps, companies working in ad-tech can better comply with the new IAB standard and drastically reduce the number of interactions they have with fraudulent ads.

The aggregated CSV file is updated weekly and is exposed to our customers via FTPS or via AWS S3. For more technical information please consult our online documentation.

Additionally, we also offer API endpoints that exposes the app-ads.txt content for a specific app (if app-ads.txt is supported by the app owner) for the most popular app stores:

How Much Does It Cost?

The CSV file containing the aggregated app-ads.txt data requires a yearly subscription, and it will automatically renew for the next year term. We provide access within 1 business day from the start of the subscription. 

Alternatively you can also access app-ads.txt data via the API service by subscribing to our Professional plan. 

To learn more about how 42matters can help your business save time and resources by accessing the highest-quality app-ads.txt data and get a quote, please contact us.

‍References and useful links:

- IAB official app-ads.txt specs

- IAB Authorized Digital Sellers

- Blog post: App-ads.txt release from IAB Tech Lab: what's in for you?

- IAB Taxonomy API for mobile apps