Getting Started with File Dumps


42matters generates file data dumps of the mobile app meta-data of apps and charts available on Google Play and iTunes and makes it available to its customers for download.

File Dumps Available

Data Format

Data is store in a single gzipped file with line delimited JSON with the following characteristics:

  • Each line is a valid JSON object
  • UTF-8 encoding
  • Line separator is '\n'

Access Credentials

In order to obtain your Access Credentials, please contact us.

Authentication

Clients require AWS S3 credentials in order to obtain the data files. Your credentials are available for you in your 42matters account under Launchpad.

To make sure that the credentials are working well try out with a tool such as Cyberduck. Enter path (bucket) ­ external.42matters.com and the access credentials from your account. You will be able to navigate to the target location such as /1/42apps/v0.1/production/itunes/ or /1/42apps/v0.1/production/playstore.

In order to programmatically download the feed files use a client library for AWS S3 in the language of your choice. E.g for Python we recommend Boto3 and more specifically S3Transfer tool for bulk downloads.

Automation Example

Here we show an example of how to use boto3 with S3Transfer to download a full playstore dump for a particular date. This code snippet is written in Python.

import boto3
from boto3.s3.transfer import S3Transfer

s3_client = boto3.client('s3', aws_access_key_id=aws_access_key_id, aws_secret_access_key=aws_secret_access_key,use_ssl=True)
s3_transfer = S3Transfer(s3_client)
s3_transfer.download_file(bucket='external.42matters.com', key="1/42apps/v0.1/production/playstore/full/2016-08-06/playstore-00.tar.gz", filename='playstorefulldump')

Last Modified: 2016-09-26


Get Free Support


We offer free email support to all our customers, with the 42matters team glad to provide you, prompt, professional support. We’re here to help.


CONTACT US