Facebook's new API for academic research is now available in early access

This week, a small set of academic research teams will have access to a new Facebook tool designed to collect almost universal real-time data from the world's largest social network.

When it comes to who has access to Facebook data and how, Meta still suffers from the impact of the Cambridge Analytica incident, where a political consulting firm collected personal data from millions of unknown Facebook users to develop comprehensive profiles of potential voters. In the three years since, the company has shut down thousands of APIs and is currently beginning to restore broad access to academic research.

FulTech had a first look at Facebook's new academic research API and spoke with Facebook's product manager Kiran Jagadeesh, who led the project with the Facebook Open Research & Transparency (FORT) team.

"This is just the beginning," Jagadeesh told TechCrunch, describing the Researcher API as a beta version of the toolkit the company plans to offer in the future. The Python-based API, originally unveiled on F8 this year, runs in JupyterLab, an open source laptop interface.

Given Facebook's many privacy issues in the past, the new Researcher API comes with some preliminary limitations. To begin with, the API will be made available to a selected number of established academics via an invitation method. In February 2022, the business intends to increase access beyond the initial test group, by combining test comments to a larger rollout to all academics.

Another precaution is that the Researcher API works in a highly regulated environment, which Jagadeesh refers to as a "digital clean room". Academic researchers with API access can enter the environment via a Facebook VPN, collect data and crunch statistics, but only the analysis may be exported.

The goal is to protect the privacy of users and prevent all data analyzed from being re-identified, but the restriction may irritate some of the company's critics given that all public data collected by the Researcher API is already available but difficult to aggregate and analyze using Facebook's existing tool.

The API first provides access to four categories of Facebook data in real time: pages, groups, events and posts. In each scenario, the program will initially solely retrieve data from public sources in the United States and the European Union. To make data available through the API for groups, events, and pages, at least one administrator must be in a recognized country.

Researchers can use the program to examine large parts of raw text with approaches such as sentiment analysis, which analyzes the valence and emotions that people show in their speech on a particular topic. Apart from the text-based posts that make up the majority of the available information, researchers can also access relevant information such as group and page descriptions, creation dates and post responses.

Multimedia data, such as raw photos, will not be included, nor will comments or user demographic information (age, gender, etc.). The API will also not collect data from Instagram, although Jagadeesh realizes that the site is extremely important to researchers, and the team is investigating methods to make Instagram data available.

The FORT team hopes to work closely with university academics to improve and expand existing tools, which Jagadeesh described as "ongoing work." Although Meta has not yet completed its first group of academic partners, the company has asked academics from 23 academic institutions around the world to test the waters.

On Monday, November 15, researchers who had completed the team's induction procedure and agreed to its privacy rules were given access. Everyone who has access to the research must agree to privacy restrictions, such as not re-identifying specific people in the data.

At present, the research API is only available to a few academic institutions, but the FORT team intends to expand access to other groups, including journalists. The goal is to produce a public roadmap that gives researchers and journalists a clear picture of what the team is working on.

The company still has a lot of work to do to earn the trust of the scientific community. In August, Facebook blocked two well-known researchers working on NYU's Cybersecurity for Democracy project from accessing advertising data, which attracted criticism from many academics and authorities. These researchers focused on tracking misinformation and political ads using Ad Observer, an opt-in browser application. In September, Facebook apologized to an elite group of academics known as Social Science One for sending them insufficient data, which undermined months of work and analysis.