⚠️ The Fediverse has been scraped, again ⚠️
Almost six million posts from 363 instances have been scraped.
"All the posts with public visibility published by users hosted on Mastodon servers [...] which support the English language" have been scraped along with their metadata, and the "policy, the code of conduct and the prohibited contents of each instance".
The dataset is an attempt at creating an open dataset for "research" into algorithms like the ones Facebook uses to identify problematic content, based around users' use of Content Warnings.
The dataset can be found here:
https://dataverse.harvard.edu/dataset.xhtml?persistentId=doi:10.7910/DVN/R1HKVS
It was created by the University of Milan, Italy, apparently for the 13th AAAI:
https://aaai.org/
The associated publishing:
https://aaai.org/ojs/index.php/ICWSM/article/download/3262/3130/ or https://likeable.space/media/30ae595a191923a1ce84a1e0feac6a3cef5b8669f44e15535ea18c7a5594b93a.pdf?name=Mastodon%20Content%20Warnings%3A%20Inappropriate%20Contents%20in%20a%20Microblogging%20Platform.pdf or DM me for a copy.
Related dataset:
https://dataverse.mpi-sws.org/dataset.xhtml?persistentId=doi:10.5072/FK2/AMYZGS
Original post:
https://likeable.space/objects/98fe7449-1776-4343-818d-aaff710a867b @tastytea
#FediAdmin #MastoAdmin #MastoDev #Privacy #OpSec #Warning #Fediverse #Mastodon #Scraping
@puffinus_puffinus @tastytea
This is, simply put, anti-ethical.
Since it is supposedly a scientific study, I would suggest contacting the review board of the university (or something like that).
Participation in scientific studies is not something trivial. Usually it is necessary to get a signed form with free and informed consent; implied consent should not be acceptable. And both the allowed uses and the handling of the data are very restricted.
Server run by the main developers of the project 
It is not focused on any particular niche interest - everyone is welcome as long as you follow our code of conduct!