rra/dafr
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity
doing applied fediverse research
14 Commits 1 Branch 0 Tags 13 MiB
Python 100%
26b4e4a868
Go to file
rra 26b4e4a868 april 2020
2020-04-28 14:30:31 +02:00
.gitignore gitignore and readme 2018-05-30 09:05:29 +02:00
fedicrawler.py updated with a way to get around gab.best enumeration and better error logging 2020-04-28 14:30:18 +02:00
instance_scrape.json april 2020 2020-04-28 14:30:31 +02:00
instances.txt first version, crawls only the announced peers 2018-05-30 08:20:46 +02:00
README.md scraper now uses parallelism 2018-06-07 23:51:07 +02:00

README.md

directorate for applied fediverse research

  • independently tries to verify fediverse statistics
  • draws conclusions from that

methodology

Currently the script starts from https://post.lurk.org and queries /api/v1/instance/peers to find servers it is peering with. For each of the peering servers it hasn't seen before it does the same and in addition it tries to query /api/v1/instance for meta data.

This method is a bit lacking because providing /api/v1/instance is voluntary and specific to later versions of mastodon/activitypub fediverse. We should study the methodology of fediverse.network for better results.

When the request fails on a given instance it just logs it as 'error' now.

Latest scrape results can be found in instance_scrape.json

TODO FIXME

  • add detailed error message to json when we get one
  • abstract the functions so we can multithread them
  • find a way to also scrape for instances that don't announce themselves