rra
/
dafr
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity
doing applied fediverse research
You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
21 Commits
1 Branch
0 Tags
13 MiB
 
Tree: 3f5d2bbad0
Branches Tags
${ item.name }
Create tag ${ searchTerm }
Create branch ${ searchTerm }
from '3f5d2bbad0'
${ noResults }
rra 3f5d2bbad0
minor tweaks to fedicrawler & scrape 05-05-2020 		5 years ago
.gitignore added new script to document mastodon about pages 5 years ago
README.md update readme 5 years ago
about_collector.py added new script to document mastodon about pages 5 years ago
fedicrawler.py minor tweaks to fedicrawler & scrape 05-05-2020 5 years ago
instance_scrape.json minor tweaks to fedicrawler & scrape 05-05-2020 5 years ago
instances.txt first version, crawls only the announced peers 7 years ago

README.md

doing frequent applied fediverse research

  • hands-on approach to verify statistics on fediverse from fediverse.network and the-federation.info
  • draws conclusions from that

methodology

Mapping the network

Currently the script starts from https://post.lurk.org and queries /api/v1/instance/peers to find servers it is peering with. For each of the peering servers it hasn't seen before it does the same. This from the assumption that getting peer lists from Mastodon & Pleroma gives enough of a view of 'known fediverse' to work with.

initial peer list

all peers of the initial peers

all peers of the peers of the inintial peers

------------------------------------------------- + the known fediverse?

Instance metadata

We try to query /.well-known/nodeinfo for instance meta-data such as software type etc. This is what both fediverse.network and the-federation.info do

When any request fails on a given instance it logs the raised Exception, if it is a HTTP error instead we currently log the answer.

Latest scrape results can be found in instance_scrape.json

TODO FIXME

  • add detailed error message to json when we get one
  • abstract the functions so we can multithread them