You can not select more than 25 topics
Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
rra
ce613426d0
|
6 years ago | |
---|---|---|
.gitignore | 7 years ago | |
README.md | 7 years ago | |
fedicrawler.py | 7 years ago | |
instance_scrape.json | 6 years ago | |
instances.txt | 7 years ago |
README.md
directorate for applied fediverse research
- independently tries to verify fediverse statistics
- draws conclusions from that
methodology
Currently the script starts from https://post.lurk.org and queries /api/v1/instance/peers
to find servers it is peering with. For each of the peering servers it hasn't seen before it does the same and in addition it tries to query /api/v1/instance
for meta data.
This method is a bit lacking because providing /api/v1/instance
is voluntary and specific to later versions of mastodon/activitypub fediverse. We should study the methodology of fediverse.network for better results.
When the request fails on a given instance it just logs it as 'error' now.
Latest scrape results can be found in instance_scrape.json
TODO FIXME
- add detailed error message to json when we get one
abstract the functions so we can multithread them- find a way to also scrape for instances that don't announce themselves