|rra e9c8e8341c rerun scrape march 2019||3 weeks ago|
|.gitignore||10 months ago|
|README.md||10 months ago|
|fedicrawler.py||10 months ago|
|instance_scrape.json||3 weeks ago|
|instances.txt||10 months ago|
Currently the script starts from https://post.lurk.org and queries
/api/v1/instance/peers to find servers it is peering with. For each of the peering servers it hasn’t seen before it does the same and in addition it tries to query
/api/v1/instance for meta data.
This method is a bit lacking because providing
/api/v1/instance is voluntary and specific to later versions of mastodon/activitypub fediverse. We should study the methodology of fediverse.network for better results.
When the request fails on a given instance it just logs it as ‘error’ now.
Latest scrape results can be found in