From 15d19e0435804e01064e79560753571bfd2f61d4 Mon Sep 17 00:00:00 2001 From: rra Date: Thu, 30 Apr 2020 12:53:04 +0200 Subject: [PATCH] update readme --- README.md | 21 ++++++++++++++------- 1 file changed, 14 insertions(+), 7 deletions(-) diff --git a/README.md b/README.md index 280041c..f703697 100644 --- a/README.md +++ b/README.md @@ -1,22 +1,29 @@ -# directorate for applied fediverse research +# doing frequent applied fediverse research -* independently tries to verify fediverse statistics +* hands-on approach to verify statistics on fediverse from fediverse.network and the-federation.info * draws conclusions from that ## methodology -Currently the script starts from and queries `/api/v1/instance/peers` to find servers it is peering with. For each of the peering servers it hasn't seen before it does the same and in addition it tries to query `/api/v1/instance` for meta data. +### Mapping the network +Currently the script starts from and queries `/api/v1/instance/peers` to find servers it is peering with. For each of the peering servers it hasn't seen before it does the same. This from the assumption that getting peer lists from Mastodon & Pleroma gives enough of a view of 'known fediverse' to work with. -This method is a bit lacking because providing `/api/v1/instance` is voluntary and specific to later versions of mastodon/activitypub fediverse. We should study the methodology of [fediverse.network](https://fediverse.network/info) for better results. + > initial peer list + >> all peers of the initial peers + >>> all peers of the peers of the inintial peers + ------------------------------------------------- + + the known fediverse? -When the request fails on a given instance it just logs it as 'error' now. +### Instance metadata +We try to query `/.well-known/nodeinfo` for instance meta-data such as software type etc. This is what both fediverse.network and the-federation.info do + +When any request fails on a given instance it logs the raised `Exception`, if it is a HTTP error instead we currently log the answer. Latest scrape results can be found in `instance_scrape.json` ## TODO FIXME -* add detailed error message to json when we get one +* ~~add detailed error message to json when we get one~~ * ~~abstract the functions so we can multithread them~~ -* find a way to also scrape for instances that don't announce themselves