logbot: adding RSS support #3

RSS support would be super nice to add, for following updates when you are not in the chat group and for adding the feed to the multifeeder https://multi.vvvvvvaria.org

You can make use of the additional routes for this I think!

https://git.vvvvvvaria.org/decentral1se/xbotlib#user-content-bot-routes

RSS support is added to LogBot!

30df62d1d5

Didn't dive into the additional routes yet.

Still work in progress!

https://validator.w3.org/feed/check.cgi?url=https%3A%2F%2Fvvvvvvaria.org%2Flogs%2Fx-y%2Ffeed.rss lists a couple of to-do's.

And the Multifeeder can't run with these feeds, it triggers an error on the data parsing, see varia/multifeeder#8.

Pasting the todo's here, to make logbot's RSS feeds valid:

line 7, column 50: lastBuildDate must be an RFC-822 date-time: Sunday, 12. December 2021 03:31PM [help]

<lastBuildDate>Sunday, 12. December 2021 03:31PM</lastBuildDate>

Help: Needs to follow this structure: https://validator.w3.org/feed/docs/error/InvalidRFC2822Date.html

line 13, column 12: pubDate must be an RFC-822 date-time: [help]

<pubDate></pubDate>

Same as above.

line 17, column 16: Undefined description element: img [help]

<description><img src="images/Screenshot_from_2020-09-21_17-28-36.png"></ ...

Help: https://validator.w3.org/feed/docs/error/UndefinedElement.html

line 17, column 76: XML parsing error: :17:76: mismatched tag

<description><img src="images/Screenshot_from_2020-09-21_17-28-36.png"></ ...

Help: https://validator.w3.org/feed/docs/error/SAXError.html

In addition, interoperability with the widest range of feed readers could be improved by implementing the following recommendation.

line 14, column 2: item should contain a guid element [help]

</item><item>

Help: https://validator.w3.org/feed/docs/warning/MissingGuid.html

Fixed the RSS build date: is an RFC-822 date-time time now!

<pubDate></pubDate> is an RFC-822 date-time time now
media files are now added in an <enclosure> tag
a <guid> is added to each item, linking to the post using an anchor link (for example: https://vvvvvvaria.org/logs/x-y/index.html#12)

And this has been an useful RSS reference: https://www.w3schools.com/xml/xml_rss.asp#rssref

RSS feeds seem to work.

It was quite a bit of work to figure out how images can be loaded in the main <description> tag. In the end, they are added both there but also in the <enclosure> tag, as recommended here: https://www.w3schools.com/xml/rss_tag_enclosure.asp

I also tried to work with the CDATA wrapper, as descripbed here https://www.w3.org/wiki/RssContent, but that didn't seem to change much.

In some RSS readers the images show up, in others unfortunately not :(. And it's hard to figure out why. For example, in Feedreader this happens (see screenshot).

Needs more testing!

Screenshot from 2022-02-18 17-16-23.png

69 KiB

When adding a logbot RSS feed to the multifeeder, this error popped up:

/var/www/multifeeder/.venv/lib/python3.7/site-packages/apscheduler/util.py:95: PytzUsageWarning: The zone attribute is specific to pytz's interface; please migrate to a new time zone provider. For more details on how to do so, see https://pytz-deprecation-shim.readthedocs.io/en/latest/migration.html
  if obj.zone == 'local':
/var/www/multifeeder/.venv/lib/python3.7/site-packages/apscheduler/triggers/interval.py:66: PytzUsageWarning: The normalize method is no longer necessary, as this time zone supports the fold attribute (PEP 495). For more details on migrating to a PEP 495-compliant implementation, see https://pytz-deprecation-shim.readthedocs.io/en/latest/migration.html
  return self.timezone.normalize(next_fire_time)

Maybe logbot's hack to hard code a timezone is causing this?

The timezone is context sensitive now, logbot saved a post based on the local timezone: c13612a69d

Although this did not solve the problem of the multifeeder (i did not figure that out in the end and left it as it is a warning message and not an error)... but it's a nice new feature.