varia/bots
13
1
Fork 1
Code Issues 24 Releases Activity

pushing a new version of logbot with delete support jeej

Browse Source
This commit is contained in:
manetta 2021-01-22 00:14:44 +01:00
parent a783c06fd8
commit 6affdd5f7d
15 changed files with 490 additions and 210 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

101
logbot/README.md
View File

@ -1,15 +1,100 @@
# logbot
# RECbot
A small XMPP bot written in Python (using the slixmpp library) that logs all images and messages with the mentioning of *@bot* to an HTML page, to allow collaborative log writing over time.
A small XMPP bot written in Python that logs XMPP conversations into a HTML page, allowing collaborative log writing over time.
To run it:
The bot is used in group chats, where it includes all images that are send to the group and all messages that include `@bot`.
$ python3 logbot.py
*work-in-progress*
## Situated tails
* Archive bot, Relearn 2017, <https://gitlab.com/relearn/relearn2017/-/tree/master/xmpp-bots/archive-bot>
* Streambot, Varia website extension 2017-2018, <https://git.vvvvvvaria.org/varia/xmpp.streambot>
* Logbot, Varia XMPP extension 2017-2020, <https://git.vvvvvvaria.org/varia/bots/src/branch/master/logbot>
## Use RECbot
* check if `RECbot` is one of the participants in the groupchat!
* send an image to the groupchat **OR** use one of the `__ACTION WORDS__` below
* the bot confirms your contribution and writes the message to a file
* check the output of RECbot (locally or online, for example: <https://vvvvvvaria.org/logs>)
RECbot works with `__ACTION WORDS__` and unique `<HANDLE>` codes.
* `__ADD__` RECbot entries with `__ADD__ <message>`, for example: `__ADD__ Logging as a form of stretching time.` or `__ADD__ https://nicelink.org`
* `__DELETE__` RECbot entries with `__DELETE__ <HANDLE>`, for example: `__DELETE__ ~+*/+-` (\*spark)
* `__BOOK__` (\*sparks)
## Install RECbot
RECbot uses the `slixmpp` library to connect to XMPP and `beautifulsoup` to parse the HTML pages.
`$ sudo pip3 install slixmpp beautifulsoup4`
## Run RECbot!
`$ python3 RECbot.py`
The bot will ask you to provide the following details:
* XMPP address of a (bot)account
* password
* groupchat address
* nickname for the bot
* output folder path
You can also run it as a oneliner, for example by writing:
`$ python3 RECbot.py -u bot@vvvvvvaria.org -p CHANGEME -g roomname@muc.vvvvvvaria.org -n RECbot -o /var/www/logs/`
* `-u` / `--use` = user / use this XMPP address
* `-p` / `--password` = password
* `-g` / `--groupchat` = groupchat
* `-n` / `--nickname` = nickname
* `-o` / `--output` = output
## \*sparks
-----------
It would be so nice to have different RECbot *modes*: `--log`, `--stream`, `--distribusi`
* `--log`: RECbot writes a growing HTML page with images and text, that can be marked up and styled in HTML/CSS.
* `--stream`: RECbot stores all images that are send to the group, and displays them as an image stream.
* `--distribusi`: RECbot saves files (images, messages as markdown, files, links as HTML pages) and generates a distribusi page of all collected material.
Under the hood the process can be cut up into two procedures:
* saving text/image/audio/video based messages as files (.txt, .png/.jpg, .ogg, .og4/.mp4)
* recbot.py
* generating different outputs, depending on the selected *mode*
* distribusi.py[\*]
* log.py[\*]
* stream.py[\*]
These modes can be changed at any moment.
[\*] These are standalone scripts. They can be used on any set of files in a folder and generate HTML pages with customizable styling.
```
RECbot <modes> <log HTML page>
│ --distribusi > distribusi.py
│ [output folder] --log > log.py [output_folder/index.html]
│ (saved as files) --stream > stream.py (saved as index.html + stylesheet.css)
│ --xxx > xxx.py
│
└── stores text/media
files in output folder
(local/server)
```
------------
How can `__ACTION WORDS__` become magical `__MAGIC WORDS__` ???
------------
Dependencies:
$ sudo pip3 install slixmpp beautifulsoup4
---
That's all for now!

BIN
logbot/__pycache__/templates.cpython-37.pyc Normal file
View File

Binary file not shown.

BIN
logbot/avatar.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 18 KiB

1
logbot/example/.handles.txt Normal file
View File

@ -0,0 +1 @@
ieiauieiauiiuioiiuioooooeooooeieoooieooouuaiauuaia

BIN
logbot/example/Screenshot from 2021-01-11 22-58-22.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 70 KiB

1
logbot/example/entries/ieooo.txt Normal file
View File

@ -0,0 +1 @@
text msg, image, video, audio

BIN
logbot/example/entries/soviet-80s-computers-EFcZsx0VUAAZZoC.jpeg Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 73 KiB

65
logbot/example/index.html Normal file
View File

@ -0,0 +1,65 @@
<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8">
<title>Log</title>
<link rel="stylesheet" type="text/css" href="stylesheet.css">
</head>
<body>
<div id="welcome">
<p>Welcome to this Log!</p>
<p>This Log file is based on chat messages exchanged in a <em>XMPP groupchat</em> and is written by <em>RECbot</em>.</p>
<hr>
<p>For the writers of this log, you can:
<br>
<br>
send an image,
<br>
<br>
<code>__ADD__</code> a message,
<br>
<br>
<code>__DELETE__</code> it by using the <code>HANDLE</code> on the left (*spark),
<br>
<br>
<!-- <code>__ANNOTATE__</code> something using the <code>~HANDLE</code>, -->
<!-- <br> -->
<!-- <br> -->
<!-- <code>__ECHO__</code> material using the <code>~HANDLE</code> or a <code>#TAG</code> (*spark), -->
<!-- <br> -->
<!-- <br> -->
Request information about a <code>__BOOK__</code> by sending a <code>TITLE</code> (*spark, almost there),
<br>
<br>
... (*spark)
</p>
<!-- <hr> -->
</div>
<!-- <div id="echo">
<label for="echo" style="display: none;">__ECHO__</label>
<select name="echo" id="echo">
<option value="~HANDLE">~HANDLE</option>
<option value="#TAG">#TAG</option>
</select>
<input type="text" name="echo">
<button><code>__ECHO__</code></button>
</div> -->
<h1>ibugev@muc.vvvvvvaria.org</h1>
<div id="entries">
<div id="{ handle }" class="post">
<small class="postid">ieooo</small>
In general terms, “transhackfeminism” refers to hacking_with_care, using hacking with a meaning of (active) resistance and transformation to generate transversal knowledge through transdisciplinary artistic, aesthetic or cultural practices/ proposals. To work on producing knowledge collectively: without differentiating between theory and practice; as well as to embrace, protect and advance in free culture. To create communities where people meet, exchange, experience and share knowledge. To work on human and non-human alliances and solidarity through DIY/DIWO/DIT biotechnology, artistic and cultural practices.
<br><br>
To stay in touch with the material-affective dimensions of doing and engaging (bio)practices.
<br><br>
https://syllabus.pirate.care/topic/transhackfeminism/
<small class="date">Added on 11 January 2021</small>
</div>
<div id="{ handle }" class="post">
<small class="postid">auiio</small>
<img src="entries/soviet-80s-computers-EFcZsx0VUAAZZoC.jpeg">
<small class="date">Added on 12 January 2021</small>
</div>
</div>
</body>

69
logbot/example/stylesheet.css Normal file
View File

@ -0,0 +1,69 @@
body{
background-color: lightgrey;
min-width: 1080px;
margin: 40px;
font-size: 20px;
line-height: 24px;
}
div#welcome{
float: right;
top:40px;
right:40px;
width: 200px;
font-size: 16px;
}
div#welcome p{
margin:0 0 1em 0;
padding:0;
}
div#welcome hr{
border:0;
border-bottom:1px dotted blue;
margin:2em 0;
}
div#echo{
position: fixed;
bottom: 0;
left: 0;
width: 100%;
padding: 0.5em;
background-color: pink;
}
div.post{
margin: 2em 5em 2em 9em;
width: 800px;
}
div.post span.tagcontainer span{
padding-left: 0.5em;
color: blue;
}
p{
margin: 1em 0;
}
code{
color: blue;
}
small{
font-size: 12px;
line-height:1.2;
}
small.postid{
float: left;
font-family: monospace;
margin: 0 0 0 -180px;
padding: 1em 1.5em;
/*border-radius: 50px;*/
/*border: 1px dotted blue;*/
color: blue;
background-color: white;
font-size: 20px;
}
small.date{
display:block;
color:magenta;
margin:1em 0;
}
img{
max-width: 100%;
}

143
logbot/index.html Normal file
View File

@ -0,0 +1,143 @@
<!DOCTYPE html>
<html>
<head>
<meta charset="utf8">
<title>*docbot*</title>
<link rel="stylesheet" type="text/css" href="stylesheet.css">
</head>
<body>
<h1>OMG</h1>
<div class="post">
<p class="id">(0)</p>
<p class="message">@docbot hello?</p>
</div>
<div class="post">
<p class="id">(1)</p>
<p class="message">@docbot hello???</p>
</div>
<div class="post">
<p class="id">(5)</p>
<p class="message">@docbot hello???</p>
</div>
<div class="post">
<p class="id">(6)</p>
<p class="message">@docbot INFO No additional plugins loaded
INFO No Redis storage discovered
INFO Serving on http://0.0.0.0:8080
INFO JID set to: bot@vvvvvvaria.org/m0Upw4m2
INFO Joining testtesttest@muc.vvvvvvaria.org automatically
INFO Joining ibugev@muc.vvvvvvaria.org automatically
@docbot hello???
^CERROR Fatal write error on socket transport
protocol: <asyncio.sslproto.SSLProtocol object at 0x7f2a4ee72588>
transport: <_SelectorSocketTransport fd=6>
Traceback (most recent call last):
File "/usr/lib/python3.7/asyncio/selector_events.py", line 857, in write
n = self._sock.send(data)
OSError: [Errno 9] Bad file descriptor
ERROR Fatal error on SSL transport
protocol: <asyncio.sslproto.SSLProtocol object at 0x7f2a4ee72588>
transport: <_SelectorSocketTransport closing fd=6>
Traceback (most recent call last):
File "/usr/lib/python3.7/asyncio/selector_events.py", line 857, in write
n = self._sock.send(data)
OSError: [Errno 9] Bad file descriptor
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib/python3.7/asyncio/sslproto.py", line 676, in _process_write_backlog
self._transport.write(chunk)
File "/usr/lib/python3.7/asyncio/selector_events.py", line 861, in write
self._fatal_error(exc, 'Fatal write error on socket transport')
File "/usr/lib/python3.7/asyncio/selector_events.py", line 670, in _fatal_error
self._force_close(exc)
File "/usr/lib/python3.7/asyncio/selector_events.py", line 682, in _force_close
self._loop.call_soon(self._call_connection_lost, exc)
File "/usr/lib/python3.7/asyncio/base_events.py", line 688, in call_soon
self._check_closed()
File "/usr/lib/python3.7/asyncio/base_events.py", line 480, in _check_closed
raise RuntimeError('Event loop is closed')
RuntimeError: Event loop is closed
</p>
</div>
<div class="post">
<p class="id">(7)</p>
<p class="message">@docbot hello???</p>
</div>
<div class="post">
<p class="id">(8)</p>
<p class="message">@docbot test</p>
</div>
<div class="post">
<p class="id">(10)</p>
<p class="message">@docbot hello?</p>
</div>
<div class="post">
<p class="id">(11)</p>
<p class="message">@docbot hello?</p>
</div>
<div class="post">
<p class="id">(12)</p>
<p class="message">@docbot NICCCCCCCCEEEE</p>
</div>
<div class="post">
<p class="id">(13)</p>
<p class="message">@docbot help</p>
</div>
<div class="post">
<p class="id">(14)</p>
<p class="message">https://xmpp.vvvvvvaria.org:5281/upload/UTVaTk5FsUNXbUOx/avatar.png</p>
</div>
<div class="post">
<p class="id">(16)</p>
<p class="message"><img src="https://xmpp.vvvvvvaria.org:5281/upload/vUQ14zav2dJqoO8R/Screenshot%20from%202021-01-05%2012-42-11.png"></p>
</div>
<div class="post">
<p class="id">(17)</p>
<p class="message"><img src="https://xmpp.vvvvvvaria.org:5281/upload/ofHJdNEZECvE7SQd/avatar.png"></p>
</div>
</body>
</html>

21
logbot/index.html.j2 Normal file
View File

@ -0,0 +1,21 @@
<!DOCTYPE html>
<html>
<head>
<meta charset="utf8">
<title>*logbot*</title>
<link rel="stylesheet" type="text/css" href="stylesheet.css">
</head>
<body>
<h1>OMG</h1>
{% for num, msg in messages.items() %}
<div class="post">
<p class="id">({{ num }})</p>
<p class="message">{{ msg }}</p>
</div>
{% endfor %}
</body>
</html>

5
logbot/logbot.conf Normal file
View File

@ -0,0 +1,5 @@
[logbot]
account = bot@vvvvvvaria.org
password = streaming
nick = logbot
rooms = testtesttest@muc.vvvvvvaria.org, ibugev@muc.vvvvvvaria.org

280
logbot/logbot.py
View File

@ -1,219 +1,95 @@
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
from xbotlib import Bot
import json
import jinja2
import re
# To run this bot:
# $ python3 logbot.py
# The output folder of this bot currently is: /var/www/logs/digital-autonomy
db = 'storage.json'
import logging
from getpass import getpass
from argparse import ArgumentParser
def readdb():
storage = open(db, 'r').read()
messages = json.loads(storage)
lastid = sorted(list(messages.keys()))
nextid = str(int(lastid[-1])+1)
return messages
import slixmpp
import ssl, os, requests, urllib
def writedb(message):
try:
with open(db, 'r') as storage:
messages = json.loads(storage.read())
if messages.keys():
keys = [int(x) for x in messages.keys()]
keys.sort()
lastid = keys[-1]
nextid = lastid + 1
else:
nextid = 0
messages[f'{ nextid }'] = message
storage = open(db, 'w')
storage.write(json.dumps(messages, indent=4))
except IOError:
with open(db, 'w') as storage:
storage.write(json.dumps('{}'))
writedb(message)
return messages
class MUCBot(slixmpp.ClientXMPP):
"""
A simple Slixmpp bot that will save images
and messages that are marked with @bot to a folder.
"""
def deletefromdb(id):
with open(db, 'r') as storage:
messages = json.loads(storage.read())
del messages[id]
storage = open(db, 'w')
storage.write(json.dumps(messages, indent=4))
return messages
def __init__(self, jid, password, room, nick, output):
slixmpp.ClientXMPP.__init__(self, jid, password)
def writelog(messages):
template = jinja2.Template(open('index.html.j2').read())
with open('index.html','w') as out:
html = template.render(messages=messages)
out.write(html)
self.room = room
self.nick = nick
self.output = output
class logbot(Bot):
# The session_start event will be triggered when
# the bot establishes its connection with the server
# and the XML logs are ready for use. We want to
# listen for this event so that we we can initialize
# our roster.
self.add_event_handler("session_start", self.start)
help = '''Oh dear, logbot is here!
# The groupchat_message event is triggered whenever a message
# stanza is received from any chat room. If you also also
# register a handler for the 'message' event, MUC messages
# will be processed by both handlers.
self.add_event_handler("groupchat_message", self.muc_message)
@delete <num>
Delete posts from the log.
For example: @logbot @delete 5
@bots
To see who is around :)
def start(self, event):
self.get_roster()
self.send_presence()
@uptime
To check how long @logbot has been around
# https://xmpp.org/extensions/xep-0045.html
self.plugin['xep_0045'].join_muc(self.room,
self.nick,
# If a room password is needed, use:
# password=the_room_password,
wait=True)
@help
Print this message
'''
# NOTE(luke): disabled for now. We'll make it possible to speak to logbot privately later
# Send a message to the room
# self.send_message(mto=self.room, mbody='Hello! Logbot here. I\'m here to log all the images that are send to this group. You can also log text messages, by including @bot in your message. Happy logging! PS. you can access the logs at https://vvvvvvaria.org/logs/', mtype='groupchat')
def group(self, message):
print(message.content)
messages = readdb()
def muc_message(self, msg):
# Some inspection commands
#print('Message: {}'.format(msg))
if message.url:
messages = writedb(f'<img src="{ message.url }">')
reply = 'Thanks for that image!'
# Always check that a message is not the bot itself, otherwise you will create an infinite loop responding to your own messages.
if msg['mucnick'] != self.nick:
elif '@delete' in message.text:
match = re.findall("@delete \d*", message.content)[0]
id = match.replace('@delete ','')
if id in messages:
print('To be deleted:', messages[str(id)])
reply = f'This message is deleted: { messages[str(id)] }'
messages = deletefromdb(id)
else:
reply = 'This message is already gone!'
# Check if output folder exists
if not os.path.exists(self.output):
os.mkdir(self.output)
elif '@help' in message.text:
print('HELP')
# Check if an OOB URL is included in the stanza (which is how an image is sent)
# (OOB object - https://xmpp.org/extensions/xep-0066.html#x-oob)
if len(msg['oob']['url']) > 0:
else:
messages = writedb(message.text)
reply = 'Added, thanks!'
writelog(messages)
return self.reply(reply, room=message.room)
# Send a reply
self.send_message(mto=self.room,
mbody="Super, our log is growing. Your image is added!",
mtype='groupchat')
# Save the image to the output folder
url = msg['oob']['url'] # grep the url in the message
filename = os.path.basename(url) # grep the filename in the url
output_path = os.path.join(self.output, filename)
u = urllib.request.urlopen(url) # read the image data
f = open(output_path, 'wb') # open the output file
f.write(u.read()) # write image to file
f.close() # close the output file
# Add the image to the log
img = '<img class="image" src="{}">'.format(filename)
log = 'log.html'
log_path = os.path.join(self.output, log)
f = open(log_path, 'a+')
f.write(img+'\n')
f.close()
# Include messages in the log (only when '@bot' is used in the message)
if '@bot' in msg['body']:
# reply from the bot
self.send_message(mto=self.room,
mbody="Noted! And added to the log. Thanks {}!".format(msg['mucnick']),
mtype='groupchat')
# Add the message to the log!
message = '<p class="message">{}</p>'.format(msg['body'].replace('@bot',''))
log = 'log.html'
log_path = os.path.join(self.output, log)
f = open(log_path, 'a+')
f.write(message+'\n')
f.close()
if '/book' in msg['body']: # Check if this is a book ...
self.send_message(mto=self.room,
mbody="Oh a book, that's cool! Thanks {}!".format(msg['mucnick']),
mtype='groupchat')
# Start of book feature
from bs4 import BeautifulSoup
import re
book = msg['body'].replace('@bot', '').replace('/book', '')
book = re.sub(' +', ' ', book) # remove double spaces
book = book.lstrip().rstrip() # remove spaces at the beginning and at the end
book = book.replace(' ', '+').lower() # turn space into + and lowercase
page_link = 'https://www.worldcat.org/search?q={}&qt=results_page'.format(book)
page_response = requests.get(page_link, timeout=5)
page_content = BeautifulSoup(page_response.content, "html.parser")
try:
book_title = page_content.findAll("div", {"class": "name"})[0].text
book_author = page_content.findAll("div", {"class": "author"})[0].text
book_publisher = page_content.findAll("div", {"class": "publisher"})[0].text
response = '<b>BOOK</b>: ' + book_title + ' ' + book_author + ' ' + book_publisher
book_found = True
except IndexError:
book_found = False
if book_found:
# Add message to log
message = '<b>BOOK</b>: ' + book_title + ' ' + book_author + ' ' + book_publisher
log = 'log.html'
log_path = os.path.join(self.output, log)
f = open(log_path, 'a+')
f.write(message+'\n')
f.close()
self.send_message(mto=self.room, mbody='Hope this was the book you were looking for: ' + book_title + ' ' + book_author + ' ' + book_publisher, mtype='groupchat')
else:
self.send_message(mto=self.room, mbody='Sorry, no book found!', mtype='groupchat')
if __name__ == '__main__':
# Setup the command line arguments.
parser = ArgumentParser()
# output verbosity options.
parser.add_argument("-q", "--quiet", help="set logging to ERROR",
action="store_const", dest="loglevel",
const=logging.ERROR, default=logging.INFO)
parser.add_argument("-d", "--debug", help="set logging to DEBUG",
action="store_const", dest="loglevel",
const=logging.DEBUG, default=logging.INFO)
# JID and password options.
parser.add_argument("-j", "--jid", dest="jid",
help="JID to use")
parser.add_argument("-p", "--password", dest="password",
help="password to use")
parser.add_argument("-r", "--room", dest="room",
help="MUC room to join")
parser.add_argument("-n", "--nick", dest="nick",
help="MUC nickname")
# output folder for images
parser.add_argument("-o", "--output", dest="output",
help="output folder, this is where the files are stored",
type=str)
args = parser.parse_args()
# Setup logging.
logging.basicConfig(level=args.loglevel,
format='%(levelname)-8s %(message)s')
if args.jid is None:
args.jid = input("XMPP address: ")
if args.password is None:
args.password = getpass("Password: ")
if args.room is None:
args.room = input("MUC room: ")
if args.nick is None:
args.nick = input("MUC nickname: ")
if args.output is None:
args.output = input("Output folder: ")
# Setup the MUCBot and register plugins. Note that while plugins may
# have interdependencies, the order in which you register them does
# not matter.
xmpp = MUCBot(args.jid, args.password, args.room, args.nick, args.output)
xmpp.register_plugin('xep_0030') # Service Discovery
xmpp.register_plugin('xep_0045') # Multi-User Chat
xmpp.register_plugin('xep_0199') # XMPP Ping
xmpp.register_plugin('xep_0066') # Process URI's (files, images)
# Connect to the XMPP server and start processing XMPP stanzas.
xmpp.connect()
xmpp.process()
logbot()

0
logbot/storage.json Normal file
View File

14
logbot/stylesheet.css Normal file
View File

@ -0,0 +1,14 @@
body{
background-color: pink;
margin: 1em;
}
.post{
margin: 1em 0;
clear: both;
}
.post p.id{
float: left;
margin: 0 1em 1em;
}
.post p.message{
}