bibliotecha-manual/docs/index.md

# Bibliotecha Manual

> [Bibliotecha - Digital books need libraries too](https://bibliotecha.info/)

> Bibliotecha is a framework to facilitate the local distribution of digital
> publications within a small community. It relies on a microcomputer running
> open-source software to serve books over a local wifi hotspot. Using the
> browser to connect to the library one can retrieve or donate texts.
> Bibliotecha proposes an alternative model of distribution of digital texts
> that allows specific communities to form and share their own collections.

[TOC]

## Introduction

Welcome to the Bibliotecha manual! This guide serves as a human-friendly
document for setting up an offline-first local library for yourself and your
community.

## Go Slow, Make a Plan

There is nothing really easy about diving into this system administration work
and getting things up and running but on the other hand there is nothing so
complicated that it cannot be figured out. In short: you can do it!

The two most important things to take into consideration before starting are

- Choosing a "route" through this documentation (see just below).
- Reading the [Prerequisites] section very closely and trying to
  understand, **before you dive in**, how things will work with your Raspberry
  Pi and how it is setup and connected to the internet.

There are 3 routes you can take through this documentation:

- **Cloning**: Skipping the entire thing and simply cloning a pre-made copy of
  a Bibliotecha SD-card onto your new SD-card. You can then plug the SD-card
  into your Raspberry Pi and Bibliotecha will be ready to go. Skip to the
  [Cloning] section for more. This is the *moving really fast and don't have
  time* option.

- **Automated**: You follow the documentation below: [Prerequisites],
  [Pre-installation] and finally [Automated Installation]. This allows you do
  to the initial setup and then a run convenient installation script which
  takes care of installing Bibliotecha. This is the *moving fast but still want
  to be involved with the process* option.

- **Manual**: You follow the documentation below: [Prerequisites],
  [Pre-installation] and finally [Manual Installation]. You do everything
  yourself to get from start to finish. Everything that would be done in the
  automated script, you do yourself. This is the *I have time and want to learn
  a lot* option.

[automated installation]: #automated-installation
[cloning]: #cloning
[pre-installation]: #pre-installation
[prerequisites]: #prerequisites

## Prerequisites

Bibliotecha is made specifically for use on the cheap and widely accessible
[Raspberry Pi] single board computer and the Debian based [Raspberry Pi OS].

You should follow the [official setup documentation] on the Raspberry Pi website
in order to get your board up and running. You will need it to have access to
the internet in order to download the necessary packages as well as access to
the command-line.

!!! warning

    You **must** use an Ethernet cable to connect your Raspberry Pi to your
    router and not use the Wifi connection. This is because the Bibliotecha
    install process will overwrite the Wifi configuration. So, you also
    **must** have access to your router before going further.

Here is an example of what we did.

*Below: Here we plug the gray ethernet cable into the router.*

![Router](img/router.png)

*Below: And then we plug that same gray ethernet cable into the Raspberry Pi.*

![Pi](img/pi.png)

!!! note

    The ethernet cable is not required after you finish the installation
    of Bibliotecha! Your Bibliotecha can now operate independent of an internet
    connection and offers its own Wifi access point. So, the ethernet cable is only
    required for the installation process. Later, if you need to do an update or
    some maintenance which requires an internet connection, you can plug it back in
    and [make sure to check this tip].

The current latest [Raspberry Pi 3 B+] model is recommended. This model is
chosen because it offers a built-in wireless card for convenient networking and
a sufficient memory allowance of 1GB. It is possible to use other models of
board but they should at least provide these guarantees.

[Raspberry Pi OS] (formerly "Raspian Buster") is the current latest recommended
version of the standard Raspberry Pi operating system. [Etcher] is a useful and
simple tool for flashing the operating system onto the SD card which you will
plug into your Raspberry Pi. You can also use the [dd command].

Here is the `dd` command we use:

```bash
$ sudo lsblk  # to see the value for the `of` argument (e.g. /dev/mmcblk0)
$ sudo dd bs=4M if=2020-08-20-raspios-buster-armhf-lite.img of=<YOUR-DEVICE-HERE> status=progress conv=fsync
```

[dd command]: https://www.raspberrypi.org/documentation/installation/installing-images/linux.md
[etcher]: https://www.balena.io/etcher/
[make sure to check this tip]: #i-cannot-connect-to-the-internet-from-the-raspberry-pi
[official setup documentation]: https://www.raspberrypi.org/documentation/setup/
[raspberry pi 3 b+]: https://www.raspberrypi.org/products/raspberry-pi-3-model-b/
[raspberry pi os]: https://www.raspberrypi.org/documentation/raspbian/
[raspberry pi]: https://www.raspberrypi.org/

## Pre-installation

Before getting started, we need to perform some preparatory steps. These steps
must be completed successfully before moving on with the rest of the guide.

You should run the following commands at the command-line interface of your
Raspberry Pi.

Firstly, we switch our user to the root account:

```bash
$ sudo -i
```

We then perform the initial system update and upgrade:

```bash
$ apt update
$ apt upgrade
```

This step may be not needed but it has been reported a few times. You need to
unblock the Wifi device. In order to do this, you'll need to run the following.

```bash
$ apt install -y rfkill
$ rfkill unblock wlan
```

We should then perform a number of steps within the [raspi-config] tool:

```bash
$ raspi-config
```

- Change the user password
    - Choose the `System Options > Password` option.
    - It is important to configure your Raspberry Pi with a secure passphrase.
      A [diceware passphrase] is a recommended approach for choosing a
      sufficiently strong passphrase.

- Choose a hostname
    - Follow the `System Options > Hostname` options.
    - The hostname will be the name that identifies the Raspberry Pi on the
      local network.

- Configure predictable network interfaces
    - Follow the `Advanced Options > Network Interface Names` options.
    - It is important to enable predictable network interface names
      so that the automatic installation script can detect which network
      interfaces are in use.

- Expand the SD card partition
    - Follow the `Advanced Options > Expand filesystem` options.
    - This allows more space on the SD card to be used. This is important
      for when you will start to place more and more digital books in your
      Bibliotecha.

- Configure the localisation
    - Follow the `Localisation Options > Change Locale` options.
    - It is recommended to ensure that the `en_GB.UTF-8 UTF-8` locale
      is selected. This is the default. Once this is selected, select `<Ok>` on the
      two following dialogs to generate the locale.

- Configure WLAN country
    - Follow the `Localisation Options > Change WLAN Country` options.
    - This is actually really important or the `rfkill` related step above
      will keep your Wireless device disabled.

The Raspi-config interface then asks you to restart the Raspberry Pi which you
should do. If not, you can also run this from the command-line:

```bash
$ reboot
```

Remember, you will need to use your new user passphrase to access the Raspberry
Pi after rebooting it. Make sure you store this passphrase somewhere safe!

Also, please note, if you did run the `rfkill` commands above, please check
that when you reboot, the message shown when you log back in does not say that
the Wifi device is still blocked. You may need to fiddle with `rfkill` and the
`raspi-config` WLAN country settings.

[raspi-config]: https://www.raspberrypi.org/documentation/configuration/raspi-config.md
[diceware passphrase]: https://www.rempe.us/diceware/#eff

## Automated installation

It is now time to run the automatic installation. This script will install and
configure all the necessary moving parts of Bibliotecha.

If you're interested in doing this process manually (for the purpose of
learning, for example), a [Manual installation] guide is provided.

If you would first like to read the script, the [source is available].

On the Raspberry Pi once again, run the following commands:

```bash
$ sudo -i
$ curl https://install.bibliotecha.info | bash
```

The script will automatically reboot your Raspberry Pi when it is finished.

If you run into any issues, please see the [Troubleshooting section].

[manual installation]: #manual-installation
[troubleshooting section]: #troubleshooting
[source is available]: https://git.vvvvvvaria.org/varia/bibliotecha-install/src/branch/master/bibliotecha.sh

## Post-installation

After rebooting, there should be a Wifi hotspot available with the name
`Bibliotecha`. You should wait a few minutes for this hotspot to become
available. This Wifi access point is being served from the Raspberry Pi. You
should be able to connect to this Wifi. It is not password protected.

Once connected you should be directed to the so-called "captive portal" of the
Bibliotecha where it explains how to enter the library and use it. The library
should be available locally at [http://bibliotecha.library].

It is recommended to customise your captive portal page to suite your own
needs. This file is available in the `/var/www/bibliotecha/index.html`
location. You should also take a look at customising your `/etc/motd` SSH
welcome banner.

You will be required to configure the [Calibre-web] installation. Extended
configuration documentation is available from the [Calibre-web wiki]. The most
important part is the "Location of Calibre database" for which you can enter
the following path:

> /opt/calibre-database

This is the default installation path used by the installation script.

You may also want to look at the "Feature Configuration" section where you can
decide whether to allow uploading, anonymous browsing and allowing public
registrations. These depend on your context and for who you will serve the
library to.

Click "Submit", "Login" and you will be redirected to the library login page.
The default adminstration password login details are:

> Username: admin
>
> Password: admin123

You should change these details to secure your adminstration account.

[http://bibliotecha.library]: http://bibliotecha.library
[calibre-web]: https://github.com/janeczku/calibre-web/
[calibre-web wiki]: https://github.com/janeczku/calibre-web/wiki/Configuration

## Maintaining a Community Library

Once your Bibliotecha is configured you can start to think about how you and
your community would like to maintain the library. You should ask yourself some
questions:

- Who will be the digital librarians? The catalogue will need care.
- Will you allow public registrations? Will you allow public uploads?
- How will you publicise the library within the local context?
- What kind of library do you want to create? What are the themes?
- Who will be responsible for maintaining the system?

## Understanding Bibliotecha Networking

Bibliotecha uses standard, venerable and stable GNU/Linux networking tools and
configuration to enable the [local-first] networking setup. Installing, running
and maintaining a network configuration is no easy topic! However, it is a
useful skill to have. Overall, Bibliotecha is made up of the following programs
and configurations:

- [/etc/network/interfaces.d/](https://manpages.debian.org/buster/ifupdown/interfaces.5.en.html): The network interface configuration
- [/etc/hosts](https://manpages.debian.org/buster/manpages/hosts.5.en.html): The hostname definitions
- [Hostap](https://wiki.debian.org/hostap): The Wifi access point provider
- [Dnsmasq](https://wiki.debian.org/HowTo/dnsmasq): The DNS and DHCP server
- [Dhcpcd](https://manpages.debian.org/buster/dhcpcd5/dhcpcd.8.en.html): The DHCP client
- [Calibre](https://calibre-ebook.com/): The library database
- [Calibre-web](https://github.com/janeczku/calibre-web/): The library web application
- [Lighttpd](https://www.lighttpd.net/): The web server

When your Bibliotecha is setup and running, it is doing a number of things. It
is first serving a Wireless access point (Hostap) which your devices can
connect to. After you connect, your device is given an IP address on the local
network (Dnsmasq and Dhcpcd) as well as a local DNS entry (mydevice.library,
for example). Once you open a web browser, it will indicate that you need to
log into the network but in fact, you are brought to a web page (Lighttpd)
which shows you how to reach the library web application (Calibre-web).

[local-first]: https://www.inkandswitch.com/local-first.html

## Troubleshooting

Because Bibliotecha is made up of a number of moving parts it is not feasible
for this manual to cover all the possible issues. However, we try our best here
to provide context, background, useful tips and tricks to help you become
familiar with fixing your Bibliotecha. We recommend a DIWO (Do It With Others)
approach!

Please make sure to take some time to read [Understanding Bibliotecha
Networking] so that you are familiar with all the moving pieces. When
troubleshooting, it is important to narrow down which piece of the puzzle is
responsible. To do this, you need to know what the pieces are.

If all else fails, please send an email to the public [mailing list].

If someone is online, you may also find us lurking in the `#bibliotecha`
channel on IRC. Here is [a web page you can use] to connect without having to do
any account sign-up.

All of the following commands should be run as the root user.

[a web page you can use]: https://webchat.freenode.net/?channel=#varia

#### I cannot connect to the internet from the Raspberry Pi

If you are connecting an Ethernet cable to your Bibliotecha in order to connect
it to your local router and have access to the internet, then you might notice
that the requests do reach their destination.

Bibliotecha is configured to capture all the network requests it receives and
point them to the library interface. You will need to temporarily disable the
`dnsmasq` service:

```bash
$ systemctl stop dnsmasq
```

It should then be possible to connect to the wider internet now.

#### The wireless access point is not available

The access point is responsibility of the `hostapd` program. You should check
the status of the running service with:

```bash
$ systemctl status hostapd
```

If there are errors, you can see the logs with:

```bash
$ journalctl -u hostapd
```

You may also attempt to restart this service afterwards:

```bash
$ systemctl restart hostapd
```

You may also want to inspect the `/etc/hostapd/hostapd.conf` configuration.

#### I do not receive an IP address when I connect

Providing IP addresses is the responsibility of the `dnsmasq` and `dhcpcd`
service. You should check the status of `dnsmasq` with:

```bash
$ systemctl status dnsmasq
```

You should make sure that `dhcpcd` is running with:

```
$ dhcpcd5
```

#### How to upgrade Calibre-web

You'll need to re-connect your Bibliotecha to the internet with an ethernet
cable first. Then, you'll need to check to see what are the latest releases
from Calibre-web on [their release page]. Once you have a fair idea of the
version you'd like to upgrade to (careful, skipping over multiple versions can
easily break things, so better go one by one and check which one works each
step of way), then start off with turning off the program and getting into the
installation directory.

```bash
$ systemctl stop cps
$ cd /var/www/calibre-web
```

Now, you'll want to pull down all the tags from the remote repository.

```bash
$ git fetch origin master -a
```

Then list all those tags.

```bash
$ git tag -l
```

At this point you should see your version number listed somewhere. You can
check check out to that tag, install the updated dependencies and restart the
program.

```bash
$ git checkout 0.6.9  # latest version as of November 2020
$ .venv/bin/pip install -r requirements.txt
$ systemctl start cps
```

[understanding bibliotecha networking]: #understanding-bibliotecha-networking
[mailing list]: https://we.lurk.org/postorius/lists/bibliotecha.we.lurk.org/
[their release page]: https://github.com/janeczku/calibre-web/releases

## Cloning

We provide a pre-made SD-card for Bibliotecha which you can copy onto your own
SD-card and get moving fast. This can be useful if you don't want to dive into
all the system adminstration work.

!!! warning

    We currently offer a 32 GB image, meaning, if you want to use it, you
    **must also have a 32 GB SD-card**. If you would like other options, please
    get in touch.

Download
[bibliotecha.tar](https://manual.bibliotecha.info/files/bibliotecha.tar). It is
greater than 30 GBs in size, so it could take some time! Then unzip the file
with the following command.

```bash
$ tar cvzf bibliotecha.tar
```

Plug in your own SD-card into your laptop. Run the `lsblk` command and take
note of the path available for the card (e.g. `/dev/mmcblk0`). See the [dd
documentation] for more help.

Start copying the image onto your SD-card with the `dd` command.

```bash
$ sudo dd bs=4M if=bibliotecha.img of=/dev/mmcblk0 status=progress conv=fsync`
```

Then plug your SD-card out of your laptop and plug it into your Raspberry Pi and turn the Pi on.

You should now be able to log into the Pi using SSH or via a screen using the following credentials:

- **username**: pi
- **password**: bibliotecha

Please change your password by running `sudo raspi-config` and choosing `System
Options > Password`.

You can now follow the [post-installation] steps but please note, you will
**not** be required to configure the Calibre-web installation because it is
already done!

[dd documentation]: https://www.raspberrypi.org/documentation/installation/installing-images/linux.md

## Manual installation

It is possible to install Bibliotecha manually. This can be useful and fun if
you would like to learn more about GNU/Linux networking tools and interfaces.
These skills are generally useful but especially so when considering community
networks.

The following guide follows the steps of the automatic installation script.

### Changing to Root

All commands should be run as the root user.

Change the user to the root user with:

```bash
$ sudo -i
```

### Update the System

We should update the system before going further:

```bash
$ apt update
```

This makes sure that we have the latest package listing from the online Debian
package list.

### Install Networking Packages

We then need to install the networking packages that we will need:

```bash
$ apt install -y \
    dhcpcd \
    dnsmasq \
    dnsutils \
    hostapd \
    wireless-tools
```

Afterwards, we'll make sure to stop these services running while we work on the
installation right now. We can do that with:

```bash
$ systemctl stop dnsmasq
$ systemctl stop hostapd
```

We do want these services to be enabled when we reboot though:

```bash
$ systemctl unmask hostapd
$ systemctl enable hostapd
$ systemctl enable dnsmasq
```

We'll also want to disable and stop the `avahi-daemon` which we won't be using
since we rely on `dnsmasq` to handle our DNS configuration and serving:

```bash
$ systemctl stop avahi-daemon
$ systemctl disable avahi-daemon
```

### Configure Network Interfaces

We now need to configure our network interfaces. The network interfaces
correspond to the Ethernet port and the Wireless card. These are how the
Raspberry Pi connect to other devices for networking uses.

We need to learn the names of our network interfaces:

```bash
$ ip a
```

The ethernet interface is the name beginning with "en" and the wireless
interface is the one beginning with "wl". These are the predictable interface
naming conventions which we rely on.

For the following steps, I assume the following:

- Ethernet: enx78e7d1ea46da
- Wireless: wlp2s0

We then configure the ethernet interface. We put the following in
`/etc/network/interfaces.d/enx78e7d1ea46da`:

```bash
auto enx78e7d1ea46da
allow-hotplug enx78e7d1ea46da
iface enx78e7d1ea46da inet dhcp
```

This configuration allows the default behaviour for the Ethernet interface.
When you plug in an ethernet cable, typically coming from your local network
router, you will receive a dynamic IP from that router. This makes it easy to
reach the internet later.

We then configure the wireless interface. We put the following in
`/etc/network/interfaces.d/wlp2s0`:

```
auto wlp2s0
iface wlp2s0 inet static
  address 10.0.0.1
  netmask 255.255.255.0
```

This configuration sets up a static IP address for the wireless interface. We
do this so as to put the IP address within the range of the addresses which we
allow in the local network. We will configure this range in the following step.

### Configure Dnsmasq

In the `/etc/dnsmasq.d/wlp2s0.conf` file, we put the following:

```bash
bogus-priv
server=/library/10.0.0.1
local=/library/
address=/#/10.0.0.1
interface=wlp2s0
domain=library
dhcp-range=10.0.0.50,10.0.0.200,255.255.255.0,12h
dhcp-option=3,10.0.0.1
dhcp-option=6,10.0.0.1
dhcp-authoritative
```

This configuration sets up a local `.library` domain and a range of IP
addresses that can be assigned in the `10.0.0.50` - `10.0.0.200` range. It
also makes sure to resolve all unknown domain requests to the `10.0.0.1` IP.
This is useful for the purposes of the captive portal configuration later on.

### Configure Hostapd

We need to set up the wireless access point too. In the
`/etc/hostapd/hostapd.conf` we add:

```bash
interface=wlp2s0
ssid=Bibliotecha
hw_mode=g
channel=11
auth_algs=1
```

We also need to make sure that `hostapd` uses this configuration. We have to
ensure that the following is uncommented and present in the
`/etc/default/hostapd` file:

```bash
DAEMON_CONF="/etc/hostapd/hostapd.conf"
```

### Configure the Hosts file

We need to register the library on the network. In the `/etc/hosts` file we put
the following at the end of the file after all the other entries:

```bash
10.0.0.1        bibliotecha.library
```

### Install and Configure Lighttpd

Moving on, we should install the web server which will respond to network
requests and return the web pages of the library interface. We can install
Lighttpd with:

```bash
$ apt install -y lighttpd
```

When then need to make sure that the following configuration is available in
the `/etc/lighttpd/lighttpd.conf` file:

```bash
server.document-root = "/var/www"
server.modules += ("mod_proxy",)
include "bibliotecha/bibliotecha.conf"
```

When then create the Bibliotecha configuration with:

```bash
$ mkdir /etc/lighttpd/bibliotecha
```

And then make sure the following is in the
`/etc/lighttpd/bibliotecha/bibliotecha.conf` file:

```bash
server.error-handler-404 = "/bibliotecha/index.html"
$HTTP["host"] == "bibliotecha.library" {
  proxy.server = ("" => (("host" => "127.0.0.1", "port" => "8083")))
}
```

This ensures that all unknown requests are pointed to the captive portal page
of Bibliotecha. And when we request the `http://bibliotecha.library` domain, we
are then pointed to the Calibre-web installation.

### Install and Configure Calibre

Calibre is responsible for maintaining the underlying database of the library. We can install it with:

```bash
$ apt install -y calibre --no-install-recommends
```

This will take some time as there are many required packages. Once it is
finished, you will then need to create a new database for the Calibre-web
installation to use.

We should run the following:

```bash
$ mkdir /opt/calibre-database
$ /usr/bin/calibredb restore_database --really-do-it --with-library /opt/calibre-database
```

### Configure the Captive Portal

When we connect to the Bibliotecha wireless access point, we will be directed
to a splash page where we are introduced to the library.

First make sure we have all dependencies:

```bash
$ apt install -y wget git ghostscript python3-venv
```

Then create a folder for our splash page:

```bash
$ mkdir /var/www/bibliotecha
```

And then we download the default page into location:

```bash
$ wget https://git.vvvvvvaria.org/varia/bibliotecha-captive-portal/raw/branch/master/index.html -O /var/www/bibliotecha/index.html
```

We should also ensure that the correct ownership permissions are configured:

```bash
$ chown -R www-data: /var/www/bibliotecha
```

### Install and Configure Calibre-web

We now setup the Calibre-web installation. We first get a copy of the source with:

```bash
$ mkdir /var/www/calibre-web
$ git clone https://github.com/janeczku/calibre-web /var/www/calibre-web
$ cd /var/www/calibre-web
```

Then, looking at the [Calibre-web release page], we can choose a version we
would like to install. As of November 2020, the latest version is 0.6.9, so
we can check that version out with the following:

```bash
$ git fetch origin master -a
$ git tag -l
$ git checkout 0.6.9
```

We then need to install the Python dependencies with:

```bash
$ python3 -m venv .venv
$ .venv/bin/pip install -r requirements.txt
```

And finally configure the service to be run by Systemd.

Create the file `/etc/systemd/system/cps.service` and add these lines to it:

```bash
Description=Calibre-Web

[Service]
Type=simple
User=root
ExecStart=/var/www/calibre-web/.venv/bin/python /var/www/calibre-web/cps.py
WorkingDirectory=/var/www/calibre-web

[Install]
WantedBy=multi-user.target
```

And we also enable this to run on reboot:

```bash
$ systemctl enable cps.service
```

We should also ensure that the correct ownership permissions are configured:

```bash
$ chown -R www-data: /var/www/calibre-web
```

[calibre-web release page]: https://github.com/janeczku/calibre-web/releases

### Setup a new MOTD

When you SSH into the Raspberry Pi, you can enable a nice welcome message.

This is possible bu putting the following in the `/etc/motd/` file:

```
  ____  _ _     _ _       _            _
 |  _ \(_) |   | (_)     | |          | |
 | |_) |_| |__ | |_  ___ | |_ ___  ___| |__   __ _
 |  _ <| | '_ \| | |/ _ \| __/ _ \/ __| '_ \ / _` |
 | |_) | | |_) | | | (_) | ||  __/ (__| | | | (_| |
 |____/|_|_.__/|_|_|\___/ \__\___|\___|_| |_|\__,_|

       Digital books need libraries too
```

### Conclusion

That's it! You should now reboot your Raspberry Pi with:

```bash
$ reboot
```

You can now follow the [post-installation] steps.

[post-installation]: #post-installation

## Contributing

Bibliotecha is made up of the following projects:

- [bibliotecha-install](https://git.vvvvvvaria.org/varia/bibliotecha-install)
- [bibliotecha-manual](https://git.vvvvvvaria.org/varia/bibliotecha-manual)
- [bibliotecha-captive-portal](https://git.vvvvvvaria.org/varia/bibliotecha-captive-portal)

All contributions are welcome!

You can also find us on the [mailing list].

[mailing list]: https://we.lurk.org/postorius/lists/bibliotecha.we.lurk.org/

## Acknowledgements

- The [Calibre] project
- The [Calibre-web] project

[calibre]: https://calibre-ebook.com
[calibre-web]: https://github.com/janeczku/calibre-web

Contributors to Bibliotecha have been Yoana Buzova, Lasse van den Bosch
Christensen, Andre Castro, Lucia Dossin, Max Dovey, Michaela Lakova, Luke
Murphy and Roel Roscam Abbing