Discourse is a web platform for hosting and moderating online discussion.

The Tor forum is currently hosted free of charge by Discourse.org for the benefit of the Tor community.

[[TOC]]

Tutorial

Enable new topics by email

Topic creation by email is the ability to create a new forum topic in a category simply by sending an email to a specific address.

This feature is enabled per-category. To enable it for a category, navigate to it, click the "wrench" icon (top right), open the Settings tab and scroll to the Email header.

Enter an email address under Custom incoming email address. The address to use should be in the format <categoryname>+discourse@forum.torproject.org.

Per the forum's settings, only users with trust level 2 (member) or higher are allowed to post new topics by email.

Use the app

The official companion app for Discourse is DiscourseHub.

Unfortunately, it doesn't appear to be available from the F-Droid repository at the moment.

Mirror a mailing list

The instructions to set up a forum category that mirrors for a mailing list can be found here.

The address that needs to be subscribed to the mailing list is discourse@forum.torproject.org.

How-to

Launch the Discourse Rails console

Log-in to the server's console as root and run:

cd /srv/discourse
./launcher enter app
rails c

Reset a user's two-factor auth settings

In case a user can't log-in anymore due to two-factor authentication parameters, it's possible to reset those using the Rails console.

First, load the user object by email, username or id:

user=User.find_by_email('email')
user=User.find_by_username('username')
user=User.find(id)

Then, simply run these two commands:

user.user_second_factors.destroy_all
user.security_keys.destroy_all

These instructions are copied from this post on the Discourse Meta forum.

Reset a user account password

Usually when there is a need to reset a user's password, the user can self-service through the forgotten password forum.

In case of issues with email, the password can also be reset from the Rails console:

First, load the user object by email, username or id:

user=User.find_by_email('email')
user=User.find_by_username('username')
user=User.find(id)

Then:

user.password='passwordstring'
user.save!

These instructions are copied from this post on the Discourse Meta forum.

Adding or removing plugins

The plugins installed on our Discourse instance are configured using Puppet, in hiera/role/forum.yaml.

To add or remove a plugin, simply add/remove the repository URL to the profile::discourse::plugins key, run the Puppet agent and rebuild the container:

./launcher rebuild app

This process can take a few minutes, during which the forum is unavailable.

Discourse has a plugins directory here: https://www.discourse.org/plugins

Un-delete a topic

As an admin user, the list of all deleted topics may be shown by navigating to https://forum.torproject.org/latest?status=deleted

Tu un-delete a topic, open it, click the wrench icon and select Un-delete topic.

Permanently destroy a topic

If a topic needs to be purged from Discourse, this can be accomplished using the Rails console as follows, using the numeric topic identifier:

Topic.find(topic_id).destroy

These instructions are copied from this post on the Discourse Meta forum.

Enter the Discourse container

It's possible to enter the Discourse container to look around, make modifications, and restart the Discourse daemon itself.

cd /srv/discourse
./launcher enter app

Any changes made in the container will be lost on upgrades, or when the container is rebuilt using ./launcher rebuild app.

Within the container its possible to restart the Discourse daemon using:

sv restart unicorn

Read-only mode

It's possible to enable "read-only" mode on the forum, which will prevent any changes and will block and new topic, replies, messages, settings changes, etc.

To enable it, navigate to the Admin section, then Backups and click the button labeled Enable read-only.

It's also possible to enable a "partial read-only" mode which is like normal "read-only" except it allows administrators to make changes. Enabling this mode must be done via the rails console:

Discourse.enable_readonly_mode(Discourse::STAFF_WRITES_ONLY_MODE_KEY)

To disable it:

Discourse.disable_readonly_mode(Discourse::STAFF_WRITES_ONLY_MODE_KEY)

The documentation for this feature is found at https://meta.discourse.org/t/partial-read-only-mode/210401/18

Access database

After entering the container, this command can be used to open a psql shell to the discourse PostgreSQL database:

sudo -u postgres psql discourse

Mass-disable email digests

If a user's account email address stops working (eg. domain becomes unregistered), and email digests are enabled (the default) Discourse will keep attempting to send those emails forever, and the delivery of each single email will be retried dozens of times, even if the chance of delivery is zero.

To disable those emails, this code can be used in the rails console:

users=User.all.select { |u| u.email.match('example.com') }
users.each do |u|
  u.user_option.email_digests = false
  u.user_option.save
end

Pager playbook

Email issues

If mail is not going out or some recurring background job doesn't work, see the Sidekiq dashboard in:

https://forum.torproject.org/sidekiq/

Email failures, in particular, are retried for a while, you should be able to see those failures in:

https://forum.torproject.org/sidekiq/retries

Dashboard warns about failed email jobs

From time to time the Discourse dashboard will show a message like this:

There are 859 email jobs that failed. Check your app.yml and ensure that the mail server settings are correct. See the failed jobs in Sidekiq.

In the Sidekiq logs, all the failed job error messages contain Recipient address rejected: Domain not found.

This is caused by some user's email domain going dark, but Discourse keeps trying to send them the daily email digest. See the Mass-disable email digests section for instructions how to disable the automatic email digests for these users.

Upgrade failure

When upgrading using the web interface, it's possible for the process to fail with a Docker Manager: FAILED TO UPGRADE message in the logs.

The quickest way to recover from this is to rebuild the container from the command-line:

 cd /srv/discourse
 git pull
 ./launcher rebuild app

PostgreSQL upgrade not working

The upgrade script may not succeed when upgrading to a newer version of PostgreSQL, even though it reports success. In the upgrade log, this message is logged:

 mv: cannot move '/shared/postgres_data' to '/shared/postgres_data_old': Device or resource busy

This is caused by a particularity in our deployment because postgres_data is a mount point, so attempts to move the directory fails.

A patch to workaround this was submitted upstream and merged.

Disaster recovery

In case the machine is lost, it's possible to restore the forum from backups.

The first step is to install a new machine following the installation steps in the Installation section below.

Once a blank installation is done, restore the Discourse backup directory, /srv/discourse/shared/standalone/backups/default, from Bacula backups.

The restoration process is then:

 cd /srv/discourse
 ./launcher enter app
 discourse enable_restore
 discourse restore <backupfilename>.tar.gz
 exit

Once that's done, rebuild the Discourse app using:

./launcher rebuild app

Reference

Installation

Our installation is modeled after upstream's recommended procedure for deploying a single-server Docker-based instance of Discourse.

First, a new machine is required, with the following parameters:

  • an 80GB SSD-backed volume for container images and user uploads
  • a 20GB NVMe-backed volume for the database

Directories and mounts should be configured in the following manner:

  • the SSD volume mounted on /srv
  • /srv/docker bind mounted onto /var/lib/docker

When this is ready, the role::forum Puppet class may be deployed onto the machine. This will install Discourse's Docker manager software to /srv/discourse along with the TPO-specific container templates for the main application (app.yml) and the mail bridge (mail-receiver.yml).

Once the catalog is applied, a few more steps are needed:

  1. Bootstrap and start Discourse with these commands:

    cd /srv/discourse ./launcher bootstrap app ./launcher start app

  2. Login to https://forum.torproject.org and create a new admin account

  3. Create an API key using the instructions below

  4. Run the Puppet agent on the machine to deploy the mail-receiver

API key for incoming mail

Our Discourse setup relies on Postfix to transport incoming and outgoing mail, such as notifications. For incoming mail, Postfix submits it to a special mail-receiver container that is used to deliver email into Discourse via its web API. A key is needed to authenticate the daemon running inside the container.

To create and configure the API key:

  1. Login to Discourse using the administrator account

  2. Navigate to https://forum.torproject.org/admin/api/keys

  3. Click the New API Key button

  4. In the Description write Incoming mail, for User Level select All Users and for Scope select Granular

  5. Locate email under topics and check the box next to receive emails

  6. Click Save

  7. Copy the generated key, then logon to the Puppet server run this command to enter the API key into the database:

    trocla set forum.torproject.org::discourse::mail_apikey plain

Upgrades

When versioned updates are available, an email is sent automatically by Discourse to torproject-admin@torproject.org.

These upgrades must be triggered manually. In theory it would be possible to upgrade automatically, but this is discouraged by community members because it can throw up some excitement every now and again depending on what plugins you have.

To trigger an upgrade, simply navigate to the Upgrade page in the Discourse admin section and hit Upgrade all, then Start Upgrade.

Sometimes, this button is greyed out because an upgrade for docker_manager is available, and it must be installed before the other components are upgraded. Click the Upgrade button next to it.

Discourse can also be upgraded via the command-line:

cd /srv/discourse
./launcher rebuild

Onion service

An onion service is configured on the machine using Puppet, listening on ports 80 and 443.

Internally, Discourse has a force_https setting which determines whether links are generated using the http or https scheme, and affects CSP URLs. When this is enabled, the forum does not work using the onion service because CSP URLs in the headers sent by Discourse are generated with the https scheme. When the parameter is disabled, the main issue is that the links in notifications all use the http scheme.

So the most straightforward fix is simply to serve the forum via https on the onion service, that way we can leave the force_https setting enabled, and the CSP headers don't prevent forum pages from loading.

Another element to take into account is that Discourse forces the hostname as a security feature. This was identified as an issue specifically affecting forums hosted behind .onion services in this meta.discourse.org forum post.

While the solution suggested in that forum discussion involves patching Discourse, another workaround was added later on in the form of the DISCOURSE_BACKUP_HOSTNAME container config environment variable. When set to the .onion hostname, the forum works under both hostnames without issue.

Directory structure

The purpose of the various directories under /srv/discourse is described in the discourse_docker README.

The most important directories are:

  • containers: contains our Docker container setup configurations
  • shared: contains the logs, files and Postgresql database of the forum

Social login configuration

GitHub

To enable GitHub authentication, you will need the github_client_id and github_client_secret codes. Please refer to the the official Configuring GitHub login for Discourse documentation for up to date instructions.

Follow these steps to enable GitHub authentication:

  1. Visit https://github.com/organizations/torproject/settings/applications.
  2. Click on "New Org OAuth App" or edit the existing "Tor Forum" app.

  3. Follow the official instructions: https://meta.discourse.org/t/13745, or add the following configuration:

    Application name: Tor Forum Homepage URL: https://forum.torproject.org/ Authorization callback URL: https://forum.torproject.org/auth/github/callback

  4. Copy the github_client_id and github_client_secret codes and paste them into the corresponding fields for GitHub client ID and GitHub client secret in https://forum.torproject.org/admin/site_settings/category/login

Design

Docker manager

The Discourse Docker manager is installed under /srv/discourse and is responsible for setting up the containers making up the Discourse installation.

The containers themselves are stateless, which means that they can be destroyed and rebuilt without any data loss. All of the Discourse data is stored under /srv/discourse/shared, including the Postgresql database.

Issues

There is no issue tracker specifically for this project, File or search for issues in the team issue tracker.

Maintainer, users, and upstream

Upstream is Discourse.org.

This service is available publicly for the benefit of the entire Tor community.

The forum hosted on TPA infrastructure and administered by the service admins which are lavamind, hiro, gus and duncan.

Monitoring and testing

Only general monitoring is in place on the instance, there is no Discourse-specific monitoring in place.

Logs and metrics

Logs for the main Discourse container (app) are located under /srv/discourse/shared/standalone/log.

The mail-receiver container logs can be consulted with:

/srv/discourse/launcher logs mail-receiver

Note that this is strictly for incoming mail. Outgoing mail is delivered normally through the Postfix email server, logging in /var/log/mail.log*.

In addition, some logs are accessible via the browser at https://forum.torproject.org/logs (administrators-only).

An overview of all logging is available on this page: Where does Discourse store and show logs?

Backups

Backups containing the database and uploads are generated daily by Discourse itself in /srv/discourse/shared/standalone/backups.

All other directories under /srv/discourse/shared/standalone are excluded from Bacula backups configured from /etc/bacula/local-exclude.

It's possible to manually trigger Discourse to create a backup immediately by entering the container and entering discourse backup on the command-line.

Other documentation

  • https://meta.discourse.org/