[[TOC]]

Tutorial

How to write a new blog post in GitLab

The steps below describe how to create a new blog post directly via GitLab using the Web IDE for users unfamiliar with the git command-line client.

Users comfortable with git are free to create and push a new branch to the repository directly, without using the Web IDE. In this case a Merge Request should still be created to allow the review process to occur. If you opt to work in this way, you will need to have git-lfs installed on your computer and have run git lfs install at least once before you start working.

Enter the Web IDE

1. Navigate to the GitLab blog project at https://gitlab.torproject.org/tpo/web/blog

• We'll create a new branch in the repository, so your account will require Developer permissions in the project

• If you're missing the permissions, please ask TPA.

• If you choose to fork the project in your personal GitLab namespace, CI will instead deploy the build to GitLab Pages, and the View app button described below will not appear on the Merge Request page. In this case, to view the build, open the fork project and open Settings -> Pages and click the URL under Access pages.

2. Navigate to Code ➡ Branches via the left-hand menu

3. Pick a name for the new branch

• The name isn't very important, it will not persist after the changes are merged because the branch will be deleted.

• In Create from, pick main.

4. Click the Web IDE button just above the file listing

• Just above the file listing, under the Code dropdown menu choose the WebIDE item. (or from the page with the file listing, you can use the keyboard shortcut .)

  • The GitLab Web IDE allows you to create a new commit right in your browser, without the git command-line client

Create the content folder / post slug

5. In the left-side navigation column, uncollapse content / blog

• Each folder corresponds to a blog post and its contents

• Folder names become post slugs during the build process

6. Click the New Folder... icon and type in <blog-post-slug>

• <blog-post-slug> will constitute the URL part after https://blog.torproject.org/ (also called the permalink)

Edit the content file

7. Still in the navigation column, click the New File... icon

8. Enter contents.lr as the new file name

• This file must contain a number of fields which will allow the build system to render the blog post

9. In the text editor which appears, draft the new blog post

• Required fields are title, author, pub_date, summary and body and categories (see Sample contents.lr section below)

• Markdown is accepted in summary and body fields

• The summary field is only displayed in the sidebar and index pages, its content is not displayed on the blog post page

• At least one category must be chosen from the list

• You can preview the formatting of various Markdown elements here (source)

Upload images and/or attachments

10. Upload the lead image and any other images or attachments in the same directory

• To upload a file, right-click on the directory in the navigation panel and choose Upload... -- or you can instead drag and drop from your file manager

• The lead image must be uploaded as lead.png, lead.jpg or lead.gif

• A default Tor logo image will be displayed if the lead image is omitted

• Image widths should not exceed 1000 pixels

• If you can, please compress your PNG files using a tool such as zopflipng

• Don't worry about uploading duplicate images from other posts, as they will be deduplicated in GitLab (just make sure not to modify them unless necessary)

• To use such uploaded images inline in the post body, use markup such as ![Image Alt-Text](image-filename.png)

• Extra attributes may be added to an image's HTML markup, eg. ![Image Alt-Text](image-filename.png "class=left")

• To add a hyperlink to an uploaded file such as a PDF, the markup would be [Link Text](document-filename.pdf)

• If you need to replace an existing image (those are tracked in LFS) using the WebIDE, first create a commit to delete the file, then a second one to upload the new version. This is a workaround for bug #217828 in GitLab.

Create and push a new commit and create a Merge Request

11. When finished, open the Source Control panel

• Look for a branching icon on the left-hand side with the purple notification dot

• Enter a short commit message, eg. Adding 2022 GSoC blog post

• Click Commit & Push

• A dialog will pop up near the top of the page asking if a new branch should be created, click No

12. Click Create MR in the dialog that appears in the bottom right

• If the dialog disappears before you can click on the Create MR button, you can see the dialog again by clicking on the bell icon at the bottom right of the editor.

• The IDE will open a new tab in your navigator to the New merge request page

• Prefix the Merge request title with Draft: to indicate the post is not yet ready to be merged

• Within a few minutes, GitLab CI should build and upload the draft version online

• Click the View app to open a browser tab to the preview URL, once the preview build is finished, it will appear at this URL

13. Add more commits if any changes or adjustments are needed

• New commits on the branch will automatically trigger a new preview build and deployment

14. Once the blog post is finalized and ready for publishing, click Mark as ready

• If the blog post is ready, but should be published at a later date, it should remain in the Draft state in order to prevent merging

• Once the merge request is reviewed and accepted by a project maintainer, a full project build will be triggered and deployed to https://blog.staging.torproject.net/

• If the staging build passes QA, a deployment to production must be triggered manually via the Pipelines or Environments project pages

Comment topic

15. When the blog post is published, visit the page with a javascript-enabled browser

• This will trigger the Discourse forum to create a new topic with the contents of the blog post in the News category.

• Adjustments to the topic source HTML might be required, notably to allow the forum to display images correctly (known Discourse bug being worked on)

How-to

Pager playbook

New blog post appears below older one

Blog posts can include a weight field to help ordering posts that are published on the same day.

By default all posts have a weight of "0". Posts which have a higher weight are sorted before those with a lower weight.

Roll-back to pevious version

In case an undesirable change has been accidentally deployed to production, it's possible to trigger a roll-back of the environment to the previous state. To do this, navigate to the Environments page, identify which version to roll-back to and click the corresponding Rollback environment button in the left-most column.

Please note that it's not possible to roll-back to an environment that was built more than 1 week ago, because environments depend on build artifacts and those are configured to expire after 1 week.

Emergency modification

To modify the site in an emergency, it's possible to bypass GitLab completely and edit the site HTML directly. The steps to do so are described here:

1. Log-in to static-gitlab-shim.torproject.org
2. cd /srv/static-gitlab-shim/blog.torproject.org/public
3. Edit the website sources files directly, in-place
4. Execute sudo -u mirroradm static-update-component blog.torproject.org

This will immediately deploy changes to the web mirrors.

Please note that using the technique, any changes made henceforth to the GitLab project repository and deployed to production will overwrite these modifications.

Disaster recovery

See howto/static-component#disaster-recovery.

Reference

Installation

See service/static-shim#adding-a-new-static-site-shim-in-puppet.

SLA

See howto/static-component#sla.

Design

Sample contents.lr

title: New Tor Blog
---
author: lavamind
---
pub_date: 2021-08-17
---
categories:

announcements
---
summary: The new Tor Blog is here!
---
body:

The new Tor Blog is here!

The new Tor Blog now runs on top of the Lektor static site generator.

Authors

If the post is written by a guest author, use the guest_author field in addition to the required author field. The guest_author will be given precedence in the rendering of the blog post.

If the post has multiple authors, simply list them in the same format as categories, for example:

---
authors:

alsmith
arturom
---

List of categories

In the process of migrating the blog from Drupal to Lektor, the number of tags has been reduced to 20 (from over 970 in Drupal). For details about this work, see tpo/web/blog#40008

The items below may now be used in the categories field:

areas of work	topics	operations
circumvention	advocacy	jobs
network	releases	fundraising
applications	relays	announcements
community	human rights	financials
devops	usability
research	reports
metrics	onion services
tails	localization
	global south
	partners

When drafting a new blog post, a minimum of one category must be chosen, with a suggested maximum of three.

Compress PNG files

When care is taken to minimize the size of web assets, accessibility and performance is improved, especially for visitors accessing the site from low bandwidth connections or low-powered devices.

One method to achieve this goal is to use a tool to compress lossless PNG files using zopflipng. The tool can be installed via apt install zopfli. To compress a PNG image, the command may be invoked as such:

zopflipng --filters=0me -m --prefix lead.png

This command will process the input file and save it as zopfli_lead.png. The output message will indicate if the image size was reduced and if so, by what percentage.

Comments embedding

When a new blog post is published, a javascript snippet included on the page will trigger the Discourse forum to create a new topic in the News category with the contents of the new post. In turn, replies to the forum topic will appear embedded below the blog post.

The configuration for this feature on the Discourse side is located in the Admin section under Customize -> Embedding

The key configuration here is CSS selector for elements that are allowed in embeds. Without the appropriate CSS selectors listed here, some parts of the blog post may not be imported correctly. There is no documentation of how this parameter works, but through trial and error we figured out that selectors must be one or two levels "close" to the actual HTML elements that we need to appear in the topic. In other words, specifying main article.blog-post as a selector and hoping that all sub-elements will be imported in the topic doesn't work: the sub-elements themselves must be targeted explicitly.

Issues

There is the tpo/web/blog project for this service, File or search for issues in the issue tracker.

Maintainer, users, and upstream

This website is maintained collaboratively between the TPA web team and the community team. Users of this service are the general public.

Monitoring and testing

For monitoring, see howto/static-component#monitoring-and-testing.

There are no automated tests such as spellchecks or dead link checking for this service. In case of malformed Lektor content files, the build job will fail.

Logs and metrics

See howto/static-component#logs-and-metrics.

Backups

Backups of this website exist both in the Bacula backups of the GitLab server (as artifacts) and backups of the static-gitlab-shim.torproject.org server. See the static components disaster recovery procedures for how to restore a site.

Other documentation

• howto/static-component
• service/static-shim
• Lektor documentation

Discussion

Drupal to Lektor migration

The Tor Project runs a blog since 2007. It's used to provide an official source of news to the community regarding software releases, fundraising, events and general Tor Project updates. However, there are several outstanding issues with the current site, including problems with comment moderation which are not easily fixed using Drupal:

• Hosting the Drupal site at a third party is a significant expense
• Technical maintenance of the blog is a challenge because upstream upgrades frequently cause breakage
• Posts must be drafted with a clunky Javascript HTML editor instead of Markdown
• Moderation is a chore for port authors, causing comments to sometimes linger in the moderation queue

It has been decided to migrate the site to a SSG (static site generator). This is currently listed as Need to have in the 2021 TPA roadmap. (The option to fix the Drupal site was on the table for a short while, but is now abandoned.)

Goals

We should migrate the site to an SSG as soon as possible.

Must have

• Migration of existing blog post and events content (title, author, date, images)
• Preservation of existing URLs (both aliases and node/* paths)
• RSS/Atom feed for blog posts and events
• Ability to edit migrated content if required (posts and events)
• Ability to comment blog posts (externally)

Nice to have

• Migration and continued use of existing blog post tags
• Straightforward content migration
• Archive of existing blog post comments (see rationale here)
• Web-based "admin" interface
• RSS/Atom feeds per-tag and per-author
• Styleguide compliant template already exists (Lektor, Hugo)

Non-goals

• Author or admin-moderated comments

Proposed solution

Migrate the site to Lektor, which is already used for https://www.torproject.org, and implement a Discourse instance for discussions, as a replacement for blog comments. This was the solution retained by @hiro for this project, as documented in https://gitlab.torproject.org/tpo/web/blog-trac/-/issues/33115.

There are two options for using Discourse as a blog comments platform:

Embedded

Using an embedded-Javascript snippet added in the site template, as documented here. When a blog post page is opened, the Javascript loads the corresponding topic on the Discourse site. New topics are added to Discourse automatically when new posts are created.

• Pro: comments are visible on the blog, no need to visit/open another site
• Pro: comments can be posted to the Discourse topic directly from within the blog
• Con: posting comments requires a Discourse account
• Con: requires Javascript

RSS/Atom feed polling

A Discourse plugin can be configured to poll the blog website RSS/Atom feed at regular intervals and create new topics automatically when a new post is published. It's possible we can predict Discouse topic URLs so that Lektor can generate the required link in the template and insert it at the bottom of blog posts (eg. a "Click here to join the discussion"-type link)

• Pro: no Javascript required on the blog
• Pro: comments not visible directly on the blog
• Con: comments not visible directly on the blog

Alternatives considered

Note that we settled on using Lektor for the blog, and Discourse as a comment backend. Those options are therefore not relevant anymore.

• Hugo is another friendly SSG, and a Tor styleguide has been made for it, however its preferable to avoid using different web stacks unless there's a compelling reason for it. There's only one known Drupal migration script by it appears to have been created for Drupal 7 and seems unmaintained. In any case it's "assembly required" which isn't much different from hacking a quick script to migrate to Lektor instead.
• Discourse might also be an option to completely replace the blog: we could configure https://blog.torproject.org to show content from a specific topic on Discourse. The challenge is that importing content is not as straightforward compared to a SSG where we just need to write text files. Maintaining existing URLs could also be a challenge and would require some form of redirect mapping on blog.torproject.org. We would also lose the flexibility to add standalone pages or other forms of content on the blog, ~~such as a calendar view of events~~ event calendar plugin. (example)