Have you ever had a repetitive task, which involves downloading files from one location, and uploading them to another? Are there "computer chores" in your routine, which simply consists of being the intermediary between two different online services? Whether it's entering spreadsheets from one program into another, or catching the output of one website and reformatting it for another tool – the proliferation of online services which do not talk to each other means that humans are left to fill the gaps. But we are more than glorified data-pipelines: to shuttle files from one place to another is the characteristic activity of an USB, not a human being. Thankfully, we can automate these tasks away using workflow automation software.

My name is Shen, and I am a Philosophy student. In this tutorial, I showcase n8n: an open source (Github link), self-hosted workflow automation platform. I will walk us through the process of authenticating with a REST API, setting up Webhooks, and finally the automation code in n8n itself. Although this tutorial is specifically concerned with two tools (Firefly III and Paperless-ngx), the general concepts are universal and broadly applicable. By automating the mundane, repetitive tasks of uploading receipts, I can devote more time to Philosophy. And it is my hope that by sharing this knowledge, you too will find more leisure to devote to contemplation.

My Receipt Management Workflow

I use two tools regularly to keep track of my spending, and to manage paper documents in general:

• Firefly III: an open source, self-hosted personal finance tool. I use this to keep track of my financial transactions and daily spending.
• Paperless-ngx: an open source, self-hosted document management platform. It offers powerful search, categorisation, and OCR capabilities that I utilise to manage all my paper documents.

Whenever I get home from the day, I scan and digitise the receipts that I have accrued. Then I enter their transaction information into Firefly: adding the description and value, before finally attaching the PDF of the relevant receipt into the transaction. Afterwards, I upload again the PDFs into Paperless, making sure once again to give them the correct description. Having to upload and enter the transaction information twice was repetitive and time consuming, and I quickly started wondering if there was some way to have the two systems talk to each other.

Was there any way to integrate Firefly with Paperless-ngx? What could I do, so that every time I uploaded a new transaction on Firefly, it automatically sent the information and attached PDF to Paperless-ngx?

Both Firefly and Paperless-ngx are open source tools, with well-documented APIs. Surely it wouldn't be too difficult to create an integration, I thought. My first approach was to open a suggestion on the Firefly III Github page. James Cole, the maintainer of Firefly III responded, declining to add a native integration due to maintenance concerns. However, he encouraged me to build my own – and with that suggestion, I stumbled upon n8n.

Why I Choose to Use n8n

n8n (pronounced as "n-eight-n") is an open source workflow automation tool. It allows one to "glue" together different services, automating the transfer of information between tools. n8n uses a visual, flow-chart-like metaphor to allow users to easily create workflows (i.e. scripts). n8n is essentially the open source version of Zapier or IFTTT. I decided to use n8n instead of an ad-hoc setup with shell scripts and crontab entries because I wanted something easy to use and understandable at a glance. After all, if you spend much more time automating a task than you'd spend performing it, your efforts would be inefficient. Once I realised it was the tool for me, I quickly installed it on to my server.

Creating an n8n workflow for automatic receipt management

How was it like creating the automation workflow? In my case, there were four steps:

1. Generate the API authentication tokens for both services.
2. Register Webhooks on Firefly III using the Firefly API.
3. Creating the automation workflow on n8n
4. Testing, and deploying the workflow to production.

Generating Authentication Tokens

Generally, the first step towards any workflow automation, is to get access to the API of the service(s) that you are trying to use. There is usually some means to authenticate yourself to the API. In my case, both Firefly III and Paperless-ngx support the use of token-based authentication. An API token is essentially a long, computer-memorised password that must be sent along with API requests. Be sure to keep these tokens safe – they allow the bearer to access your account!

We will need API tokens from both Firefly III and Paperless-ngx in order to interact with their APIs. In fact, the next step is for us to register webhooks using the Firefly III API directly.

Registering and Managing Webhooks with Firefly III

Webhooks are the foundation of our automation workflow. A webhook is like a custom function callback: you can give Firefly a special URL that it will send data to upon a certain trigger. The Firefly III documentation page on Webhooks has a more extensive description of the functionality available. We will be creating two webhooks, using the following triggers:

• TRIGGER_STORE_TRANSACTION : For when we store a new transaction.
• TRIGGER_UPDATE_TRANSACTION: For when an existing transaction is updated.

Each time a transaction is added or updated, we want Firefly to take a certain action: namely send the data of the transaction involved to our Webhook URL. This action is defined in Firefly as:

• RESPONSE_TRANSACTIONS: Respond to the Webhook URL with the transaction that was added/updated.

Where do we get the Webhook URL? These are provided by n8n! Now is the time to create a new n8n Workflow. Be sure to create two Webhook nodes using the graphical GUI. We will need two, as we want our Workflow to be triggered both when a new transaction is added, and when an existing transaction is updated:

Each of these webhook nodes will give you a unique webhook URL. Save those. Make sure to use the Test URL for now instead of the production URL, while we still build the workflow. With these URLs, it is now time to register the webhooks with Firefly III.

Most applications have an admin page where you will be able to register new webhooks. Unfortunately as of 2022-06-09, Firefly III does not have such a page. Instead, in order to register new webhooks, we must interact directly with it's API. We will be using the CURL command to do so, via the command line. You'll need the following information handy:

• [Firefly III domain]: The domain name of your Firefly III installation.
• [Auth Token] The API Authentication Token for Firefly III.
• [Webhook URL]: The URL from your webhook nodes. Make sure to use the test URL, I will show you how to switch them later.

Registering Webhooks

We will be registering two webhooks, one for TRIGGER_STORE_TRANSACTION and one for TRIGGER_UPDATE_TRANSACTION, respectively. Here is the command to register the first one:

curl -X POST 'https://[Firefly III domain]/api/v1/webhooks' \
  -H 'accept: application/vnd.api+json' \
  -H 'Authorization: Bearer [Auth Token] \
  -H 'Content-Type: application/json' \
  -d '{
  "active": true,
  "title": "Paperless-ngx Integration (via n8n) On New",
  "trigger": "TRIGGER_STORE_TRANSACTION",
  "response": "RESPONSE_TRANSACTIONS",
  "delivery": "DELIVERY_JSON",
  "url": "[Webhook #1 URL]"
}'

And the command to register the second one:

curl -X POST 'https://[Firefly III domain]/api/v1/webhooks' \
  -H 'accept: application/vnd.api+json' \
  -H 'Authorization: Bearer [Auth Token] \
  -H 'Content-Type: application/json' \
  -d '{
  "active": true,
  "title": "Paperless-ngx Integration (via n8n) On Update",
  "trigger": "TRIGGER_UPDATE_TRANSACTION",
  "response": "RESPONSE_TRANSACTIONS",
  "delivery": "DELIVERY_JSON",
  "url": "[Webhook #1 URL]"
}'

Getting List of Existing Webhooks

Once you have registered both webhooks successfully, you should be able to query the /api/v1/webhooks endpoint to get a list for all of them. Here is the command to list all webhooks.

curl -X GET 'https://[Firefly III domain]/api/v1/webhooks' \
  -H 'accept: application/vnd.api+json' \
  -H 'Authorization: Bearer [Auth Token]' \
  -H 'Content-Type: application/json'

Every successful command will return information in the form of an minified json string. You can "prettify" the information using an online tool like json prettify, which will make it easier to read. Next, I will also go through some other useful API calls to manage these webhooks, for future reference.

Updating Existing Webhooks

In order to update an existing webhook, you must make a PUT request towards the correct {id} of the webhook that you want to modify. You can get webhook IDs by using the previous API call to list webhooks.

curl -X PUT 'https://[Firefly III domain]/api/v1/webhooks/{id}' \
  -H 'accept: application/vnd.api+json' \
  -H 'Authorization: Bearer [Auth Token] \
  -H 'Content-Type: application/json' \
  -d '{
  "active": true,
  "title": "Paperless-ngx Integration (via n8n) On New",
  "trigger": "TRIGGER_STORE_TRANSACTION",
  "response": "RESPONSE_TRANSACTIONS",
  "delivery": "DELIVERY_JSON",
  "url": "[Webhook URL]"
}'

Whatever values you put in -d '{ ... }' field will overwrite existing values. Keep this update command in mind, as we will be using it to set the url to the production webhook URLs later.

Deleting Webhooks

In order to delete a webhook, you simply make a DELETE request towards the {id}.

curl -X DELETE 'https://[Firefly III domain]/api/v1/webhooks/{id}' \
  -H 'accept: application/vnd.api+json' \
  -H 'Authorization: Bearer [Auth Token] \
  -H 'Content-Type: application/json'

Saving authentication tokens from Firefly and Paperless-ngx into n8n

Now that we registered the webhooks, Firefly III is able to send information to n8n. But right now, n8n has no way to send or retrieve information from Firefly, nor from Paperless-ngx. In order for n8n to use their respective APIs, we must first save our authentication credentials into n8n. To do this, go to the Credentials tab on the left, and click New. n8n offers some pre-built authentication types to work with popular services, but it does not have one for Firefly III nor Paperless-ngx yet. Hence, we'll use the generic Header Auth credential type.

n8n's generic Header Auth credential is a reference to authenticating a HTTP request by using a HTTP Header. Any manner of header can be used, but for both Firefly III and Paperless-ngx, they use the standard HTTP Authorization Header.

Go ahead and name your credential appropriately (e.g. "Firefly III Header Auth"), and add Authorization as the name of the Header used. The value has to be Bearer [Auth Token, which is the string Bearer followed by your authentication token.

These values in the n8n form correspond to the line -H 'Authorization: Bearer [Auth Token] \ from our earlier CURL commands.

Similarly, we'll have to create another set of Header Authentication credentials for Paperless-ngx. The only difference is that the value of the header is slightly different, as the Paperless-ngx documentation specifies. We will enter Authorization for Name, and Token [Auth Token] for Value. Make sure that you are using the authentication token for Paperless-ngx here, of course!

After saving both authentication methods into n8n, we are ready to begin constructing the workflow itself.

Building the n8n Workflow to Retrieve Attachments from Firefly and Upload them to Paperless-ngx

Now we must construct the automation workflow. The remainder of this article will be a tutorial that walks us through the main parts of building such a workflow. You are welcome to either follow along, or otherwise download the complete .json code for the workflow at the following link. Be aware that the code is not "plug-and-play" – you will have to edit it to specify your own URLs and endpoints.

Our goal is that every time a transaction is added (or updated) on Firefly, the new (or updated) transaction is checked for any attachments. If the transaction has attachments, it will be downloaded by n8n, and then uploaded to Paperless-ngx. This sounds like a simple, and completely linear task– however, there are two complexities that we must handle:

1. The workflow must account for multiple attachments in the same transaction, and upload them all correctly to Paperless-ngx.
2. The workflow must preserve the transaction description, and send it to Paperless-ngx.

In order to do the first, we must create a loop. In order to perform the second, we will be renaming the downloaded attachments before POST-ing them to Paperless.

I will walk us through the basic steps involved in building this workflow, as understanding the process will be helpful in creating future workflows. The bulk of the nodes that we use are to make HTTP requests (the ones with the blue '@'). Hence, the techniques we learn here are fairly universal and generally applicable to all services that have REST-ful APIs.

Retrieving New or Updated Transactions, and Checking them for Attachments

First, every time a webhook fires, we must retrieve the transaction that was associated with the webhook. Remember, Firefly will only send data to n8n if a transaction is added or updated. However, the information that it sends is only a reference to what transaction was added/updated, and does not contain much information on the transaction itself. Hence, the first step that we must take, is to retrieve the full information of the webhook transaction.

To do this, we add two new HTTP Get nodes (which have the blue '@'). Then, click on the node itself, and configure it to use the header authentication credentials that we added earlier.

Now, when this node makes an HTTP request towards the Firefly III API, it will have the right credentials to be authenticated. What kind of request should this node make? This is the moment when we get into the heart of building automation workflows. We must consult the Firefly III API to see what kinds of information can be retrieved:

• Firefly III API Intro
• Firefly III API Reference (List of endpoints)

In our case, we must retrieve the information associated with the transaction that triggered the webhook. According to the Transactions section of the Firefly III API Reference, a request made to the following endpoint:

https://[Firefly III domain]/api/v1/transactions/{id}

Will return with the JSON-formatted information associated with the transaction bearing the ID of {id}. Hence, that's the call that we must make.

At this point, we still don't know what information is given to us by the webhooks. Hence, we need to call the webhook at least once, in order to capture it's input. Press the large, orange "Execute Workflow" button at the bottom of the n8n editor so we may begin capturing data. Now go ahead and create a new (test) transaction on Firefly III. If you registered the webhooks correctly, the following information should be sent to your HTTP node.

One of the JSON fields that the Webhook gives us is the id of the transaction. We now have the information that we need to make the request! Now, all we have to do is to craft an expression in the URL field. We do this by clicking on the grey gear icon to the right of the URL field (not visible in the above screenshot), and clicking "Add expression."

The window that now opens up is n8n's expression editor. Here, using a combination of JavaScript and templating, you can craft expressions that take information from nodes which feed into your current node, or any node executed so far.

In order to find the right values to insert into the expression, browse the expandable value list to the left of the expression editor. Find the value labeled id and click on it– it will insert the appropriate expression into the editor. You should see the value become highlighted in green (unlike my screenshot), and the resulting substitution will be displayed on the Result field of the screen.

Finally, combine the outputs of both nodes into an IF node, to check if the transactions contain attachments. If they do, then we must get the attachment information, and begin downloading all attachments from the transaction. To do this, you will also use the expression editor– taking the value from the inputs of the IF node and comparing if there has_attachments attribute is true. The expression that I used is:

{{$json["data"]["attributes"]["transactions"][0]["has_attachments"]}}

This process of creating expressions in different nodes forms the core of the n8n workflow creation process. Feel free to experiment and understand the expression editor! You will use it in ways greater or less to create all the remaining nodes of the workflow.

Downloading Attachment(s) from the Transaction in a Loop.

After getting the information about the attachment on the Firefly III transaction, we must be able to download the attachment and send it on to Paperless-ngx. However, there can be multiple attachments in a transaction – hence we must create a loop that downloads every attachment in the transaction.

Just like in regular programming, every loop in n8n has three elements:

1. A Set node (with the blue pencil icon) which sets a loop counter before the loop.
2. A second Set node which increments the counter for every iteration of the loop.
3. An IF node which checks whether the counter is greater than a chosen value. If false, it repeats the loop.

Here's a screenshot of how a simplified loop looks like. We would implement the actual "business logic" of what we want the loop do in the middle of it.

Do you see the similarity between the loop above, and the loop in our workflow implementation? The only flourish that our workflow contains is that a second path branches out from the Download Attachment(s) node.

Our loop instructs the Download Attachment(s) node to download every i-th attachment, starting from 0, until all attachments have been downloaded. This is accomplished by having the If Done node compare the current value of the counter, against the overall length of the attachment information array, returned by the Get Attachment Information node.

Renaming Attachments Using a Function Node

For every attachment, we utilise a function node to rename the attachment, before POST-ing them to Paperless. A function node basically receives information from the preceeding nodes as an JavaScript object, to which we can modify with code.

• Function node documentation.
• "How to modify filename?" n8n community forum post.

Our renaming code has some logic to check if this attachment is the only attachment of the transaction, or if it is only one of a series. Depending on the condition, it appends an appropriate suffix.

// Rename the input binary file to new_file_name. Contains some logic to determine how to
// name the files, depending on if there are multiple attachments.
if ($node["Get Attachment Information"].json["data"].length === 1) {
  // If the transaction only has one attachment
  var new_file_name = 
  $node["If Transaction has Attachment(s)"].json["data"]["attributes"]["transactions"][0]["description"];
} else {
  // Else if the transaction has multiple attachments
  // We will name each file with a number, starting from 1. The number comes from the counter.
  let suffix = String($json["counter"] + 1);
  var new_file_name =
  $node["If Transaction has Attachment(s)"].json["data"]["attributes"]["transactions"][0]["description"]
  + " " + suffix;
}

const data = Object.assign(items[0].binary.data, {fileName:new_file_name});
return [
  {
    json:{},
    binary: {
      data:data
    }
  }
]

Posting Renamed Documents to Paperless-ngx

Finally, the attachment is sent to Paperless using a POST request. The way that this is done in n8n is slightly non-intuitive, so it's helpful for us to consult the documentation first to understand how POST requests are made.

• HTTP Request Node, n8n documentation
• Send a binary file to an API endpoint, n8n documentation

We will use the same HTTP Request Node, but instead of a GET request, we must use a POST request. Additionally, as per the example in the documentation, set the JSON/RAW Parameters toggle to true (green), and select Form-Data Multipart for the body content type. Also set the Send Binary Data toggle to true (Green).

Finally, add document:data into the Binary Property field. Note that this does not have to be a expression, just the string document:data.

Now all that remains is to test out the workflow. Click on the orange "Execute Workflow" button once more, and create a test transaction on Firefly III with an attachment. You should see the workflow trigger, and your attachment automatically appear on Paperless-ngx!

Once you have verified that the workflow works correctly in Development mode, you must activate it on n8n so it runs automatically upon triggers. Then, you must update the webhook URLs on Firefly III to point towards the production URL, instead of the test URL. To do this, simply use the update command listed earlier.

Conclusion

We have created an n8n automation workflow which allows attachments to be automatically uploaded to Paperless-ngx. I hope my tutorial was helpful in covering the steps towards creating such workflows. As of always, the the complete .json code for the workflow at the following link.

Tools like n8n are useful for automating mundane tasks, and I hope that the time it saves us will allow us to do more purposeful, and interesting things.

Thank you for reading my tutorial. For further content, feel free to subscribe to my RSS feed. If you have any thoughts, feedback, or got stuck in any way, feel free to contact me. I always enjoy meeting interesting strangers! Be sure to check out my other content: I also write about Galileo, Special Relativity, and NixOS.

Hello there. My name is Shen, and I'm a Philosophy and Computer Science student. I write articles about Metaphysics and Ontology. Recently, a reader contacted me about making my website available as an Onion Service over the Tor Network. Using EOTK (the Enterprise Onion Toolkit), I was able to mirror my blog as a .onion site, and make it accessible to users over Tor.

This article is a guide to the process of using EOTK to mirror an existing website as an Onion Service. It is be applicable to any small website, whether a static site, or one running common Content Management System like WordPress, Drupal, or Ghost. Both this guide, and EOTK in general are designed to be self-contained, and "drop-in" – you do not need to modify your existing website configuration.

This guide is suitable for the intermediate Linux user or hobbyist web admin. You do not need to know any technical details about Tor. My intended audience is someone who is comfortable managing a WordPress site on the command line, or self-hosting a webapp using a VPS, but otherwise does not know about Unix sockets, Nginx modules, or advanced networking.

What Is Tor? What Are Onion Sites?

The Tor Network is a free and open source volunteer network that mitigates against tracking, surveillance, and internet censorship. Users connect to the Tor Network using the Tor Browser, which lets them browse websites anonymously. Tor users are often people with advanced privacy and security needs – such as journalist, whistleblowers, and citizens of politically oppressive regimes.

People can visit regular internet websites (i.e. the "clearnet") using the Tor Browser, but there are special sites called onion sites, that are only visible using Tor. These Onion Services provide the highest degree of anonymity and protection for Tor users. This is why many news organisations run onion sites. The New York Times, Washington Post, BBC, and others all maintain onion sites for users in countries where internet censorship is pervasive and freedom of speech is limited. Even Facebook and Twitter have onion sites. One can think of an onion site as the "shortwave radio" equivalent of a regular internet website.

Why Should I Make My Website Available as an Onion Site over Tor?

Onion sites are typically used by people with exceptional privacy and cybersecurity requirements. These users may come from politically oppressive regimes, or marginalised communities where surveillance is widespread and dangerous. By making your regular, "clearnet" website accessible as a onion site, you can help readers and visitors who may otherwise be unable to reach your content. I believe that even if only a single individual benefits from the onion mirror of my blog, it would be all worth it – because the pursuit of Philosophy is a universal, human activity – that should be accessible to all.

What Is the Difference between Making My Website an Onion Site, and Visiting My Website over Tor?

For Tor Network users, native onion sites offer far better security, privacy, as well as performance – than simply visiting a clearnet website over Tor. In order for a Tor Network user to visit a clearnet site, her connection must exit the Tor Network through an Exit Node, which makes the final connection over the Internet to the clearnet website. This last step can be subject to surveillance and monitoring, as well as interception if the clearnet site does not use TLS (i.e. SSL, HTTPS). Finally, the capacity of the Tor Network's exit nodes is limited – congestion at Exit Nodes typically make visits to clearnet web pages slow.

Alec Muffet, the creator and maintainer of EOTK has made a video comparison on the performance between visiting Facebook over Tor, and Facebook as a native Onion Service. The Facebook Onion Service loads around an order of magnitude faster.

Making an Existing Website Available as an Onion Site

We now begin the technical portion of the guide. We will be using Alec Muffet's EOTK (The Enterprise Onion Toolkit) to create a transparent mirror for our existing clearnet website. EOTK is a best-in-class, industry standard toolkit for creating onion sites for existing clearnet websites. It has been used by the BBC, the Intercept, and many other large organisations. This tutorial will guide you in deploying EOTK for your existing website, to create a secure and low-maintenance onion mirror. This tutorial will also guide you in the process of generating a "vanity" .onion domain name using mkp224o, adding a Onion-Location header for your clearnet site, and installing signed TLS certificates with HARICA.

As an overview, the process of making a onion mirror will be as follows:

1. Generate a "vanity" .onion domain name using mkp2240.
   We do this first because it can take a long time – this task can run in the background while you complete the subsequent steps.
2. Install and configure EOTK on our remote web server.
   This step will include daemonising the process and running via systemctl.
3. Install trusted TLS certificates from HARICA into EOTK.
   This step is optional. It is only applicable if you decide to purchase signed certificates from the HARICA certificate authority.
4. Configure the Onion-Location header in Nginx for our clearnet site.
   This better facilitates discovery for your onion site.

This process will take roughly 2 hours to complete. It is a good choice for a weekend-project for the hobbyist web admin. We will begin with the first step of the process, by using the mkp2240 tool to generate a vanity onion domain name.

Using mkp224o to Generate “Vanity” Onion Domain Names

GitHub - cathugger/mkp224o: vanity address generator for tor onion v3 (ed25519) hidden services

vanity address generator for tor onion v3 (ed25519) hidden services - GitHub - cathugger/mkp224o: vanity address generator for tor onion v3 (ed25519) hidden services

GitHubcathugger

The Tor Network is a decentralised network. As a result, there is no central authority to issue or register .onion domain names. Rather, all .onion domain names consist of a 56 character-long hash of letters and numbers, which is the base32 encoding of the site's public key. In practice, this means typical onion domain names look something like this:

http://5ekxbftvqg26oir5wle3p27ax3wksbxcecnm6oemju7bjra2pn26s3qd.onion/

This is of course, rather ugly and entirely unmemorable. But it is possible to generate partially human-readable onion domain names, by repeatedly creating many candidate domain names, and filtering them for patterns. This is how organisations like Twitter, or the New York Times have .onion domains which look like this:

https://twitter3e4tixl4xyajtrzo62zg5vztmjuricljdp2c5kshju4avyoid.onion/

https://nytimesn7cgmftshazwhfgzm37qxb44r64ytbb2dj3x62d2lljsciiyd.onion/

These are called "vanity" onion domain names, and the brute-force process to create them is analogous to cryptocurrency mining – since we are repeatedly making many hashes, and filter them according to an arbitrary criteria.

Security Considerations of Using Vanity .onion Domain Names

Although vanity .onion domain names are more recognisable, they may also have potential security disadvantages. If your users become accustomed to simply remembering your domain name as name<random-hash>.onion, an attacker may potentially impersonate your site by generating a similar onion domain name – relying on how users will likely only glance at the first part of the domain name and ignore the unreadable hash. For instance, the following two domain names look indistinguishable at the first glance.

legiteun3agp5zpw2zgcrjih4flq4ipp4dbpveqhlcyuas67wfqwmmyd.onion
legiteun3abbemz2u6zkfhiqemr6gurrekbjfppmdg2ud57sz6dr4mqd.onion

Hence, it is up to the implementer to decide whether or not the user-friendliness of a vanity .onion domain name will outweigh the small, but possible security disadvantage. I personally believe that the risk is quite small, so I choose to use a vanity .onion domain name – but it is important to make an informed choice.

We can generate these vanity onion domain names using a tool called mkp224o (Github Link). We will do this step first, since it can take some time – we can run it in the background while we accomplish the subsequent steps.

Downloading mkp2240

Let us download and compile mkp224o from its authors Github Repository. You'll want to do this on a local machine (i.e. your desktop), preferably one which has a powerful, multi-core CPU.

git clone https://github.com/cathugger/mkp224o.git

The utility is a C program, which we must configure and then compile on our machine. First, let us download some tools necessary to compile it:

sudo apt install gcc libsodium-dev make autoconf

Now we run ./autogen.s in order to generate a ./configure autoconf script. ./configure is the script which generates the Makefile that allows us to compile the C program.

cd mkp224o
./autogen.s

Configuring and Compiling mkp224o with Optimal Settings

This is the point where we must consider carefully the different configuration options that we can compile the program with. The options are documented in the OPTIMISATIONS.txt file available in the project root (here's a link to it on Github).

The mkp224o program is essentially an "onion miner" which mines onion domain names. We must ensure that we compile it with the optimal options, so that it performs as fast as possible. Specifically, mkp224o uses a library from NaCl to generate Ed25519 signatures very quickly. This library (called SUPERCOP) has different versions (i.e. implementations) that are better optimised for different CPU architectures. The mkp224o tool supports the following implementations:

The author offers advise and a discussion of the various options in OPTIMISATIONS.txt. In practical terms, if you are using a modern, 64-bit (x86_64) Intel or AMD CPU, you should use the --enable-amd64-30k configure flag. If you don't know what x86_64 means, you should certainly use --enable-amd64-30k.

Likewise, there are an additional set of configure options for mkp224o's filtering algorithm. The options are described in detail in OPTIMISATIONS.txt – and a proper selection is important, if you have hundreds of filters. However, if you are using only a few, the following flag will suffice:

--enable-intfilter=native

This enables simple integer filters, which is fastest if you have only a dozen filters.

We the above considerations in mind, we can now configure mkp224o with the following flags:

./configure --enable-amd64-51-30k --enable-intfilter

Now we run make to compile the program, and now mkp224o should be available as a native binary within the git repository.

make

# Run mkp224o and print help
./mkp224o -h

Mining Onions with mkp224o

Now we are ready to mine onions with mkp224o! This process is CPU bound, so more threads your CPU has, the faster your process is. Before we begin, make sure to check out all the available options for mkp224o by running ./mkp224o -h. In particular, we'll be using the following flags:

First, we'll create a ./my-filters.txt file with a list of "filters" that we want our onion domain to be. For my website https://shen.hong.io/, my filters were the following:

hongonion
shenhong
hongblog
shenblog
shensite
hongio
shenio

With these filters, you will be able to find candidate onion domain names like:

hongio267dx4o2ofkn4ddsztu4ok2vq4euc7sxumbi7kcfd64ije62ad.onion

Alec Muffet, the Facebook Engineer who created EOTK, has written some excellent advise about the process of mining and choosing onions. For inspiration and guidence, I highly recommend checking out his post (available as within the EOTK git repository as docs.d/TIPS-FOR-MINING-ONIONS.md).

Create a directory called ./candidates/ which will contain all the matching domains. We will also specify a --checkpoint and a passphrase -p. This is to allow us to cancel and resume the search process conveniently. For the passphrase, make sure it is a long, > 64 character random string. We will need it to resume the search! Finally, we will be using the -B flag to enable batched mode, which speeds up performance significantly. With the above settings, we may begin the search process with the following command:

./mkp224o -s --checkpoint checkpoint.txt -d candidates -f my-filters.txt -B -p PASSPHRASE

You are now mining onions! Candidate domain names which match your filter criteria will be stored in the ./candidates/ directory. Depending on how many characters your filters are, this process can take a long time. Keep it going in the background, while we continue the rest of the tutorial.

How Long Does It Take to Find a Onion Domain Name of a Given Length?

I'm glad you asked! There is a very informative discussion in the Issues section of mkp224o's Github page, which essentially asks the same question. As one commentor has presented, number of calculations (hashes) \(t\) needed to find an onion address with \(n\) characters with probability \(p\) is governed by the following equation:

\[t = \frac{\log(1-p)}{\log(1-\frac{1}{32^n})}\]

Each additional character in our vanity onion domain increases the term \(32^n\), where 32 is the base32 encoding of a onion domain. When we run ./mkp224o, it outputs its own performance statistics, including its calc/sec rate which represents the number of hashes \(t\) which it performs every second. Hence, given the calc/sec value, you will know the \(t\) that your computer is capable of performing every second.

I've recreated a table of \(t\), given some values of \(n\) and \(p\):

Here are some typical values of \(t\) which I have measured:

• Framework Laptop with 11th Gen Intel Core i7-1185G7 (8 threads): \(\approx 13,417,732\) calc/sec.
• Hetzner Dedicated CPU Server with AMD EPYC 7003 Milan (48 threads): \(\approx 93,313,394\) calc/sec.

(Feel free to contact me if you have your own measurements! I'll add them to this list)

As you can see, for a 75% chance to "mine" a single 8-character vanity onion domain name, you will need approximately 31 hours on a modern Intel i7 CPU. That same 8-character vanity onion domain name will take less than 5 hours on a high-end AMD EPYC 7003 server CPU with 24 cores. But even that same powerful server CPU will take more than 4 days to mine a 9-character onion, and more than 191 days to mine a 10-character onion!

Hence, keep your onions mining, while we proceed on to the next part of the guide: installing and configuring EOTK.

Introduction to EOTK: The Enterprise Onion Toolkit

GitHub - alecmuffett/eotk: Enterprise Onion Toolkit

Enterprise Onion Toolkit. Contribute to alecmuffett/eotk development by creating an account on GitHub.

GitHubalecmuffett

We'll be using Alec Muffet's EOTK in order to make our website available transparently over the Tor Network. This section of the guide will introduce EOTK, install it on the server, and go through common configuration options and settings.

EOTK is the best approach to making an existing clearnet website available as an onion site. It offers the following advantages:

• It is a transparent, drop-in solution: You do not need to alter your existing web app configuration or stack.
• It is widely used and actively developed: The toolkit has been deployed by the BBC, The New York Times, and other large organisations. The Git project has a consistent history of commits.
• It packages the tor service with it: You do not need to separately install and configure tor. The solution installs a standalone tor service that is preconfigured with secure defaults.

Hence, we will use EOTK to make our website available over the Tor Network as an onion site.

How EOTK Works

EOTK works as a man-in-the-middle rewriting proxy. It accepts inputs from the Tor Network, rewrites them for your web application, and then sends them to the application itself. Likewise, when the application responds, it will rewrite the response and then route it to the correct user in the Tor Network. Essentially, EOTK serves as a translator which allows Tor Network users to visit your site as an onion site, and for your site to respond as an onion site.

Here's a diagram of a regular web setup, where you have the web application behind a Nginx reverse proxy. The web application can be any CRM such as WordPress, Drupal, or Ghost – and the Nginx reverse proxy performs traditional tasks such as compression and TLS.

When we install and configure EOTK successfully, it sits between our Nginx reverse proxy, and the Tor Network.

Note the substantive similarity between the role and position of EOTK, and our regular "clearnet" Nginx reverse proxy. EOTK is effectively a modified Nginx reverse proxy, which "translates" requests between the Tor Network, and your application's regular reverse proxy. In fact, we will also have to install TLS certificates at the EOTK level, in order to allow our onion sites to be served over HTTPS. But also observe that the connection between EOTK and your Web Application is not direct, but rather EOTK is a layer "on top of" your regular Nginx reverse proxy. This means that all the benefits and optimisations from your regular Nginx configuration is preserved. For example, if you configured Nginx to use compression, your site will likewise be compressed as an onion site.

Now that we understand the the architecture of EOTK within our existing web stack, let us begin with the installation process.

Installing EOTK on Ubuntu

EOTK's installation instructions are available within the ./docs.d folder of its git repository (link here). The toolkit is available on a variety of different platforms, such as Ubuntu, Debian, CentOS, Mac OS, and FreeBSD. We'll be following the latest Ubuntu 20.04 instructions. First, clone the EOTK repository onto your remote webserver.

git clone https://github.com/alecmuffett/eotk.git

Be very mindful where you place the ./eotk directory. The folder is not disposable, but rather it will serve as the root which all of our site-specific configuration will reside. The ./eotk directory that we just cloned is analogous to our /etc/nginx directory – it will later contain all of our site-specific configuration, TLS certificates, and .onion site keys. Make sure to move it to a safe place! Consider placing it in /etc/.

Once you are ready, we run the included build script.

cd ./eotk
./opt.d/build-ubuntu-20.04.sh

This build script does the following:

1. It first downloads and installs a set of platform-specific build dependencies, such as build-essential, libssl-dev, and curl.
2. It then calls out to the ./opt.d/lib.sh script, which contains instructions common to all platforms.
3. Within the ./opt.d/lib.sh script, it downloads, configures, and builds two programs. The tor daemon itself (which provides us access to the Tor Network), as well as OpenResty, a Nginx plugin that provides Lua scripting support for Nginx configuration. OpenResty is the core of the EOTK – we use OpenResty to "translate" requests to and from the Tor network for our web application.

The build process can take some time. Once it is complete, you should have a ~/eotk/eotk binary that is ready to use, as well as a directory structure that is populated like this:

root@remotehost:~/eotk# tree -L 3
.
│   # Directories
├── demo.d
├── docs.d
├── lib.d
├── opt.d
├── projects.d
├── secrets.d
├── templates.d
├── tools.d
│   ├── openresty.d
│   ├── tor -> /root/eotk/opt.d/tor.d/bin/tor
│   └── tor.d
│
│   # Executables
├── eotk
├── eotk-housekeeping.sh
├── eotk-init.sh
│
│   # Other files
├── LICENSE
├── Makefile
└── README.md

EOTK Directory Structure and Reference

Here's a brief explanation of the EOTK directory structure. Many of the directories are only generated once you build the eotk binary.

Out of all of these directories, the most relevant are the projects.d and the secrets.d folders. These will contain the configuration files for your site, and the public key pair for your onion domain name. Every website you choose to make available as an onion site will have its own folder within projects.d:

projects.d
├── website1.com.d # Includes all subdomains of website1.com
├── website2.net.d
└── website3.org.d

Now that our EOTK binary is built and available, we are ready to make our website available on Tor.

Understanding the EOTK Binary and Command Reference

The EOTK binary, available within its folder as ./eotk is equivalent to our nginx command. Just as you might run nginx -t to test your nginx configuration file, you will run ./eotk -a to check your EOTK configuration. However, it is important to note that you must always run EOTK locally, within the root folder (e.g. ~/eotk/eotk, or /etc/eotk/eotk), because unlike the nginx executable, ./eotk is not available globally. The entirety of our EOTK installation is self-contained within its folder, and we must always be inside that folder when we interact with it.

For a full listing of the available commands, run ./eotk -h. Partial documentation for the command syntax is also available at eotk/docs.d/EOTK-COMMAND-SYNTAX.md (Github link). For our purposes, the most relevant and common commands will be the following:

As you can see, there is a strong similarity between EOTK and our regular Nginx reverse proxy. And just like Nginx, every website (and its subdomains) begin as a configuration file. The websites that we wish to make available over Tor are projects, in EOTK's terminology. Given a website example.com with its subdomains www.example.com, blog.example.com, portal.example.com, we must create a EOTK configuration file called example.com.conf, within which we will configure all the details.

Creating EOTK Configuration Files for Projects

Within the main ~/eotk/ folder, create a .conf file with the name of your project. Your project can be anything, but I suggest naming it after the domain name of your clearnet website. I will use the domain name example.com for the purpose of our tutorial. Within the file, write:

set project example.com

The set project directive is essential for EOTK to know which project this configuration file belongs to. EOTK will use the value we set to create a folder within ~/eotk/projects.d/, such as ~/eotk/projects.d/example.com.d. Save and exit out of that file for now, we are not done with it.

We must next configure a series of hardmap directives to map our .onion domain name to the clearnet domain and subdomains of our website. But before we can do that, we must install the cryptographic keys of our vanity onion domain name, which we just mined. At this point, hopefully you have already mined some memorable vanity onion domain names!

Installing Mined Vanity Onion Domain Names

Back on your local (and powerful) machine, you should stop mkp224o (Ctrl + C) and examine the candidates/ directory. It should contain folders with the name <domain>.onion/, each one being a domain name which matches your filters. Select one which you wish to keep, and move the folder to a safe place.

Within it, you will find the following files:

hostname
hs_ed25519_public_key
hs_ed25519_secret_key

The hostname is a text file containing your .onion domain name (in case you forget it). The hs_ed25519 files are the public key pairs of your domain. Make sure to protect these files. If you lose your hs_ed25519_secret_key, you will effectively lose control over your domain name. Do not under any circumstances lose or publish the hs_ed25519_secret_key. If an adversary obtains your private (secret key), they will be able to take over your domain name.

We will need to upload both the public key, and the private key to our remote web server (where we installed EOTK). A simple and secure way to do this is to use rsync. On your local machine, where these files are located, run:

rsync -a hs_ed25519_*_key username@remote:/path/to/destination

This command will let rsync to upload both files to your remote machine to a destination of your choice. Once they are on your remote machine, we must rename them and move them to the ~/eotk/secrets.d directory.

Note: If you have been following this guide along with the EOTK project documentation (~/eotk/docs.d/HOW-TO-INSTALL.md, Github link), please note that as of 2022-03-26 (commit e4a4ca4), the instructions for installing pre-mined onion domain keys are incorrect. Until the documentation is updated, follow the instructions given here.

We must take the hs_ed25519_public_key and hs_ed25519_secret_key files, and move them into the ~/eotk/secrets.d directory, with the following names:

• Rename hs_ed25519_public_key to <domain>.v3pub.key
• Rename hs_ed25519_secret_key to <domain>.v3sec.key

The <domain> must be your onion domain name without the .onion suffix. For example, if the vanity onion domain of your site was exmple27tdg5nesxaao6khfuk5ac723n7tpqkzhyytnjm2ltt6tgkhyd.onion, then your ~/eotk/secrets.d directory looks like this:

secrets.d
├── exmple27tdg5nesxaao6khfuk5ac723n7tpqkzhyytnjm2ltt6tgkhyd.v3pub.key
└── exmple27tdg5nesxaao6khfuk5ac723n7tpqkzhyytnjm2ltt6tgkhyd.v3sec.key

The ~/eotk/secrets.d/ directory will contain the public-key pairs of all of your onion domain names for all of your projects, in the forms of <domain>.v3pub.key and <domain>.v3sec.key files. If you wish to commission any future sites, simply mine the onion domain names with mkp224o and install them in ~/eotk/secrets.d. We are now ready to go back to our project configuration file ~/eotk/example.com.conf, and map our domain name to the project.

Configuring Hardmaps for EOTK Projects

Move back to our main ~/eotk/ folder, and reopen the example.com.conf EOTK configuration file. We will now define a hardmap which maps the onion domain name to our clearnet domain. Fun fact: according to Alec Muffet, the main developer of EOTK, the reason that the mapping directive is called hardmap is due to reasons of backward compatibility:

'Hardmap' originally was just 'map' and expected a filename, and then 'softmap' arrived and took just an onion address ... which is nicer; but I don't want to change the code very much, so if you are 'hardmap' and specify only an onion address, let's fill in the filename and then continue by stripping it away again. Now we just clean it up and assume the key materials are in secrets.d; le huge sigh, hindsight is 20-20.

– Alec Muffet, ~/eotk/lib.d/do_configure.pl comments (Source).

The hardmap directive takes the following form:

hardmap <onion domain> <clearnet domain> <subdomains>

Where the <onion domain> is the domain name (including the .onion TLD) of your newly mined vanity domain name. Hence, our project configuration file will look like:

set project example.com

hardmap exmple27tdg5nesxaao6khfuk5ac723n7tpqkzhyytnjm2ltt6tgkhyd.onion example.com

Subdomains are listed after the clearnet domain, seperated by whitespace. If our example.com website has www, blog, and portal as subdomains, then our example.com.conf file will look like the following:

set project example.com

hardmap exmple27tdg5nesxaao6khfuk5ac723n7tpqkzhyytnjm2ltt6tgkhyd.onion example.com www blog portal

An individual project configuration file may contain multiple hardmap directives, potentially mapping multiple different .onion domain names to the same, or different clearnet domain names. The ~/eotk/demo.d directory (Github link) contains multiple example configuration files. In particular, the wikipedia.conf file has an example with multiple hardmaps.

The above example.com.conf EOTK configuration file is now sufficient for us to bring our onion site online.

Activating, Deactivating, and Reloading EOTK configuration

We will now generate our example.com project from the example.com.conf file. This is where the magic happens. Run the following:

./eotk config example.com.conf

The EOTK binary will now parse the provided project configuration file, and setup the project. It will do the following:

• EOTK will create a directory in ~/eotk/projects.d named example.com.d, which will contain all our project-specific configuration, and startup/shutdown scripts.
• EOTK will instantiate a project-specific Nginx instance, with an autogenerated Nginx configuration file at ~/eotk/projects.d/example.com.d/nginx.conf. This configuration file will contain the OpenResty rewriting rules that "translate" our clearnet site to an onion site, and vice versa.
• EOTK will configure the tor daemon service with the provided onion domain keyfiles, to create a hardmap to our EOTK Nginx instance. It will autogenerate a Tor configuration file at ~/eotk/projects.d/example.com.d/tor.conf. Incoming Tor Network requests for our onion site will be handed over to the EOTK Nginx instance, translated, and passed through to our regular Nginx instance (and vice versa).

Now our example.com project is ready! To start it, all we have to do is to run:

./eotk start example.com

And our project is live! All you have to do is to navigate to the resulting .onion URL in the Tor Browser, and after accepting the self-signed certificate warning, you will be able to see the site. In the case of our tutorial, I actually deployed a real onion mirror for the IANA Example.com site. If you visit the following link on the Tor Browser, you can see that it is live!

https://exmple27tdg5nesxaao6khfuk5ac723n7tpqkzhyytnjm2ltt6tgkhyd.onion/

But Wait, How Did You Deploy an Onion Mirror for IANA’s Example.com?

Recall that EOTK functions by essentially running a tor service, in conjunction with a Nginx rewriting proxy. This makes EOTK analogous to an open proxy server which translates requests between a Tor Network client, and any remote host. You may in fact point EOTK to any clearnet website, and it will be able to translate requests from your hardmapped onion domain name to the clearnet domain.

In practice, you should almost never deploy EOTK to websites or web applications that you do not own. For one, an EOTK installation on a different host from the clearnet service will achieve lower performance, as proxied requests must traverse through the open internet, instead of only internally. Secondly, your EOTK server will function effectively as an open proxy for Tor users, and you will be responsible for any malicious traffic which passes through your EOTK deployment. In fact, your server will be the source of all Tor traffic, malicious or otherwise. And of course, it will be impossible to find the ultimate source of any malicious Tor traffic, since the very nature of the Tor Network makes Tor clients anonymous.

Dealing with potentially malicious Tor traffic is an important consideration with any EOTK deployment. In particular, we would want to protect the administrative portions of our website, and render them inaccessible via Tor. We will discuss how to accomplish this with EOTK in the following section.

Whitelisting, Blacklisting, and Access Control with EOTK

Our website (example.com) is now accessible on its onion domain over the Tor Network. EOTK seamlessly and transparently translates requests back and forth between Tor Network users and our webserver. However, oftentimes we do not wish to make the entire functionality of our website available over Tor. For example, we may not want to expose our admin login portal to the Tor Network, as it will be impossible to ban malicious login attempts. There may be certain subdomains that contain functionality (typically relating to CORS (cross-origin resource sharing) that will not function once rewritten.

EOTK comes with a set of whitelisting and blacklisting directives that will allow us to selectively manage access to our site from the EOTK project configuration file (e.g. example.com.conf). EOTK provides three different types of syntax for access control,:

• Blacklist: Denies access to the specified resources. Blocked resources will return a 500 Internal Server Error response code.
• Whitelist: Allows access only to the specified resources. If you add whitelist directives, all requests that do not fit the whitelist will be rejected with a 500 Internal Server Error response code.
• Block: Rejects specified resources with a 403 Access Forbidden response code. They allow the admin to specify a block_err message which will be displayed on the blocked request. However, block directives can only accept a single value.

I don't recommend using block directives, as the EOTK project maintainer states:

The [block] variables are single-valued so be careful of polluting multiple projects. If your site needs different blocking for different onions, consider splitting your config into multiple files and using "foreignmap" to stitch the hostname rewrites together.

– Alec Muffet, ~/eotk/demo.d/example.tconf comments (Source).

Instead, we will use the blacklist and whitelist directives, which perform the same function, and as they take multiple values, are much more powerful.

Block, blacklist, and whitelist directives allow the administrator to specify matches based upon multiple criteria. The following three are probably the most useful and relevant:

• Host: A HTTP Header. The requested domain name or subdomain. We will specify host directives if we want to whitelist/blacklist domain names or subdomains.
• Parameter: The HTTP query string (i.e. what comes after a ? symbol in the requested URL.
• Path: The path of the requested resource.

Furthermore, EOTK also provides the additional criteria, which are probably less applicable.

• Origin: A HTTP Header. The origin of the request. This is rarely useful, unless you are working with CORS, e.g. loading resources over a onion CDN.
• Referer: A HTTP Header. This is rarely useful for the purpose of access control, since the Referrer header is easily spoofed.
• User-Agent: The User-Agent. This is likewise rarely useful, since the User-Agent can be easily spoofed.

All of these directives also have an alternate form with the _re suffix, which accepts regular expressions as a matching parameter.

In summary, the EOTK blacklist and whitelist access control directives are:

EOTK Blacklist Directives

EOTK Whitelist Directives

Keep in mind that all of the above whitelist and blacklist directives accept multiple values, separated by a whitespace. Value lists can also be split over multiple lines using the \ character. With the above directives as a reference, let us take a look at an example configuration.

Example Access Control Configuration for Example.com

Let us go through a practical example of using the EOTK access control directives to limit our website's exposure over Tor. Given our example.com project, let us suppose our website exposes the following sensitive functionality:

• An admin portal at admin.example.com and remote.example.com
• A staff-only login page at example.com/login/
• A rate-limited API at example.com/api/

We can blacklist all of the above by writing the following in our EOTK configuration file. Note that we may split parameters over separate lines using the forward slash (escape) character: \:

set project example.com

set host_blacklist \
admin.exmple27tdg5nesxaao6khfuk5ac723n7tpqkzhyytnjm2ltt6tgkhyd.onion \
remote.exmple27tdg5nesxaao6khfuk5ac723n7tpqkzhyytnjm2ltt6tgkhyd.onion

set path_blacklist /login/ /api/

hardmap exmple27tdg5nesxaao6khfuk5ac723n7tpqkzhyytnjm2ltt6tgkhyd.onion example.com www blog portal

Observe carefully how the set host_blacklist parameter must accept full host strings, in the form of <subdomain>.<onion domain>.onion. As the blacklist functions based on matches from the Host header in the incoming request, specifying only the subdomain will not work. This was an unexpected source of confusion when I implemented hong.io's onion mirror.

# set host_blacklist admin             # WILL NOT WORK
# set host_blacklist admin.example.com # WILL NOT WORK

# Only the following works
set host_blacklist admin.exmple27tdg5nesxaao6khfuk5ac723n7tpqkzhyytnjm2ltt6tgkhyd.onion

After making these changes, we must regenerate our configuration by running ./eotk configure <filename.conf>, and then reload the built-in Nginx server in EOTK using ./eotk nxreload <project name>:

./eotk configure example.com.conf
./eotk nxreload example.com

Be sure to not forget to configure and then nxreload every time you make changes in your project configuration file. Otherwise, whatever modifications you have made will not take affect! Likewise, carefully note that the ./eotk nxreload command takes the project name as its argument, not the configuration file name. The project name is defined by the set project <projectname> directive in your configuration file!

Now that we have implemented blacklist directives for our mirror, whenever a Tor Network client attempts to visit one of the specified resources, EOTK will reject the request with a 500 Internal Server Error response code. This will yield the following error page:

You may attempt to visit those links yourself. Open the Tor Browser, and try navigating to the following addresses. They will all result in a 500 Internal Server Error response!

https://admin.exmple27tdg5nesxaao6khfuk5ac723n7tpqkzhyytnjm2ltt6tgkhyd.onion/
https://remote.exmple27tdg5nesxaao6khfuk5ac723n7tpqkzhyytnjm2ltt6tgkhyd.onion/
https://exmple27tdg5nesxaao6khfuk5ac723n7tpqkzhyytnjm2ltt6tgkhyd.onion/login/
https://exmple27tdg5nesxaao6khfuk5ac723n7tpqkzhyytnjm2ltt6tgkhyd.onion/api/

As per the reason why blacklisted resources return a 500 Internal Server Error response code, we may view Alex Muffet's explanation, located once again as a comment in the EOTK git repository:

[Access] failures are generally 500 because it presents the least attack surface to a penetration tester.

– Alec Muffet, ~/eotk/demo.d/example.tconf comments (Source).

Host-based whitelisting and Allowing only HTTP GET Requests

In its default configuration, EOTK naively forwards all requests with the specified hardmap domain name, regardless of subdomain. When we configure our example.com website with the hardmap directive:

hardmap exmple27tdg5nesxaao6khfuk5ac723n7tpqkzhyytnjm2ltt6tgkhyd.onion example.com www blog portal

We would have expected only requests to example.com, and the subdomains www, blog, and portal be forwarded on to our web application. However, in my testing, this was not the case – but instead, EOTK translates all requests made to *.example.com. Hence, all services available as a subdomain under example.com will be available as an onion site on the Tor Network – even if these services are hosted on entirely different servers.

This default behaviour may be undesirable, particularly if you only intend to expose one service as an onion site. In order to prevent this behaviour, we must explicitly whitelist subdomains using the host_whitelist directive. Here is a modified example.com.conf file which uses the host_whitelist directive:

set project example.com

set host_whitelist \
exmple27tdg5nesxaao6khfuk5ac723n7tpqkzhyytnjm2ltt6tgkhyd.onion \
www.exmple27tdg5nesxaao6khfuk5ac723n7tpqkzhyytnjm2ltt6tgkhyd.onion \
blog.exmple27tdg5nesxaao6khfuk5ac723n7tpqkzhyytnjm2ltt6tgkhyd.onion \
portal.exmple27tdg5nesxaao6khfuk5ac723n7tpqkzhyytnjm2ltt6tgkhyd.onion

set path_blacklist /login/ /api/

set suppress_methods_except_get 1

hardmap exmple27tdg5nesxaao6khfuk5ac723n7tpqkzhyytnjm2ltt6tgkhyd.onion example.com www blog portal

Make sure that the list of whitelisted hosts matches the list of subdomains in the hardmap directive. Finally, do not forget to add the root (subdomain-less) host to the whitelist as well!

There is one final EOTK directive that is relevant for some use-cases. On line 9, we added set suppress_methods_except_get 1. This effectively makes our onion site a read-only mirror of our clearnet website. It informs EOTK to only allow HTTP GET requests. This will disable any functionality on the website which takes user input, such as search bars, file uploads, and form submissions. This directive will only make sense if your website does not require any server-side interaction at all for Tor Network clients.

In my use case, I know for certain that the administrative interfaces of my blog will never be accessed over the Tor Network. Hence, I set suppress_methods_except_get to 1 for my blog.

No matter your choices, remember to run ./eotk config <filename.conf> on your configuration file, followed by ./eotk nxreload <projectname>. Now your changes are live!

Other EOTK Configuration Options

As of the time of this article's writing (2022-03-28), the EOTK configuration file documentation is incomplete. A listing and partial explanation of the different options can be found in the ~/eotk/demo.d/example.tconf file (Github link). I have gone through the directives with the most common use-cases in this article, but I highly encourage any administrator to review the example.tconf file for further information.

Some interesting, and potentially useful options that this article does not cover include:

• Injecting request headers for upstream use.
• Managing the rewriting process, mangling SRI tags.
• Caching requests, redirects, and Nginx tuning.

Note: Do not worry about the TLS-related directives for now. This tutorial will also present a complete guide to purchasing, and installing a TLS certificate later.

Activating EOTK as a Systemd Service and Starting Automatically on Boot

By following the above instructions, you should have a secure, nearly-production-ready Onion version of your website, accessible over a custom vanity .onion domain over the Tor Network. We must make sure that our EOTK service runs automatically as a Systemd daemon, which will ensure that it always starts up when our server boots, and restarts if the service stops or crashes.

We do this by generating, and then installing the EOTK startup scripts. These scripts are generated with each installation, because they contain hardcoded paths to our EOTK binary. To generate them, simply run:

./eotk make-scripts

This will create the eotk-init.sh and eotk-housekeeping.sh files in the same location as your ./eotk binary. The eotk-init.sh file is a System V-style init script which we can move to the /etc/init.d directory, and then convert it automatically into a Systemd-style service unit. To do this, run:

cp eotk-init.sh /etc/init.d
update-rc.d eotk-init.sh defaults

Now, all we have to do is to start it via Systemd.

systemctl start eotk
systemctl status eotk

The status for the eotk-init.service should show as active (running) with a green status semaphore. Now we are certain that our EOTK installation will autostart on boot, and always be available. We can still manage our specific EOTK projects using ./eotk start <project> or ./eotk stop <project> within the /eotk directory, and our workflow for updating project-specific configuration files is the same.

Note that we do not do systemctl enable eotk as we might with a native Systemd service unit. start-ing the service is enough to have it autostart on boot, since the EOTK is using a System V-style init script which is only managed by Systemd through a compatibility layer.

Our EOTK installation is almost production-ready. We only have one last task left, which is purchasing, and installing a TLS certificate.

Purchasing, and Installing a Trusted TLS Certificate

By now, your website's .onion mirror is nearly production-ready. However, we must address one final issue. EOTK creates self-signed TLS certificates by default, which are not trusted by the Tor Browser. When any visitor visits your website's .onion link, they will be greeted with a prominent certificate warning. This warning persists upon every restart of the Tor Browser, even if the user accepts to continue.

Self-Signed TLS Certificates Break Cross-Domain Resource Loading

The warnings prompted by the self-signed certificate are not just cosmetic in nature, but can signifcantly break site functionality. If your website loads resources for any subdomain (such as static.example.com), then resources from the subdomain will not load unless the user manually navigates to the subdomain, and adds an exception. This means that an onion site mirror with self-signed certificates can present a broken experience, with images, scripts, and resources not loading at all.

Hence, for any production-quality onion mirror, we must deploy a signed TLS certificate for use with EOTK.

There Are Only Two Options for Trusted TLS Certificates

Unfortunately, as of the time of this article's writing (March 2022), LetsEncrypt does not support TLS certificates for .onion domain names. Hence, we must purchase a commercial TLS certificate. As of March 2022, there are only two commercial certificate vendors which provide TLS certificates for .onion domain names:

• DigiCert: Offers only Extended Validation (EV) Certificates (Link to Press Release)
• HARICA: Offers Domain Validation (DV) Certificates, including wildcard Certificates (Link to Press Release).

DigiCert only offers EV certificates, which must be purchased via an order form, and are only available to registered organisations. Although the cost is not stated, EV certificates generally cost thousands of dollars per year. This makes DigiCert generally inaccessible to small websites and hobbyists like myself.

The only vendor of DV certificates is HARICA, which is a much more accessible and affordable option. HARICA has been featured on the Tor Project's blog as an option for .onion TLS certificates, and we will be using them for this tutorial. Note: I do not have any affiliation with HARICA, and this article is not sponsored by them in any way.

About the HARICA Certificate Authority

HARICA is a Greek-based Certificate Authority founded by a non-profit civil organisation. HARICA is an acronym which stands for the Hellenic Academic and Research Institutions Certification Authority. They seem to be active in the Greek academic sector and work with Greek Universities. Although there are not any choices, I felt comfortable about using HARICA as a certificate authority, because they come from a familiar academic background.

Purchasing a Onion TLS Certificate from HARICA

I'll take us on a quick walkthrough of the process it takes to purchase, validate, and install a TLS certificate from HARICA. I have documented this process in unusual detail, because it was entirely unfamiliar to me. I'm young enough that I only ever used Lets Encrypt, so I never had the experience of purchasing and deploying a commercial TLS certificate – I grew up using Let's Encrypt's certbot tool. For other hobbyist web admins who are likewise unfamiliar with the certificate purchasing process, I hope the following guide will be helpful.

All screenshots were taken on March 2022.

First, the user must navigate to HARICA's website, and register for an account. Once registration is complete, you will be directed to login to the HARICA CertManager (direct link). Before you begin, make sure to check your settings and enable Two Factor Authentication (2FA), which is good practice for any important service.

Next, you'll want to create a Certificate Request for a "Server Certificate," which is HARICA's term for a server-side TLS certificate. It will present a screen where you can add domain names. Simply paste in the .onion domain name directly. You can add additional domain names, as well as wildcard domains:

HARICA does not seem to have a public pricing page. Prices are displayed in the requesting process. As of March 2022, the pricing for DV TLS Certificates appear to be the following:

• DV Certificate: €30.00 EUR per year.
• DV Wildcard Certificate: €150.00 EUR per year

They offer an additional 10% discount for every additional year, up to 4 years (30% discount).

Next, you'll be asked to either generate, or submit a Certificate Signing Request (CSR). This is a text file which contains the information about your certificate, such as the Common Name, Organisational Name, et cetera. These were the questions that Let's Encrypt's certbot tool would ask, when you provision a new certificate. We will select the Auto-generate CSR option. This will instruct HARICA to generate a CSR in your browser, using a private key for your certificate that it will also create.

For the private key, it is important to choose ECDSA as your algorithm, and 384 as your key size. These are the most secure options, followed by RSA with 4096 key size. Between ECDSA and RSA, you should always choose ECDSA which is the more secure option, especially since you don't have to worry about backward-compatibility as all recent versions of Tor Browser support ECDSA.

You will also be prompted for a passphrase used to encrypt your certificate's private key. This is similar to the kind of passphrase that you might set for your SSH key. Set a long passphrase, and keep it safe – we will need it later when we install these certificates.

Your browser will freeze for a few seconds, as it generates the CSR and a private key. Then, you will be prompted to download your private key, as a privateKey.pem file. Make sure to save this file – we will need it when we install the certificate into EOTK. Protect your private key carefully – losing it will allow your certificate to be compromised!

We have now successfully submitted a request for a DV TLS Certificate. Now we must prove to HARICA that we own and control the .onion domain name. This is the next step.

Obtaining an Onion Domain Proof of Ownership

When using Let's Encrypt, domain ownership is proven automatically when certbot places a file in the ./well-known (defined by the RFC 5785 IETF standard). With HARICA, the process is a little different. You may either manually place a file in a similar location, which is the Validate via "Agreed-Upon Change to Website," or use the Validate via "Onion CSR" option.

We will go with the second Validate via "Onion CSR" option. This is because if you are attempting to purchase a certificate with multiple subdomains, the first validation option will not be available anyways. Likewise, the first option is not available for wildcard certificates. Finally, it requires making modifications to our EOTK project config (e.g. ~/eotk/example.com.conf) which may be annoying, if you have multiple projects.

When we do this, we will be asked to confirm and submit our validation request. Then, we must download and compile a tool called onion-csr from the official HARICA Github repository.

We will do this in the following section.

Cloning and Building HARICA's Onion CSR Tool

GitHub - HARICA-official/onion-csr: Tool to create CSRs signed with Tor Hidden Service private keys

Tool to create CSRs signed with Tor Hidden Service private keys - GitHub - HARICA-official/onion-csr: Tool to create CSRs signed with Tor Hidden Service private keys

GitHubHARICA-official

We must download and build the onion-csr tool. We must do this on the remote machine, where we run EOTK. This is a source-available Ruby script which essentially creates and signs a CSR using the private key of your .onion domain name. By signing the CSR using your .onion domain's private key, HARICA can confirm that you possess ownership over the domain by verifying it using the domain's public key.

The entirety of onion-csr's source code is located in the onion-csr.rb file (Github link). The code is only 100 lines long, and is understandable enough to be audited and read. The instructions for using the tool are included in the README.md file of the repository (Github link), but I will present the process here in this guide as well.

First, we install dependencies for the onion-csr tool. This is only the ruby intepreter, and some compilation dependencies. We will need build-essential because we will build the ED25519 library from the original C source code.

apt install ruby build-essential

Fun fact: the ED25519 library that we will be building is actually the same library that was used by mkp224o from earlier! HARICA's onion-csr repository includes it as a submodule which will also be downloaded when we clone the repository. The ED25519 library can be found here (Github link), and if desired the code can be audited as well.

git clone --recurse-submodules https://github.com/HARICA-official/onion-csr.git
cd onion-csr
gem install ffi
gcc -shared -o libed25519.so -fPIC ed25519/src/*.c

We are done. All that we need now is to run ./onion-csr.rb with the appropriate CA Signing Nonce, that is provided by HARICA in the preceding step. Likewise, we specify the path to our tor directory. EOTK creates project-specific tor directories at the following location:

~/eotk/projects.d/<projectname>.d/<truncated onion domain>-v3.d/

For our example.com project, we would run the following command:

./onion-csr.rb -n <NONCE> -d ~/eotk/projects.d/example.com.d/exmple27tdg5nesxaao6-v3.d/

The utility should immediately output the onion CSR to the screen. We copy this and submit it to HARICA. When pasting the onion CSR, make sure that there are no extra newlines at the end, which will cause it to be rejected.

Once our "Onion CSR" is submitted, domain ownership validation is complete. All that's left is to accept the certificate, make payment, and then download the signed TLS certificate files.

Naturally, I pressed "decline" at this stage, since I was going through the process only for example. But once you click accept, you'll be able to pay for, and then download the certificate – which will be made available as a certificate.pem file.

We will now go through the final step of this guide, which is installing these signed, trusted TLS certificates into EOTK.

Installing Signed TLS Certificates into EOTK

You should have the signed certificate available as a .pem file. It will have a 32-digit hexidecimal name like AF6EC6C9FEE27EB9769B47B040A71832.pem (a random example, not a real key). I'll call it certificate.pem for easier reference. You should also have a password-protected private key file named privateKey.pem. We must first transfer both of these files to our remote server:

rsync -a certificate.pem username@remote:/path/to/destination
rsync -a privateKey.pem username@remote:/path/to/destination

These files must be installed within the ssl.d folder of our EOTK project directory. These files must be renamed to the truncated 20-character long onion domain name, followed by the -v3 suffix. Your public key must be given the .cert file ending, and the private key must be given the .pem file ending. The specific naming of these files is important – if you misname them, EOTK will not be able to use them!

The best way to not mess up is to simply copy them on top of (i.e. replace) the existing self-signed TLS certificates which EOTK automatically generates. We can do this with our certificate file first:

cp certificate.pem eotk/projects.d/example.com.d/ssl.d/exmple27tdg5nesxaao6-v3.cert

Our privateKey.pem is passphrase protected. Before we copy it, we must first unlock it with our passphrase. We do this by using the openssl command to create an unlockedKey.pem file. The command will prompt you for your passphrase.

openssl ec -in privateKey.pem -out unlockedKey.pem
# Enter private key passphrase

Once that's done, we can copy unlockedKey.pem on top of the existing self-signed private key.

cp unlockedKey.pem eotk/projects.d/example.com.d/ssl.d/exmple27tdg5nesxaao6-v3.pem

Now, our signed TLS certificates are installed! In order for them to take effect, all we have to do is to instruct EOTK to reload its internal Nginx server:

./eotk nxreload example.com

Now navigate to our .onion domain in the Tor Browser. As you can see, there are no longer any certificate warnings, and our connection is verified by the Hellenic Academic and Research Institution Cert. Authority.

Now, with a trusted TLS certificate installed, your website's Onion Service is truly production-quality. By using EOTK, you have made your website available as a native onion site using the same framework that large organisations and media outlets use!

Configuring the Onion-Location Header in Nginx

One last fun step. Now that your website is available as an Onion Service, how do you publicise this information? You can put a link in the footer of your website (as I have done), but there exists a standard called the Onion-Location header. You can configure your clearnet website to pass the Onion-Location header with every request. Browsers which know how to inteprete the header (such as the Tor Browser, and the Brave Browser), will detect the header and inform the user that an .onion link is available.

The Tor Project maintains instructions for both Nginx and Apache. For Nginx, all you have to do is to add the following directive to your clearnet nginx.conf file:

add_header Onion-Location https://<your-onion-address>.onion$request_uri;

Likewise, you can also add a <meta> tag to your website's HTML code, should that be more convenient:

<meta http-equiv="onion-location" content="https://<your-onion-service-address>.onion" />

Make sure to specify the protocol as https://, since EOTK always enforces TLS connections. No matter the method, when a user navigates to your website using either the Tor Browser or Brave, they will see the following button:

Congratulations, now your website's Onion Service is complete!

Conclusion

By following this guide, you should have been able to create a production-quality mirror which enables you to make your website available as a native Onion Service over the Tor Network. The Onion Service mirror will allow users to access your website securely and anonymously, delivering your content to audiences that are previously inacessible.

For me, I know that on a practical sense, few readers will access my blog over the Tor Network. After all, my website is a small, independent blog dedicated to Philosophy (and the odd Computing article), with a readership measuring in the dozens. But I believe in empowering my readers whenever I can, and even if a single reader is to benefit from my Onion Service, all this effort will have been worth it. For the pursuit of philosophy is an universal activity, among the characteristic activities of the human being.

I hope this guide has been useful to you, and that it was able to help you deploy your own instances of EOTK. More Onion Service sites will help empower users, especially those who face censorship and oppression.

Thank you for reading my tutorial. For further content, feel free to subscribe to my RSS feed. If you have any thoughts, feedback, or got stuck in any way, feel free to contact me. I always enjoy meeting interesting strangers! Be sure to check out my other content: I also write about Galileo, Special Relativity, and NixOS.

Hello there. I am Shen, and I study Philosophy and Computer Science. This is a guide on creating fully reproducible, PDF/A-compliant documents in LaTeX. LaTeX documents are the gold-standard in academic publishing and typesetting, and this guide will help you make your documents more accessible, better structured, and suitable for long-term preservation for the future.

This guide is suitable for the intermediate LaTeX user. You do not need to know any TeX programming at all. The method described here is compatible with existing documents and templates.

Introduction to PDF/A-Compliance, and Fully Reproducible PDFs.

We will begin with an introduction on PDF/A compliance, reproducible builds, and why they are beneficial for LaTeX documents. If you are familiar with these topics already, feel free to visit the next section instead.

What is the PDF/A standard, and why should my document be PDF/A-compliant?

The PDF/A standard is an ISO standard that aims to make PDF-documents better suited for long-term preservation and archiving. A PDF is supposed to be a paper-replacement, after all. And documents (whether digital, or paper) must be readable, even decades after they are created. LaTeX is used for many documents those long-term preservation is important, such as theses, dissertations, and documentation. This is especially true for documents that are submitted to digital libraries and university archives, which make them available for posterity.

Unfortunately, regular PDF documents are unsuitable for long-term preservation. Fonts used in the document are often linked from the operating system, and media files could be compressed using compression algorithms that are proprietary, with intellectual property constraints. If you have any colours in your document at all, they can only be displayed with a device-dependent color-profile, which may change or become unavailable decades down the line. All of these factors mean that a PDF created today is not guaranteed to look the same, or even be readable at all, in the future.

The PDF/A standard solves this issue, by requiring the PDF to include all components necessary for its display. Font files are embedded into the PDF, and only open source compression algorithms are used for media. A device-independent color profile is embedded into the document, so the PDF-viewer of the future will know how to render the content, even if we stop using RGB.

More importantly, PDF/A compliance is the first step towards creating more accessible documents. PDF/A compliant documents include rich metadata, which allows the authorship, topic, and cataloguing information of the document to be inferred automatically. Users of screen-readers and braille displays will be better accommodated by PDF/A compliant PDFs. Although this guide does not present the creation of fully-accessible (i.e. structured) PDFs, a compliant-PDF is the first step towards universal accessibility.

What are fully reproducible builds, and what benefits does it have for LaTeX documents?

A fully reproducible build, is a compilation (i.e. build) process that guarantees a specific output, given a specific input (i.e. source code). The professional discipline of software development has made significant efforts towards reproducible builds for software, due to many well-founded security benefits. However, for LaTeX we are interested in reproducible PDFs primarily for the purposes of long-term preservation and archiving.

How can you be sure that your LaTeX source code will generate the exact same PDF, both on your laptop, and on a colleague's computer? If a historian of the future were to compile your document, could she be certain that her result was the same as your PDF from the past? LaTeX documents are not bit-for-bit reproducible. If you build your document twice, the resulting PDFs will have different hashes, even if the source code remained unchanged.

There can be many reasons why a PDF will need to be re-compiled from it's original LaTeX. If we know that the build process is fully reproducible, we can be certain that we will have the same document as the original, no matter how many years have passed.

Creating PDF/A Compliant documents in LaTeX

This tutorial will take the form of a dialectic, where I walk the reader through the steps and decisions involved. A complete configuration will only be provided at the end. This way, you will understand the reasoning behind the choices made, and gain knowledge, rather than merely correct opinion.

Levels of PDF/A Compliance:

There are three different levels of PDF/A compliance:

• PDF/A-3a The a level denotes the highest level of compliance. These are structured, tagged, and fully-accessible documents capable of text-reflowing (e.g. like an ebook). LaTeX is not capable of creating these types of PDFs at this point in time.
• PDF/A-3b The b level denotes a basic level of compliance. These documents are self-contained, and archival grade.
• PDF/A-3u The u level is simply PDF/A-3b, with the additional bonus of keeping the text of the PDF as Unicode. This is a good idea, so we will be using pdfx with the a-3u option, to create this type of document.

We will be using LuaLaTeX as the underlying LaTeX engine. Here's a quick overview of the differences between LuaLaTeX and other engines (XeLaTeX, etc.) LuaLaTeX is the successor to pdfLaTeX, and offers native Unicode support, and packages like microtype offers more features for it.

We will be loading the pdfx package in order to add PDF/A support to our document.

Note on mmap, cmap, and why we do not need them.

Please note that unlike many other guides, we will not be using mmap or cmap. These packages are used to provide character map files for our documents. However, lualatex and fontspec support modern OpenType fonts in the first place, which comes with character-maps embedded within the font files themselves. mmap and cmap do not even support the modern TU encoding (TeX Unicode) which lualatex uses.

What is the mmap package, and why do some people use it? Have you ever had the experience where you tried to copy some text from a PDF, but when you pasted the selected text, the paste was incorrect? This is because documents that contain ligatures (i.e. 'th,' 'fl,' or 'ae') render them as a single glyph, or 'character.'

If the PDF does not contain a character map table, than when you select the text, it will not know that the ligature represents a combination of two actual characters. Having character-map tables is especially important for screen-reader support, or automatic text-search. Once again, we will not encounter this problem, using LuaLaTeX and modern unicode OpenType fonts.

Finding the Correct Load Order for the pdfx Package

Let's begin with a regular LaTeX document, that is similar to ones that you may have right now.

\documentclass[
  12pt,
  a4paper
]{article}

% Packages
\usepackage{fontspec}
\usepackage{libertinus-otf}
\usepackage{geometry}
\usepackage{microtype}
\usepackage{amsmath}
\usepackage{xcolor}
% ...
\usepackage[style=mla, backend=biber]{biblatex}

% Hyperref must be loaded last!
\usepackage{hyperref}

\title{A LaTeX Document}
\author{Shen}
\date{\today}

\begin{document}
\maketitle
\tableofcontents

\section{Introduction}
The body of the document begins here.

\printbibliography
\end{document}

This is the kind of document that any intermediate LaTeX user will have. We use common packages like geometry, microtype, or amsmath – and we manage our citations using biblatex. We enable unicode and font support with fontspec. This is not some contrived minimal example, but rather a realistic example – one that reflects the complexity of a paper or article. We even use hyperref and xcolor, to give our citations and footnotes blue-colored hyperlinks.

How do we make this document PDF/A compliant? Do we simply load the pdfx package at the top of our load-chain?

% Packages
\usepackage[a-3u]{pdfx}

\usepackage{fontspec}
\usepackage{libertinus-otf}
\usepackage{geometry}
\usepackage{microtype}
\usepackage{amsmath}
\usepackage{xcolor}
% ...
\usepackage[style=mla, backend=biber]{biblatex}

% Hyperref must be loaded last!
\usepackage{hyperref}

The above attempt will not work. The document will compile, but if your paper has any footnotes, figures, or internal links at all, hyperref would experience errors in creating hyperlinks. This issue occurs because the pdfx package loads hyperref internally, and by loading pdfx at the top of our load-chain, we end up loading hyperref first.

This is a problem. The hyperref package modifies and patches many LaTeX commands (such as \section, \footnote, etc), and this is why it must be loaded last:

Make sure comes last of your loaded packages, to give it a fighting chance of not being over-written, since its job is to redefine many LATEX commands
– Section 3: Implicit Behaviour, Hyperref manual

The pdfx package also loads xcolor internally, hence we have further duplicated our efforts with loading xcolor. Let's change our code now, to account for pdfx, and move it towards the end of our load chain.

% Packages
\usepackage{fontspec}
\usepackage{libertinus-otf}
\usepackage{geometry}
\usepackage{microtype}
\usepackage{amsmath}
% ...
\usepackage[style=mla, backend=biber]{biblatex}

% pdfx loads both hyperref and xcolor internally
% \usepackage{hyperref}
% \usepackage{xcolor}
\usepackage[a-3u]{pdfx}

The above attempt will solve issues with hyperref, and now footnotes, citations, and labels will be linked properly. Hence, the first complete, working configuration, would look like this:

\documentclass[
  12pt,
  a4paper
]{article}

% Packages
\usepackage{fontspec}
\usepackage{libertinus-otf}
\usepackage{geometry}
\usepackage{microtype}
\usepackage{amsmath}
% ...
\usepackage[style=mla, backend=biber]{biblatex}

% pdfx loads both hyperref and xcolor internally
% \usepackage{hyperref}
% \usepackage{xcolor}
\usepackage[a-3u]{pdfx}

\title{A LaTeX Document}
\author{Shen}
\date{\today}

\begin{document}
\maketitle
\tableofcontents

\section{Introduction}
The body of the document begins here.

\printbibliography
\end{document}

Managing PDF Metadata, and hyperref Configuration

The above example will work, but our document is not complete yet. PDF/A requires us to specify certain document metadata, which will be used to help cataloguing, as well as make it easier for automated tools to injest our PDF. The pdfx documentation manual states:

Standards-compliant PDF documents require document-level metadata to be included. This, known as an ‘XMP packet,’ is like having a library catalog card included within the PDF itself.
...
[The advantages of this metadata are:]

For a librarian: cataloguing information is available within the file itself, without the need to search explicitly in the visual layout of the content or elsewhere;

All actual libraries cataloguing this PDF can have consistent information; including web-based indexing sites such as Google.

For the author(s): who can specify the kind of information most appropriate to help readers understand the nature and purpose of the document.

This metadata is also read by PDF-viwers, which allows them to display the title of the document in the window bar of the PDF viewer, for example.

So how do we include this XMP-packet of metadata? Users who are familiar with hyperref may specify their document's metadata like this, alongside other hyperref-specific configuration options:

\usepackage[
    % First specify PDF metadata with hyperref
    pdftitle   = {A PDF-A Tutorial},
    pdfauthor  = {Shen},
    pdfsubject = {A guide on creating PDF-A compliant documents},
    pdfcreator = {LuaLaTeX},
    % Now configure settings for hyperref itself
    colorlinks = true,
    allcolors  = blue,
]{hyperref}

You might also use hyperref's command \hypersetup to specify these options separately from the package load (which is good practice):

\usepackage{hyperref}

% ...

\hypersetup{
    % First specify PDF metadata with hyperref
    pdftitle   = {A PDF-A Tutorial},
    pdfauthor  = {Shen},
    pdfsubject = {A guide on creating PDF-A compliant documents},
    pdfcreator = {LuaLaTeX},
    % Now configure settings for hyperref itself
    colorlinks = true,
    allcolors  = blue,
}

Some clever people may even the pdfusetitle option in hyperref at load, which automatically defines the pdftitle and pdfauthor, attributes from the document's \title and \author declarations:

\usepackage[pdfusetitle, colorlinks=true, allcolors=blue]{hyperref}

Unfortunately, none of the above methods will work using the pdfx package. The pdfx package requires the PDF's metadata to be specified in a separate .xmpdata file, because it needs to do transform the metadata into a specific XMP packet that is then embedded into the PDF.

This .xmpdata is given the same name and in the same location as our primary .tex file. If your document is called paper.tex, than you'll have to create a paper.xmpdata file in the same location. An example .xmpdata file will look like this:

\Author{Shen}
\Title{
  Default PDF Document Title (For PDF/A Compatibility)
}
\Language{English}
\Keywords{keyword1\sep keyword2\sep keyword3}
\Publisher{Self-Published}
\Subject{
  Description of the PDF's subject
}
\Date{2022-02-26}
\PublicationType{pamphlet}
\Source{https://github.com/ShenZhouHong/latex-essay}
\URLlink{https://github.com/ShenZhouHong/latex-essay}

It uses the same syntax as TeX, and we specify our metadata in this document. The pdfx documentation on CTAN.org lists other fields and options. Now, when we compile the document, the pdfx package will automatically seek out and find the .xmpdata file.

It is important to keep in mind that we cannot use the pdfusetitle option for hyperref anymore, and attempting to pass that option will result in an error.

Now, the only thing we must do is to pass the regular options we use for hyperref through \hypersetup, since we are not loading hyperref directly anymore, and cannot pass options to it from the \usepackage line.

Complete Example of PDF/A-Compliant Document

Hence, the complete example of our PDF-A compliant document will look like this:

\documentclass[
  12pt,
  a4paper
]{article}

% Packages
\usepackage{fontspec}
\usepackage{libertinus-otf}
\usepackage{geometry}
\usepackage{microtype}
\usepackage{amsmath}
% ...
\usepackage[style=mla, backend=biber]{biblatex}

% pdfx loads both hyperref and xcolor internally
% \usepackage{hyperref}
% \usepackage{xcolor}
\usepackage[a-3u]{pdfx}

% We use \hypersetup to pass options to hyperref
\hypersetup{
  colorlinks = true,
  allcolors  = blue
  % ...
}

\title{A LaTeX Document}
\author{Shen}
\date{\today}

\begin{document}
\maketitle
\tableofcontents

\section{Introduction}
The body of the document begins here.

\printbibliography
\end{document}

And here is the corresponding paper.xmpdata file. Remember to place it in the same directory as your main .tex file (i.e. the one that you run lualatex on).

\Author{Shen}
\Title{
  Default PDF Document Title (For PDF/A Compatibility)
}
\Language{English}
\Keywords{keyword1\sep keyword2\sep keyword3}
\Publisher{Self-Published}
\Subject{
  Description of the PDF's subject
}
\Date{2022-02-26}
\PublicationType{pamphlet}
\Source{https://github.com/ShenZhouHong/latex-essay}
\URLlink{https://github.com/ShenZhouHong/latex-essay}

Congratulations! Now you have created a fully PDF-A compliant document!

Checking for PDF-A Compliance using PDF Validation Tools

The last step that we can take is to check for our document's compliance. Adobe Acrobat Pro's Preflight tool is still the gold standard in this area, as Adobe has been instrumental in developing the PDF standard as a whole. However, there are also quality open-source validators that function nearly as well, if not identically.

We will be checking our resulting PDF document for PDF/A-compliance using the Open Preservation Foundation's VeraPDF tool. VeraPDF is an open source (GPLv3+), cross-platform PDF standard validator that's available on Windows, Mac OS, and Linux. It's a Java program, so you will have to install a Java Runtime. I recommend using the OpenJDK JRE. The latest version is available on Ubuntu simply as the default-jre apt package:

sudo apt install default-jre

The VeraPDF tool can be downloaded from the Open Preservation Foundation's website, which is linked here. It comes as a .zip file with an verapdf-install shell script and verapdf-install.bat file. For linux, you'll want to execute the verapdf-install script, by first granting it executable permissions:

chmod +x verapdf-install
./verapdf-install

This would start the GUI installer, and allow you to install the validator into a directory /verapdf at a location of your choice. We can run the GUI validator by simply executing it from the terminal like before:

chmod +x verapdf-gui
./verapdf-gui

The only slightly tricky part left is that if you are using a modern, high-resolution display, the Java program might not have a GUI that's large enough. The first time I ran the program, it looked like this, and was unusable:

A simple fix for that is to open the ~/verapdf/verapdf-gui file. It's a simple shell script that launches the Java GUI program, and all you have to do is to change the SCALE_FACTOR=1.0 line to a larger integer, like 1.5 or 2.0.

Now the GUI is appropriately sized. All we have to do is to select our PDF, and click Execute. VeraPDF will tell us if the document is compliant with our variant of the PDF/A standard.

Congratulations on creating your first PDF/A-compliant LaTeX document!

Fully Reproducible LaTeX Documents using Makefile

All that remains for us to do now, is to make this document fully reproducible, so that compilations of the same source code will always result in a PDF with the same hash.

Creating fully bit-by-bit reproducible programs is quite difficult for regular software programming, but for LaTeX it is quite easy. First, we must make sure we are not using any LaTeX commands that change over time. The only common one would be the \date{\today} line. We'll remove that to specify a specific date.

Finally, the pdfx package embeds the creation date and modification date of the file into the PDF itself. As that changes every time we run lualatex, this will result in PDF files with different hashes. Thankfully, lualatex respects the SOURCE_DATE_EPOCH environment variable.

SOURCE_DATE_EPOCH is a standardised environment variable that is read by compilers and build toolchains to set the modification date of a binary (or a PDF file) to a specific date and time. It's a specification made by the Reproducible Builds project.

The variable takes an integer that represents the number of seconds elapsed since the Unix epoch, and we can get that integer for today by simply running:

date +"%s"

If you are running lualatex directly in the terminal, all you have to do to make your document reproducible, is to export this variable (for a set value, of course), and have it be available every time you run lualatex:

export SOURCE_DATE_EPOCH=1646013411
lualatex paper.tex

This is a little burdensome, and besides, you'll have to write down whatever epoch you started with. Furthermore, it would be strange having the creation-date and modification-date of the document fixed permanently.

A much better way to use SOURCE_DATE_EPOCH to make our LaTeX documents reproducible, is to use a Makefile, and tie the SOURCE_DATE_EPOCH to the time of our latest git commit.

SOURCE_DATE_EPOCH=$(git log -1 --pretty=%ct)

This way, we'll always have a realistic modification date for our PDF documents, that'll remain constant for every given commit. By doing so, you'll be certain that your git repository will produce definite PDFs for every commit in it's history, that will be bit-for-bit identical. We'll create a Makefile in the directory where our paper.tex and paper.xmpdata files are located:

COMMIT_EPOCH = $(shell git log -1 --pretty=%ct)

# Makes sure latexmk always runs
.PHONY: paper.pdf all clean
all: paper.pdf

paper.pdf: paper.tex
	SOURCE_DATE_EPOCH=$(COMMIT_EPOCH) latexmk -pdf -lualatex -use-make $<
clean:
	latexmk -c
delete:
	latexmk -C

Now, every time you run make to compile your PDF, you can be certain that the resulting document is fully reproducible. And of course, if you are using this makefile in a project that does not use git, simply set COMMIT_EPOCH to a fixed value.

Conclusion

Now your document is both PDF/A compliant, as well as fully reproducible. You can be reassured that your documents will be available, even many decades into the future. Should you ever need to recompile the PDF from source, you'll get to verify that the outputs are the same by comparing the hashes of the new PDF, with the old PDF.

Creating reproducible, PDF/A compliant documents is increasingly important. As organisations and universities expand their digital archives, PDF/A compliance is increasingly becoming a requirement for many types of documents.

LaTeX Template for the Humanities

If you are interested in creating PDF/A compliant, fully-reproducible documents, make sure to also check out my LaTeX essay template! Over the course of four years, I have created and refined a simple LaTeX template for writing in the academic humanities. It implements the contents of this tutorial, and more, offering features such as:

• Custom Microtype Protrusion Settings for Hanging Punctuation.
• Typography Tweaks, Adjustments, and Custom Headers and Footers.
• BibLaTeX Integration for Citation Management

The template is available on Github, and it's open source. Feel free to check it out for inspiration!

Thank you for reading my LaTeX guide! I also write essays on Philosophy, Metaphysics, and NixOS. Let me know if you have any suggestions, comments, or advice – I always enjoy conversations with interesting strangers! If you want to be notified of future content, feel free to subscribe to my RSS feed.

Hello there. I'm Shen, and I study Philosophy and Computer Science. This is the start of a three-part series on the Theory of Relativity, from a philosophical perspective.

All Philosophy is a pursuit towards being. Whether it is political philosophy (what is legitimacy?), epistemology (what is the truth?) or even just the personal contemplations of an individual life (see Montaigne's Essays), we must have some sense of the ontology of our human-reality, in order to conduct philosophical inquiry in the first place. But yet, it is important to ground our inquiries in the facticity of our human-reality – for if we let ourselves get carried away, we might end up talking about a metaphysics which have no correspondence to our human-reality at all – in other words, we would be simply "building castles in the clouds."

Hence, any rigorous metaphysical inquiry of metaphysics must at minimum acknowledge the insights of the empirical sciences. In particular, in order to investigate space and time, we must consult not only our phenomenological experience of spatiality and temporality – but also the physical foundation of space and time. After all, Physics is a component of our human-reality, for we live in an objective reality in which the phenomena that we experience are not simply subordinate to our will. With these thoughts in mind, I began a study of Einstein's Relativity – so that I can better understand the principles of our physics, and derive insights that will better inform me on further inquiries in metaphysics as a whole.

In this process, I completed a reading of Einstein's 1917 book Relativity: The Special and The General Theory, as well as a study of his 1905 paper On the Electrodynamics of Moving Bodies. These were highly satisfying readings, which I enjoyed greatly – and with these thoughts on my mind, I find myself ready to present some contemplations on Einstein's Special and General Relativity. This article is partly an explanation of Relativity, partly a review of Einstein's book – and wholly a reflection of my contemplations, as a Philosophy student reviewing Physics.

What is the Theory of Relativity?

To begin, what is the Theory of Relativity? What solutions does it offer, and what problems does it aim to solve?

Our human-reality is a spatial and temporal reality. We exist as beings with definite extension in space, as well as a location in time. Our existence is subject to empirical principles which apply to us irrespective of our internal desires – hence we live in an objective reality. One of the necessary conditions of this objectivity, is for the same empirical principles to apply to any human being, irrespective of their perspective. We must all be subject to the same natural laws, in order for these laws to be objective in the first place.

The student of Physics can surely agree to this conclusion, for it is essentially a formulation of a more familiar principle. This is the principle of Galilean Relativity, which states that the laws of Newtonian mechanics are true for observers in any reference frame. Whether you are "at rest," on a train, or jetting across the globe on an aircraft at 300 kilometers per hour – the kinematic equations and laws of motion are equally true for all of us.

Why do we call this principle Galilean Relativity? It is a principle of relativity, insofar that the events (i.e. the phenomena, for the student of Philosophy) of a system in any reference frame is directly relatable to any other reference frame. There will always exist some method of transformation, some function, that will allow you to take the event(s) of one reference frame, and translate it into the terms of another – in a way where this transformation will preserve the underlying laws and equations of Newton. And it is Galilean, because this principle was first expressed by Galileo Galilei, in his Dialogues Concerning the Two Chief World Systems – in the most charming little thought experiment, that I will reproduce for the pleasure of my readers below:

Shut yourself up with some friend in the main cabin below decks on some large ship, and have with you there some flies, butterflies, and other small flying animals. Have a large bowl of water with some fish in it; hang up a bottle that empties drop by drop into a wide vessel beneath it.

With the ship standing still, observe carefully how the little animals fly with equal speed to all sides of the cabin. The fish swim indifferently in all directions; the drops fall into the vessel beneath; and, in throwing something to your friend, you need throw it no more strongly in one direction than another, the distances being equal; jumping with your feet together, you pass equal spaces in every direction.

When you have observed all these things carefully (though doubtless when the ship is standing still everything must happen in this way), have the ship proceed with any speed you like, so long as the motion is uniform and not fluctuating this way and that. You will discover not the least change in all the effects named, nor could you tell from any of them whether the ship was moving or standing still.

In jumping, you will pass on the floor the same spaces as before, nor will you make larger jumps toward the stern than toward the prow even though the ship is moving quite rapidly, despite the fact that during the time that you are in the air the floor under you will be going in a direction opposite to your jump. In throwing something to your companion, you will need no more force to get it to him whether he is in the direction of the bow or the stern, with yourself situated opposite. The droplets will fall as before into the vessel beneath without dropping toward the stern, although while the drops are in the air the ship runs many spans. The fish in their water will swim toward the front of their bowl with no more effort than toward the back, and will go with equal ease to bait placed anywhere around the edges of the bowl. Finally the butterflies and flies will continue their flights indifferently toward every side, nor will it ever happen that they are concentrated toward the stern, as if tired out from keeping up with the course of the ship, from which they will have been separated during long intervals by keeping themselves in the air.

And if smoke is made by burning some incense, it will be seen going up in the form of a little cloud, remaining still and moving no more toward one side than the other. The cause of all these correspondences of effects is the fact that the ship's motion is common to all the things contained in it, and to the air also.

– Galileo, Dialogues Concerning the Two Chief World Systems.

What is the Problem of Galilean Relativity, which Einstein wishes to Solve?

Galilean Relativity is reasonable and convincing, for it broadly agrees with both our lived experience, as well as rigorous (and accessible) experiments. But yet, at the turn of the 19th century, Physicists began to encounter seemingly inexplicable phenomena, which Galilean Relativity is unable to account for. What were these difficulties, which pointed towards the insufficiency of Galilean Relativity? They laid chiefly in the field of electrodynamics – the study of the behaviour of moving electromagnetic bodies. And the two most apparent difficulties were on the electrodynamics of moving bodies, and the isotropy of the speed of light.

The Electrodynamics of Moving Bodies

Move a magnet in motion past a conductor at rest, and you'll generate a moving magnetic field. As the moving field perturbs the conductor, the moving magnetic field will induce an electric field in the conductor. The electric field in the conductor becomes observable as a measurable current on our multimeter. This is Faraday's Law of Induction in effect.

\[\nabla\times\vec E=-\frac{\partial \vec B}{\partial t}\]

But yet, there exists a puzzling asymmetry. For when we move a conductor in motion past a magnet at rest, there is no moving magnetic field. And yet, an electrical current is induced in the moving conductor nonetheless. There is no electric field anywhere to be found, as there is no magnetic field in motion.

Now, it seems reasonable for us to observe a current in the second case as well as the first, for moving a magnet past a conductor and a conductor past the magnet are the same interaction, seen from different perspectives (i.e. reference frames, for the Physics student). And yet, classical electrodynamics was unable to provide a singular account for both cases. For the latter case, a different mechanism of action has to be proposed, relating to the electromotive force instead. The details of both cases are not necessary for our investigation – but the key takeaway is that the theory of classical electrodynamics required two different accounts for the same underlying phenomena. The laws of our physical reality are not the same irrespective of our reference frame, but somehow, we have fundamentally different laws for what is essentially the same phenomena.

The (Anti-)Isotropy of the Speed of Light

Imagine a train, an embankment, and two observers – one on the train, and one on the ground. The observer on the train will perceive herself to be stationary, while the observer on the ground will perceive the observer on the train to move at the speed of the train itself.

If the observer on the train were to throw a ball towards the front of the train, the speed of the ball will appear to be the speed of the train plus the speed of the ball, for the observer on the ground. And likewise, if her ball were to be thrown towards the rear of the train, than the observer on the ground will perceive the ball to be the speed of the train minus the speed of the ball.

We use the word anti-isotropic to describe this phenomena, where the speed of the ball (as observed from the ground) is different, depending on whether the ball is thrown in one direction, or another. Naturally, for the observer on the train, the ball will move at the same velocity no matter which direction she throws it – but that is only true if we were on the reference frame of the train. As observers on the ground, the ball's velocity has to be anti-isotropic.

This anti-isotropy is not a violation of Galilean Relativity. For the velocities of the ball in respect to the train and in respect to the ground are merely quantities that refer to different reference frames. And there exists definite functions to "translate" the values from one reference frame to another – namely adding or subtracting the velocity of the train itself. This transformation preserves the kinematic equations and Newtonian laws of motion, so this anti-isotropy is a welcome sign that all is well in our physics.

At the close of the 19th century, Physicists tried to discover this anti-isotropy in the speed of light, and failed. Light, as an electromagnetic wave – must propagate within a definite medium, an electromagnetic field. This is the train from our analogy. And whatever our velocity is in relation to this field, we must be able to observe a similar anti-isotropy in the speed of light depending on whether we are moving with the motion of the field, or against the motion of the field.

Are we moving with the motion of the electromagnetic field, or against the motion of the electromagnetic field? Well, since the Earth orbits the sun in a circle, we would be moving both with the field, and against the field, at different points of our orbit.

Hence, a good way to uncover the anti-isotropy of the speed of light, is to measure it during different times of the year. This way, we will get to measure the speed of light while we are moving in different directions within this "lumiferous aether," and whenever we are aligned with it, light would move slightly faster, or slower, if we are moving against the aether.

But yet, despite the best and most exacting experiments of the Physcists of the time, we were unable to discover any anti-isotropy in the speed of light. The speed of light was the same, no matter our reference frame, in respect to the "aether." This culminated in the famous Michelson-Morley experiment of 1887, in which the existence of the "aether wind" was conclusively proven to be false.

This is a fatal problem for the Galilean Relativity of Classical Mechanics. Because now, there is a phenomena – the speed of light – which is the same no matter the observer's reference frame, or perspective. You could be traveling on a rocket at 95% the speed of light, and shoot a laser beam forwards – and that laser beam will move at exactly \(c\), for both you on the rocket – as well as me, an observer on the ground. Somehow, the speed of light is no longer objective, insofar Classical Mechanics is concerned. It has become a constant those constancy seems subjective to the observer's perspective – a metaphysical solipsism that is deeply troubling for our system of physics as a whole.

These are the two difficulties of Classical Mechanics, at the end of the 19th century. Somehow, the Galilean Relativity of Newton are unable to account for the theory of electrodynamics at all.

This is the background behind the project of Einstein's theory. We are faced with an incomplete system of physics, in which our laws of motion are invariant in respect to reference frame, but yet our laws of electrodynamics are not. But yet, it cannot possibly be the case that our laws are not invariant in respect to reference frame, for the laws of physics privilege no perceptual subject – it is an objective science.

Examples of this sort, together with the unsuccessful attempts to discover any motion of the earth relatively to the “light medium,” suggest that the phenomena of electrodynamics as well as of mechanics possess no properties corresponding to the idea of absolute rest. They suggest rather that, as has already been shown to the first order of small quantities, the same laws of electrodynamics and optics will be valid for all frames of reference for which the equations of mechanics hold good.
– Einstein, On the Electrodynamics of Moving Bodies

Concluding Remarks

What does it even mean to have physical laws which are not invariant in respect to a reference frame? Is this a possibility that we can tolerate in any case?

We cannot tolerate it. It would be a disastrous flaw within the ontology of our physics, for it destroys all possibility of objectivity. Physical laws must be the same no matter our perspective – both on the practical grounds of experimental physics, as well as on a priori principles of metaphysics too. To paraphrase Jean-Paul Sartre, any theory of knowledge presupposes a metaphysics. Ontologically, to state that physical laws are variant in respect to perspective, is to reduce our system of metaphysics to a dangerous solipsism – the world no longer has properties which are independent of my will.

This is where Einstein begins his pursuit. An attempt to furnish an account of physics, one in which both the laws of electrodynamics, as well as the laws of kinematics, are valid and invariant, for all reference frames. This is the problem of Galilean Relativity which Einstein's Relativity aims to solve.

We will examine Einstein's Special Relativity in the next installment of this series, followed by an account of Einstein's General Relativity as a whole. In particular, we will look at what Relativity means, in terms of the spatial and temporal condition of our human-reality.

Thank you for reading my article. For updates on the series, feel free to subscribe to my RSS feed. If you have any thoughts, comments, or feedback, feel free to contact me. I am particularly interested in conversations with Physicists and Physics students, who can help me further my understanding of Einstein. Write to me, and let me know of your thoughts!

Hello there. I'm Shen, and I'm a student of Philosophy and Computer Science. The subject of my undergraduate thesis is on Metaphysics, and I often come across friends and colleagues in the Sciences, who are unfamiliar with the topic. What is Metaphysics? What sort of questions does it investigate? What is its purpose and utility, as a subject in the first place? These are all worthy questions, so let us explore them today.

As fellow explorers of the world, we are all united by our dedication to the pursuit and understanding of our human-reality. Hence, the work of a Mathematician, Computer Scientist, Physicist, or Philosopher all stem from our fascination with the world, and its details.

This article is an introduction and an overview of metaphysics as a Philosophical discipline. But more importantly, I hope that through this understanding, it will present an understanding and appreciation of the beauty and unique perspective that metaphysics offers, as a means to comprehend the world.

Metaphysics is a study of Being

Metaphysics is the study of Being. What is the study of Being, you ask? That is an excellent question, and for us to come to an answer, it will be the most useful to examine the study of Being from the perspective of different fields.

We live in a world of "things", which exist as components of our reality. These "things" can be physical objects, such as a chair, a rock, a tree, or the laptop in front of you. But these "things," (which at this moment, we can only afford to speak loosely and imprecisely about) do not necessarily have to be physical. In fact, many of them are not. A computer algorithm, a mathematical equation, even your friend's phone number – all of these are examples of "things" which have no physical existence. But yet, they are all definite components of our human reality – for we can think of them, act with them, and reason about them in definite, and well-defined manners.

So, what do all of these "things" share in common? An equation is quite different from a table or a tree, after all! At this stage of our discussion, we can only discern one commonality between all of them, which is that they exist as components of our reality – even though this existence is not always physical. They exist, because they are.

Hence, these "things" are beings – components of our reality. An explanation or theory about the nature or behaviour of being is called ontology – the study of the ontos – being, or "that which is."

Now, we only have to ask: what is a study of being like?

The Sciences as an (Empirical) study of Being

The study of being may sound abstract and metaphysical, but it does not have to be. To begin, let us look at the empirical Sciences. Science, in its own way, is a study of being! In fact, each Science has a category of being that it investigates, and different methods to which this investigation is pursued.

Let us take the humble rock, a being that was first presented in our examples. What is the being of this rock? Hand it over to a Geologist, and she will examine it– and using the methods of her discipline, come to a rigorous account of the rock's being.

She could begin with an examination of the rock's material, by holding it up to the light. And marvel at the way that it's rough surface glimmers, as its details appear. There will be feldspar and muscovite, wholesome in their dullness. Amidst tiny inclusions of quartz, and amphibole, which lends a subtle sparkle, to the texture of the stone. A sample could be chiselled and pulverised for analysis – yielding realms of graphs and data, showing the exact distribution of silicates and carbons within. This is a material account of the rock's being – and the Geologist can conclude from it that the rock's being is granite – as determined by its constitutive minerals and chemistry.

But this is not the only account possible. She may reach the same conclusion, through entirely different means. Lacking the means and facility of a field laboratory, the Geologist can equally attempt an account by looking at the source of the rock instead, by discerning clues on how it came to be. She could look at the site of the quarry, situated against weathered bedrock of a weary continent. And examine the slopes of the landscape, and see the ruins of an ancient caldera in the formations around. With that, she can conclude that the efficient cause of the rock is through volcanic activity – that this sample was once a part of a lava column, of an eruption long ago, lost in geologic time. Hence, she will conclude that the rock's being is igneous, specifically igneous granite. The same being, but accounted for differently, in different means.

Both of these accounts are valid, even though they take on slightly different characteristics, and address facets of the rock's being in slightly different ways. It is a demonstration on how the study of being is one of the primary concerns of the Sciences, and not just the being of material objects either.

For is it not the task of a Computer Scientist, to discern the being of an algorithm? Do we not talk about different categories of algorithmic complexity, and rank programs by their behaviour? Given the same end, a programmer may use either a recursive function, or an iterative one – and in order to understand the differences in their performance, we must know the being of the algorithm, sometimes in deep, information-theoretic ways.

It is clear now, that we pursue the study of being in all of the constitutive sciences. So now it remains for us to ask: how does metaphysics differ, when it comes to the study of being?

Metaphysics as a study of all beings

Now, we approach the essential difference between metaphysics and the physical sciences. Metaphysics is a study of not only a being, within the method and discipline of a specific science. But rather, metaphysics is a study of the nature of all beings, in our human-reality.

What does it mean to be a discipline which aims to study all beings? After all, the being of a rock, and a table, seems fundamentally different from the being of an algorithm or equation. But yet, the fact that they all exist, implies some deep and fundamental commonalities.

Thus lies the difficulty and abstractness of metaphysical inquiry. If we want to provide an account for being, it must be able to satisfy not just one being, or a category of beings – but all beings, from oak trees to search trees, and everything in between. To quote the words of Martin Heidegger (a metaphysician of the first order):

The question is the broadest in scope. It comes to a halt at no being of any kind whatsoever. The question embraces all that is, and that means not only what is now present at hand in the broadest sense, but also what has previously been and what will be in the future. The domain of this question is limited only by what simply is not and never is: by Nothing
...
Given the unrestricted range of the question, every being counts as much as any other. Some elephant in some jungle in India is in being just as much as some chemical oxidation process on the planet Mars, and whatever else you please.
– Martin Heidegger, Introduction to Metaphysics.

Wow! Isn't it audacious, to think that we can attempt to account, or theorise, about the nature of all beings? Isn't it both scary, but also exciting? To think that there is something which puts me into a common relation, with a tree, with a boulder, with equations and phone numbers and elephants too – some elusive foundation behind our human-reality.

This is the pursuit of metaphysics. It is the study of the broadest question, the question of all beings. But along with it, metaphysics is also the study of the deepest question. The question of why are there beings at all. To quote Heidegger once again:

Why are there beings at all? Why—that is, what is the ground? From what ground do beings come? On what ground do beings stand? To what ground do beings go? The question does not ask this or that about beings—what they are in each case, here and there, how they are put together, how they can be changed, what they can be used for, and so on. The questioning seeks the ground for what is, insofar as it is in being.
– Martin Heidegger, Introduction to Metaphysics.

That question is the "elusive foundation" behind our human-reality, that metaphysics aims to pursue. We are all united by our being, by the fact that we are, as opposed to we are not. Why are there beings, asks Metaphysics. Why are there any beings at all?

The question of being is the fundamental question of metaphysics.

And just like with the Riemann hypothesis or the Standard model, or the apogee of any art and scientific endeavour, it remains an open question.

It is, one of the Unsolved Problems of Philosophy.

List of unsolved problems in philosophy - Wikipedia

Wikimedia Foundation, Inc.Contributors to Wikimedia projects

The Practical Necessity of Metaphysics

We will conclude today's discussion, with a quick examination of the utility of metaphysics in practice. I hope my sketch of metaphysics has awakened your wonder and curiosity, the same aesthetic appreciation that unites all of us in our studies and inquiries. At the same time, metaphysics – like any abstract scientific pursuit – is also a bountiful source of practical utility and applications. And although metaphysics is more abstract than most – the pursuit of its study opens up more derivative fields of inquiry, that can have surprising relevance to our day-to-day lives.

For you see, the relationship between metaphysics and philosophy, is akin to the relationship between physics and engineering – or computer science, and software development. The abstract, formal laws of the former, will allow us to derive the facts and methods of the latter.

The relationship between philosophy and metaphysics is keener than most, for many of the most interesting and practical questions of philosophy, find their ground eventually, in the question of being.

Consider the question of happiness, for one! What does it take, to live a happy live? If you were to ask Aristotle, he would have said that eudaemonia (a sense of virtuous flourishing, happiness) comes from acting according to the "characteristic activity of the human being." What is the characteristic activity of a human being? This is, ultimately – a question of ontology. It is a question about the being of a human!

Likewise, consider the question of politics. How does one create a good government? For Thomas Hobbes, a good government is one held by a strong, authoritarian sovereign, who holds absolute power over all men. How did he come to this conclusion? It's because Hobbes believes that the being of man is fundamentally selfish, with "the life of man [in nature being] solitary, poor, nasty, brutish, and short." But someone like Rousseau would have entirely different views on government, all due to a different conception of the ontology of humanity.

Finally, consider our personal lives, and our struggles with living. What is authenticity? What is true love? All of these are questions, which we will face in our own journey. And the answer to these questions, all have a foundation, in examining the being of our nature, as well as the being of love.

This is why metaphysics is so exciting for me. It is the theoretical physics of Philosophy, the understanding of which will further our exploration in other fields of the human condition.

Metaphysics is a study of being, which serves not only as the broadest question, but also the deepest question.

And through this study of being, not only will we come to a better understanding of the other beings in our reality.

But also examine our own beings, and learn what it means to be.

Thank you for reading my article on metaphysics. If you have any comments questions, or philosophical musings of your own, feel free to reach out and contact me! I am always looking for conversations with interesting strangers, and pen-pals of philosophy and contemplation.

Hey there! I'm Shen, and I'm a Philosophy and Computer Science student. My area of interest is in metaphysics and ontology – and in this tutorial, I will show you how to setup a workstation for Philosophy using NixOS.

This tutorial is designed for the Philosophy student who is familiar and comfortable with Linux in general, but new to NixOS. It may also be suitable for Computer Science students and Software Developers, with some modifications. The tutorial will walk you through the process step-by-step and provide ample links and explanations. We will be continuing from where the previous guide left off, starting with wireless networking, Home Manager, and a basic Sway graphical environment. In this guide (Part II), we will install and configure the software needed to practice philosophy:

• Firefox – For reading papers, and writing blog posts! Unlike Wittgenstein, we can (thankfully) publish more than one work in our lifetime.
• TeX/LaTeX – Used to write essays and longer-form content. LaTeX allows us to focus on the formal cause of our essay, instead of grappling with the material cause. Plato didn't write his dialogues on Microsoft Word, and neither should you.
• VSCodium – We will use VSCodium (an open-source fork of VSCode) to write LaTeX documents. We use VSCodium because it's extensibility allows us to use a custom language engine. The syntax highlighting will be helpful for students of the Analytic tradition as well. (Full disclosure: I am a student of the Continental tradition.)

For Part III, we will also do the following:

• Element Messanger – Needed for correspondance with other philosophers and students of philosophy. I'm available on Matrix – feel free to reach out to me about this post!
• Bluetooth (+HiFi Music Support) – Music is philosophy set to rhymth, and the contemplation of harmony is essential for the attunement of our soul. Just remember the advice of Socrates, and avoid listening to the Ionian and Lydian modes!

The ontology of this tutorial is dialectical in nature. Hence we will not provide a complete configuration (which would only be true opinion, and not knowledge) – but take the reader step-by-step through the process of building a NixOS workstation.

Installing and Configuring Firefox

Let's begin with installing and configuring Firefox. We'll be adding Firefox to our configurations.nix file using Home Manager. Having Firefox available on our NixOS system is as easy as adding:

programs.firefox = {
    enable = true;
};

Followed by running sudo nixos-rebuild --switch. But wait! We are not done yet. For this configuration would only give us a bare-bones, default Firefox installation. Could we manage our settings (and even browser plugins) through our NixOS config as well? That is certainly possible. In fact, Home Manager offers a wide array of options for configuring Firefox. All of these settings are held within the (Firefox) user profile, so we can begin by setting up a default profile for Firefox:

programs.firefox = {
    enable = true;
    profiles.default = {
        id = 0;
        name = "Default";
    };
};

It is within this profiles.default scope where we can customise our Firefox installation. Here, you can add directives such as userChrome or userContent, which allows you to change the look and interface of the Firefox UI in powerful ways. For example, you can use the userChrome directive to add custom CSS styles which can remove the Firefox tab bar. This would be useful if you are using a browser extension such as Tree Style Tabs.

Configuring Firefox Settings Through NixOS

For our tutorial, we will add the settings directive within profiles.default. This allows us to configure Firefox with our settings and preferences – which can be both as mundane as setting a homepage, to disabling the default telemetry, debugging, and the sponsored Pocket integration. Previously, you would have had to dig through Firefox's settings page, but with the power of NixOS, we can configure it all in one place! Let's begin by setting our homepage:

programs.firefox = {
    enable = true;
    profiles.default = {
        id = 0;
        name = "Default";
        settings = {
            # Browser settings go here
            "browser.startup.homepage" = "https://shen.hong.io/";
            
        };
    };
};

Note how each setting directive is a string, followed by an equal sign that points to it's value, and the line ends with a semicolon. Don't mess this up! The rest of the settings will all go within this scope.

Next, let's enable HTTPS Everywhere. This will ensure that our browser connects to websites only via HTTPS, and show a warning should it not be available. Don't worry, you can still visit (and whitelist) HTTP sites, but it's a good security measure for 2022.

# Enable HTTPS-Only Mode
"dom.security.https_only_mode" = true;
"dom.security.https_only_mode_ever_enabled" = true;

Continuing on this theme of security and privacy, lets enable some privacy settings such as the Do Not Track header, as well as configure Firefox's built-in tracking protection. These measures are no replacement for a dedicated browser extension such as UBlock Origin, but it's a good start.

# Privacy settings
"privacy.donottrackheader.enabled" = true;
"privacy.trackingprotection.enabled" = true;
"privacy.trackingprotection.socialtracking.enabled" = true;
"privacy.partition.network_state.ocsp_cache" = true;

Remember! Privacy is essential to the conduct and practice of true philosophy.

# Disable all sorts of telemetry
"browser.newtabpage.activity-stream.feeds.telemetry" = false;
"browser.newtabpage.activity-stream.telemetry" = false;
"browser.ping-centre.telemetry" = false;
"toolkit.telemetry.archive.enabled" = false;
"toolkit.telemetry.bhrPing.enabled" = false;
"toolkit.telemetry.enabled" = false;
"toolkit.telemetry.firstShutdownPing.enabled" = false;
"toolkit.telemetry.hybridContent.enabled" = false;
"toolkit.telemetry.newProfilePing.enabled" = false;
"toolkit.telemetry.reportingpolicy.firstRun" = false;
"toolkit.telemetry.shutdownPingSender.enabled" = false;
"toolkit.telemetry.unified" = false;
"toolkit.telemetry.updatePing.enabled" = false;

# As well as Firefox 'experiments'
"experiments.activeExperiment" = false;
"experiments.enabled" = false;
"experiments.supported" = false;
"network.allow-experiments" = false;

I admit, number of directives in the previous section was distressingly long. And I'm not even sure if the options listed are complete. If you happen to know of any other settings that will help improve the privacy of a Firefox user, please reach out and let me know! In any case, we are not done yet. The last thing that we must do in our Firefox configuration, is to remove and disable the Pocket extension. These settings should completely neutralise it – we both remove Pocket from the new tabs page, as well as delete it's API URL and the unique per-user authentication key.

# Disable Pocket Integration
"browser.newtabpage.activity-stream.section.highlights.includePocket" = false;
"extensions.pocket.enabled" = false;
"extensions.pocket.api" = "";
"extensions.pocket.oAuthConsumerKey" = "";
"extensions.pocket.showHome" = false;
"extensions.pocket.site" = "";

With all of the above settings, you should have a Firefox installation that's secure, privacy-friendly, and ready for use. Remember to run sudo nixos-rebuild --switch!

Where do I find additional settings for Firefox?

All of the settings above came from Firefox. Specifically, they are entries from Firefox's about:config page. If you enter that into your Firefox URL bar, it should open a special internal page which lists all the settings and their values. As far as I understand, these settings are how Firefox configures itself internally. What is an easy way to browse, or find more settings?

Unfortunately, I can't seem to find an official listing or directory of all the settings and explanations for their effects. I was able to find an user-contributed Wiki which lists many settings, but the entries appear to be a little dated. If someone can help me find an official list of all about:config entries and their usage, I would be very grateful. Let me know if you know where to look!

How do I configure extensions like uBlock Origin through NixOS?

This... isn't actually a rhetorical question. I genuinely wish to know! Right now, I have simply installed my Firefox extensions manually from within Firefox. I'd like to be able to manage my extensions through NixOS and Home Manager as well, but I'm not sure how. If you know of a way to manage Firefox extensions through Nix, I would be grateful if you can write to me.

Installing LaTeX on NixOS

After FireFox, installing LaTeX is a relatively simple affair. LaTeX is a document typesetting engine used to produce academic and technical documents, and it is an essential part of my personal philosophy workflow. LaTeX under NixOS is offered by the texlive scheme, with different schemes for different numbers of LaTeX packages. If space or installation time is a constraint, it may be useful to go with a smaller scheme like texlive.combined.scheme-medium, but I personally didn't worry and went with the full installation with all LaTeX packages:

# Packages installed (under Home Manager)
home.packages = with pkgs; [
    # Full LaTeX installation with all packages
    texlive.combined.scheme-full
];

The above directive, followed by a quick sudo nixos-rebuild --switch – and LaTeX is installed! It's available via latex or lualatex. Isn't NixOS magical?

LaTeX support in NixOS is important to me. I write all my papers in LaTeX – as it allows me to focus on the content of my discourse, rather than it's layout. That's not to say LaTeX documents aren't beautiful on their own, however! Over the years, I've made my own LaTeX template which comes with all the typographic bells and whistles of professional typesetting. It's also PDF/A compliant (archival grade), for the day I die and become one with the facticity of historicity. And of course, LaTeX's powerful TKZ-Euclide package allows the Galileo's Geometra filosofico to construct demonstrations of the same apodeictic certainty as Euclid.

The completely text-based format of LaTeX makes it easy to commit one's work to version control, which makes it easy to keep track of the progression of one's thoughts and changes through time. Immanuel Kant certainly wished that he had git while he was working on his Critique of Pure Reason. Alas, he didn't – and that's why modern editions of his text have to put up with very strange marginal page numbering schemes like 14[b] and 188[a], just to keep track of the differences between the first and second editions. Whatever the metaphysics of the future is, it will be version-controlled with git and paginated with diff.

Installing VSCodium, and Configuration for the Java Language Engine

Now that we have LaTeX installed, it's time to install and configure VSCodium so we have an editor to work with. VSCodium is an open-source fork of Microsoft's VSCode, a powerful and highly configurable text editor. Named in honor of Charles Cotin, the 15th century French philosopher and polyglot, both Cotin and VSCode are known for their support of many languages. VSCodium is used for academic writing, but it is also popular amongst Programmers and Software Developers as well.

# Install VSCodium under Home Manager
programs.vscode = {
    enable = true;
    # Make sure to use the open source vscodium
    package = pkgs.vscodium;
};

Add the above within Home Manager's scope in configurations.nix, and VSCodium should be available. You can verify it by running codium in the command line to launch it. However, we are not finished with it yet. We still have two plugins to install:

• LaTeX Workshop for LaTeX Syntax Highlighting and Support
• LTeX for Grammar and Spell-checking.

We'll install both plugins in the next step through VSCodium's GUI. Although Home Manager does offer a way to define VSCodium plugins through the configurations.nix file, the two plugins above do not appear to be packaged. If the situation has changed, and the plugins are available – please let me know!

Installing the LaTeX Workshop and LTeX Plugin through VSCodium GUI

First, we will install the LaTeX Workshop plugin. This plugin adds syntax highlighting for .tex files, as well as useful autocompletion for LaTeX and packages. Simply search for it in the Extensions tab of VSCodium. It should appear with the name LaTeX Workshop, with the extension ID of james-yu.latex-workshop. Install it and we're done – it does not require any additional configuration.

Next, we install the LTeX VSCode plugin. LTeX is a powerful grammar and spelling checker, with a deep integration with LaTeX and LaTeX packages. This is not a simple dictionary by any means – it uses the same syntax engine that Software Developers and Programmers use for computer code. Furthermore, advanced users can configure LTeX to use n-Gram based language models, pre-trained neural networks, and Word2Vec language models. It's certainly a plugin that the younger Wittgenstein would be proud of – just imagine how useful symbol definition lookups and keyword autocompletion would have been, when writing the Tractatus Logico-Philosophicus!

For the actual installation, likewise navigate to the Extensions tab of VSCodium, and search for LTeX. It should appear with the name LTeX – LanguageTool grammar/spell checking and the extension ID is valentjn.vscode-ltex. Install this now – we will have to configure it next.

Configuring Java for the LTeX plugin's Local LSP Server

The LTeX plugin works by running a local instance of a Language Server that implements the LSP (Language Server Protocol), which requires Java. Unfortunately, due to NixOS's idiosyncratic ontology, the included Java executable does not work, so we must first install Java using Home Manager and have it available to our user.

# Install the Open Source Java Runtime under Home Manager
programs.java = {
    enable = true;
    package = pkgs.jdk11;
};

We use the package directive to simply tell NixOS to use the open source implementation of the Java Runtime, instead of Oracle's version. Run a sudo nixos-rebuild --switch. Now, we must restart our workstation. Restarting is essential in this process: NixOS must not only install Java, but the Java executable must be included in the $PATH variable of our user. The $PATH is essentially a list of places that your computer searches through, when you run a program like vim or git. Likewise, NixOS must set the $JAVA_HOME variable, which is a variable that Java Programs expect. As far as I can tell, these variables are only set properly when the computer is restarted after installation.

Once your computer has rebooted, we must retrieve the $JAVA_HOME path. Do this by simply running:

echo $JAVA_HOME

In the terminal. You should see a path which leads to your Java executable. Copy this path, and navigate into the settings page of LTeX in VSCodium. Find the setting listed as ltex.java.path, and paste your $JAVA_HOME output there.

Now restart VSCodium, and the LTeX plugin should work! Now, you should have completed the configuration of your writing space. With VSCodium and LTeX, you have a solid tool for writing essays – the material of philosophical inquiry. And oh, what wonderful essays you will write!

But what kind of essays can I write? I don't know any philosophy! I don't know anything at all!

Socrates knew nothing, and he was the wisest man in Athens.

No, but seriously – what should I write about?

Why not write about yourself? Philosophy is fundamentally an act of contemplation – and there is no better form of contemplation than the contemplation of the self. Know thyself, was the inscription at the Temple of the Oracle of Delphi. There can be great pleasure and virtue, in writing essays about your world and what you know.

This is not a sophistry – I genuinely mean it. Write about yourself. There's no topic that you know better than your own being. And in this process, you will follow the path of Michel de Montaigne, who wrote essays too. Montaigne sat down one day, in the evening of his life. In front of a philosophy workstation not too dissimilar to our own. He had also pondered, about the question of what to write about.

He realised very quickly that he knew little. He saw himself before all the experts of the world – the geometers and theologians, and realised he can hope to offer nothing useful in that regard. He was a courtier and a statesman – an occupation usual for a nobleman of his rank. But his career is uninteresting, and his hair is greying, and he knew he was not going to create a treatise of political philosophy or any lofty work of grandeur and acclaim.

"What do I know?" Asked Michel de Montaigne. And so he decided to write, about the only thing that he knew well. He wrote about himself. He wrote for himself. He examined the world through his eyes, and tried to contemplate about its questions – without thought as to approval or greatness. He wrote essays On Friendship, and On Solitude. On the Education of Children. He wrote about the little things that he knew well. And throughout this all, he wrote with humility and contemplation, with genuine curiosity and goodwill. Above all, he wrote with frankness and sincerity. When the court was abuzz with sensation and scandal, upon hearing stories On Cannibals and "savages" from the New World, Montaigne wrote about how:

I do not believe, from what I have been told about this people, that there is anything barbarous or savage about them.

But directed the eye of his fellows towards the cruelty of the Inquisition, and the affairs of their own "civilised" states.

I consider it more barbarous to eat a man alive than to eat him dead; to tear by rack and torture a body still full of feeling, ... a practice which we have not only read about but seen within recent memory, not between ancient enemies, but between neighbours and fellow-citizens and, what is worse, under the cloak of piety and religion - than to roast and eat a man after he is dead.

It's hard to write, because we all ask ourselves Montaigne's question: "What do I know?" And just like Montaigne, we find ourselves facing the experts of the world, the great mathematicians and computer scientists – who have written volumes and monographs, irreproachable in their perfection. But you are still the foremost expert of yourself, if you strive towards contemplation and curiosity. And like Michel de Montaigne, you can write about your world, and what you know – in bits and dabbles and little blog posts too.

Ultimately, Montaigne's writings – his attempts at understanding his world – gave us the literary genre of the essay itself (essayer: French, "to try") . It catapulted him to fame, and he is considered one of the foremost philosophers and humanists of the European Renaissance. But this all began very slowly – with curious person, sitting in front of a workstation. A Philosophy workstation.

Who picks up their pen, and began to write.

sudo nixos-rebuild --switch
touch  my-essay.tex
codium my-essay.tex

Thank you for reading Part II of my guide in building a Philosophy Workstation with NixOS. If you have any questions or comments, feel free to contact me – I would love to know if my guide has helped you in any way! I enjoy exchanging emails with interesting strangers, and look forwards to your correspondence.

This tutorial is Part I in a series on configuring a default Minimal installation of NixOS to a state where productive philosophy can be done. We will start from a text-only environment that has minimal amenities (e.g. where the previous installation guide left off), and slowly build up our configuration.nix file until we have a graphical user interface, as well as all the applications that are conductive towards producing philosophy. In this series, we will complete the following:

• Configure networking using NetworkManager (e.g nmcli)
• Install low-level systems applications (e.g. git, wget, parted, and vim)
• Learn about and use Home Manager to manage our user-level applications.
• Install and configure the Sway display manager with the Wayland display server
• Fine tune the Sway graphical environment, enabling buttons for brightness and volume control, and enabling audio (Part II).
• Install and configure VSCodium, LaTeX, Firefox, and other tools needed to produce philosophy (Part II).

This tutorial is designed for the philosophy student who's familiar and comfortable with Linux in general, but new to NixOS. It may also be suitable for Computer Science students and Software Developers, with some modifications. It will walk you through the process step-by-step and provide ample links and explanations.

The ontology of this tutorial is dialectical in nature. Hence we will not provide a complete configuration (which would only be true opinion, and not knowledge) – but take the reader step-by-step through the process of building a NixOS workstation.

Installing and Using NetworkManager for WiFi Connections

The first, and one of the most important steps of any Linux installation, is to ensure that we have WiFi connectivity. After all, the Internet is one of the most direct embodiments of our Mitsein!

Adding NetworkManager to configurations.nix

There are many ways to manage Internet connectivity in Linux. We will use NetworkManager, a high-level command line interface that simplifies the process – as well as comes with many GUI interfaces which we can add later on. In your /etc/nixos/configuration.nix, add the appropriate options in the first level of nested brackets (right after imports and account configuration).

{ config, pkgs, ... }:

{
  imports =
    [ # Include the results of the hardware scan.
      ./hardware-configuration.nix 
    ];

  # ...
  
  # Networking Configuration
  networking.hostName = "my-philosophy-workstation"; # Define your hostname.
  networking.networkmanager.enable = true;
  # networking.networkmanager.wifi.backend = "iwd";

  # The global useDHCP flag is deprecated, therefore explicitly set to false here.
  # Per-interface useDHCP will be mandatory in the future, so this generated config
  # replicates the default behaviour.
  networking.useDHCP = false;
  networking.interfaces.eno0.useDHCP = true;
  networking.interfaces.wls6.useDHCP = true;
};

We will enable DHCP on both of our wired en0 as well as wireless wls6 interfaces – these options should already be there from your original installation.

Note how all of our options (e.g. such as hostname, networkmanager.enable, etc) are within the networking namespace. This makes sense, as they all relate to configuring the networking of your NixOS installation. Do you wonder what other options are available? NixOS has a wonderful options search on their website, where you can look up any options. This is a good resource to keep in mind as you progress through this guide!

Now run a quick rebuild to allow NixOS to install NetworkManager.

sudo nixos-rebuild --switch

Now Network Manager should be available on the terminal via the nmcli command. If you run sudo nmcli, you should receive a colorful status output, where it lists various devices that are available (such as ethernet, wifi, and the odd loopback interface).

Adding the networkmanager group to our User Account

Is using sudo (or root) a neccessary condition for accessing nmcli? Or is the need privilege escalation only contingent on our account's lack of permission, which can be remedied in other ways? In fact, you can add the networkmanager group to your user account, so you can access nmcli without needing elevated privileges. Simply modify the extraGroups directive in your users.users.[username] option, and add networkmanager as a group for your account:

  users.users.myUserAccount = {
    isNormalUser = true;
    shell = pkgs.zsh;
    extraGroups = [ "wheel" "networkmanager"];
  };

Connecting to Wifi using nmcli: Practical Steps

Now our configuration of NetworkManager is complete. All that remains is to learn the practical steps of connecting to wifi. I'll guide you through the commands that I use to connect to WiFI – but keep in mind that you'll always have more certain references at hand, through nmcli --help and man nmcli.

Scan for WiFi networks in a new location:

nmcli device wifi rescan

List available WiFi networks (these are the ones you just scanned!):

nmcli device wifi list

You should now see a colorful array of WiFI names (SSIDs), as well as their signal strength. Find the name of the network you wish to connect to, and then type:

nmcli device wifi connect WiFiName --ask

NetworkManager should ask for your password, and now you should be successfully connected to the network. You can test your connectivity using nmcli networking connectivity check, or even just by pinging a nearby server (i.e. ping 8.8.8.8).

Why do we invoke the command with the option --ask? Check out the Linux manual entry for nmcli in order to find out!

man nmcli

NetworkManager auto-magically saves all of your WiFi connections. In the future, when you return to a location where you have connected before, you don't need to enter the credentials again. Simply run the following in order to list all your previous connections:

nmcli connection show

And then run:

nmcli connection up <WiFi Name>

NetworkManager should actually connect to known networks automatically – the above instructions are useful as a backup.

Connecting to Enterprise, Education, and Other Networks: A Complication

The above workflow works well for simple networks, namely the sort that you'll find at a coffee shop, or a friend's apartment. That's because they utilise a form of authentication called WPA-Personal (or WPA2, WP3). That's the type of WiFi network where you only need to enter a password.

Unfortunately, most Universities, Colleges, and Institutions of Philosophy utilise a more sophisticated WiFi authentication method, called WPA-Enterprise. This is the type that requires an username and password, or even additional settings like domain.

NetworkManager supports these more complicated types of connections, but you'll need to setup a connection profile. I'll write a seperate guide detailing this more involved procedure at a later date.

(If you really need help with this step, feel free to email me, and I'll let you know how I made my connection profile)

Installing Low-level Systems Applications

Congratulations! If you have successfully completed the previous steps, you should be able to access the internet. We're still in the command line right now, but we can make it more friendly, by installing some familiar tools. Right after your networking configuration, add the following stanza:

  # List packages installed in system profile.
  environment.systemPackages = with pkgs; [
    vim
    parted
    git
    wget
  ];

After a quick sudo nixos-rebuild --switch, these additional packages should become installed, and available system-wide. Now all users will get to access vim, parted, git, and wget. This means we'll have a powerful editor available in the command line, as well as a way to check out git repositories and download files. Any number of programs can be installed on Nix this way! In fact, the NixOS website has a search function for packages analogous to the options search. It's a good way to find out what programs are available!

Introduction to Home Manager

But wait, is it really a good idea to install all programs on a system-wide basis? Perhaps there are programs that should only be available for the user who installs it. Likewise, many of the more advanced programs like git or firefox comes with a dizzying array of configuration options that are personal (such as setting an account name or email address). Is there any way we can install and configure programs only for the individual user?

The NixOS ecosystem does provide a powerful tool for managing settings and programs (in other words, an environment) on a per-user basis. This tool is called Home Manager, and it has many sophisticated uses. For our purpose, we will only use it to manage programs and configuration files for our user. Here's how to use Home Manager in your configurations.nix.

Installing and Configuring Home Manager

First, install Home Manager following the instructions in it's manual. As with both Computing and Philosophy, we always follow upstream sources. You wouldn't read someone else's opinion about a Platonic dialogue, would you? After adding the Home Manager channel, we can then modify our configurations.nix accordingly, adding the following directives:

  # Begin home-manager directives
  home-manager.useUserPackages = true;
  home-manager.useGlobalPkgs = true;
 
  home-manager.users.myUserAccount = { pkgs, ... }: {
    # Everything inside here is managed by Home Manager!
    
    programs.home-manager = {
      enable = true;
    };
  };

Why do we set home-manager.useUserPackages and home-manager.useGlobalPkgs to true? Check out the reasoning in the Home Manager documentation to find out. You may set them both to false, if you like – Philosophy is practiced by contrarians, after all.

You should have noticed by now that all directives in your configurations.nix file have different levels of scope, sort of like a domain name or Aristotelian category. That's because NixOS configuration is written in a programming language called the Nix Expression Language. It's closely related to Lisp and Scheme, which are other programming languages that are very keen on brackets, parantheses, and Philosophy. In practical terms, this means that you can group together, and re-arrange the Nix expressions as you wish. The above configuration can also be written like this, for example:

  # Begin home-manager directives
  home-manager = {
  	useUserPackages = true;
    useGlobalPackages = true;
    users.users.myUserAccount = { pkgs, ... }: {
      # Everything inside here is managed by Home Manager!
      programs.home-manager = {
        enable = true;
      };
      
    };
  };

As you work on and build up your configurations.nix, keep this perculiar property in mind. It'll help you re-order your configuration files as you amass more options in time.

How to use Home Manager to Install and Configure Programs for your User

Now that we've setup NixOS to use home-manager, we can begin installing programs and configuring their settings on a per-user basis. Make sure to have rebuilt your system (sudo nixos-rebuild --switch!), and follow along as we add some programs with Home Manager. Keep in mind that the following directives will be added within the home-manager.users.users.myUserAccount brackets (i.e. where the # Everything inside here is managed by Home Manager! comment lies). I'll let you know once we add files in the other bracket (where we originally installed the systems packages).

Installing using home.packages

As far as I can tell, there are two ways to install programs using Home Manager. Some programs like fortune or gnumake are simple, low-level, and do not have configuration options associated with them. These programs are installed just like the system-wide packages we installed earlier, but they will only be available for our current user:

home.packages = with pkgs; [ 
  fortune
  gnumake
];

From my observations, it seems that we use home.packages to install these programs, because they have not been yet "wrapped up" by Home Manager contributors in a form that exposes all of their settings. These programs might be so simple and low-level that they don't have any options to be made available in the first place.

Installing via a program directive

For more sophisticated programs, we install them by declaring a program directive, where we state that they are enabled (enable = true;). I do not know why a priori some programs are available this way, and others not – but assume it is because the Home Manager contributors have not packaged everything in this manner yet.

    programs.git = {
      # Install git
      enable = true;
      
      # Additional options for the git program
      package = pkgs.gitAndTools.gitFull; # Install git wiith all the optional extras
      userName = "myGitUsername";
      userEmail = "myGitEmailAddress@users.noreply.github.com";
      extraConfig = {
        # Use vim as our default git editor
        core.editor = "vim";
        # Cache git credentials for 15 minutes
        credential.helper = "cache";
      };
    };

As you can see, programs that have been packaged this way have all of their configuration options exposed. This means we can use Home Manager to manage their settings for us. This is very powerful, because if you configure your programs with all their customisations using Home Manager, then you can replicate your setup anywhere by simply copying your configurations.nix file and running a sudo nixos-rebuild --switch. Imagine making a very fancy, sophisticated setup – and even putting your configuration file into version control. Wouldn't that be a dream?

Putting our two examples together, this is how the Home Manager snippet of your configurations.nix would look like, with both git and some packages installed:

  # Begin home-manager directives
  home-manager.useUserPackages = true;
  home-manager.useGlobalPkgs = true;
 
  home-manager.users.myUserAccount = { pkgs, ... }: {
    # Everything inside here is managed by Home Manager!
    programs.home-manager = {
      enable = true;
    };
    programs.git = {
      enable = true;      
      # Additional options for the git program
      package = pkgs.gitAndTools.gitFull; # Install git wiith all the optional extras
      userName = "myGitUsername";
      userEmail = "myGitEmailAddress@users.noreply.github.com";
      extraConfig = {
        # Use vim as our default git editor
        core.editor = "vim";
        # Cache git credentials for 15 minutes
        credential.helper = "cache";
      };
    };
    home.packages = with pkgs; [ 
      fortune
      gnumake
    ];
  };

Remember to rebuild you configuration, using nixos-rebuild --switch!

Finding More Home Manager Programs

How do we know what programs can be installed using the programs.[name].enable directive? For a given program like program.git, how do we find all of it's options? Regular NixOS has it's options search, and package search, as we've seen earlier. Does Home Manager possess a similar search function?

Yes, it does! There is the Home Manager Options Search, a community project maintained by other Home Manager users. It works the same way like the options and package search for regular NixOS.

Home Manager's manual also has an appendix which lists all the programs that it has packaged, along with their options. You can visit the link above and take a look! It's not as easy to navigate, but you can simply use Ctrl+F to search within the page.

For any new program that you want to install in your Home Manager, a good workflow seems to be:

1. Search via the Options Search or look through the manual appendix, to see if it's available as a packaged program that you can enable, and set it's configuration options.
2. If it is not within the Appendix, find it in NixOS's packages search, and then simply add it in the home.packages list.

I developed this workflow through a process of a trial and error. I'm still very new to NixOS myself, and this guide documents my understanding of Nix – I may be wrong in regards to best practices. If you know of a better way to find programs for Home Manager, please let me know!

Now you should be reasonably confident in adding new programs using Home Manager. Why not add tmux, and neovim? Here's how they might look like – this snippet is copied out of my configuration, but don't let my settings inform yours – check out their options in the Home Manager appendix and modify them your own way!

    programs.tmux = {
      enable = true;
      keyMode = "vi";
      shortcut = "a";
      terminal = "screen-256color";
      clock24 = true;
      shell = "/etc/profiles/per-user/myUserAccount/bin/zsh";
    };

    programs.neovim = {
      enable = true;
      vimAlias = true;
    }; 

Likewise, I installed the following packages. Do you recognise them?

    home.packages = with pkgs; [ 
      fortune 
      file
      htop
      exa
      zenith
      neofetch
      irssi
      gnumake
    ];

With all of these, you should have a pretty comfortable command-line environment with NixOS. Make sure to run nixos-rebuild --switch!

Installing and Configuring Sway with Wayland

Now that we are confident with configuring NixOS on the command line, let's move on towards installing and building a graphical environment. There are as many schools of Desktop environments as there are Philosophy – but for this tutorial, we will go with Sway. Sway is a tiling window manager that uses the Wayland display server.

Adding a display server and window manager to our NixOS installation would give us a graphical user interface, and allow us to take our first steps away from the analytic consoles and command-prompts of yesteryear – and move into the brave new world of Postmodernism!

There are various steps involved here. Many of which I have sourced from different guides and mailing-lists. I admit I do not understand all the configuration – unfortunately, I can only offer the appearance of knowledge here, and not knowledge itself. If my explanations are lacking in any way, or you have some clarifications – please reach out and let me know.

Enable Hardware Support for Wayland

First, we must enable some hardware support options for Wayland and Sway. OpenGL is a graphics API that our hardware uses to render accelerated graphics. If we do not enable this option, we will run into a bug and Sway will not run.

Please note that these directives go into the top-level (i.e. OUTSIDE) of your home-manager bracket. Place them on the same level as your other system-wide configuration.

  # Hardware Support for Wayland Sway
  hardware = {
    opengl = {
      enable = true;
      driSupport = true;
    };
  };

XDG Integration

Next, also in the top-level we must add the following XDG directives. XDG is a standard originating from freedesktop.org which formalises a set of common variables and methods for desktop applications to interact with each other. The following settings enable what's called a XDG Portal, which improves communication between bundled flatpak apps, and Wayland.

  xdg = {
    portal = {
      enable = true;
      extraPortals = with pkgs; [
        xdg-desktop-portal-wlr
        xdg-desktop-portal-gtk
      ];
    };
  };

Adding packages to HomeManager in Support of Sway

Now, it is time to go to the HomeManager level and add some packages that Sway will need to use. I found the list of these packages from the NixOS Wiki page on Sway, which you should also review as it contains additional information.

    home.packages = with pkgs; [ 
      # ...
      # All of the below is for sway
      swaylock
      swayidle
      wl-clipboard
      mako
      alacritty
      wofi
      waybar
    ];

Each of these packages serve a specific purpose.

• swaylock is a idle screen locker.
• swayidle is an idle timer.
• wl-clipboard allows you to copy-paste (i.e. a clipboard).
• mako is a "notification daemon," which creates little floating popups when you receive messages.
• alacritty is a rust-based terminal. This is very important, as it is the terminal that you'll be using within the Sway graphical environment.
• wofi is an application launch menu. It's like the dash on Mac OS, where you get to search for the name of a program, and run it. wofi is a Wayland descendent of rofi, which was a descendent of dmenu. Isn't Open Source interesting?
• waybar is a status bar. Ultimately I did not use it, but uncomment it for your own configuration

Please be very careful with swaylock right now. If you run it in the command line, you will get locked out of your computer. Even if you type your password correctly, it will not let you in. That's because swaylock must be properly configured to accept your password and authenticate to your computer using PAM, which is like an API but for authentication. In order to properly get swaylock working, add the following directive to your system-wide configuration (i.e. not inside home-manager!):

  # Allow swaylock to unlock the computer for us
  security.pam.services.swaylock = {
    text = "auth include login";
  };

Later on, once you are inside Sway – you'll be able to lock your computer by running swaylock on any terminal!

Adding and Configuring Sway with Wayland as a Program in Home Manager

Now we are ready to add and configure sway with Wayland. The instructions here are sourced from the NixOS Sway wiki page, once more. The following snippet goes inside your Home Manager (i.e. on the same level as programs.git).

This is a lot of configuration, and it's not easy to understand at first. So just like with reading Hegel, we'll go through it line-by-line and unpack it.

    # Use sway desktop environment with Wayland display server
    wayland.windowManager.sway = {
      enable = true;
      wrapperFeatures.gtk = true;
      # Sway-specific Configuration
      config = {
        terminal = "alacritty";
        menu = "wofi --show run";
        # Status bar(s)
        bars = [{
          fonts.size = 15.0;
          # command = "waybar"; You can change it if you want
          position = "bottom";
        }];
        # Display device configuration
        output = {
          eDP-1 = {
            # Set HIDP scale (pixel integer scaling)
            scale = "1";
	      };
	    };
      };
      # End of Sway-specificc Configuration
    };

First, we enable sway. We also enable the wrapperFeatures for gtk, because it lets applications decorate their windows using the gtk toolkit, which is a common library used to make GUIs for programs.

Next, we begin the sway-specific configuration in config. Sway has a large number of configuration options, all of which are documented in the Home Manager manual's appendix. If you have time, I highly recommend browsing it – but the following three are the most important:

We set the default terminal as alacritty, otherwise you'll be faced with a horribly ugly default terminal. We'll also set the application launcher menu to wofi. We specifically call wofi with the --show run option, since that's for listing applications and running them. wofi has plenty of other options, such as --show ssh for starting an SSH menu – but that's not what we need. The bars option is a list [ ] of multiple brackets { }, each of which contains the configuration for a separate status bar. We will only need one status bar, which will be located towards the bottom of the screen. My configuration does not use waybar, but you are welcome to uncomment it for yours.

Finally, we set some settings for our laptop's default display eDP-1. Specifically, we set the pixel-scaling factor to 1. If you have a high-resolution screen, you may wish to set it to 2 (so that the text is not too small).

Adding and Configuring Alacritty with Themes

Finally, the last thing we need to do is to configure our graphical terminal emulator, alacritty. Although the configuration looks difficult at first, you can see that it is all entirely contained within the settings brackets. The options are documented in Home Manager's manual, once again.

    # Rust-based terminal emulator
    programs.alacritty = {
      enable = true;
      settings = {
        env.TERM = "alacritty";
        window = {
          decorations = "full";
          title = "Alacritty";
          dynamic_title = true;
          class = {
            instance = "Alacritty";
            general = "Alacritty";
          };
        };
        font = {
          normal = {
            family = "monospace";
            style = "regular";
          };
          bold = {
            family = "monospace";
            style = "regular";
          };
          italic = {
            family = "monospace";
            style = "regular";
          };
          bold_italic = {
            family = "monospace";
            style = "regular";
          };
          size = 14.00;
        };
        colors = {
          primary = {
            background = "#1d1f21";
            foreground = "#c5c8c6";
          };
        };
      };
    };

The most important setting here is probably the size setting – it determines how big the text is in the terminal. If you have a high-resolution screen, the default size will probably be too small. As you build up your configuration.nix file, this would be a good place to theme and make a really nice-looking terminal.

Rebuilding NixOS and Running Sway

Now everything should be all done! If your configuration file is correct (make sure that the top-level and home-manager-level directives are in their proper places), you should be able to rebuild your system and run now.

sudo nixos-rebuild --switch

After this is done, I recommend restarting your computer, just in case.

sudo reboot now

After logging in (making sure to select the newest version of you NixOS install from the boot manager), you will be greeted once again by a command line environment. But don't fear! All you need to do now is to run:

sway

And you should be in a graphical environment! Congratulations, you have successfully transcended the command prompt, and reached a transcendental aesthetic!

Help! I'm Stuck! How do I... get around in Sway?!

If you're like me, and you have never used a tiling window manager before, you probably have no idea how to get around. At all. And even worse, when you search for "sway documentation", you'll find no instructions on how to control the Window Manager!

Fortunately, Sway is based on i3, a venerable tiling Manager hailing from the Archlinux days. And it has both excellent documentation on it's project site, as well as a quick-reference cheat sheet to help you get around. Print out a copy and keep it with you – you'll become an expert in tiling window managers in no time!

Conclusion of Part I

You should now have a graphical environment in NixOS, that you put together yourself. It's still a very bare and basic space, but should serve as a good foundation for further customisation. In Part II of this guide, we will take this foundation and build upon it further. Specifically, we will install and configure FireFox together, as well as other applications like Element (Matrix Client) and Signal Desktop. I will also show you how to configure audio using the Pipewire service, and most importantly – install LaTeX and VSCodium.

I hope you enjoyed my tutorial. If you have any questions or comments, feel free to contact me – I would love to know if my guide has helped you in any way. I also write about Philosophy in my own time – let me know of your thoughts!

I recently finished reading Galileo Galilei’s (1564 – 1642) Dialogue Concerning the Two Chief World Systems, a treatise on geocentric and heliocentric astronomy. Published in Italy during the February of 1632, the Dialogue was both revolutionary and controversial for its time – earning Galileo a condemnation at the hands of the Italian Inquisition, and a life-long house-arrest that lasted until his death. But yet, despite it’s contemporaneous infamy – it’s valid to ask ourselves:

“Why are the Dialogues relevant today? What can we gain from reading it?” As a purely scientific work of observational astronomy, the Dialogues would only be relevant as a matter of historical interest. It may be a milestone or a landmark in the studies of the history of science, but heliocentrism as a doctrine is neither new nor controversial today. For a book to be considered perennial, it generally has to touch upon some truth or thesis that is universal to the human experience – to transcend the material (and ephemeral) circumstances of it’s historicity. That’s why the Iliad, with its themes of rage and justice – is just as valid and relevant today, as it was during the days of Homer. But as a work of scientific non-fiction, it is not readily apparent how Galileo’s Dialogue has appeal to universal validity – when the subject-matter of the treatise itself is a field which has advanced in the 400 years since. Why do we read Galileo’s Dialogues? To what extent can it teach us, as students of a culture nearly four centuries antecedent to Galileo’s death? These were the questions that puzzled me, as I engaged in a month-long preceptorial of the Dialogue with my colleagues.

The Dialogue Concerning the Two Chief World Systems is an unusual work for a scientific monograph. The Dialogue is first and foremost, a dialogue. Set as a conversation between Salviati, Sagredo, and Simplicio, the three characters meet over the course of four days, and converse amongst themselves like characters in a play. The book is organised not by subject or theme, but chronologically – literally divided into four days: Sagredo and Simplicio arrive Salviati’s house in the morning – and refreshments are served at the afternoon, when the hot Venetian summers conclude the conversation. Yet, despite its casual form, Galileo’s work is a serious, empirical presentation of the merits of Heliocentric astronomy, complete with geometrical proofs and demonstrations – accoutrements of the highest rigour by the standards of Renaissance science. Why did Galileo choose the form of a dialogue for his presentation? Was the structure of the dialogue essential to Galileo’s demonstration, or only an incidental trapping of fancy and style? Modern authors have flirted with different forms of presentation for scientific inquiry, after all. Shalev Ben-David’s 2017 paper on quantum cryptography featured an abstract in the form of an Aesopian fable, and Rovelli’s Dialogue on Quantum Gravity even re-imagines the familiar characters of Salviati, Sagredo, and Simplicio in the cafetaria of an American university. The key difference, which I realised as I immersed myself in Galileo’s treatise – is that its formal nature as a dialogue is not incidental to its presentation, but rather essential to the very philosophy of scientific inquiry that Galileo’s Dialogues embody.

What do I mean by philosophy of scientific inquiry? Does a work of academic literature necessarily embody an underlying philosophy at all? I think it is fairly uncontroversial to claim that all forms of inquiry must assume some wider set of principles or axioms which serve as a foundation for its study – even if such a philosophy is unstated, and as omnipresent as the water to which a fish calls home. To quote Sartre, “If any metaphysics presupposes a theory of knowledge, it is equally true that any theory of knowledge presupposes a metaphysics.” And as a philosophy of scientific inquiry, these underlying assumptions define the ways we gather knowledge, the ways we reach valid truths. The claim of my essay is that in our study of Galileo’s Dialogue, we come to a better understanding about the philosophy of scientific inquiry. Not just a method of scientific inquiry in the Renaissance, or the particular scientific inquiry as it was conducted by Galileo – but scientific inquiry as we know it. I cannot apodictically assert whether Galileo’s Dialogue Concerning the Two Chief World Systems was the first presentation of the scientific method, but it is decidedly the most influential example – and it’s publication, controversy, and subsequent condemnation set the stage for the development of science as a distinct field from Philosophy and Theology.

What is this philosophy of scientific inquiry which Galileo’s Dialogue embody? In order to answer this question, we must start by taking a look at the text itself. Galileo’s dialogue is the gathering of three close friends at Sagredo’s house, in the midst of a hot Venetian summer. The book prefaces the beginning of the first Day with the following:

It has happened that several discussions had taken casually at various times among these gentlemen, and had rather whetted than satisfied their thirst for learning. Hence very wisely they resolved to meet together on certain days during which, setting aside all other business, they might apply themselves more methodically to the contemplations of the wonders of God in the heavens and upon the earth. They met in the palace of the illustrious Sagredo; and, after the customary but brief exchange of compliments, Salviati commenced as follows.

Following that introduction, we meet the three characters. The worldly and successful Salviati, listens actively and with admiration, as his friends Sagredo and Simplicio take on the roles of advocates for the titular “two chief world systems.” Sagredo is presented to us as a keen and sophisticated advocate for heliocentrism, substantiating his claims with both experiments and geometrical demonstrations. Simplicio takes on the mantle of the Ptolemaic geocentric model, quoting learnedly from Latin as he supports his claims with Aristotelian doctrine.

I admit, for the longest time I could not understand the character of Simplicio. At first, knowing his role as the speaker for geocentricism – my perceptions of him were coloured by the fact that he was the proverbial Devil’s advocate – the very absurdum in the reductio ad absurdum that the work seemed to be. Biased by my modern sensibilities of universal gravitation, Simplicio’s quaint, spirited quotations from Aristotle seemed like hidebound dogmatisms. In brief, my perception of him was summed up by his very name: a Simpleton. How could his interlocutors not feel the my same feelings of frustration, as they argued with him over the course of several days? His dogged following of Aristotelian physics, and the way he reluctantly and begrudgingly conceded points only after rigorous argumentation, made him appear foolish and asinine. Was Simplicio’s very inclusion in the text, merely a rhetorical strawman, a personified concession to the Church?

It was only much later, did I realise my misjudgement of Simplicio’s character – and his place in the dialogue. Simplicio’s no simpleton – but rather, he is a young man trained and educated by the finest standards of Classical education in the Italian Renaissance. Named after Simplicius of Cilicia, a noted Neoplatonic commentator and scholar; Simplicio quotes and refers to Aristotle with a scholastic fluency worthy of his namesake. The arguments that Simplicio makes in refutation of Heliocentricism may appear unusual to our modern-day empiricism – but they are entirely consisted with an Aristotelian system of physics and substance. Simplicio is not a simpleton. If anything, his comprehensive and intimate knowledge of Aristotle would earn him a professorship in the Classics department of any University in the world today. I had misjudged Simplicio’s character out of no advantage other than the benefit of hindsight.

So what is the role of Simplicio in Galileo’s Dialogue? How does he fit in the rhetorical narrative of the work as a whole? I think it is perhaps my own initial misunderstanding of Simplicio’s character, which may best shed light on Galileo’s underlying method. We are all products of our environment, our path a trajectory of the sum of the forces acting upon us. And it is my own bias, as a member of a face-paced, results-oriented society – to be interested only in the answers, and not the process that leads us to them. Simplicio is wrong, empirically speaking – the Earth is not the center of the universe, nor the Sun a satellite of our globe. But Galileo’s Dialogue is not a mere presentation of facts as they stand – but rather, a work that presents the scientific method, as a process of discursive inquiry.

Science is fundamentally dialectical in nature, where findings are not totems of authority to be appealed to – but components of a longer, overarching conversation – a dialogue between scholars and scientists as the field progresses. It would be unthinkable for a modern Physicist to base their findings on naught but the authority of Einstein – but such a underlying conception of science as an empirical pursuit was not always the case in our past. A Natural Philosopher of the 15th century can publish a paper explicating the aetiology of a novel phenomena based solely upon the authority and dogma of Aristotle, and such a work would be considered a valid method of inquiry. Its merits would be analysed on the validity of the interpretation alone.

Galileo’s Dialogue is a proposal and demonstration, of an alternative system. Where the validity of any unit of scientific truth is not assessed based on its consistency with established authorities – but on empirical experiments and demonstration. More importantly, Simplicio’s role in the dialogue is to illustrate the discursive role of scientific inquiry in action. Sagredo presents his arguments to Salviati, and Simplicio in turn – and Salviati plays his own role in the resulting synthesis of thesis and antithesis. As key players in this dance of scientific inquiry, the characters embody qualities required for the very process to work.

What are these qualities? Sagredo, Salviati, and Simplicio are many things – they are educated, leisurely, perhaps even aristocratic. But the theme that was most present on my mind, as I worked through the Dialogue, as the fact that all three characters – despite their differences – are friends. Friendship: or at the very least, a good-natured benevolence and goodwill – seem to be amongst the essential qualities necessary for the discursive process of inquiry to work. Throughout the dialogue, no matter how much the characters disagree with each other – an atmosphere of good faith and understanding prevails. Simplicio may misunderstand the reasoning of Sagredo, but his friends are quick to correct him. And despite his own dogged nature and faith in Aristotle, Simplicio does accept the counterarguments of his friends, and profess the need for further study and contemplation, with every concession that he makes. This genuine friendship between the three characters – is more than just a narrative scene-setting or facade. They are truly friends. Salviati and Sagredo fret and worry over their companion, when he shows up late to their meeting due to the tides. And likewise, Sagredo, who leans towards the side of Salviati’s arguments throughout the work – is quick to spring to Simplicio’s defense, when Salviati misunderstands or critiques Simplicio on unfair grounds.

In conclusion, as I read Galileo’s Dialogue Concerning the Two Chief World Systems, I realised that there are many lessons to be learnt from it – that are valid not just in it’s own historical context, but universally valid today. The Dialogues are a great presentation of the scientific system – a process and method of inquiry based on neither Gods nor Man, but evidence alone. And it is a heartening and sincere rendition on the necessity of goodwill and friendship, in the process of any intellectual inquiry. There are many things that I had wished to address in this essay which I ultimately left unspoken, due to a lack of time of my own. But I would like to conclude with the following passage from the latter half of the Dialogue:

Salviati: “Sagredo, please let us weary ourselves no longer with these particulars, especially since you know that our goal is not to judge rashly or accept as true either one opinion or the other, but merely to set forth for our own pleasure those arguments and counter-arguments which can be adduced for one side and for the other.”

Let us remind ourselves that Sagredo, Salviati, and Simplicio – busy individuals with personal and professional lives full of happenings of their own – agreed to take a break from their worldly affairs, and set aside four days amongst themselves for a philosophical discussion. During which, the friends argued not for profit, or even to necessarily change each other’s opinions. But rather gathered in the company of each other’s friendship, to set forth opinions and arguments for the sake of the pleasure of inquiry alone.

There’s a question on my mind, that is perhaps best represented in the manner of a dialogue. It’s a question on the nature of consciousness, where we ask not only: What is consciousness? and Where is consciousness located? But in order to do so, we must first explore adjacent questions in ontology, like what is being? or what is meaning? I will develop these questions in the course of this dialogue, which I present as follows.

Imagine one day, a Visitor arrives to your office. You offer them tea, and after all the introductions and customary salutations, they inquire of you the following question: “What is chess?”, they ask.

Of course, you would be astonished – for how could they not know of such a common game? But your Visitor hails from a faraway land, with no knowledge of your customs. So despite your bemusement, you proceed with an impromptu demonstration. You find a dusty chess-board beneath some papers, and a box of chess-pieces behind a shelf. Presenting these materials to your Visitor, you arrange the pieces in their starting order, and explain the movements of the chess-pieces. You showcase the forward marching of the Pawns, and the odd, L-shaped pounces of the Knight. Thus you demonstrate the pieces one-by-one, as well as other nuances of the rules, such as checkmate and en passant.

After you conclude your explanation, you look expectantly at your visitor. They seem surprised. “What a marvellous invention of yours, this ‘chess!’”, they exclaim.

“And surely, you have a prized possession indeed, for many strangers to speak of its renown. But should anyone else wishes to see this game, must they also come visit you at your office, just as I have done?”

You laugh, bemused by your Visitor’s peculiar misunderstanding. You quickly explain. No, you were not the inventor of chess (if only!). Nor is your worn-out chess board the ‘Chess’ of the world. In fact, you clarify – your visitor has confused the singular with the general. There are such things which are singular, such as You and I (or the Eiffel Tower). And in English, we often denote their singularity with the definite article – “the Eiffel tower, not an Eiffel tower.” But in any case, chess is a game that is enjoyed by millions, of which you only possess a particular instance – namely that battered chess-board which is now in front of you both.

Your Visitor looks thoughtful, and glances at the chess-board, with an odd intensity. After they moment, state pensively: “So chess is not this particular board and these particular pieces which you have in front of you.” You nod, encouragingly. They pick up a Rook, and examine it – before proceeding: “But rather, chess is a game of black and white pieces, made from ebony and boxwood. It is played on a varnished, mahogany board, with black and white squares made from oak and birch.”

“And hence, anyone may possess a particular instance of chess, should they own a board and a piece-set constructed by these materials and dimensions,” they conclude.

You object, stating that they have misunderstood once again. No, chess is not defined by these material characteristics. A game of chess is not defined by ebony and boxwood pieces, nor must a chessboard be made from mahogany, oak, and birch. Rather, there can be chessboards made from plastic or canvas, chess-pieces from metal or glass. Indeed, there is no necessity for the pieces to be black and white either – all of these are merely material characteristics which are accidental to the essence of chess altogether. To define the being of chess as a set of material properties would miss what is essential about chess. To drive forth the point, you make sure to recount a rather charming miniature chess-set you saw, made from frosted and transparent crystal – at a colleague’s office down the hall.

The Visitor remains silent for a few moments, taking this all in. “Chess is not defined by its material characteristics, but rather it’s material characteristics are accidents from any particular embodiment of it’s being.” You nod along, reasonably. They look up at you, with a look of increasing bafflement. “But then, what is chess?!”

You lean back in your armchair, taking a moment this time to ponder. You arrive at an answer, presenting it this time with more care. Chess is not defined by any material characteristics of its components, you admit. But rather, a board game is chess, when its pieces follows certain definite rules. A chess-game is a game where there is a board composed of 64 alternating squares, arranged in an eight-by-eight grid. Where each player controls sixteen pieces, eight of which are Pawns. You generalise even further, making sure to point out that it is not necessary for the pieces and squares to be black and white – nor any colours at all, provided that they are differentiated from each other. And likewise, it is not necessary for the pieces to be ‘Pawns,’ ‘Bishops,’ ‘Knights,’ and ‘Kings’ – the customary names that we give them are also accidental. Instead, their true natures are defined by their movements.

A contemplative silence settles over the both of you. Your Visitor considers your explanation with an unusual attentiveness. After a while, they speak.

“You state that chess a general being, which manifests in particular instances. It is not defined by the material characteristic of these instances, but rather by a set of underlying rules which define certain behaviours.”

The Visitor looks puzzled, and pauses for a moment before they exclaim: “But now that you divorce the definition of chess from the material basis of its embodiment, you risk a progression into absurdity!”

Astonished, you interject – and ask what them what they mean.

“If chess were to be defined as a set of logical rules, these rules may be present under any representation. You would not need a chess board, or chess pieces at all, in order to have chess!”

You laugh, surprised by the Visitor’s particular objection. Why, you say– that is not a problem at all. Indeed, chess is represented – and played – often without chess boards nor chess pieces. You mention the existence of correspondence chess – matches played via post or internet forums, where nary a chess-board is in sight. You present the concept of Descriptive Chess Notation, a terse, succinct language that’s often used to represent chess matches in print. To better illustrate this, you even print off an example, of the opening moves from a match between two Grandmasters in Berlin:

Your Visitor studies the printout closely. “I do not see the chess-pieces,” they state. “Nor how there is any movement in space.”

Ah, but you are simply unaccustomed to reading Descriptive Chess Notation, you reassure. For you see, each numbered line in the printout represents a pair of moves, the first belonging to white, the second to black. Each move contains a letter and a destination square, where P represents ‘Pawn’, N represents ‘Knight’, and so on. Given the expression “P-K4”, it would mean “move the (white) Pawn to the fourth square in front of the (white) King”. It’s all a little obscure, but to an experienced chess player reading such notation is as clear as seeing a movie play out in their mind.

“I see,” nods the Visitor. They revisit the printout, which now appears with a newfound significance.

“From what I understand so far, the essence of chess is not in its material embodiment. But rather, it is a set of rules, and relations, which may manifest within an arbitrary set of symbols which represent them. These symbols may be totems of carved ebony and boxwood, or simply letters upon a page.”

“But in both cases, the simple symbols by themselves are not enough to be chess. To a person that does not know about Descriptive Chess Notation, your printout will have the appearance of random, meaningless characters. Likewise, boxwood chess- pieces would look like simple wood-turnings to a person ignorant of their use.”

You agree to your Visitor’s elaboration. Perhaps, you muse provisionally – there are two components to the being of chess. There is, by necessity – a set of symbols which represent the state of the game. But there must also be a corresponding interpretation, which inscribes meaning to the symbols. In fact, you hypothesise, there is probably a formal method to define the being of chess, as a set of symbols and a suitable interpretation.

“Like a formal language. Where the alphabet is a set composed of the chess pieces, and a grammar which defines valid combinations of moves.”

Or a geometrical system, you reply. One which contains definitions of the chess-pieces, and postulates which allow their moves.

(Or even a Turing-machine, I add, narrating from the margins. But that is neither here nor there. The dialogue continues on.)

You feel there’s a certain oddity with this definition of chess, and your Visitor notices it too, who takes it upon themselves to elaborate:

“We’ve arrived at a puzzling stopping-point, in this progression of abstraction,” they state. “We began with wooden chess-pieces, physical and embodied – as real objects with extension in space and time. We than divorce chess from it’s material representation, and ultimately conclude that it’s essence lays in a set of symbols and a suitable interpretation.”

The Visitor picks up a pen and makes a few notes on the printout. “For the sake of convenience, let us denote the set of symbols as \[\{\text{Symbols}\}\], and their interpretation as a function \[I(x)\]. The process of interpreting symbols to yield meaning can be written as:”

\[{\text{Symbols}} \Rightarrow I(x) \Rightarrow {\text{Meaning}}\]

So given a collection of symbols like the printout from above (Figure 1), there exists a suitable interpretation that yields meaning from otherwise obscure lines. You grant this step of reasoning.

“But these symbols and their interpretation may be of arbitrary complexity,” they frown. “By going further in abstraction, we encounter the same absurdity that I feared earlier.”

What sort of absurdity? You ask the Visitor to clarify, and they provide an example:

“Let me take your earlier representation of a chess match, in Descriptive Chess Notation (Figure 1). I will make some minor changes to the representation, for the sake of demonstration.” They pick up your printout and a proffered pen, and begin to write. “I will replace the arabic numbers with their numerals, and remove any dashes and space. With these minor, purely notational substitutions, the match takes on the following form.”

You glance at their amendations:

“With this string of characters (i.e. symbols), there exists an interpretation such that they yield a chess game. This is not difficult to grant, for the string is a simple, idiosyncratic variation on Descriptive Chess Notation,” the Visitor explains.

What an ungainly representation. But it can be done, you allow.

“But now I will do the following. I will take the above string and encrypt them with a password according to a cypher, yielding an encrypted output.”

You raise a puzzled eyebrow.

“A cypher can be understood as a simple, reversible function, which takes a key and an input.” The Visitor clarifies. “If you input plaintext and the key, it’s output will be ciphertext. If you input ciphertext and the (correct) key, it will output the original plaintext.” They helpfully scribble down some notation:

\[C(k, i) = O\]
\[C(k, \text{plaintext}) = \text{ciphertext}\]
\[C(k, \text{ciphertext}) = \text{plaintext}\]

What does this have to do with symbols and interpretation, you ask.

“The philosophical import of this example comes from the following cryptographic principle. Any good cypher is capable of producing seemingly random ciphertext, given an input and an appropriate key. Given the modified notation as input, and a well-chosen cipher and key , I can produce the following ciphertext as output:”

You examine the result, comparing to the earlier plaintext string (Figure 2). It does look pretty random, you admit. Any appearance of structure and meaning seems to be obliterated.

“But here’s the thing. This string does not only have the mere appearance of randomness. But it is truly random, by all mathematical measures. You can give it to a Mathematician, who could subject it to exacting statistical analysis. They will find that the frequency distribution (i.e. occurrences) of letters will be uniform and evenly distributed. The encrypted string will be mathematically indistinguishable from random noise.”

You let the Visitor’s words sink in for a moment.

But this is such a pedestrian objection! You protest.

You still have a set of symbols, and a corresponding interpretation. The only difference is that the symbols are obfuscated, by adding a cryptographic key and decryption step to the interpretation. Sure, one may add any number of steps and contrivances in the interpretation, but it doesn’t change the essential nature of chess as symbols and a means to interpret them.

“I apologise, but you misunderstood my point”, replies the Visitor good- naturedly.

“For the essential structure of my syllogism is thus. Given a set of symbols and an interpretation, I can yield meaning, as agreed upon before.”

\[{\text{Symbols}} \Rightarrow I(x) \Rightarrow {\text{Meaning}}\]

“With the deciphering process, I essentially add an additional step in the process. First, the cipher-function \[C(x)\] takes a meaningless input, which yields the symbols that are then interpreted by \[I(x)\].”

\[{\text{No Meaning}} \Rightarrow C(x) \Rightarrow {\text{Symbols}} \Rightarrow I(x) \Rightarrow {\text{Meaning}}\]

“You are right to admit, however – that the additional deciphering process is not essentially different from interpretation. It is merely a continuation (or rather, an extension) of the interpretive act. Indeed, it can be simplified into it, where \[I'(x) = I(C(x))\], just like functions can be composed (i.e. nested)”

\[{\text{No Meaning}} \Rightarrow \underbrace{ \big[ C(x) \Rightarrow {\text{Symbols}} \Rightarrow I(x) \big] }_{I'(x)} \Rightarrow {\text{Meaning}}\]

“And thus, we yield the conclusion of our reductio ad absurdum. Given something that is meaningless, there exists a suitable interpretation \[I'(x)\] which yields a meaning.”

\[\therefore {\text{No Meaning}} \Rightarrow I'(x) \Rightarrow {\text{Meaning}}\]

“Not only that, but given something which has no meaning, there will always exist a suitable interpretation of it which can yield an arbitrary meaning of one's choice. For any string of random characters, there can be an interpretation that yields a riveting novella. Within any noise of radio static, there is a suitable interpretation that yields a lost concerto of Bach.”

But this is an absurdity! You protest. By stretching the nature of symbols and their interpretation to such an extreme, we risk losing meaning entirely. There has to be some paralogism in this chain of reasoning, or some illusion at hold. Otherwise, it seems that the very structure of being loses its shape. How can we affect an explanation?

“We can try. For one, the absurdity of this conclusion comes an imbalance between the symbols and interpretation.” The Visitor sighs. “We hollow out the symbols until there is nothing left within them, just as we simultaneously over-stuff the interpretation so that it does all the work. But yet, somehow meaning is conserved in the two parts as a whole – it is merely distributed unevenly.”

You nod. There’s a similar meta-theorem in the study of Hilbert-style logic systems, you recall. Given a logical system with axioms, and rules of inference – there is always some tenuous balance in play. You can either have a system that has few axioms, but many rules of inference. Or a system that has many axioms, but few rules of inference. It seems like a similar dynamic is at play here.

But still, there is no reconciliation yet. If anything, all this goes to illustrate is that the conception of chess (or anything) as a set of symbols and an interpretation is an illusion. After all, meaning is still conserved – no matter how you distribute the share of work between symbols and the interpretative function (Figure 4).

You both ponder now, in silence. All of a sudden, the office feels chilly and strange. There is such a thing as chess, that is certain. Chess is a well-defined being which exists. It’s trivial to dismiss the material cause as a definition, but now it seems that the formal cause is just as useless in explaining what being (or chess, for that matter) is. You feel a sense of aporia set in, as your Visitor politely shifts in the armchair.

“If I may,” the Visitor offers gently. “I think there’s something hidden here. Not in the interpretative function itself, but rather – in the very act of interpretation. The interpretive process is not a static one, but an active one of some sort. We have not explored it properly, so we can only speak loosely about it. But perhaps the being of chess is neither in its materials, nor in its rules. But in the mind of a person who desires to play – in the interpreter, not interpretation.”

“This is not an answer, for the terms I use are nebulous, and ill-defined. It would take another dialogue to properly explicate this hypothesis, let alone test it for validity. But to stake the foundation of being behind the interpreter, is to claim that being is teleological in nature.”

Chess is chess, because two players want to play, you surmise. The final cause. It’s not a satisfying answer by any means, especially since there exists a similar progression into absurdity with this answer. To say that chess is endowed with meaning because meaning comes from the mind of the interpreter, is just as awkward of a contortion. What is being? What is meaning? Where are they located?

Ultimately, these questions trouble me, just as it troubles the two interlocutors of my dialogue. For while I presented them in the form of an analogy with chess, the true object of my investigation is actually consciousness. What is consciousness? Where is it located? These are the questions that I seek to know. To state the consciousness is the emergent phenomena of an interaction between neurotransmitters and synapses, is to likewise recourse to the material explanation of being – one that is unsatisfying, as illustrated above. Likewise, there seems to be a similar absurdity if we were to abstract consciousness away as a construction of logic, or a sterile computational act.

There are several potentially fruitful avenues of discussion, that I have not explored in this dialogue. I could examine the role of the interpreter, behind meaning and being. The role of desire (the teleological aim) in ascribing meaning to non-meaning can be important. Likewise, even though the symbol-interpretation hypothesis of meaning is inadequate, there may be a different (perhaps more nuanced) presentation of it that solves the absurdity that I critique. Maybe there is a self-referential component to the being of consciousness, which I have not addressed at all. I would like to be able to explore these questions in greater precision and detail, over the course of a subsequent essay. But this is a concern of mine alone, and it will not be fitting to end the dialogue on such a conclusion. So we turn back, to the Visitor, for the final time.

“Thank you so much for the tea,” the Visitor remarks gratefully. “And the explanations. I am very grateful for your time.”

Would you, like to play some chess? You ask. The Visitor smiles warmly, and as the sun sets outside your office window, you begin a game.

I hope you enjoyed my dialogue. If you have any comments, or remarks, feel free to contact me – I'm always happy to have conversations with interesting strangers!

This tutorial is a guide on installing NixOS, with a separate /boot partition and full-disk-encryption (FDE) using LUKS2 with a detached header. This guide is different from other tutorials, because it offers the following features:

• Full disk encryption using the newer LUKS2 container format, instead of LUKS1.
• Separate /boot partition on portable device (i.e. USB, MicroSD Card).
• Detached LUKS2 header on separate portable device, offering deniable encryption.

Example Setup

Using this guide, I installed NixOS on a Purism Librem 14 laptop, running Coreboot with Tianocore, an open-source implementation of UEFI. Using the laptop's on-board MicroSD Card reader, I installed both the \boot partition and the LUKS2 header located on a MicroSD card.

When the MicroSD card is removed from the laptop, the hard drive appears to be unformatted, without the appearance of an operating system at all.

Prerequisites

You must have an USB stick imaged with the appropriate NixOS installation image.

In order to follow this tutorial, two devices are necessary. You will need a primary device (e.g. hard drive, SSD) - which we will denote as /dev/sda. This device holds the LUKS2 container.

You will also need a second, portable device (e.g. USB, MicroSD card) - which we will denote as /dev/sdb. This device will hold the /boot partition, as well as the detached LUKS2 header.

If you plan to use a SD or MicroSD card, make sure to get a high-endurance version that uses SLC or MLC flash. These cards are slower, but allows more write-cycles.

This tutorial is written for a user that has an intermediate familiarity with Linux operating systems, but who is new to NixOS specifically. You do not need to know how to manually install an operating system. Every single step will be presented in this tutorial, including steps for partitioning.

Warning: be extremely careful with disk labels when following this tutorial. Devices and partitions may appear under different labels on your system. Using commands like dd or cryptsetup luksFormat will cause permanent data loss! Use lsblk in order to check disk labels if at all unsure.

All commands must be run as the root user. This may be done using sudo, or by opening a root shell. In order to open a root shell, simply run:

sudo su root

Prepare Disks

This step is entirely optional. We will prepare the disk devices by erasing them. We will first zeroise /dev/sdb, in order to completely wipe it.

dd if=/dev/zero of=/dev/sdb status=progress

Next, we will wipe /dev/sda by filling it with random data.

dd if=/dev/urandom of=/dev/sda status=progress

The reason we wipe the primary device by filling it with random data, instead of simply zero-ising it, is to make the size of the encrypted content undeterminable.

Now /dev/sda and /dev/sdb are fully prepared. We are ready to create partition tables, and partitions within them.

Create Partitions

We will have the following partition scheme. The specific method (or scenario) that we will implement for our Full Disk Encryption (FDE) is to mount an LVM (Logical Volume Manager) on top of our LUKS2 Container. LVM volumes are then created to reflect the different partitions on our root filesystem. This method is referred to as LVM on LUKS and is a common method of implementing FDE on Linux devices.

/dev/sda Primary device

• /dev/sda1 LUKS2 Container crypted
• LVM Volume Group vg
• LVM Logical Volume vg-swap
• LVM Logical Volume vg-nixos
• and any other partitions...

/dev/sdb Portable device

• /dev/sdb1 boot Partition
• /dev/sdb2 LUKS2 Detached Header

The underlying root \, \home, and swap partitions will be created as LVM virtual volumes within the LUKS2 container.

We will first proceed to make the partition table and partition for the primary /dev/sda device, then for the portable /dev/sdb device. We will be using parted as the tool to create partitions.

Make root partition on primary device /dev/sda

We will be using the parted command-line tool to create all of our partitions.

Create a gpt partition table for /dev/sda. Other partition table formats like msdos are not necessary.

parted /dev/sda -- mklabel gpt

Create primary partition for /dev/sda. This will take up all of the space on the primary /dev/sda device, since it will hold a LUKS2 encryption container. Other partitions (such as swap, or /home) will be created within the encryption container.

parted /dev/sda -- mkpart primary 0% 100%

The partition setup for the primary /dev/sda device is now complete. We will now proceed to partition the /dev/sdb portable device (i.e. the USB, MicroSD card).

Make boot partition on portable device /dev/sdb

Create a gpt partition table for /dev/sdb.

parted /dev/sdb -- mklabel gpt

Create boot partition for /dev/sdb. This is the unencrypted partition that will contain the bootloader for the operating system. It will reside on the portable device, which should be stored securely when not in use.

parted /dev/sdb -- mkpart ESP fat32 0% 50%

We must set some flags to indicate that this partition contains the bootloader.

parted /dev/sdb -- set 1 boot on

Now we will use the remaining space on the portable device to create a partition for the LUKS2 detached header. This partition will not actually contain a filesystem, since the LUKS2 detached header will be read as a raw header.

parted /dev/sdb -- mkpart primary 50% 100%

I choose to devote the remaining 50% of the device's space, simply because the MicroSD card will not be used for any other purpose. However, in practice a smaller partition can be allocated for the LUKS2 detached header. Unlike LUKS1, the detached header does not have a static, fixed size. However in practice a partition of 16MB should be more than enough.

Make filesystem for boot partition

We will make a FAT32 filesystem for the boot partition. We are using FAT32 instead of ext4 for greater compatibility with bootloaders.

mkfs.fat -F 32 -n boot /dev/sdb1

It is not necessary to make a filesystem for the LUKS2 header partition /dev/sdb2. This is because the LUKS2 header will reside as a raw header without any underlying file system. This avoids the complications of the bootloader having to mount a filesystem before reading the header file.

Make LUKS2 Encryption Container with detached headers

We will now use the cryptsetup command to create the LUKS2 Encryption container. We will invoke the luksFormat action on /dev/sda1 in order to do so. The command comes with a set of sensible and sane cryptographic defaults, however you may choose to explore further configuration options if you desire.

The location of the LUKS2 wrapper is on the primary hard drive /dev/sda1.

The LUKS2 header which contains the metadata is on raw partition /dev/sdb2

This command will prompt you to enter a password. This password will be used to unlock the primary device when the computer boots.

cryptsetup luksFormat /dev/sda1 --type luks2 --header /dev/sdb2

Open LUKS2 Container

A LUKS2 container has been created on /dev/sda1, with it's corresponding header file located at /dev/sdb2.

In order to use this container, we must unlock it using the following command. You will be asked to input the decryption password from the previous step.

cryptsetup luksOpen /dev/sda1 crypted --header /dev/sdb2

Now the container will be available as crypted, at /dev/mapper/crypted.

Create a LVM Volume within the LUKS2 Container

We will now create a set of LVM volumes within the LUKS2 container. This way we may have other multiple logical partitions within the container itself.

First, we will initialise the physical volume crypted. This step is necessary in order to create logical volumes within it.

pvcreate /dev/mapper/crypted

Next, we will create a volume group upon the newly-initialised physical volume. This volume group will be called vg.

vgcreate vg /dev/mapper/crypted

Now that the volume group is made, we can make arbitrary logical volumes. These logical volumes correspond to the unencrypted partitions of a traditional Linux installation.

Although it is possible to make multiple partitions corresponding to different mount points (such as /home, /etc, var, etc), in practice this is not necessary. This is because the main benefit of having a separate /home partition is easier recovery, where the /home partition can be mounted separately in a rescue process.

Because all partitions reside within the encrypted LUKS2 container, which must be unlocked for access, there is no benefit to having separate partitions. Hence we will only create a root and swap partition.

Create a swap partition within the LVM volume

Create a swap partition with the label swap. Note that we use the option -L which denotes a numerical size. In the following command, a 16 GB swap partition is created.

lvcreate -L 16G -n swap vg

There are differing opinions on the appropriate size for a swap partition on modern Linux. In particular, if you wish to to enable hibernation you must have the same amount of swap as your RAM.

Create a root partition within the LVM volume

Create a root partition with the label nixos using the remaining free space. Note that we use the option -l which denotes a percentage.

lvcreate -l '100%FREE' -n nixos vg

Now the LVM volumes are created, and ready to be used.

Create Filesystem and Swap

After creating the volumes, we must create filesystems that will reside on them. For this guide, we will use a relatively ordinary ext4 filesystem. You may substitute more exotic filesystems such as ZFS or btfs if desired.

mkfs.ext4 -L nixos /dev/vg/nixos
mkswap -L swap /dev/vg/swap

After creating the filesystems, we will mount them (and activate swap).

Mount filesystems

Mount the root partition

mount /dev/disk/by-label/nixos /mnt

Mount the boot partition

mkdir -p /mnt/boot
mount /dev/sdb1 /mnt/boot

Activate swap

swapon /dev/vg/swap

Now our filesystems are mounted. The future NixOS installation's root will be accessible at /mnt, and it's boot partition at /mnt/boot.

Configure NixOS

We can now instruct NixOS to generate a set of configuration files for our installation. Make sure to pass the --root /mnt flag, in order to indicate where the root filesystem resides.

nixos-generate-config --root /mnt

Now the configuration files will be available at /mnt/etc/nixos. We must modify this file in order to add the appropriate settings.

Configure configurations.nix

We can now specify the configuration of the NixOS installation using configurations.nix. The included example configuration file has various options that can be explored. In particular, you should check out the following:

• User configuration
• Enabling NetworkManager for WiFi

Once generic configurations are complete, we must add the specific boot.initrd.luks.devices settings which will allow the system to boot.

The following section must be added. We are specifying for NixOS that there is a dictionary of boot.initrd.luks.devices, where there exists a device crypted with configuration options enclosed in another dictionary.

# Configuration options for LUKS Device
boot.initrd.luks.devices = {
  crypted = {
    device = "/dev/disk/by-partuuid/<PARTUUID of /dev/sda1>";
    header = "/dev/disk/by-partuuid/<PARTUUID of /dev/sdb2>";
    allowDiscards = true; # Used if primary device is a SSD
    preLVM = true;
  };
};

We must specify the device directive which is a path that points to the encrypted primary partition /dev/sda1, as well as header which is a path that points to the LUKS2 Header located on /dev/sdb2.

These paths can be specified in more than one way. You can see the various ways that this path is specified by listing the subdirectories in /dev/disk.

I recommend specifying the paths using /dev/disk/by-partuuid instead of /dev/disk/by-uuid, because /dev/sda1 does not have a uuid assigned to it, since it doesn't have a filesystem.

Hence in order to find the UUIDs of /dev/sda1 and /deb/sdb2, run:

blkid /dev/sda1 -s PARTUUID

Or you may simply run ls -l /dev/disk/by-partuuid

Installation

After setting the configuration options, it is time to install the device. You will need to have an internet connection, as NixOS will be downloading and compiling sources. Either connect to an ethernet cable, or a WiFi network.

Now run nixos-install. We will specify the option --cores 0 to let NixOS use all CPU cores when compiling binaries.

nixos-install --root /mnt --cores 0

This command will take anywhere from 5 to 25 minutes, depending on the configuration of the system and the speed of the internet connection.

Once the installation is nearly complete, it will prompt you to set a root password for the newly installed system. After setting the password, the installation is complete.

reboot

Post-Installation

When the installation is complete, perform a reboot. Now you must enter the BIOS/UEFI interface of your computer's firmware, and change it's boot settings to use the bootloader located on the portable device /dev/sdb1 by default.

Now your NixOS installation is complete!

This tutorial is a guide on flashing (i.e. installing) Coreboot firmware on to a Purism Librem 14 laptop. Coreboot is an open source firmware platform that supports multiple payloads. We will be flashing the TianoCore payload, which is an open source implementation of UEFI.

We will do this by first booting into a 'Live' Linux operating system that is installed on a USB disk. From this environment, we will download the ChromeOS Firmware Utility Script from Mr.Chromebox. Using the firmware utility script, we will flash the laptop with Coreboot and TianoCore.

Why is Flashing Coreboot with TianoCore Necessary?

Modern (circa 2017) Purism devices ship with Pureboot firmware by default. This firmware is based upon Coreboot, and offers additional tamper-evident features such as Heads. However, Pureboot is a BIOS firmware, not UEFI. As a result, it does not support EFI boot.

NixOS 21.05 does comes with instructions for installing on a BIOS using GRUB. However, I was unable to accomplish this successfully. As a result, I decided to use Coreboot with TianoCore as my firmware instead.

Prerequisites

You must have a USB stick with an 'Live' Linux operating system, a stable power supply, as well as an internet connection. I used an USB stick with an installation image of Ubuntu 21.10, but other Debian-based operating systems should also work well.

This tutorial is written for a user that has an intermediate familiarity with Linux operating systems, but who has never flashed their device firmware before. The firmware installation will use a script, hence you do not need any advanced technical knowledge. A familiarity with the Linux command line is beneficial.

Warning 1/3: modifying your laptop's firmware is potentially hazardous. Should the process be interrupted (e.g. due to unexpected power loss), your device may be unable to boot at all. Recovering from such an incident requires the use of a hardware programmer, and may be technically challenging.

Warning 2/3: changing from a BIOS-based firmware to UEFI will render your existing operating system unbootable. A pre-existing installation will expect a BIOS boot process. Changing the underlying firmware will naturally render the bootloader non-functional. This tutorial is meant for a brand-new NixOS installation, and does not contain any information on recovering any pre-existing operating systems.

Warning 3/3: be extremely careful with disk labels when following this tutorial. Devices and partitions may appear under different labels on your system. Commands like dd will cause permanent data loss! Use lsblk in order to check disk labels if at all unsure.

All commands must be run as the root user. This may be done using sudo, or by opening a root shell. In order to open a root shell, simply run:

sudo su root

Preparing Installation Medium

First, download a 'Live' Linux operating system, and image it on to a spare USB stick. Make sure to verify the disk label of your USB stick using lsblk. I used a desktop installation image of Ubuntu 21.10. Once the image is downloaded, you may image it using dd:

dd if=/~Downloads/ubuntu-21.10.iso of=/dev/sdX status=progress

After you have imaged the installation iso on the USB, restart the laptop and boot into the 'Live' operating system on the USB.

Download ChromeOS Firmware Utility Script

Once you have booted into the 'Live' environment of the USB operating system, start a root shell. We will now download the ChromeOS firmware utility script. The ChromeOS firmware utility scripts are by Mr. Chromebox, which offers a convenient method to flash and update the Coreboot-based firmware of ChromeOS-based computers. Mr. Chromebox is a Purism employee and firmware developer, and his firmware utility script is unofficially available for Purism devices as well.

The download page for the firmware utility script is available on his website, with the source files also available at his Github Repository. We will download the script from his Github repository using the commands below.

Change to home directory, and download script:

cd
curl -LO https://raw.githubusercontent.com/MrChromebox/scripts/master/firmware-util.sh

Next, make sure to inspect the downloaded script. Scroll through the file carefully. Press q to exit the reader:

less firmware-util.sh

Running the Firmware Utility Script

Run the script using the bash shell:

bash firmware-util.sh

The script will proceed to download other auxiliary helper scripts. When it is ready, it will present a visual TUI (Text User Interface), with conveniently labeled menu buttons. It will present a menu with a selection of choices.

Select the one labeled Install/Update UEFI (Full ROM) Firmware. Depending on your version of the script, it should be the option #2.

Once you have selected the option, follow the on-screen prompts to confirm. The firmware will then proceed to flash Coreboot with Tianocore on your laptop.

Make sure to not interrupt the process, and that your laptop is connected to power at all times. Once the firmware flashing process is complete, you may restart your computer by pressing r on the prompt.

Verifying that the new TianoCore UEFI firmware is complete.

Restart your laptop in order to boot into the new firmware. Press Esc while the laptop is booting, in order to enter the UEFI interface. Please note that it may take up to 5 minutes for your laptop to reboot for the first time, as the firmware has to complete some initialisation and hardware auto-detection routines. This is perfectly normal, and you do not need to worry. Subsequent boots will take only a few seconds.

You should be now in the new TianoCore UEFI firmware menu! You have successfully completed this tutorial.

Next Steps: Installing NixOS, or any other OS.

Now your Purism Librem 14 laptop is successfully flashed with a high quality, open source UEFI implementation. You can proceed to install any other operating system on top of it.

This guide was specifically written to install NixOS on the Purism Librem 14 laptop. The subsequent guide is available here.