Deploy the DeityGuard

Step 0: Build environment

The standard build environment assumes you have three computers more or less dedicated to managing your DeityGuard installation (in addition to the actual client machines that will be running DeityGuard). In the network diagram below, the exterior machine is your main workstation (where you edit configuration files and such), the interior machine is the build host (notice the lack of Internet connection), the httpserv machine is used to serve DeityGuard output files to the LAN segment, and the client machine is an example client (you would connect each client machine to the interior machine in turn for flashing).

                            /(DVD reader)          /(DVD burner)
                           /                      /
[httpserv]<===ithird===[interior]<===ithird===[exterior]
    |                     | |                     |
    |                     |F|                     |
    |                     |L|                     |
    |                     |A|                     |
    |                     |S|                     |
    |                   \ |H| /                   |
    |                    \| |/                    |
    |                     \ /                     |
    |                      |                      |
    |                   [client]                  |
    |                      |                      |
    |                      |                      |
    |                      |                      |
    \--------------------------------------------------- LAN (DHCP + Internet)

Note: For playing around with DeityGuard, used hardware is more than adequate (there are no 3D texture maps to load!). It’s recommended that the interior machine be a used

Dell
Optiplex

Quad Core desktop with 16GB RAM and a 7200 RPM internal SATA hard disk drive (typically retails for around $220 on eBay). It’s recommended that the exterior and httpserv machines be used Core 2 Duo laptops with 8GB RAM (typically retailing for around $150 each on eBay).

The links labelled ithird in the diagram should be unidirectional ethernet cables that are connected to the machines with a USB ethernet adapter at each end. If you don’t have (and don’t want to make) unidirectional ethernet cables (see Third Idea), you can use regular ethernet cable for the ithird links (with a reduction in the security factor).

The interior machine must also have a USB DVD (reader) drive, and the exterior machine must have the capability to burn a DVD. The DVD reader pictured in the diagram is in addition to the built-in drive used to boot the LiveCD. The DVD is just used once (per boot of the interior machine) to bootstrap the ithird link that will be used to transfer more data.

The 4-deploy script fully manages the interior and httpserv machines. Boot up the Ubuntu 7.10 LiveCD (link here) on these machines and select Try Ubuntu. This guide assumes that these machines have exactly one hard disk that appears as /dev/sda in the OS.

Warning: Following the instructions in this guide will overwrite any existing data on the hard disk drives of the interior machine and the httpserv machine.

The exterior machine is not fully managed; since it is assumed that the exterior machine is a multi-purpose workstation, dg-setup does not take the liberty of installing software on it. The exterior machine must be able to run debootstrap and xorriso so install the appropriate packages in your OS (yes, debootstrap is available on at least some non-Debian distributions; e.g., Gentoo).

Step 1: `exterior` machine

Starting on the exterior machine, let’s unpack the 4-deploy directory from the DeityGuard source archive into the current directory. (Don’t just cd into 4-deploy as the dg-setup.sh script will expect ./deityguard.tar.xz to exist).

tar -xf ./deityguard.tar.xz --strip-components=2 deityguard/4-deploy/

Next, you must create a configuration file for ./dg-setup.sh:

nano -w ./dg-config.sh

And write in:

DG_SETUP_ROLE=exterior
DG_SETUP_EXTERIOR_ISO_DEV=/dev/disk/by-id/...
DG_SETUP_EXTERIOR_ITHIRD_INTERFACE_TO_INTERIOR=e...
DG_SETUP_INTERIOR_DISK=/dev/sda
DG_SETUP_INTERIOR_ITHIRD_INTERFACE_TO_EXTERIOR=e...

Fill in the dots as appropriate. DG_SETUP_EXTERIOR_ISO_DEV must point to the DVD burner drive on the exterior machine; use ls -alh /dev/disk/by-id to discover the appropriate device name. The DG_SETUP_..._ITHIRD_INTERFACE_... variables must be network interface names; use busybox ifconfig -a on the appropriate machines to discover these.

Note: In accordance with the specifications of the Third Idea, dg-config.sh changes the hardware MAC address of some of the ethernet interfaces. At the same time, predictable network interface naming in Linux is based on the MAC address. Thus you can run into problems if you run dg-setup followed by non-cold reboots (which do not necessarily clear the assignment of the hardware MAC address from the volatile memory of the USB ethernet adapter). So, basically, always power cycle the interior and httpserv machines; don’t “reboot” them).

Also, you may have to tune the following file:

nano -w ./overlay/2-fingen/config/fetch-local.sh

Change 192.168.0.2 to an address on your LAN segment that will never be assigned by DHCP (for example, on my LAN the DHCP server is configured to assign only addresses in the range 192.168.0.100 to 192.168.0.200). If you don’t know how to configure your LAN’s DHCP server, you can just pick a random address and hope there won’t be a conflict, or that conflicts will be avoided by some form of duplicate address detection at some level. Further on this guide, you will later have to configure httpserv to use the same address as that set in fetch-local.sh.

Finally, edit the following file:

nano -w ./overlay/2-fingen/local/vars-test.sh

Set the ITHIRD_INTERFACE variable to the network interface name corresponding to the httpserv side of the ithird link to the interior machine. As before, you can use busybox ifconfig -a on the httpserv machine to determine the interface name.

Now, the interior machine needs to have some additional packages installed that aren’t present on the LiveCD. The “no Internet” security rule prevents us from simply using apt-get install. Instead, we will have the exterior machine prepare a DVD disc with the required packages (and additional needed DeityGuard-related files). First, let’s build the package:

./dg-setup.sh

Next, insert a blank DVD+/-R disc or an erasable DVD+/-RW disc into the DVD burner of the external machine and do:

./dg-setup.sh isowren

Step 2: `interior` machine

Switching to the interior machine, insert the above written DVD in the external drive, wait for it to automount, open a terminal, and do:

cd /media/ubuntu/ISOIMAGE
./dg-setup.sh

After installing some packages, the script should drop you to a shell instructing you to “format /dev/sda”. These steps are considered too dangerous to automate, so, the first time only, enter the following:

sudo cryptsetup luksFormat /dev/sda
YES
x
x
sudo cryptsetup luksOpen /dev/sda sda
x
sudo mkfs.ext4 /dev/mapper/sda
sudo cryptsetup luksClose sda

The passphrase must be literally a single x character (unless you update GLOBAL_STORAGE_PASSPHRASE in the script as well). Encryption is mainly used here to prevent things like automount, and not for any real security benefit – though if you are concerned about malicious drive firmwares tampering with the build data, then using a strong passphrase may help with that. Once you are finished, type Control-D to exit the shell. On subsequent invocations of dg-setup.sh you should not reformat and just type Control-D to continue the script (unless you really want to start the build over from a blank slate).

Next, the script should configure the “third idea” receiver and pop up a dg_ithird xterm window that allows you to view third idea transfer progress. It will then drop you to a shell instructing you to “wait for transfer completion”.

Step 3: `exterior` machine

Back to the exterior machine. It’s now necessary to prefetch the sources for DeityGuard. This essentially runs the world_dryrun target of DeityGuard:

./dg-setup.sh prefetch

Now we’re ready to transfer the fetched source files to the interior machine:

./dg-setup.sh transfer

Note: By default, the transfer command is incremental - it’ll only send files that have newer inode or modification times than the last time it was invoked. In a “third idea” setup, a transfer failure is not detectable by the sending host. If you encounter a transfer failure, you can force all files to be transferred with ./dg-setup.sh transfer clean.

Step 4: `interior` machine

Once you see the line “tar_exit_0” in the dg_ithird xterm window, you know that the transfer completed successfully. Hit Control-D in the shell to continue the script. It will then drop you to a shell instructing you to “interface with gengen”. Here you do:

. ./aliases.sh
dg_locally_gengen_gentoo_stage3 world-libressl
dg_locally_gengen_brarm
dg_locally_gengen_linux
dg_locally_gengen_arm_linux_bpi
dg_locally_gengen_arm_linux_rok
dg_locally_gengen_coreboot
dg_locally_gengen_arm_u_boot
dg_locally_gengen_tools
dg_locally_gengen_flashpagan
touch ./local/unikitmaybeyes
dg_locally_gengen_unikit
dg_locally_gengen_gentoo_world world-{purge,libressl,selfhost,anybase,netsurf,workstation,misc,firefox,chromium}

See the Recipe page for details on what these commands do. You can also chain them together into a single command with the && operator. This is great for letting the build complete unattended but may make it more difficult to determine what is going on if you do encounter an error.

Note: As of 25 DEC 2017, you must drop netsurf to build successfully, as the netsurf build appears to be broken in Gentoo. Hopefully this will be fixed upstream soon.

Now hit Control-D in the shell to continue the script. It will then drop you to a shell instructing you to “interface with fingen”. However, fingen needs to be able to upload DeityGuard files to httpserv which we haven’t configured yet! On to …

Step 5: `httpserv` machine

Since there is Internet on this machine, you can download the DeityGuard archive. Then unpack the 4-deploy contents just like on the external machine:

tar -xf ./deityguard.tar.xz --strip-components=2 deityguard/4-deploy/

Next, you must create a configuration file for dg-setup.sh:

nano -w ./dg-config.sh

And write in:

DG_SETUP_ROLE=httpserv
DG_SETUP_HTTPSERV_DISK=/dev/sda
DG_SETUP_HTTPSERV_LAN_INTERFACE=e...
DG_SETUP_HTTPSERV_LAN_FIXED_ADDRESS=192.168.0.2
DG_SETUP_HTTPSERV_LAN_FIXED_NETMASK=255.255.0.0
DG_SETUP_HTTPSERV_ITHIRD_INTERFACE_TO_INTERIOR=e...

By now, filling in the dots should be a walk in the park. You should also change the ADDRESS and NETMASK as appropriate for you LAN configuration. The script will create an interface alias with the given address, so the httpserv machine will end up with two addresses on the LAN - one assigned by DHCP and the “fixed” address used for communicating with DeityGuard clients.

Next, do:

./dg-setup.sh

It will then drop you to a shell instructing you to “format /dev/sda”. Same as above and hit Control-D. This should result in a pair of xterm windows - one for monitoring the “third idea” transfers from the interior machine and one for monitoring the busybox httpd output to see the requests made by DeityGuard clients. Now we’re ready for …

Step 6: `interior` machine

Do:

ls ./config

to see the available configurations. The test configurations helpfully begin with "test-". Attach the flashing apparature for a particular client and do:

./fingen.sh ./config/... flash

When this completes, you should be ready to boot the DeityGuard client!

Note: Once again, there is the possibility of a bad transfer with “third idea”. To reset fingen’s idea of what has already been uploaded, do rm -f ./local/persist/already/test/*.

A few more tips …

Surprise! dg-setup.sh isn’t just for one-time automation a default installation of DeityGuard. Beyond that, it’s also designed to resume properly after you make your own changes/updates to DeityGuard.

The overlay directory is fully general mechanism for overriding any file in the DeityGuard source distribution - simply copy a source file to the corresponding place under overlay and edit it.

Here are some more tips on how to do all that. (This page focuses on the mechanics of using dg-setup.sh - for tips on how to actually make modifications to the DeityGuard code, see the Internals page).

How to rerun …

If you made changes that affected the portage world package selection or portage USE flags, do, on the external machine:

./dg-setup.sh prefetch

This will update the source package cache on the external machine. Then:

./dg-setup.sh transfer

This will (incrementally) transfer any new files to the internal machine. After the transfer completes, you’re ready to continue the script on the internal machine and interface with gengen and fingen as necessary. Whether it worked or not, rinse and repeat!

Faster debugging cycles

For the common case of debugging failures in the gengen phase, even spinning through dg-setup.sh can be too much work. Just run ./dg-setup.sh transfer on the exterior machine to transfer in your new code and use the following alias on the internal machine to update the local copy of gengen in-place:

. ./aliases.sh
dg_setup_reload

The equivalent update command for the fingen phase is:

./maint-dg-setup-reload.sh

Also, in the gengen phase, you can do:

. ./aliases.sh
dg_setup_reload both

to update both gengen and fingen in one command.

Alternating between `gengen` and `fingen`

Once you’ve run through dg-setup a few times and got the feel of things you can mostly forget about dg-setup. After the first run of dg-setup gets everything mounted and running you can just cd into the appropriate gengen and fingen directories in two separate terminals and take it from there without dg-setup. When entering the gengen directory without dg-setup about the only thing that you should need to do extra is:

export GENGEN_NETESC=

It’s really only necessary to run dg-setup again if the “third idea” client loses sync (which should be extremely rarely now since the repeated transmission parameters have been tuned to extremely conservative values).

Running `fingen` only after a restart

If you restart the interior machine, and you don’t enter a gengen command, fingen won’t be able to run because the backing image that stores all of gengen’s build products will not have been mounted. So here’s how you can achieve a “nop” in the gengen interfacing phase that does nothing more than open up backing image properly:

. ./aliases.sh
dg_locally_gengen_ex

By the way, “ex” stands for “exact” - because any appended arguments are passed through to gengen without any mediation by the aliases layer. Without arguments, gengen will just open the backing image and ensure that some subset of the source archives are present (specifically, those hardcoded into gengen and those required for building buildroot – but not those required for building Gentoo; only the world_dryrun target does that because emerge is so slow).

Upgrading the portage snapshot

Gentoo releases a new portage snapshot daily. DeityGuard never upgrades the portage snapshot of its own volition. Thus you must take action to upgrade the portage snapshot regularily or you will soon be running outdated software with known vulnerabilities. To upgrade the portage snapshot, do, on the exterior machine:

./dg-setup.sh prefetch -kill-portage

As usual, this would be followed by:

./dg-setup.sh transfer

to transfer any new/changed files. Then, on the interior machine, in the gengen interfacing phase, do:

. ./aliases.sh
dg_locally_gengen_gentoo_portage

The above command will unpack the new portage snapshot and also run the world target and thus rebuild everything that has newer versions in the fresh portage tree. You can also mutate the set of active wunst keywords by appending world- arguments to the portage target just the same as you can do using the world target directly.

Pruning unused files

By default, DeityGuard keeps any downloaded source files indefinitely (apart from portage snapshots, which are always overwritten). This is because you may sometimes want to drop certain targets from your build and later add them back and this should not necessitate downloading the sources of those targets again. So with DeityGuard you must explicitly specify when you want source file pruning to happen (and you should probably only do so at times when you have everything you want building properly).

To prune unused source files on the exterior machine, do:

./dg-setup.sh prefetch prune

To prune unused source files on the interior machine, do, in the gengen interfacing phase:

. ./aliases.sh
dg_gengen_locally_gentoo_world_dryrun prune

Deploy the DeityGuard

Step 0: Build environment

Step 1: exterior machine

Step 2: interior machine

Step 3: exterior machine

Step 4: interior machine

Step 5: httpserv machine

Step 6: interior machine