The big picture
The Tails ISO build system downloads a set of Tor Browser tarballs from a location specified in config/chroot local-includes/usr/share/tails/tbb-dist-url.txt, and compares their hash with previously verified ones found in config/chroot local-includes/usr/share/tails/tbb-sha256sums.txt.
Once released officially, Tor Browser tarballs can be found in
a permanent (?)
location.
However, when upgrading Tor Browser for an imminent Tails release, we
generally have to use Tor Browser tarballs that are under QA and not
officially released yet. So, we have to retrieve them from another,
temporary location, such as
http://people.torproject.org/~mikeperry/builds/. If we hard-coded
this temporary URL in tbb-dist-url.txt
, then our release tag would
only be buildable for as long the tarballs stay in that place, which
at best is a few months.
To solve this, we host ourselves the Tor Browser tarballs we need, and point to this permanent location for anything that we tag.
Still, one can set an arbitrary download location in
tbb-dist-url.txt
, which should provide all the flexibility needed
for development purposes.
Upgrade Tor Browser in Tails
Have a look at
- https://archive.torproject.org/tor-package-archive/torbrowser/
- https://www.torproject.org/dist/torbrowser/
- https://people.torproject.org/~mikeperry/builds/
- https://people.torproject.org/~gk/builds/
- https://people.torproject.org/~boklm/builds/
- https://people.torproject.org/~linus/builds/
and see if the desired version is available. Set TBB_DIST_URL
to the
chosen URL, and set TBB_VERSION
to the desired Tor Browser version, for
example:
TBB_DIST_URL=https://people.torproject.org/~mikeperry/builds/4.5-build5/
TBB_VERSION=4.5-build5
Fetch the version's hash file and its detached signature, and verify with GnuPG:
wget ${TBB_DIST_URL}/sha256sums-unsigned-build.txt{.asc,} && \
gpg --verify sha256sums-unsigned-build.txt{.asc,}
Filter the tarballs we want and make them available at build time, when the tarballs are fetched:
grep --color=never "\<tor-browser-linux64-.*\.tar.xz$" sha256sums-unsigned-build.txt \
| grep -v '\<tor-browser-linux64-debug\.tar\.xz$' \
> config/chroot_local-includes/usr/share/tails/tbb-sha256sums.txt
Then update the URL to the one chosen above:
echo "${TBB_DIST_URL}" | sed "s,^https://,http://," > \
config/chroot_local-includes/usr/share/tails/tbb-dist-url.txt
We cannot use HTTPS due to limitations/bugs in
apt-cacher-ng
, which often is used in Tails build
environments. However, it is of no consequence since we verify the
checksum file.
Lastly, commit:
git commit config/chroot_local-includes/usr/share/tails/tbb-*.txt \
-m "Upgrade Tor Browser to ${TBB_VERSION}." && \
git show
If this new Tor Browser is meant to be included in a Tails release, then that's not enough: as explained above, we need to host the corresponding tarballs ourselves, so read on the next section.
Sync with the upstream wrapper scripts
Adapt our config/chroot_local-includes/usr/local/bin/tor-browser
and/or
config/chroot_local-includes/usr/local/lib/tails-shell-library/tor-browser.sh
for recent changes made in the
Tor Browser build Git repo:
git log -p \
projects/firefox/abicheck.cc \
projects/firefox/start-firefox \
projects/tor-browser/RelativeLink/start-tor-browser
Then apply any relevant change, e.g. to:
- environment variables;
- commandline options passed to the
firefox
executable; - required libstdc++6 version bumps; if there's been any change upstream,
look for
abicheck
inconfig/chroot_local-hooks/10-tbb
and adjust that hook as needed.
Self-hosted Tor Browser tarballs archive
Initial setup
First, install git-annex.
Then, make sure you have an entry for git.puppet.tails.boum.org
in
your ~/.ssh/config
. See systems/ISO_history.mdwn
in the internal Git repo
for details.
Then, clone the metadata repository and initialize git-annex:
git clone gitolite@git.puppet.tails.boum.org:torbrowser-archive.git && \
cd torbrowser-archive && \
git annex init
You now have a lot of (dangling) symlinks in place of the files that are available in this git-annex repo.
To synchronize your local git-annex metadata with the remote, run:
git annex sync
Set up environment variables
Make sure you still have the environment variables defined in the previous section set.
Make
TAILS_GIT_REPO
point to the main Tails Git repository checkout wheretbb-dist-url.txt
is being worked on, for example:TAILS_GIT_REPO="$HOME/tails/git"
Make
TBB_ARCHIVE
point to your local git annex working copy of our Tor Browser archive, for example:TBB_ARCHIVE="$HOME/tails/torbrowser-archive"
Make
TBB_IMPORT_BRANCH
point to the branch where you want to import the new Tor Browser's metadata, for example:TBB_IMPORT_BRANCH=feature/123456-torbrowser-42.3.4
Import a new set of Tor Browser tarballs
Download and verify all the tarballs we need:
DL_DIR=$(mktemp --tmpdir -d "tor-browser-${TBB_VERSION}.XXXXXXXXXX") CHROOT_INCLUDES="config/chroot_local-includes" TBB_SHA256SUMS_FILE="${CHROOT_INCLUDES}/usr/share/tails/tbb-sha256sums.txt" TBB_DIST_URL_FILE="${CHROOT_INCLUDES}/usr/share/tails/tbb-dist-url.txt" cd "$TAILS_GIT_REPO" && git checkout "$TBB_IMPORT_BRANCH" TBB_TARBALLS_BASE_URL="$(cat "${TBB_DIST_URL_FILE}" | sed "s,^http://,https://,")" current_branch=$(git -C "$TAILS_GIT_REPO" branch | awk '/^\* / { print $2 }') for branch in "$current_branch" ; do git -C "$TAILS_GIT_REPO" show "$branch:$TBB_SHA256SUMS_FILE" \ | while read expected_sha256 tarball; do ( cd "$DL_DIR" echo "Retrieving '${TBB_TARBALLS_BASE_URL}/${tarball}'..." curl --remote-name --continue-at - \ "${TBB_TARBALLS_BASE_URL}/${tarball}" ) done ( cd "$DL_DIR" && \ git -C "$TAILS_GIT_REPO" show "$branch:$TBB_SHA256SUMS_FILE" \ | sha256sum -c - ) done
Move the tarballs into your local Git annex:
cd "$TBB_ARCHIVE" && \ mkdir "$TBB_VERSION" && cd "$TBB_VERSION" && \ git annex import --duplicate "$DL_DIR/"* "$TAILS_GIT_REPO/"sha256sums-*
Commit and push your changes
cd "$TBB_ARCHIVE" && \
git commit -m "Add Tor Browser ${TBB_VERSION}." && \
git annex sync && \
git annex copy --to origin -- "${TBB_VERSION}"
Wait for the synchronization
Once you've gone through these steps, a cronjob that runs every 5 minutes will download the tarballs and make them available on http://torbrowser-archive.tails.boum.org/.
Wait for this to happen before you proceed with the next steps.
In the meantime, you might want to import the new Tor Browser tarballs
into your apt-cacher-ng
local cache.
Adjust the URL in the main Git repository
cd "$TAILS_GIT_REPO" && \
git checkout "$TBB_IMPORT_BRANCH"
current_branch=$(git branch | awk '/^\* / { print $2 }')
for branch in "$current_branch" ; do
git checkout "$branch" && \
echo "http://torbrowser-archive.tails.boum.org/${TBB_VERSION}/" > \
config/chroot_local-includes/usr/share/tails/tbb-dist-url.txt && \
git commit config/chroot_local-includes/usr/share/tails/tbb-dist-url.txt \
-m "Fetch Tor Browser from our own archive." && \
git show
done
Clean up
cd "$TBB_ARCHIVE" && \
git annex drop -- "${TBB_VERSION}" && \
rm -rf "$DL_DIR"
Update the htpdate User Agent
We want to use the same user agent in our htpdate script (see the Time syncing design for more info on that) as in Tor Browser.
To find out the User Agent of the new Tor Browser:
- Start Tor Browser (outside of Tails, if there is no ISO yet with the new Tor Browser)
- Open the Network tab in the Developer Tools (Ctrl+Shift+E)
- Load a website (e.g. https://tails.boum.org)
- Select one of the GET requests in the Developer Tools
- Scroll down to
User-Agent
in the Request headers section
Now replace the User Agent in config/chroot_local-includes/etc/default/htpdate.user-agent
with the one you found above.