Test Web Server Documentation

Alex's notes from creating a test web server that will eventually host important stuff (aka a test run on a cheap Dell Inspiron).

Installing Ubuntu (12/18/15 - 1/1/16)

I chose the 14.04.3 (aka "Trusty Tahr", sometimes abbreviated as "trusty" in online package documentation) Ubuntu Server ISO image for the installation process. I did so after unsuccessfully trying to install from the Minimal ISO image. I found this Ubuntu documentation page helpful during the install, and I even did a MD5 checksum verification for the first time to make sure I downloaded the ISO image correctly.

The menus for both installations were almost identical (go figure), but the Server ISO image offered a subset of the choices presented by the Minimal ISO image (for example, the Minimal installation asks about shadow passwords and Linux kernels), in a slightly different order. Here are some of the less obvious choices that I made when installing from the Server ISO image; the remaining choices were either locale-based (e.g. time zone, keyboard layout, etc.) or network configuration (I chose the wired connection as the primary interface):

• hostname: mcnairtestwebserver
• encrypted home directory? no
• how to partition disk? Guided - use entire disk and set up LVM
• how much of volume group should be used for guided partitioning? 75%
• automatic updates? no

As a note, I did have to turn off Secure boot in the Dell's UEFI firmware menu (the one that you get to by mashing F2 when the computer just turned on and shows you the Dell logo) because Ubuntu kept having Kernel panics about something attempting to kill init. This probably won't be a problem on the production server, but for replicatability's sake, now you know.

Installing the LAMP stack (1/1/16)

With the Server ISO image, one of the options during installation is to select packages that you might want to install. Obviously, I checked the LAMP server box. The Minimal installation has a similar screen, except with more options (mostly desktop-related packages, like GUIs and fonts). And now you have all that you need. Don't forget to update the package manager and upgrade all your packages:

$ sudo apt-get update
$ sudo apt-get upgrade

Network troubleshooting (1/2/16 - 1/3/16)

I'm not sure if this is my house's ethernet or my own fault, but I spent a lot of time digging into Ubuntu network configuration. I know it can work: I installed packages during the installation of Ubuntu! Anyways, with the help of the internet (this thread and this SO question were useful, as was this Ubuntu documentation) and man pages, here's a quick troubleshooting guide:

First, some diagnostics commands:

$ ping google.com
$ ping localhost
$ hostname -i
$ ifconfig

Check some relevant configuration files (note that my ethernet connection interface is named "p3p1" and is configured for DHCP instead of a static IP address):

$ cat /etc/resolv.conf
$ cat /etc/hosts
$ cat /etc/network/interfaces
$ cat /etc/dhcp/dhclient.conf
$ cat /var/lib/dhcp/dhclient.p3p1.leases

Next, try editing /etc/network/interfaces with sudo vi. I added two lines (Google's public DNS addresses and the DNS domain name of my network) into the p3p1 interface block:

    dns-nameservers 8.8.8.8 8.8.4.4
    dns-search attlocal.net

To make the changes, use sudo ifdown p3p1 and sudo ifup p3p1 to take down and bring back the network interface and try the above diagnostics again.

Also, for some reason, I was able to connect to the internet upon rebooting, but after trying the ifdown/ifup commands above, I wasn't able to get to the internet anymore. But then I rebooted again, and now I'm able to connect to the internet even after ifdown/ifup.

Edit (1/3/16): Dr. Egan notes (possibly from reading this SO post) that the following command to restart the network service is equivalent to the ifdown/ifup commands:

$ sudo service network-manager restart

Setting up SSH remote connection (1/3/16)

I got the brilliant idea to set up a remote connection to the Ubuntu box so that I could continue working on the box despite not being physically able to access it. Dr. Egan suggested SSH, and the adventure began. First, I installed the OpenSSH server, which receives SSH connections from SSH clients (I installed PuTTY as my SSH client on my Windows laptop):

$ sudo apt-get install openssh-server

Then, according to suggestions from this Ubuntu help page, I backed up the sshd_config file to a read-only copy:

$ sudo cp /etc/ssh/sshd_config /etc/ssh/sshd_config.original
$ sudo chmod a-w /etc/ssh/sshd_config.original

Now the real fun begins. I wanted to use SSH keys (specifically RSA keys) for authentication instead of password authentication (as suggested by this other Ubuntu help page), but I needed a way to copy the public RSA key on my laptop (the SSH client) onto the Ubuntu box (the SSH server). I basically decided to strip all forms of authentication off of the SSH connection by editing sshd_config and then restarting the SSH service to apply the changes:

$ sudo vi /etc/ssh/sshd_config
$ sudo service ssh restart

The sshd_config man page helped a lot (especially in noting which options were on or off by default), but I basically disabled password authentication, RSA authentication, and pubkey authentication. Then, with my laptop connected to the same network as the Ubuntu box, I opened the SSH connection and copied my key into the authorized keys list (I had to make a new authorized_keys file):

$ vi ~/.ssh/authorized_keys
$ [copy my public key to ~/.ssh/ajiang_rsa.pub]
$ cat ~/.ssh/ajiang_rsa.pub >> ~/.ssh/authorized_keys
$ rm ~/.ssh/ajiang_rsa.pub

Then I went back to sshd_config and enabled RSA and pubkey authentication, kept password authentication off, allowed TCP and X11 forwarding, set the port to 23 (according to Dr. Egan's suggestion), and explicitly specified the authorized keys file (though the default would have worked too), restarting the SSH service again to apply the changes.

$ sudo vi /etc/ssh/sshd_config
$ sudo service ssh restart

I checked that the sshd service was running and which ports it was listening to with:

$ ps -A | grep sshd
$ sudo ss -lnp | grep sshd

Now I had to configure my network's router firewall to allow port forwarding from outside the network (aka a pinhole). I fixed the IP address assigned to the box to a single IP address that I knew would work, and then I went to the port forwarding configuration page to allow TCP port forwarding on port 23 to the Ubuntu box on port 23. The router gave me a public IP address, and I used that in my PuTTY client (along with my private key and port 23) to try a SSH connection, and it worked!

Installing Mediawiki (1/4/16)

Mostly going to be following steps from this page on installing Mediawiki.

Make a directory for the stable version of Mediawiki (1.26.2), which isn't available through apt-get, so we're downloading the official tarball!

$ mkdir ~/Downloads
$ cd ~/Downloads
$ wget https://releases.wikimedia.org/mediawiki/1.26/mediawiki-1.26.2.tar.gz
$ tar -xvzf /pathtofile/mediawiki-*.tar.gz

Copy the extracted files to /var/lib/mediawiki:

$ sudo mkdir /var/lib/mediawiki
$ sudo mv mediawiki-1.26.2/* /var/lib/mediawiki

Mediawiki Security (1/15/16)

Mediawiki advises against implemented security measures because, if you're trying to make a publicly-editable wiki, you should need any user access restrictions at all (but you'll need to combat spam). We do want to make some pages publicly-editable (aka community-maintained) in the future, but much of the content should only be edited by us or a specified group of registered users. In addition, some pages should not even be viewable by unregistered or even registered users, whether that's on search results or through internal or external links.

The old webserver uses an extension called SimpleSecurity, but it's no longer maintained and has some known security issues (including allowing users to be able to see the titles of pages for which they do not have read access). These issues may be fixed by another extension, RemoveProtectedContent, but it doesn't seem like the best option.

Installing IntraACL (1/20/16)

I looked over some of the information on Mediawiki authorization extensions, including common security issues that many of the extensions have trouble fixing and a table listing several of the more popular authorization extensions and what features each supports, and a new extension, IntraACL seems to offer the most features and is the most recently maintained.

I followed the installation instructions for IntraACL pretty much line-for-line, including the patch.

Understanding IntraACL (1/22/16)

So I must have set something during the installation or configuration of Mediawiki that revoked the 'edit' page permission from all users (even anonymous users), which I think applies before the IntraACL. The line in LocalSettings.php looks like:

$wgGroupPermissions['*']['edit'] = false;

and I commented it out so that the IntraACL can manage permissions. I also found a special page in Mediawiki that lists the user group rights. Mediawiki has a page listing the different user rights and user groups.

The thing is, IntraACL has its own system of groups. So we can either focus entirely on IntraACL groups or try to apply broad changes with Mediawiki groups and LocalSettings.php $wgGroupPermissions. I think we should try going with IntraACL's groups...

Finding Holes in IntraACL (1/22/16)

Going through the issues on this Mediawiki page

• Inclusion/Transclusion

• Preloading

• XML Export (Special:Export)

• Atom/RSS Feeds

• Listings & Search

Pages that can't be read still have their titles show up in search auto-complete, but not in search results. Can disable search-box autocomplete, as shown on this Mediawiki page

• Diff & Revision Links

• API

• Action Links

• Related Rights

• Author Backdoor

• Caching

• Files & Images

• Redirects

• Edit Section

• Watching Pages

• Other Extensions

Creating Users (1/25/16)

Assuming that you have root, you can create user accounts and give them root too. The process is:

First create the users group, checking the last group number:

$ cat /etc/group
$ /usr/sbin/groupadd -g 515 username 

Then add the user

$ /usr/sbin/useradd -g username -G root -s /bin/bash -p xxxx -d /home/username -m username
      where g is the primary group, G is other groups, p sets a password, 
      d declares a home directory and m makes the directory

Change the user's password:

$ passwd username

And add the user to the sudoers file

$ echo 'username ALL=(ALL) ALL' >> /etc/sudoers

To delete a user:

$ /usr/sbin/userdel -r roger 
      where r removes the home directory

And to remove their group

$ /usr/sbin/groupdel username

Labeled Section Transclusion (1/25/16)

Ed wanted me to look into Labeled Section Transclusions, which would allow two different sections of a single page to be transcluded onto different locations. There's an extension for that, but the extension download is only compatible with Mediawiki version 1.19 or newer, and our current webserver is on Mediawiki 1.13.3 (can check Mediawiki version on the Special:Version page). But I got the LST extension on the test webserver and successfully transcluded different sections. Installing the extension is pretty straightforward: wget the tarball link, untar it, and copy to the /var/lib/mediawiki/extensions directory:

$ cd ~/Downloads
$ wget https://extdist.wmflabs.org/dist/extensions/LabeledSectionTransclusion-REL1_26-60037a2.tar.gz
$ tar -xzvf LabeledSectionTransclusion-REL1_26-60037a2.tar.gz
$ cd /var/lib/mediawiki/extensions
$ cp -r ~/Downloads/LabeledSectionTransclusion ./LabeledSectionTransclusion

Then add this line to LocalSettings.php:

require_once "$IP/extensions/LabeledSectionTransclusion/LabeledSectionTransclusion.php";

This Wikipedia page documents a method for selective transclusion that doesn't require the extension, but I was unable to replicate the results on the test webserver, so I assume that it requires some package or code that is specific to Wikipedia.

Responsive Design (1/25/16)

According to this mediawiki page, the extension that makes Wikipedia look pretty on mobile/tablet is called MobileFrontend (and it requires another extension called Mantle for Mediawiki versions 1.24 or older). Since the extension only serves up downloads for MW 1.19 or newer, I have to test this on the test webserver (MW 1.26).

As with the LST extension, the installation of the extension is pretty standard stuff: wget the tarball, untar it, and copy it to the extensions directory:

$ cd ~/Downloads
$ wget https://extdist.wmflabs.org/dist/extensions/MobileFrontend-REL1_26-187dae8.tar.gz
$ tar -xzvf MobileFrontend-REL1_26-187dae8.tar.gz
$ cd /var/lib/mediawiki/extensions
$ cp -r ~/Downloads/MobileFrontend ./MobileFrontend

Then, per the installation instructions on the extension page, add these two lines to LocalSettings.php:

require_once "$IP/extensions/MobileFrontend/MobileFrontend.php";
$wgMFAutodetectMobileView = true;

Short URLs (1/27/16)

The goal is to get the URLs to look like www.mcnaircenter.org/wiki/Page_Title. Mediawiki has a page for configuring short URLs on Apache.

First, enable the mod_rewrite Apache module (if it isn't enabled already). In Apache 2.4.7, this can be done from the command line:

$ sudo a2enmod rewrite

You can also check that mod_rewrite has been enabled by creating a phpinfo page and looking for mod_rewrite under the apache "Loaded Modules" section as described in this Stack Overflow answer.

As described on the Mediawiki manual page (and also by the web utility that the manual page recommends), add these lines to the end of the VirtualHost block in the Apache configuration file that defines the wiki's DocumentRoot (I added them to /etc/apache2/sites-available/000-default.conf): (You may also want to make a backup of the original configuration file in case you want to revert)

## http://www.mediawiki.org/wiki/Manual:Short_URL/Apache 
# Enable the rewrite engine
RewriteEngine On

# Short url for wiki pages
RewriteRule ^/?wiki(/.*)?$ %{DOCUMENT_ROOT}/mediawiki/index.php [L]

# Redirect / to Main Page
RewriteRule ^/*$ %{DOCUMENT_ROOT}/mediawiki/index.php [L]

Then restart Apache:

$ sudo service apache2 restart

Find the line in LocalSettings.php that sets the $wgScriptPath and add these two lines below that line:

$wgScriptExtension = ".php";
$wgArticlePath = "/wiki/$1";

Infoboxes (2/1/16)

Here's the Mediawiki page about importing infoboxes from Wikipedia (here's a list of infobox templates and a help page on infoboxes and another help page on designing infoboxes from Wikipedia).

Installation procedure begins with installing the Scribunto extension:

$ cd ~/Downloads
$ wget https://extdist.wmflabs.org/dist/extensions/Scribunto-REL1_26-4d4766f.tar.gz
$ tar -xzvf Scribunto-REL1_26-4d4766f.tar.gz
$ cd /var/lib/mediawiki/extensions
$ cp -r ~/Downloads/Scribunto ./Scribunto

Then, per the installation instructions on the extension page, add these two lines to LocalSettings.php:

require_once "$IP/extensions/Scribunto/Scribunto.php";
$wgScribuntoDefaultEngine = 'luastandalone';

The ParserFunctions extension comes with Mediawiki 1.18 or newer, but you still need to load the extension in LocalSettings.php:

wfLoadExtension( 'ParserFunctions' );

Finally, give all users execute permissions on the Lua binaries that come with the extension (note that you should choose the correct OS in the binaries directory; you can use the uname -m command to determine whether the Linux kernel is 64- or 32-bit)

$ sudo chmod a+x /var/lib/mediawiki/extensions/Scribunto/engines/LuaStandalone/binaries/lua5_1_5_linux_64_generic/lua

And check if SELinux is enabled with sestatus -v. If it is enabled, set the type:

$ chcon -t httpd_sys_script_exec_t /var/lib/mediawiki/extensions/Scribunto/engines/LuaStandalone/binaries/lua5_1_5_linux_64_generic/lua

Go to Wikipedia's Special:Export page and list the infobox templates that you want to export. For example, I chose to export the following templates, and checked the options "Include only the current revision, not the full history", "Include templates", and "Save as file":

Template:Infobox
Template:Infobox/doc
Template:Infobox person
Template:Infobox person/doc
Template:Infobox company
Template:Infobox company/doc

Then go to the test wiki's Special:Import page and choose the XML file that you got from the step above. I chose to "Import to default locations".

The import threw a 500 Server Error, but when I checked the Special:AllPages list of all pages in the Template namespace, the entries for the Template:Infobox, etc. pages show up but when I try to see them, I get another 500 Server Error.