Category Archives: software

Migrating gitlab projects

Posted on by

If you’re migrating a gitlab project from one server to another, unless the two gitlab instances are the same major revision you may run into a couple of problems with the export/import procedure.

The first error you’re likely to hit is something like:

The repository could not be imported.
Error importing repository into pp-computing/todo-list - Import version mismatch: Required 0.1.8 but was 0.1.6

This is because whenever there is a potentially “dangerous” change to the import script, gitlab “fails safe” and refuses to import the project. If the two numbers are reasonably close together (and your project is straight forward enough that you can carefully check the users, permissions, wiki pages, issues and milestones etc then you can try this to pretend that your export tarball is newer than it really is:


mkdir project_export
tar xfv old_export_file.tar.gz -C project_export
cd project_export
echo '0.1.8' > VERSION
tar czf experimental_new_project_export.tar.gz *

If you have milestones in your project, you may hit another error if you’re migrating from a gitlab instance that is older than 9.5 is:


Error importing repository into my-group/my-project - Validation failed: Group milestone should belong either to a project or a group.

The workaround for this one appears to be to import your project into your personal gitlab space, and then “move” it to your group space.

If you hit any errors not covered in the above, let us know below!

(And don’t forget you’ll need to update your remotes in any checked out working copies you have!)

Molly-guard for RHEL/CentOS – protect your hosts from accidental reboots!

Posted on by

Molly-guard is a very useful package which replaces the default halt and reboot (and other related) commands with a version which prompts you to type the hostname of the host you intended to halt/reboot before it continues to do so. For example:

root@testhost:~# reboot
I: molly-guard: reboot is always molly-guarded on this system.
Please type in hostname of the machine to reboot: [type incorrect hostname]
Good thing I asked; I won't reboot testhost ...
W: aborting reboot due to 30-query-hostname exiting with code 1.

This is invaluable if you use a lot of different systems and they are often in use by other users whom you don’t want to anger with accidental reboots…

For Debian-based distros (including Ubuntu), it’s available via a simple apt-get install molly-guard. On RHEL-based distros, unfortunately, it’s not in the base repositories and I was unable to find a suitably-trustworthy repository which contains it.

So this leads to asking some questions:

What does it do?

Simply put, it copies the existing /sbin/halt and related commands to a separate directory (by default /lib/molly-guard), and replaces them with symlinks to /lib/molly-guard/molly-guard to ensure that the new executable is used.

By default it only requires hostname confirmation when you are logged in via ssh, but this can be changed to always ask for the hostname by setting the ALWAYS_QUERY_HOSTNAME variable in the /etc/molly-guard/rc configuration file. Further customisations are possible by adding scripts to run to the /etc/molly-guard/run.d directory, and if any of these exit with a non-zero exit code then the reboot is aborted. (This is how the hostname check is done, but you can add whatever logic you want via this method)

How can we make this work on RHEL / Why are there no packages for RHEL?

Someone kindly ported a version of molly-guard from Debian to RHEL, and a github repo of this is available here. Unfortunately this doesn’t quite solve the problem, as creating a package from this (having updated it for molly-guard 0.6.2) creates an RPM which gives us errors when we try to install it:

Running Transaction Test


Transaction Check Error:
  file /sbin/halt from install of molly-guard-0.6.2-1.1.noarch conflicts with file from package upstart-0.6.5-13.el6_5.3.x86_64
  file /sbin/poweroff from install of molly-guard-0.6.2-1.1.noarch conflicts with file from package upstart-0.6.5-13.el6_5.3.x86_64
  file /sbin/reboot from install of molly-guard-0.6.2-1.1.noarch conflicts with file from package upstart-0.6.5-13.el6_5.3.x86_64
  file /sbin/shutdown from install of molly-guard-0.6.2-1.1.noarch conflicts with file from package upstart-0.6.5-13.el6_5.3.x86_64

RPM really doesn’t like replacing files which are owned by another package, so an alternative (you’ll see what I did there in a minute) strategy is required.

Handily there’s a tool called alternatives which can handle selecting which of a set of binaries to use, via managing a directory of symlinks. (See!)

If we re-create the RPM without the explicit symlinks and instead use a post-install script snippet which copies the halt/reboot binaries to the molly-guard directory and sets up alternatives to point at them then this might just work!

(There are a bunch more reboot/halt commands which are in /usr/bin/... on RHEL, so we need to turn those into links as well:

alternatives --install /sbin/halt halt              /lib/molly-guard/molly-guard 999 \
  --slave /sbin/powreoff          poweroff          /lib/molly-guard/molly-guard \
  --slave /sbin/reboot            reboot            /lib/molly-guard/molly-guard \
  --slave /sbin/shutdown          shutdown          /lib/molly-guard/molly-guard \
  --slave /sbin/coldreboot        coldreboot        /lib/molly-guard/molly-guard \
  --slave /sbin/pm-hibernate      pm-hibernate      /lib/molly-guard/molly-guard \
  --slave /sbin/pm-suspend        pm-suspend        /lib/molly-guard/molly-guard \
  --slave /sbin/pm-suspend-hybrid pm-suspend-hybrid /lib/molly-guard/molly-guard \
\
  --slave /usr/bin/reboot         usrbinreboot      /lib/molly-guard/molly-guard \
  --slave /usr/bin/halt           usrbinhalt        /lib/molly-guard/molly-guard \
  --slave /usr/bin/poweroff       usrbinpoweroff    /lib/molly-guard/molly-guard

# Ensure we're using this by default:
alternatives --set halt /lib/molly-guard/molly-guard

At this point we’ve got a set of alternatives and a post-install RPM scriptlet which copies the required commands into the /lib/molly-guard directory, but there is still a problem that RPM will clobber these when the clashing packages are updated! So we definitely need to ensure that our alternatives get re-added after any updates. To do this we can do one of the following:

  • Have a cron job which runs regularly and enforces the alternatives setting for “halt” (the “slave” entries trigger all the others to update when this is done)
  • Use Puppet or some other configuration management to enforce the alternatives setting (will have a small window of the binaries being the new version until the alternatives are reset by Puppet on the next run)
  • Scrap the alternatives method and use something else like modifying the PATH to ensure our reboot/halt versions are before the system versions (will break if scripts use full paths to the commands, which is pretty common)
  • Drop this approach and create a PAM module which does the same as molly-guard, then use consolehelper and PAM instead (untested)

I’ve decided to go for the second of these and have configuration management ensure that the “alternatives” settings are enforced.

Will this have any side-effects?

One side-effect is that on RHEL6 the reboot-related commands in /usr/bin use “consolehelper” to control access to these commands through PAM. Without some additional jiggery-pokery this functionality will be broken by this update.

Updates to the packages will need to be handled somehow, perhaps by detecting that the binaries have been put back in place and updating the molly-guard-controlled versions. (Eek!)

I’ve not looked at updating the package for RHEL7, where the reboot and related commands all link to systemctl for systemd control. This is likely to need some different shenanigans to get it to work…

Where can I get the packages?

These are a work in progress but the updated git repoistory is available here. RPMs will be available from http://packages.bris.ac.uk/centos/6/zone_d/ (UoB internal only access) once finalised. It it works well these should be published more widely 🙂

No RHEL7 packages yet I’m afraid — I need to investigate how to get this to work with systemd!

(Bonus Q: Why is it called Molly-guard? See this definition for an explanation)

Linux VMs on Hyper-V – be sure to install and run hyperv-daemons!

Posted on by

A short post, just to say that if you are running Linux VMs on Hyper-V hypervisors you really should install and run the hyperv daemons.

On RHEL7-based distros this leads to :

yum install hyperv-daemons
systemctl enable hypervvssd
systemctl enable hypervkvpd

Ubuntu users should install the packages specified on this Microsoft TechNet page (I’ve not tested this myself, as I don’t yet have any Ubuntu VMs on Hyper-V).

Once you’ve done this a whole host of important features will work, including:

  • Live migration of VMs
  • IP injection (?)
  • Dynamic memory sizing
  • etc..

This will avoid any surprise reboots when hypervisor nodes are taken down for maintenance (which is what happened to me before I installed these..).
Obviously the complete failure of a hypervisor will still cause VM downtime,

I leave it as an exercise to the reader to use configuration management to add these to all their Hyper-V VMs automatically 🙂

Further reading : Best Practices for running Linux on Hyper-V

Packaging mod_auth_cas for CentOS 7

Posted on by

Mark wrote a useful post about building mod_auth_cas for CentOS 7. It works, but I prefer to build RPM packages on a build server and deploy them to production servers, rather than building on production servers.

Basically I took the spec file from the mod_auth_cas source package for CentOS 6 from the EPEL 6 repository, tweaked it, and replaced the source tarball with the forked copy Mark recommended. This built cleanly for CentOS 7.

I’ve sent a pull request back to the upstream project with the Red Hat build files and documentation and for those who are keen, here are the EL7 RPM and source RPM:

Update

There’s an even better way of building this for CentOS 7. Fedora 21 includes Apache 2.4 and mod_auth_cas and it is really easy to backport this source package.

First grab the latest version of the source package from the Fedora mirror onto your CentOS 7 build server. Always make sure there isn’t a newer version available in the updates repo.

Rebuild is as simple as issuing:

rpmbuild --rebuild mod_auth_cas-1.0.8.1-11.fc22.src.rpm

It will spit out an RPM file suitable for deployment on CentOS 7. Add --sign if you routinely sign your packages with an RPM-GPG key.

mod_auth_cas on CentOS7 / Apache 2.4

Posted on by

For CentOS, this is now available in the EPEL repo

mod_auth_cas (https://github.com/Jasig/mod_auth_cas) is an Apache module that plugs into the Apache mod_auth framework, to provide authentication against a Jasig CAS SSO server.  Unfortunately development on it seems to have stalled, and it currently doesn’t support Apache 2.4 – https://github.com/Jasig/mod_auth_cas/issues/49.

There is a fork that does support Apache 2.4, so this willl cover how to build and install it.

# Clone the forked github repo
git clone https://github.com/klausdieterkrannich/mod_auth_cas.git .

# install development libraries
yum install gcc httpd-devel openssl-devel libcurl-devel automake

# Run configure
./configure

# Run Make
make

# If you get errors like:
/opt/mod_auth_cas/missing: line 81: aclocal-1.12: command not found
WARNING: 'aclocal-1.12' is missing on your system.
# then symlink the binaries as follows:
ln -s /usr/bin/aclocal /usr/bin/aclocal-1.12
ln -s /usr/bin/automake /usr/bin/automake-1.12
# and run make again

# Assuming it built OK, then install it:
make install

# On CentOS this will put the binaries into:
#/usr/lib64/httpd/modules
# and you can then copy them from here to your production systems.