My Profile Photo

Sheogorath's Blog

Config management at MadIRC

• on

A week ago I’ve talked about configuration management of an InspIRCd instance in the #InspIRCd support channel. So today I’ll provide you a little insight into the configuration management I build for MadIRC.

But first of all:

What is InspIRCd and IRC in general

InspIRCd is an IRC daemon written in C++ and one of the most popular IRC daemons in the world. IRC allows users to communicate in Chatroom all over the world in multiple IRC networks. MadIRC is one of those networks, but a young one. MadIRC exists since 2 years but IRC itself started in 1988. 1993 the first RFC of the protocol was published.

The population of IRC networks was growing fast and until 2004 IRC was the most popular chat protocol ever seen. Even today it’s a big player in chatting but not in its classical use. Twitch’s whole chat system is based on IRC and also internal chatting of Minecraft and Counter Strike.

Why is IRC interesting? It’s a really simple, text-based protocol. You can write a bot in less than 10 Minutes and there is no need to register an account or similar. You can use a plain TCP socket and send you commands.

But back to topic:

The idea behind the configuration structure

MadIRC as an IRC network needs to have some parts of its configuration shared. Means there are common parts inside our configuration which need to be identical on all servers but also some individual settings which can be different on each server. It’s also important to have a history of our configuration. And at least we want to be able to refresh the configuration on all servers without the need to access the servers by using SSH or similar.

What we did: At first we created a private git repository for each server on a central Gogs instance. Those repositories are filled with the individual settings of the servers.

Every git repository has the same git submodule. This submodule is another repository named ‘common’ and stores all network wide settings.

What we covered now is:

  • history of configuration
  • individual and common parts of our configuration

Updating the configuration using a script

To allow updating our configuration we added SSH keys without a password on each host and added the public key fingerprint as deployment key to the common git repository and to their individual one.

So we are ready to use git commands to update our local configuration repositories.

InspIRCd allows to run a script on rehash and startup if you place an <include executable>-tag inside the configuration.

<include executable="conf/common/update-config.sh &servername;">

This line calls a shell script inside our common directory which updates our configuration. And as long as we don’t update the file which includes the line we use right now it’s working on every reloading process.

Let’s see the content of conf/common/update-config.sh:

#!/bin/sh
cd conf/
git pull >> update.log 2>&1
git submodule foreach git pull origin master >> update.log 2>&1
rev=`git rev-parse HEAD`
echo '<alias text="CONFREV" replace="NOTICE $nick :Configuration rev ID is '$rev' for '$1'" operonly="yes">'
cd ..

What we do is we enter the configuration directory, update the individual configuration and then update all submodules. At least we add an alias for our current configuration reference. Means the git commit hash. (Please notice this requires m_alias to be loaded)

Configuration handling

Handling round about 5000 lines of configuration isn’t fun, especially if spread across 51 files. InspIRCd doesn’t allow include-directories. So you need to add an <include>-tag for each file. Horror!

So we started with adding includes to the configuration files but we quickly ended up in a real jungle of includes where some hosts have individual files to override settings while others don’t have and so on.

So we added a script to generate <include>-tags automatically to include files from the individual and common settings if they exist.

After a while we discovered that having configuration directories - like apache2 has - is much more useful, so we added this ability to our script, too. So today we have the following script running:

#!/bin/bash


function includeCommonFile {
 if [ -z "$1" ]
  then
   return
 fi
 echo "<include file=\"conf/common/$1\">"
}

function includeFile {
 if [ -z "$1" ]
  then
   return
 fi
 if [ -f "conf/$1" ]
 then
  echo "<include file=\"conf/$1\">"
 fi
}

function includeCommonFirst {
 if [ -z "$1" ]
  then
   return
 fi
 includeCommonFile "$1"
 includeFile "$1"
}
function includeCommonLast {
 if [ -z "$1" ]
  then
   return
 fi
 includeFile "$1"
 includeCommonFile "$1"
}

function includeConfFolder {
 for moduleConfFile in $1
 do
   echo "<include file=\"$moduleConfFile\">"
 done;
}

includeCommonFile "general.conf"
includeCommonFirst "binds.conf"
#-#-#-#-#-#-#-#-#-#-  CONNECTIONS CONFIGURATION  -#-#-#-#-#-#-#-#-#-#-#
#Servers
includeCommonLast "connectblockserver.conf"
includeCommonLast "connectblockexception.conf"
includeCommonFile "connectblocksasl.conf"

#Users
includeFile "connectblock.conf"
includeCommonFile "connectblock.conf"


#Opers.conf
includeCommonFirst "opers.conf"

#-#-#-#-#-#-#-#-#-#-#-#-#-#-  BAN OPTIONS  -#-#-#-#-#-#-#-#-#-#-#-#-#-#
includeCommonLast "badnicks.conf"

# exception: Hosts that are exempt from [kgz]lines.  
includeCommonLast "eline.conf"
#other badhosts
includeCommonLast "badhosts.conf"

#-#-#-#-#-#-#-#-#-#-#-    Server Links     -#-#-#-#-#-#-#-#-#-#-#-#-#-#
includeCommonLast "links.conf"

#-#-#-#-#-#-#-#-#-#-#-#-#-#-#-#-# MODULES #-#-#-#-#-#-#-#-#-#-#-#-#-#-#
includeCommonFile "modules.conf"

includeConfFolder "conf/common/modules/*.conf"
includeConfFolder "conf/modules/*.conf"

This handles the whole configuration file jungle and is really really useful. The reason why we have the normal modules configuration and the modules directory includes: Well, there are a lot of modules which only need to be loaded. So having each of them in an own file would be a bit too much. In other words we were lazy ;)

Just a side-note: The reason for having includeCommonFirst and includeCommonLast is that sometimes the order is important. Especially for connect block configuration.

Message of the day

The next problem we were running into was synchronizing the MOTD of all servers. Because MOTD doesn’t have includes, we had to catch this also using a script.

We just used cat and echo to generate individual MOTD files on each server using the common and the individual configuration directories.

#!/bin/bash
###############
## $1 = Servername
###############

rm -r conf/cache/ > /dev/null
mkdir conf/cache/ > /dev/null
for motdFile in conf/common/motd/*.motd
do
 for insertFile in conf/common/motd/*.txt
 do
  cat $insertFile >> conf/cache/`basename $motdFile`
  echo "" >> conf/cache/`basename $motdFile`
  echo "" >> conf/cache/`basename $motdFile`
 done;
 cat $motdFile >> conf/cache/`basename $motdFile`
 echo "" >> conf/cache/`basename $motdFile`
 echo "Generated at: "`date` >> conf/cache/`basename $motdFile`
done;
if [ `ls -1 conf/motd | wc -l` -gt 0 ]
 then
  for motdFile in conf/motd/*.motd
  do
   for insertFile in conf/common/motd/*.txt
   do
    cat $insertFile >> conf/cache/`basename $motdFile`
    echo "" >> conf/cache/`basename $motdFile`
    echo "" >> conf/cache/`basename $motdFile`
   done;
   cat $motdFile >> conf/cache/`basename $motdFile`
   echo "" >> conf/cache/`basename $motdFile`
   echo "Generated at: "`date` >> conf/cache/`basename $motdFile`
  done;
fi
sed -i 's/&servername;/'$1'/g' conf/cache/*.motd
sed -i 's/&fingerprint;/'`openssl x509 -in conf/cert.pem -noout -fingerprint | cut -d'=' -f 2`'/g' conf/cache/*.motd
sed -i 's/&fingerprint256;/'`openssl x509 -in conf/cert.pem -noout -sha256 -fingerprint | cut -d'=' -f 2`'/g' conf/cache/*.motd
sed -i 's/&fingerprintmd5;/'`openssl x509 -in conf/cert.pem -noout -md5 -fingerprint | cut -d'=' -f 2`'/g' conf/cache/*.motd

echo "<files motd=\"conf/cache/common.motd\" rules=\"conf/common/rules.txt\""
for finishedMotdFile in conf/cache/*.motd
do
 echo `basename $finishedMotdFile .motd`"=\""$finishedMotdFile"\""
done;
echo ">"

This is a lot of black magic for a few lines of MOTD but we have to say that we have multiple MOTD for different connect classes and we include our SSL Fingerprints and server name into the MOTD. So maybe you could cut it a bit to fit your needs.

For all of you interested in calling this file:

<include executable="conf/common/dynamic-motd.sh &servernetworkname;">

So what we got now is nearly our whole configuration structure.

The basic configuration

So as we got the whole configuration thing now the last question is: How to use it?

Well, the answer is really simple. I’ll show you our inspircd.conf:

<config format="xml">
<include file="conf/common/inspircd.conf">

Yeah, that’s it. It’s nearly empty. And now let’s see conf/common/inspircd.conf:

<include file="conf/define.conf">
<include executable="conf/common/update-config.sh &servername;">
<include executable="conf/common/dynamic-motd.sh &servernetworkname;">
<include executable="conf/common/dynamic-include.sh">

So we add the local define configuration to get the servername, updating our configuration, generate and include the MOTD files and load configuration by using the dynamic include script.

Conclusion

That’s it! Managing IRCd configuration is so easy if you only know how to!

You may insist and say “Why aren’t you using Ansible or puppet to mange the configuration?” Yeah… To be honest: When we were writing those scripts I didn’t know about Ansible :D

On the other hand I now know Ansible and really don’t want to create something similar using templates. It’s crazy.

And we use this setup for over 1½ year and it’s so lovely because it just works. Since the last few month our network structure has changed and I have no access to some IRCd host systems by SSH but I can maintain their configuration without problems.

All I need to do is /rehash * and all servers in our network are updating their configuration and reloading it.

So I hope you enjoyed the read and learned something useful. If you have suggestions or questions send me an email, message me on Mastodon or write a comment :) Don’t forget to share this post if you like it and tell your friends!

Further information:

MadIRC: