boompty boomp

Please note that this is not a production-ready installation document! As the title suggests, this is a fast and dirty installation of Graphite for testing, and assumes you’re installing on a new virtual machine dedicated to Graphite. This will break stuff if you run on an existing server!

These installation instructions are basically the steps from http://graphite.wikidot.com/installation, with a few minor adjustments to work on Debian Squeeze.

First things first:

# cd /root
# apt-get update
# apt-get install bzr

Once bzr’s installed, we can:

# bzr branch lp:graphite

Install Whisper:

# cd graphite/whisper
# python setup.py install

At the time of writing, Whisper can be installed from apt-get on Squeeze, but the version doesn’t match what we’ve pulled Launchpad.

Install Carbon:

# cd ../carbon
# python setup.py install

Now we copy some sample configurations into place:

# cd /opt/graphite/conf
# cp carbon.conf.example carbon.conf
# cp storage-schemas.conf.example storage-schemas.conf

Graphite depends on a number of other packages, and I’ve made every attempt to grab stuff from stock apt repos rather than building (unnecessarily) from source. To see what’s missing, run:

cd /root/graphite
python check-dependencies.py

This should return something like this:

# python check-dependencies.py
[FATAL] Unable to import the 'cairo' module, do you have pycairo installed for python 2.6.6?
[FATAL] Unable to import the 'django' module, do you have Django installed for python 2.6.6?
[FATAL] Unable to import the 'tagging' module, do you have django-tagging installed for python 2.6.6?
[WARNING] Unable to import Interface from zope.interface.
Without it, you will be unable to run carbon on this server.
[WARNING] Unable to import the 'mod_python' module, do you have mod_python installed for python 2.6.6?
mod_python is one of the most common ways to run graphite-web under apache.
Without mod_python you will still be able to use the built in development server; which is not
recommended for production use.
wsgi or other approaches for production scale use are also possible without mod_python
[WARNING]
Unable to import the 'memcache' module, do you have python-memcached installed for python 2.6.6?
This feature is not required but greatly improves performance.
 
[WARNING]
Unable to import the 'ldap' module, do you have python-ldap installed for python 2.6.6?
Without python-ldap, you will not be able to use LDAP authentication in the graphite webapp.
 
[WARNING]
Unable to import the 'twisted' package, do you have Twisted installed for python 2.6.6?
Without Twisted, you cannot run carbon on this server.
[WARNING]
Unable to import the 'txamqp' module, this is required if you want to use AMQP.
Note that txamqp requires python 2.5 or greater.
3 necessary dependencies not met. Graphite will not function until these dependencies are fulfilled.
6 optional dependencies not met. Please consider the warning messages before proceeding.

Now, to get this stuff installed:

# apt-get install python-cairo
# apt-get install python-django-tagging
# apt-get install python-twisted
# apt-get install python-memcache
# apt-get install libapache2-mod-wsgi

I don’t know enough about mod_python (which we’re supposed to install), but the sample vhost configuration below refers to mod_wsgi, so I installed that instead.

Now we configure Apache and modify the sample vhost configuration file provided:

# rm /etc/apache2/sites-enabled/000-default
# cp -a examples/example-graphite-vhost.conf /etc/apache2/sites-enabled/graphite
# cp -a conf/graphite.wsgi.example /opt/graphite/conf/graphite.wsgi
# sed -i s%"@DJANGO_ROOT@/contrib/admin/media/"%"/usr/share/pyshared/django/contrib/admin/media/"% /etc/apache2/sites-enabled/graphite
# sed -i 's%WSGISocketPrefix /etc/httpd/wsgi/%WSGISocketPrefix /var/run/apache2/wsgi%' /etc/apache2/sites-enabled/graphite

Once done, go ahead and restart Apache:

# /etc/init.d/apache2 reload

Finally:

# cd /opt/graphite/webapp/graphite
# python manage.py syncdb
# chown -R www-data:www-data /opt/graphite/storage/
# cd /opt/graphite/
# ./bin/carbon-cache.py start

If that’s all worked, you should be able to pump data into Graphite:

# echo "carbon.installation.test $RANDOM `date +%s`" | nc -w 1 localhost 2003

On the above, if I don’t pass -w 1 to nc, nc just sits there doing nothing.

Now, browsing your server’s http://x.x.x.x/ should load up the Graphite app and hopefully you can see a graph for the data you’ve sent in.

The minimum and maximum speeds can be set via:

/proc/sys/dev/raid/speed_limit_min

and:

/proc/sys/dev/raid/speed_limit_max

These are system-wide values and affect all md block devices. If you wish to alter the speeds for a specific block device (overriding the system-wide values), this can be done through:

/sys/block/md0/md/sync_speed_min

and:

/sys/block/md0/md/sync_speed_max

(replacing md0 with the name of the block device in question)

From the md man page:

While this recovery process is happening, the md driver will monitor accesses to the array and will slow down the rate of recovery if other activity is happening, so that normal access to the array will not be unduly affected. When no other activity is happening, the recovery process proceeds at full speed.

My screen fu is a bit weak; I’ll be using this post to document some useful commands for future reference.

Firstly, the following key combo brings up a listing of windows, allowing you to scroll up or down and select a window to switch to:

C-a "

This is great if you have a number of windows open and want to quickly switch to another window when you don’t know the number of that window. Now, to benefit from this window menu, you need to set window titles appropriately. This can be done by hitting:

C-a A

… and then entering a title for the window in question.

Every now and again, I close windows & re-open and this disrupts my screen window flow when using the previous / next window key combinations. In this situation, you can re-order your windows by going to a window and then:

C-a :

This enters command line mode, and here you can type the following to change the current window’s number:

number 0

In this example, the current window’s number will be changed to 0.

I recently found myself needing to update a new Debian install using apt-get through a proxy server. You can run either of the following to force apt-get to use an HTTP or FTP proxy:

# export http_proxy=http://username:password@proxyserver.net:port/

# export ftp_proxy=http://username:password@proxyserver.net:port/

However, I didn’t want to start messing about with proxies but use ssh’s SOCKS5 feature instead. A quick Google search turned up the following link, which worked beautifully. After pulling the tsocks .deb package via ssh from another box and installing, it was simply a matter of dumping the following into /root/.tsocks.conf:

server = 127.0.0.1
server_type = 5
server_port = 8888

At that point, I launched the SOCKS5 proxy using:

# ssh -D 8888 user@remotehost

Finally, I was able to:

# tsocks apt-get update
# tsocks apt-get upgrade

It is possible to use tsocks a bit more transparently, but as I only had to send those two commands through the proxy I didn’t have any need. Look at the tsocks man page for more information.

Lastly, make sure you unset $http_proxy or $ftp_proxy if you’ve set either. I forgot that I tried setting $http_proxy and spent far too long trying to debug why tsocks wasn’t working!