HOWTO: Install iClassify on Ubuntu and Mac OS X Leopard

Update:

The folks who developed iClassify have come out with a total framework that is an alternative to iClassify/Puppet called opscode-chef. I will be looking into that soon and not do anymore work with iClassify (unless Chef turns out to suck or something, but at first glance it looks pretty good!).

[After playing with Chef, my conclusion is forget about iClassify / Puppet and just use Chef unless you are already using Puppet]

iClassify Description

From the creator’s of iClassify HJK Solutions website:

iClassify allows for the easy registration and classification of nodes. Most of the time, a node is a server. With iClassify:

• Nodes register themselves with a central web service, including reporting Facter facts.
• You can then tag those nodes, and add manual attributes.
• You can search the nodes with a full text search engine
• You can write recipies for icagent to auto-classify and auto-attribute your nodes.
• You can tie it in to Puppet as an external node classification tool, enabling you to easily configure hundreds of nodes at a time.
• You can tie it in to Capistrano, and have a dynamic ad-hoc configuration tool.

We are considering using it along with Puppet and Amazon EC2 for deployment of some of our infrastructure.

Install iClassify

Original Instructions are at HJK Solutions. Some of the following quotes liberally from that site but adds the things I learnt along the way as well as how to do it on a Mac.

Prerequisites

Ruby Gems

• Rails 2.0.2
• Rake
• Builder
• UUID Tools
• Mongrel
• Highline
• Net-LDAP

sudo gem rails rake install builder uuidtools mongrel ruby-net-ldap

Non-Gems

• MySQL
• Factor
• Git
• Java
• Runit
• Mongrel Runit

Configure MySQL

The folks at HJK say they use MySQL and it should work with PostgreSQL and sqlite3 as well. We went wtih MySQL.
First create the database iclassify_production

mysqladmin -u root -p  create iclassify_production

Grant it the correct privleges (set yourpass to the password you want to use):

mysql -u root -p iclassify_production
mysql> GRANT ALL ON iclassify_production.* TO 'iclassify'@'localhost' IDENTIFIED BY 'yourpass';

You should have the ruby mysql gem installed

sudo gem install mysql

on the Macintosh OS X Leopard I had to say:

sudo env ARCHFLAGS="-arch i386" gem install mysql -- --with-mysql-config=/usr/local/mysql/bin/mysql_config

Git

You can install from ubuntu packages

sudo apt-get install git-core

On the Mac Download and install the git OS X package for the Mac from Git OSX Installer on Google Code or

port -uR install git-core

Java

Java is needed for the Solr package that is bundled in the iClassify distro. It can be installed via apt and/or downloaded from the Sun site. Java is already installed on Mac OS X.

Runit

Runit is an alternative / addition to the standard /etc/init.d “systemV” init system. I guess the HJK folks like it and seem to have dependencies on it. They say it should work without it, but I haven’t tried putting this together without the Runit/Mongrel_runit dependencies yet.

Some good info and tips on setting up / running runit on various systems can be found at runit – a UNIX init scheme with service supervision

Runit can be safely installed with apt. It will not replace the standard init system.:

sudo apt-get install runit

For the Mac:

port install runit

On the Mac, you’ll have to start the runit system with the command

sudo launchctl load -w /Library/LaunchDaemons/org.macports.runit.plist

iClassify itself

In a directory you want to keep the source of iclassify:

git clone git://git.hjksolutions.com/iclassify iclassify
cd iclassify

You need to know where you want to install the actual working rails app of iclassify and what user id/group you want to run it under.

The default location and the one we’ll use on Ubuntu is /srv/iclassify and the user id/group is usually the same as the one that runs the apache web services (www-data). Change yourpass to the password used for the iclassify user in MySQL.

sudo rake iclassify:install ICBASE=/srv/iclassify ICUSER=www-data ICGROUP=www-data DBUSER=iclassify DBPASS=yourpass

For the Macintosh:

sudo rake iclassify:install ICBASE=/usr/local/iclassify ICUSER=_www ICGROUP=_www DBUSER=iclassify DBPASS=yourpass

This will create a new iClassify instance in /srv/iclassify, set the right ownership to run iClassify, and set run the migrations to prepare your database instance. I found that I had to run this as root so that it will create the directories properly.

You can test that the iClassify rails app was installed properly by testing it with the built in Rails Server (on the Mac use the /usr/local/iclassify directory and _www user id):

$ cd /srv/iclassify
$ sudo -u www-data env RAILS_ENV=production ./script/server
=> Booting Mongrel (use 'script/server webrick' to force WEBrick)
=> Rails application starting on http://0.0.0.0:3000
=> Call with -d to detach
=> Ctrl-C to shutdown server
** Starting Mongrel listening at 0.0.0.0:3000
** Starting Rails with production environment...
** Rails loaded.
** Loading any Rails specific GemPlugins
** Signals ready.  TERM => stop.  USR2 => restart.  INT => stop (no restart).
** Rails signals registered.  HUP => reload (without restart).  It might not work well.
** Mongrel available at 0.0.0.0:3000
** Use CTRL-C to stop.

You can now point your browser to the local instance of iClassify at http://localhost:3000, hit CTRL-C when you are done to terminate the script/server.

Mongrel Runit

You can download the mongrel_runit gem from the Mongrel Runit page at HJK

Then install it with (on the Mac you’ll have to use the /usr/local/iclassify directory instead of /src/iclassify):

For some reason the HJK folks set their runit_service_dir to be /var/service but the runit ubuntu package puts it in /etc/service. So you might want to edit /srv/iclassify/examples/mongrel_runit_iclassify.yml and set runit_service_dir to /etc/service. You can also change the number of mongrels you want to run in that file. Similarly, the DarwinPorts port install of runit expects it to be in /opt/local/var/service. You can change the /srv/iclassify/examples/mongrel_runit_iclassify.yml or you can make a symbolic link from /opt/local/var/service to /var/service

sudo gem install mongrel_runit-0.2.1.gem
sudo mkdir /etc/mongrel_runit
sudo cp /srv/iclassify/examples/mongrel_runit_iclassify.yml /etc/mongrel_runit/iclassify.yml
sudo mongrel_runit -c /etc/mongrel_runit/iclassify.yml create

You should be able to then run the command

mongrel_runit -v status -c /etc/mongrel_runit/iclassify.yml

and see something like (there should be as many lines as you have set for mongrel servers. I changed the iclassify.yml from 5 to 3):

5000: true: run: /etc/sv/mongrel-iclassify-5000: (pid 4403) 119s; run: log: (pid 4402) 119s
5001: true: run: /etc/sv/mongrel-iclassify-5001: (pid 4401) 119s; run: log: (pid 4400) 119s
5002: true: run: /etc/sv/mongrel-iclassify-5002: (pid 4399) 119s; run: log: (pid 4398) 119s

Solr

First create some directories that will be needed for Solr to run its index as the www user (on the Mac replace /srv with /usr/local).

sudo mkdir -p /srv/iclassify/vendor/plugins/acts_as_solr/solr/solr/data/production
sudo chown -R www-data:www-data /srv/iclassify/vendor/plugins/acts_as_solr/solr/solr/data/

You can then test it with the command:

sudo -u www-data env RAILS_ENV=production rake solr:start

It should start and run with no errors.

You can stop it if you want with:

sudo -u www-data env RAILS_ENV=production rake solr:stop

Then set up runit to run it automatically (use /usr/local instead of /srv on the Mac and /var/service or whatevever your system uses for runit service dir if not on ubuntu and using /etc/service):

sudo mkdir -p /etc/sv/iclassify-solr/log/main
sudo cp /srv/iclassify/examples/solr-run /etc/sv/iclassify-solr/run
sudo cp /srv/iclassify/examples/solr-log /etc/sv/iclassify-solr/log/run
sudo chmod a+x /etc/sv/iclassify-solr/run /etc/sv/iclassify-solr/log/run

The following will start the solr process immediately as well as in the future reboots

sudo ln -s /etc/sv/iclassify-solr /etc/service

Apache

iClassify is best configured as a virtual host under Apache, running with SSL and mod_proxy_balancer. Follow the proper steps for configuring your platforms Apache to use mod_ssl, mod_proxy_balance r and mod_rewrite. Create a virtual host config which resembles the following (works with Mac and Ubuntu, On the Mac just change the refs to /srv to /usr/local and the EXAMPLE.com to your domain in any case)

<VirtualHost *:443>
  DocumentRoot /srv/iclassify/public
  LimitRequestBody 8388608
  ServerName iclassify.EXAMPLE.COM
  ServerAlias iclassify
  <Directory />
    Options FollowSymLinks
    AllowOverride None
  </Directory>

  <Location /server-status>
    SetHandler server-status
    Order Deny,Allow
    Deny from all
    Allow from 127.0.0.1 192.168.0.0/255.255.0.0
  </Location>

  <Proxy balancer://iclassify>
    BalancerMember http://localhost:5000
    BalancerMember http://localhost:5001
    BalancerMember http://localhost:5002
    BalancerMember http://localhost:5003
  </Proxy>

  LogLevel info
  ErrorLog /var/log/apache2/iclassify-error.log
  CustomLog /var/log/apache2/iclassify-access.log combined

  RewriteEngine On
  RewriteLog /var/log/apache2/iclassify-rewrite.log
  RewriteLogLevel 0
  RewriteCond %{DOCUMENT_ROOT}/%{REQUEST_FILENAME} !-f
  RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f
  RewriteCond %{SCRIPT_FILENAME} !maintenance.html
  RewriteRule ^.*$ /system/maintenance.html [L]
  RewriteRule ^/server-status$ /server-status$1 [L]

  RewriteCond %{DOCUMENT_ROOT}/%{REQUEST_FILENAME} !-f
  RequestHeader set X_FORWARDED_PROTO 'https'
  RewriteRule ^/(.*)$ balancer://iclassify%{REQUEST_URI} [P,QSA,L]
  ProxyPassReverse / balancer://iclassify
  SetEnv proxy-nokeepalive 1

  SSLEngine on
  SSLCertificateFile /etc/apache2/ssl/iclassify.crt
  SSLCertificateKeyFile /etc/apache2/ssl/iclassify.key

  BrowserMatch ".*MSIE.*" \
    nokeepalive ssl-unclean-shutdown \
    downgrade-1.0 force-response-1.0
</VirtualHost>

<VirtualHost *:80>
  DocumentRoot /srv/iclassify/public
  LimitRequestBody 8388608
  ServerName iclassify.EXAMPLE.COM
  ServerAlias iclassify

  RewriteEngine On
  RewriteCond %{HTTPS} !=on
  RewriteRule ^/(.*) https://%{SERVER_NAME}/ [R,L]
</VirtualHost>

Basic instructions for setting up ssl certificates can be found at Just Samuels blog post HOWTO: Create a self-signed (wildcard) SSL certificate

Take the resulting hosts.cert and copy it to /etc/apache2/ssl/iclassify.crt and hosts.key to /etc/apache2/ssl/iclassify.key (or whereever you put your ssl keys and make sure the SSLCertificateKeyFile and SSLCertificateFile are set the same in your vhosts conf file.

Conclusion

That should get you up and running with iClassify. In a future post I will install Puppet and then figure out how to use these together to deploy to Amazon EC2.