Reseting the Opscode Chef Server Validation key/pem

In upgrading from my own custom hacked pre-0.8.x Chef server/clients to the official new and shiny 0.8.2 release, I wanted to make everything vanilla. One issue was somewhere along the line I set the validation_client_name to “validator”. The vanilla setting is “chef-validator”.

To do this I had to get rid of the “validator” and “chef-validator” authentication client entries I had on the chef-server. It turns out you can’t just delete them with knife or the web-ui. You have to edit the couchdb to delete the entries.

The fantastic Chef IRC channel came to the rescue in the usual personage of Josh Timberman (jtimberman) who paused from I’m sure one of his most hectic days of his life (they were still cleaning up all the loose ends of todays release of 0.8.2) to help me. The steps are:

• Create an ssh tunnel from your local machine to the chef-server:

ssh -L 5984:localhost:5984 fqdn-of-chef-server

• Then with a browser on your local machine access:

http://localhost:5984/_utils

That will connect you to futon, a web interface to couchdb running on the chef-server.

• Click on “chef”

• Select the View to be Client->All

• Select the “chef-validator”

Then delete the “chef-validator”

Once you have removed the client authentication from the couchdb, you need to remove the validation.{pem,key,crt} from /etc/chef on the chef server (there may just be validation.pem). Then restart the chef server (/etc/init.d/chef-server restart).

You should now have a fresh clean valid validation.pem in /etc/chef on the chef-server. You can then copy that to the /etc/chef on your chef client[s]. Remeber to also remove the client.pem in /etc/chef on the client. If client.pem is there, the chef-client will not try to re-validate with the new valdation.pem.

Once I had the proper validation.pem I  used the Chef upgrade bootstrapping process to update all my server and clients and use the new validation.pem to create new client authentication on the server. Everything was clean and fresh after that.

Note that this is an unusual situation that requires clearing out the old validator client.  You should not have this problem in any normal situation. If you do have this problem, make backup copies of any validation.pem or client.pem until you make sure everything is cool.