Moves ceph.conf settings about running a ceph-mon

[hufman]

Only ceph-mon ceph.conf files need fsid and initial nodes.
Nodes that aren't ceph-mon don't need them.
This is particularly important for nodes that aren't in the same
environment as the ceph cluster.

[hufman]

http://tracker.ceph.com/issues/3253 says that they are also needed for OSD volumes, to handle drive hotplugs. Tweaking now!

[hufman]

How does it look?

[guilhem]

@hufman can you test f48fa95 ?
I may fix it while debugging aio

[hufman]

I ran test-kitchen on that commit. The .kitchen.yml file wanted to test 10.04 and 12.04, I needed to change it to test 12.04 and 14.04, because Ceph doesn't host deb packages for the 10.04 version.
The Centos-6 test failed because it wanted to fetch http://ceph.com/rpm-dumpling/el6/noarch/ceph-release-1-0.el6.noarch.rpm/repodata/repomd.xml. There is a file at .../ceph-release-1-0.el6.noarch.rpm, and there is also a file at .../repodata/repomd.xml. I suspect either the repository attribute has to be changed to just be the directory, or the yum_repository invocations have to point to the directory itself. To continue the test, I used File.dirname inside the recipes/rpm.rb file.
Centos 5.9 had to be bumped to Centos 5.10.
After those change, running "kitchen test" completed successfully. It just seemed to install the software but not actually configure or run it. Running kitchen test 'aio-ubuntu-1404' seemed to run the full installation process and it seemed to work, I'm going to let the full test suite run tonight.

[guilhem]

@hufman thank you for this tests, do a PR when your work is done :)

[hufman]

I've rebased this PR onto the new master. I see that the cookbook automatically generates an FSID if there isn't one, in the conf.rb recipe. However, I think this will be confusing for ceph clients that don't need to have the FSID set, because they will generate their own FSID independent from the rest of the cluster. This won't cause problems, it will just look confusing in the ceph.conf file.
Should a similar approach to the monsecret (#121) be used? The first mon node could generate and save an FSID, and other mon and osd nodes will automatically use it if they don't have it but need it.