Writing a new SELinux policy module for a standard init daemon

This is going to be a summary of my experience writing new policy modules for Ganglia gmetad and gmond on RHEL5. Ganglia is a "scalable distributed monitoring system for high-performance computing systems." I downloaded the package source distribution, and built RPMs myself.

In case you are looking to apply this to something else, here are a couple of the underlying assumptions:

• the service is a standard init-launched daemon
• each service only has one executable, the daemon program

In the case of gmetad and gmond, the daemon programs are, respectively, /usr/sbin/gmetad and /usr/sbin/gmond.

I have written about creating new SELinux policies before, but I think this is better in that it wraps things up into a module that may be removed or updated more easily than a monolithic policy. Note, however, that rules governing network ports are not bundled into the module. (See below.)

This is going to be an iterative process. Before even starting, one needs to know which files/directories the daemons will write to, and if they run non-root. If the package one is working with is well-documented, this may be obtained from the documentation. If not, some trial and error will be needed. Also, for most programs, these file/directory locations are configurable.

We use the GUI Selinux Policy Generation tool, system-config-selinux. There is a good article on using this tool by Dan Walsh dating back to 2007.

We will start with gmetad. In the case of gmetad, the default location for the RRD files is /var/lib/ganglia/rrds. So, the policy should allow write access to /var/lib/ganglia.

In the Selinux Policy Generation tool, these are the entries used:

• Name: gmetad
• Executable: /usr/sbin/gmetad
• Standard Init Daemon
• Incoming network ports, both TCP and UDP: 8651,8652
• Common Application Traits
• Application uses syslog to log messages
• Application uses /tmp to Create/Manipulate temporary files
• Application uses nsswitch or translates UID's (daemons that run as non root)
• Add Directory: /var/lib/ganglia

This generates 4 files in whatever directory you specify at the end of the druid: gmetad.fc, gmetad,if, gmetad.sh, gmetad.te. If you examine gmetad.sh, you will see:

#!/bin/sh
make -f /usr/share/selinux/devel/Makefile
/usr/sbin/semodule -i gmetad.pp

/sbin/restorecon -F -R -v /usr/sbin/gmetad
/sbin/restorecon -F -R -v /var/lib/ganglia
/usr/sbin/semanage port -a -t gmetad_port_t -p tcp 8651
/usr/sbin/semanage port -a -t gmetad_port_t -p tcp 8652
/usr/sbin/semanage port -a -t gmetad_port_t -p udp 8651
/usr/sbin/semanage port -a -t gmetad_port_t -p udp 8652

Note that the ports are not bundled into the "compiled" module file, gmetad.pp. The port rules are added "manually". The module merely defines the type gmetad_port_t.

The gmetad.te file is what we will be editing in the iterative steps below.  The first line determines a version number, that allows you to update a policy using "semodule -u gmetad.te".

policy_module(gmetad,1.0.0)

Make sure the gmetad service is not running. Now, turn off the auditd service, and move away the audit log file to simplify finding incremental changes in policy that are needed:

# service gmetad stop
# service auditd stop
# cd /var/log/audit< # mv audit.log audit.log.20130313-1500

Then, start up the audit daemon, followed by gmetad. Wait for a few minutes (or much longer) for gmetad to do its thing, and for auditd to accumulate all or most of the AVC denials that would affect gmetad. Once a sufficient amount of time has passed:

# grep gmetad /var/log/audit/audit.log | audit2allow -R > audit.out

The output should look like:

require {
        type gmetad_t;
        class capability { setuid setgid };
}

#============= gmetad_t ==============
allow gmetad_t self:capability { setuid setgid };
kernel_read_kernel_sysctls(gmetad_t)

Next, edit gmetad.te, and increment the version number. Append to the end of gmetad.te the contents of audit.out. Then, generate the policy file, and load the updated policy:

# make -f /usr/share/selinux/devel/Makefile
# semodule -u gmetad.pp

Next, shut down gmetad, shut down auditd, move the audit log away, start auditd, and start gmetad. Wait a bit, and look for new denials in the audit log by doing

# grep gmetad /var/log/audit/audit.log | audit2allow -R > audit2.out

To append any new rules, you have to manually pick out the new unique lines from audit2.out and put them in the appropriate sections (the 'require' section, or the block of allows) of gmetad.te. For gmetad.te, I found there wasn't much change between iterations. For gmond, however, there were quite a few, mostly the addition of file getattr permissions. This involved changing many lines like:

allow gmond_t lvm_t:file read;  -->  allow gmond_t lvm_t:file { getattr read };

This iteration may have to include alternating gmond and gmetad since gmetad has to connect to the gmond port, which means something like:

allow gmetad_t gmond_port_t:tcp_socket name_connect;

Here at the Wake Forest University HPC facility, we have a combination of cfengine and Puppet to manage machine configurations: cfengine for the RHEL5 nodes, and Puppet for the RHEL6 nodes. The policy .pp file is distributed via cfengine, and a shellcommand is run by cfengine to load/update the module, and additional commands do the file system relabelling and the port rules. Basically, reproducing the .sh file that the Policy Generation Tool creates.

UPDATE 2013-03-22: If you have a cyclic dependency in your policy modules -- in this case, gmond refers to gmetad, and gmetad refers to gmond -- you will find that you can't load the modules individually. All you have to do is load them all in one command line:

semodule -i gmond.pp gmetad.pp

More Puppet and SELinux

Remember my previous post about Puppet and SELinux? Well, it turns out it wasn't complete. The policy file was missing a couple of policies. This happened because I didn't completely start from scratch at each iteration of testing, and at some point, I turned SELinux to permissive, so client certificates were being signed with no problem.

In moving to our production server, there were error messages on the client side:

err: Could not request certificate: Error 400 on SERVER: Permission denied - /var/lib/puppet/ssl/ca/serial
Exiting; failed to retrieve certificate and waitforcert is disabled

On the production puppet master, AVC denials looking like:

type=1400 audit(1328213559.254:21031): avc:  denied  { remove_name } for  pid=5901 comm="ruby" name="serial.tmp" dev=dm-2 ino=131791 scontext=unconfined_u:system_r:httpd_t:s0 tcontext=system_u:object_r:puppet_var_lib_t:s0 tclass=dir

with corresponding items in /var/log/messages (why not in /var/log/audit/audit.log? I have no idea):

puppet-master[13193]: Could not rename /var/lib/puppet/ssl/ca/serial to /var/lib/puppet/ssl/ca/serial.tmp: Permission denied - /var/lib/puppet/ssl/ca/serial.tmp or /var/lib/puppet/ssl/ca/serial

(Still unsolved mystery: on the production server, ausearch did not show any AVC denials; the denials were logged to /var/log/messages. I did not try "semodule -DB" to disable all dontaudits.)

On the test system, there were also denials like:

type=AVC msg=audit(1328221549.372:27539363): avc:  denied  { unlink } for  pid=29452 comm="ruby" name="serial.tmp" dev=dm-2 ino=134565 scontext=unconfined_u:system_r:httpd_t:s0 tcontext=unconfined_u:object_r:puppet_var_lib_t:s0 tclass=file

What happens is when a certificate signing request (CSR) comes in to the puppet master from a client, a file /var/lib/puppet/ca/serial.tmp is created. At the end of the signing process, that file is moved to serial. I think it just does a cp and rm. (My suspicion is based on the unlink policy that it needs.)

In any case, here is an updated policy file. Note the version number compared to the previous one.

module puppet_passenger 1.15;

require {
        type httpd_t;
        type httpd_passenger_helper_t;
        type port_t;
        type puppet_var_lib_t;
        type puppet_var_run_t;
        type puppet_log_t;
        type proc_net_t;
        type init_t;
        type user_devpts_t;
        class dir { write getattr read create search add_name remove_name rename unlink rmdir };
        class file { write append relabelfrom getattr setattr read relabelto create open rename unlink };
        class udp_socket name_bind;
}

#============= httpd_passenger_helper_t ==============
allow httpd_passenger_helper_t httpd_t:dir { getattr search };
allow httpd_passenger_helper_t httpd_t:file { read open };

#============= httpd_t ==============
#!!!! This avc can be allowed using the boolean 'allow_ypbind'

allow httpd_t port_t:udp_socket name_bind;

allow httpd_t proc_net_t:file { read getattr open };

allow httpd_t puppet_var_lib_t:dir { write read create add_name remove_name rename unlink rmdir };
allow httpd_t puppet_var_lib_t:file { relabelfrom relabelto create write append rename unlink };

allow httpd_t puppet_var_run_t:dir { getattr search };

allow httpd_t puppet_log_t:file { getattr setattr };

allow httpd_passenger_helper_t init_t:file { read };
allow httpd_passenger_helper_t init_t:dir { getattr search };

Puppet, Apache, mod_passenger, and SELinux

At work, we are currently working on deploying Puppet with Apache on RedHat Enterprise Linux 6 to replace our cfengine on RHEL4/5 setup.

We install Puppet direct from Puppetlabs, and mod_passenger from Stealthy Monkeys.

There are quite a few issues with directory permissions and SELinux. The directory permission issues are fairly easy to diagnose because the httpd log files, and the error messages that httpd sends back generally tell you what permissions it expected.

SELinux is a different kettle of fish. After doing ausearch and using audit2allow, plus a little bit of pruning, this seems to be a minimal set of permissions that allow puppet to run under Passenger and Apache (the following is a .te file):

module puppet_passenger 1.5;

require {
        type httpd_t;
        type httpd_passenger_helper_t;
        type port_t;
        type puppet_var_lib_t;
        type proc_net_t;
        class dir { write getattr read create search add_name };
        class file { write append relabelfrom getattr read relabelto create open };
        class udp_socket name_bind;
}

#============= httpd_passenger_helper_t ==============
allow httpd_passenger_helper_t httpd_t:dir { getattr search };
allow httpd_passenger_helper_t httpd_t:file { read open };

#============= httpd_t ==============
#!!!! This avc can be allowed using the boolean 'allow_ypbind'

allow httpd_t port_t:udp_socket name_bind;

allow httpd_t proc_net_t:file { read getattr open };

allow httpd_t puppet_var_lib_t:dir { write read create add_name };
allow httpd_t puppet_var_lib_t:file { relabelfrom relabelto create write append };

To install these changes:

# mkdir -p /usr/share/selinux/packages/puppet_passenger/

# cp puppet_passenger.te /usr/share/selinux/packages/puppet_passenger

# cd /usr/share/selinux/packages/puppet_passenger

# checkmodule -M -m -o puppet_passenger.mod puppet_passenger.te

# semodule_package -o puppet_passenger.pp -m puppet_passenger.mod

# semodule -i puppet_passenger.pp

And if you ever want to remove the permissions, just do:

# semodule -r puppet_passenger.pp