Docker, SELinux, Consul, Registrator

Wednesday, 04. 29. 2015  –  Category: sw

Dear Google,

On RHEL / CentOS SELinux can get in the way of setting up a progrium/consul / gliderlabs/registrator network:

  • consul needs somewhere to store persistent state on the host, and SELinux won’t allow the container to write to arbitrary locations
  • registrator needs access to /var/run/docker.sock to monitor container events

The first is easy: add the required svirt_sandbox_file_t on the host to whatever mount you’re passing to consul for /data


# chcon -Rt svirt_sandbox_file_t /var/lib/consul

The second needs a custom SELinux policy, and the policycoreutils-python RPM to compile it:


# cat > docker-socket.te <<EOT
module docker-socket 1.0;
 
require {
        type docker_var_run_t;
        type docker_t;
        type svirt_lxc_net_t;
        class sock_file write;
        class unix_stream_socket connectto;
}
 
#============= svirt_lxc_net_t ==============
allow svirt_lxc_net_t docker_t:unix_stream_socket connectto;
allow svirt_lxc_net_t docker_var_run_t:sock_file write;
EOT

# checkmodule -M -m -o docker-socket.mod docker-socket.te
# semodule_package -m docker-socket.mod -o docker-socket.pp
# semodule -i docker-socket.pp

ZFS performance on FreeBSD

Tuesday, 09. 16. 2014  –  Category: sw

For a combined application and database server with considerable per-database MySQL buffers and lots of synchronous IO on SATA drives:

  • vfs.zfs.prefetch_disable="1" – disable prefetch, even on systems with decent amounts of RAM. With prefetch enabled the server freezes under moderate database activity. The associated arc_summary stats are mixed, with almost 100% miss for some reads.
    
File-Level Prefetch: (HEALTHY)

DMU Efficiency:                                 4.36b
        Hit Ratio:                      82.85%  3.61b
        Miss Ratio:                     17.15%  748.47m

        Colinear:                               748.47m
          Hit Ratio:                    0.02%   170.47k
          Miss Ratio:                   99.98%  748.30m

        Stride:                                 3.48b
          Hit Ratio:                    100.00% 3.48b
          Miss Ratio:                   0.00%   68.37k

    An old mailing list posts suggest this is a “more or less known problem”.
  • vfs.zfs.arc_max="16G" – explicitly set a maximum amount of memory for ARC after working out what headroom is needed elsewhere. Otherwise ARC will cause pageouts for userland processes and it isn’t as good as getting out of the way as it is supposed to be. Sometimes this is visible as THROTTLED in arc_summary, with a non-zero throttle count.
    
ARC Summary: (THROTTLED)
        Storage pool Version:                   5000
        Filesystem Version:                     5
        Memory Throttle Count:                  8

Controlling Exim SMTP behaviour from Dovecot password data

Wednesday, 09. 3. 2014  –  Category: stash

Given this Dovecot PasswdFile with a homegrown smtp ExtraField:


$ head -1 /data/example.org/etc/passwd
foo@example.org:{MD5-CRYPT}$1$HASH-U-LIKE::::::updated=1409712878 smtp=no

Then this Exim ACL snippet forbids the user from sending mail. Dovecot will allow them to login (allowing them receive the mail asking them to change their password) and so will ancillary systems that authenticate with the same data (eg: the password changing facility):

  deny    authenticated = *
          message       = User must change password before sending any new mail. See https://example.org/notices
          set acl_c_auth_sender_address = $authenticated_id
          set acl_c_auth_sender_domain  = ${extract{-1}{@}{$acl_c_auth_sender_address}}
          set acl_c_user_passwd_entry   = ${lookup{${acl_c_auth_sender_address}}lsearch{/data/${acl_c_auth_sender_domain}/etc/passwd}}
          set acl_c_user_passwd_fields  = ${extract{-1}{:}{$acl_c_user_passwd_entry}}
          set acl_c_user_smtp_field     = ${extract{smtp}{$acl_c_user_passwd_fields}}
          condition     = ${if eq{$acl_c_user_smtp_field}{no}}

  • The use of ACL variables is a bit gratuitous but allows the massive expansion to be built up piece by piece.
  • We check the authenticated_id not the sender_from since we permit senders to set their own From address, but they still need to authenticate with their real account.

« Older Entries