zomo technology

Solaris fssnap and realtime processes

Wednesday, 12. 21. 2005  –  Category: all, vague

fsnap(1M) is most woo-esque for backing up filesystems, in a

• snapshot filesystem
• dump filesystem to tape
• release snapshot
• check dump

kind of manner.

It does, however, have problems with realtime priority processes. If the executable of such a process resides on the filesystem being snapshotted, things don’t work:
snapshot error: File system could not be write locked  

Quick solution: identify these processes and halt them across the snapshot:

#   priocntl -d -i class RT
REAL TIME PROCESSES:
    PID    RTPRI       TQNTM
    577       0           0       

Perhaps a better attempt would be to temporarily jigger the priority away from realtime.

Highly available filesytems

Thursday, 11. 24. 2005  –  Category: all, sw

A hosting project I’m working on needs a large filesystem to be available on multiple hosts and to be continually mirrored.

• A pair of NFS servers rsyncing master to slave – feeble
• Coda – looks good but project appears dead (though has current kernel support on both FreeBSD and Linux), 32bit limitation on metadata restricts fs size.
• Lustre – shiny, very current, Linux-only

I will play with Lustre on a pair of lab boxes

Really, I want to do something with ZFS – it has features galore.

Apache stuckage

Thursday, 11. 24. 2005  –  Category: all, sw

• mod_status shows loads of weird lines, eg:
  

  29-1  -  0/0/1  .   0.00  10322  28  0.0  0.00  0.00   x.y.z.a  (unavailable)  GET /reports/latest.html HTTP/1.0
  30-1  -  0/0/1  .   0.00  10312  34  0.0  0.00  0.00   x.y.z.a  (unavailable)  GET /cgi-bin/certificate.cgi/732.gif HTTP/1.1 

• eventually the number of these reaches MaxClients, wedging the server
• hung process backtrace like this:
  

  (gdb) bt
  #0  0x28129b0f in write () from /lib/libc.so.5
  #1  0x281babb7 in config_log_transaction () from /usr/local/libexec/apache/mod_log_config.so
  #2  0x281bac24 in multi_log_transaction () from /usr/local/libexec/apache/mod_log_config.so
  #3  0x08051e9e in run_method ()
  #4  0x08051f3e in ap_log_transaction ()
  #5  0x080628e6 in ap_process_request ()
  #6  0x0805b6ca in child_main ()
  #7  0x0805b8e5 in make_child ()
  #8  0x0805bb9b in perform_idle_server_maintenance ()
  #9  0x0805c09c in standalone_main ()
  #10 0x0805c613 in main ()    

Which leads me to this nugget one of the <VirtualHost> blocks:

CustomLog | /dev/null  

Classy! It’s stranger still since it’s next to a CustomLog directive that does indeed log to disk