ESXi: PXE install

  • Copy installer iso contents to tftp server, like $TFTPROOT/esxi/installer/5.1u1/.
  • Modify boot.cfg:
$ sed -i 's@/@@g' boot.cfg

label esxi51u1
  kernel esxi/installer/5.1u1/mboot.c32
  append -c esxi/installer/5.1u1/boot.cfg

ESXi: boot from PXE server

  • Install ESXi server on a disk as usually
  • Copy files from /bootbank to the tftp server
  • Copy mboot.c32 file as well (from the isntalled partition mounted (FAT32) externally to a system or from the installer iso)
  • modify boot.cfg
$ sed -i 's@/@@g' boot.cfg
  • edit pxcelinux configuration

label esx01
   kernel esx01/mboot.c32
   append -c esx01/boot.cfg

Continue reading

ESXi: Install (Areca) storage driver from command line

# enter maintenance mode
~ #  vim-cmd hostsvc/maintenance_mode_enter

# install storage driver
# !!! ABSOLUT PATH must be used !!!!
~ # esxcli software vib install -v /VMware_bootbank_scsi-arcmsr_1.20.00.15.vmk5.120930-1vmw.500.0.0.472560.vib
Installation Result
   Message: The update completed successfully, but the system needs to be rebooted for the changes to be effective.
   Reboot Required: true
   VIBs Installed: VMware_bootbank_scsi-arcmsr_1.20.00.15.vmk5.120930-1vmw.500.0.0.472560
   VIBs Removed:
   VIBs Skipped:

# system restart: takes some time
~ # reboot
~ #

# exit maintenance mode
~ # vim-cmd hostsvc/maintenance_mode_exit

jenkins eats up cpu and memory resources

In the last two years we were suffering from jenkins CPU and memory utilization. The following switches made miracle:

JAVA_ARGS="-Djava.awt.headless=true -Xms1g -Xmx6g -XX:MaxPermSize=1G -XX:+UseParNewGC -Dorg.kohsuke.stapler.compression.CompressionFilter.disabled=true"

We see ~60% CPU usage at peaks(!) on a Xeon E5504 2.00GHz, instead of constant 400%.
Currently there are 80 executors.

Convert squid.log timestamp to readable format

Squid log looks like this:

Tue May 21 07:32:33 2013     25 TCP_MISS/200 68240 GET - DIRECT/ image/jpeg

Use this one-liner to convert unix timestamp to readable dates:

#!/usr/bin/perl -p
s/^\d+\.\d+/localtime $&/e;

HP iLO lost password recovery

There are number of howtos about changing a lost password in iLO. They’re all talking about properly formatted xml file and hponcfg.

But what if you get this message?

$ hponcfg 
HP Lights-Out Online Configuration utility
Version 4.0.0 Date 12/08/2011 (c) Hewlett-Packard Company, 2011
ERROR:  Could not find the Management Processor Device Node.
  Please check if the iLO driver module is installed and running properly.
  ACTION REQUIRED: Install/Re-install hp Lights-Out driver package('hp-ilo' RPM) for iLO. 

It’s worth verifying, that it’s really iLO not Lights-Out 100!

Lights-Out 100 (or LO100) is a standard IPMI device.

It’s easy to manage:

$ ipmitool user list
# if admin is 6th user
$ ipmitool user enable 6
$ ipmitool user set password 6 admin

Upgrade LXC container from Ubuntu Lucid to Precise

Through the release upgrade process there is an error:

unable to make backup link of `./lib/init/fstab’ before installing new version: Invalid cross-device link

$ apt-get -f install
Reading package lists... Done
Building dependency tree       
Reading state information... Done
Correcting dependencies... Done
The following extra packages will be installed:
The following packages will be upgraded:
1 upgraded, 0 newly installed, 0 to remove and 246 not upgraded.
8 not fully installed or removed.
Need to get 0B/67.8kB of archives.
After this operation, 5,120B of additional disk space will be used.
Do you want to continue [Y/n]? 
(Reading database ... 18882 files and directories currently installed.)
Preparing to replace mountall 2.15.3 (using .../mountall_2.36.4_amd64.deb) ...
Moving obsolete conffile /etc/init/mounted-varrun.conf out of the way...
Unpacking replacement mountall ...
dpkg: error processing /var/cache/apt/archives/mountall_2.36.4_amd64.deb (--unpack):
 unable to make backup link of `./lib/init/fstab' before installing new version: Invalid cross-device link
Reinstalling /etc/init/mounted-varrun.conf that was moved away
Errors were encountered while processing:
localepurge: Disk space freed in /usr/share/locale: 0 KiB
localepurge: Disk space freed in /usr/share/man: 0 KiB

Total disk space freed by localepurge: 0 KiB

Stop the container and run this command from the host:

$ lxc-unshare -s MOUNT -- chroot rootfs apt-get -f install

rm * does not work

sudo rm -f’ does not remove directory content.

$ sudo rm -f /1/*
$ echo $?
$ sudo ls /1
1 2 3 4 5

How can this happen?
Check permissions:

$ ls -ld /1
d-wx--x--x 1 user user 10 Mar  1 16:24 /1

If user doesn’t have permission to list the directory, the wildcard matches to nothing in despite of sudo.

Note: in zsh an error message is displayed:

zsh: no matches found: /1/*

Liar Zimbra

Notify Zimbra of your installation? [Yes] no
Notifying Zimbra of installation via

Notification complete


Nokia N9: “connect to wi-fi network to download maps”

When I wanted to install new or update available maps I saw the following error message:

Connect to Wi-Fi network to download maps.

Solution is on this page:

1) Turn off Power saving – Device|Battery|Power Saving. If this is on it will turn off the WiFi when it thinks it is not being used! You can turn it back on when finished the map downloads.

2) In the Map application go to Settings and turn ‘On-line’ on. This seems to turn itself off…

However for me the first step was not necessary.
Whereas restarting the Maps application was a must.

IPA err=53

Login does not work in the application. In ipa access log there are some entries:

[26/Aug/2012:16:10:49 -0400] conn=183 fd=68 slot=68 connection from to
[26/Aug/2012:16:10:49 -0400] conn=183 op=0 BIND dn="admin" method=128 version=3
[26/Aug/2012:16:10:49 -0400] conn=183 op=0 RESULT err=53 tag=97 nentries=0 etime=0
[26/Aug/2012:16:10:49 -0400] conn=183 op=-1 fd=68 closed - B1

Official error code shows:

#define	EBADR		53	/* Invalid request descriptor */

Not really useful.

Solution: connection was initiated by the user (bind dn) ‘admin’, but wrong credential was provided.

glusterfs: mismatching layouts

There are messages like this in glusterfs mount point’s log:

W-Projects.log:[2012-05-14 19:13:14.283459] I [dht-layout.c:682:dht_layout_dir_mismatch] 0-w-vol-dht: subvol: w-vol-client-3; inode layout - 0 - 0; disk layout - 858993459 - 1717986917

W-Projects.log:[2012-05-14 19:13:14.283475] I [dht-common.c:524:dht_revalidate_cbk] 0-w-vol-dht: mismatching layouts for /97_KAMU/60_Elements/Prod/KAMU_ARNOLDTESZT/KAMU_ARNOLDTESZT_LGS/CG/KAMU_ARNOLDTESZT_LGS-cg_li_v004/masterLayer/Kettle_Indirect_Diffuse_raw

Although it’s not identified as an error, it is. Some of the clients cannot access the affected files.

For fix, run this command on each(?) cluster node:

gluster volume rebalance w-vol fix-layout start

RHEV: Error from Kerberos: Integrity check on decrypted field failed

There is an almost brand new RHEV 3.0 installation.
Old users can authenticate and can be listed on the administration interface, but a new user cannot be added.
In the IPA log no relevant information or error message, only the same as in the rhev.log:

2012-02-01 09:51:13,189 ERROR [org.ovirt.engine.core.bll.adbroker.GSSAPIDirContextAuthenticationStrategy] (pool-10-thread-3) Error from Kerberos: Integrity check on decrypted field failed (31) - PREAUTH_FAILED
2012-02-01 09:51:13,189 ERROR [org.ovirt.engine.core.bll.adbroker.DirectorySearcher] (http- Failed ldap search server LDAP:// due to org.ovirt.engine.core.bll.adbroker.RhevmDirectoryServiceException. We should not try the next server

Solution: by default IPA password policy require to change the passwords from time to time. RHEV was installed before changing the password of the user, who used accessing IPA.
So change back the password and error disappear.

Ubuntu Lucid 10.04 + KVM + Windows XP virtio block device: “Windows could not start because of computer disk hardware configuration problem.”

With this issue Windows 7 is not affected.
I installed Windows XP with virtio block device fine. After rebooting the OS cannot be started from the disk with this message:

Windows could not start because of computer disk hardware
configuration problem.
Could not read from the selected boot disk. Check boot path
and disk hardware.
Please check the Windows documentation about hardware disk
configuration and your hardware reference manuals for
additional information.

Solution: upgrade seabios package to 0.6.2-0ubuntu2 or above.


It’s possible to install IPA into an LXC container.

First time I got this error message:

Failed to create semaphore for stats file

Solution: set up tmpfs for /tmp

none    /tmp    tmpfs   nodev,nosuid,noatime,size=1000M,mode=1777   0   0

nginx: https proxy for http backend

This come up in many cases. Typical secanario for tomcat based applications.

location / {
            proxy_set_header             Host            $host;
            proxy_set_header             X-Real-IP       $remote_addr;
            proxy_set_header             X-Forwarded-For $proxy_add_x_forwarded_for;
            client_max_body_size      10m;
            client_body_buffer_size   128k;
            proxy_connect_timeout     90;
            proxy_send_timeout          90;
            proxy_read_timeout           90;
            proxy_buffers                       32 4k;
            proxy_cache             off;

            proxy_pass   http://localhost:8080;
            proxy_redirect   default;
            proxy_redirect  /;
            proxy_redirect   http://localhost:8080/;



ulimit -m 6291456
ulimit -v 6291456
ulimit -d 6291456
ulimit -n 4196
ulimit -f 1048576

exec 2>&1

if pgrep -u tomcat -f Dcatalina.home=/opt/java/apache-tomcat-7 > /dev/null; then
        echo "Tomcat 7 instance is already running!"
        logger -i -p user.err -- "Tomcat 7 instance is already running!"
        ## prevent runit storming
        sleep 30
        exit 0


chgrp -R ${G} supervise
chmod g+rw supervise/*
chmod g+rwxs supervise

# set in $CATALINA_BASE/bin/
#JAVA_OPTS="-XX:MaxPermSize=256m -XX:-OmitStackTraceInFastThrow -Djava.awt.headless=true"


exec > >( chpst -u ${U}:${G} rotatelogs -l -f /opt/java/apache-tomcat-7.0.20/logs/catalina-%F.out 86400 )
exec 2>&1

exec chpst -u ${U}:${G} /opt/java/apache-tomcat-7.0.20/bin/ run \
        >> /opt/java/apache-tomcat-7.0.20/logs/catalina.out 2>&1

send_nsca: Error: Timeout after 10 seconds

The official wiki page at says, the system can be configured by telnet. But no, you will get timeout.

Actually it can be tested via send_nsca like this:

# echo "test;test;0;test-output" | send_nsca -H -p 5667 -c /etc/send_nsca.cfg -d ";"
1 data packet(s) sent to host successfully.

In syslog this must be appeared if debug is enabled:

Jun 27 17:46:40 nagiospro nsca[6122]: Connection from port 48773
Jun 27 17:46:40 nagiospro nsca[6122]: Handling the connection…
Jun 27 17:46:50 nagiospro nsca[6122]: End of connection…
Jun 27 17:46:54 nagiospro nsca[6129]: Connection from port 49285
Jun 27 17:46:54 nagiospro nsca[6129]: Handling the connection…
Jun 27 17:46:58 nagiospro nsca[6129]: SERVICE CHECK -> Host Name: ‘test’, Service Description: ‘test’, Return Code: ‘0’, Output: ‘test-output’
Jun 27 17:46:58 nagiospro nagios3: EXTERNAL COMMAND: PROCESS_SERVICE_CHECK_RESULT;test;test;0;test-output
Jun 27 17:46:58 nagiospro nagios3: Warning: Passive check result was received for service ‘test’ on host ‘test’, but the host could not be found!
Jun 27 17:47:04 nagiospro nsca[6129]: End of connection…
Jun 27 17:49:32 nagiospro nagios3: Auto-save of retention data completed successfully.

RTSP DNAT rules for Vivotek cameras


WEB [] – fw1 [] —– INTERNET —– [] fw2 – CAMERA []

On fw1:

-A PREROUTING -t nat -p udp -d --sport 5500:6000 --dport 30000:31000 -j DNAT --to

On fw2:

-A PREROUTING -p tcp -d --dport http -j DNAT --to
-A PREROUTING -p tcp -d --dport rtsp -j DNAT --to
-A PREROUTING -p udp -d --dport rtsp -j DNAT --to

zimbra: logger service does not start after installing custom certificate

After installing custom openssl certificate it cannot be properly restarted with this failure message:

Starting logger…Failed.
Starting logswatch…ERROR: service.FAILURE (system failure: ZimbraLdapContext) (cause: PKIX path building failed: unable to find valid certification path to requested target)
zimbra logger service is not enabled! failed.

Continue reading

dirvish sophisticated/custom expire rule

Keep everything for 1 month.
Keep every sunday backup for 3 months
Never erase a backup of the the first day of months.

#MIN    HR      DOM     MON         DOW     EXPIRE
*       *       *       *           *       +1 month
*       *       *       *           Sun     +3 months
*       *       1       *           *       never