Why I love Virtualisation...

I love VMware products in general, and ESX in particular, I just solved a problem in under 30 minutes, without any service interruption, that on the same physical box would have required a reboot... Amazing you say? Well yes, when your /var partition is dying on your mail gateway, it is!

So our /var was getting full, and has been on and off for a while, it needs to be bigger, about double it's current 4GB. What to do? Well fire up the VI Client (yes, we are still running ESX3.5i), and add a 10GB disk to the VM.

Now, this is linux, the drive doesn't automatically show up, wo what to do next? Well a quick Google said:

echo "- - -" > /sys/class/scsi_host/host0/scan

Sure enough, an ls of /dev now shows an sdb that wasn't there before!

So, we add it into lvm:

pvcreate /dev/sdb
vgextend vg /dev/sdb

That gets us lots more space available, but I don't really want /var split over two disks, so we will dedicate this pv to /var, and leave some free space on the old pv...

pvmove -n var /dev/sda4 /dev/sdb

That does the trick, /var is now on the new pv. But still only 4GB...I've never managed to make the lvextend command automatically use all of the available free space, so I looked at how many extents were free in pvdisplay -m and then used that figure to do the extension.

lvextend -l+1535 /dev/vg/var

Ok, so the partition is now 10GB, but the file system (etx3) is only 4GB..

resize2fs /dev/vg/var

And there we go, all done!

Edit: Apparently the rescan above only picks up new disks. It won't notice a change on old disks, so if you increase a LUN, and want Linux to pick up the change you need to do

echo 1 > /sys/bus/scsi/devices/3:0:7:1/rescan

Where the 3:0:7:1 is the device ID.