My Profile Photo

Sheogorath's Blog


Depending on the time of the day a friend, a colleague, a wise guy. The beauty of the world is its sense of humor to show humans their way by letting them search their own.


Zero downtime LUKS + LVM volume resize

On my server backends, all volumes are encrypted. The raw block storage is provided by my hosting provider. Therefore it on top of the block storage, a LUKS volume is created to ensure secrecy of all data stored there. To keep management easy and allow easy filesystem snapshots for backups on top of this encrypted volume, there is a LVM volume set up.

From time to time, the volume, which are kept rather minimal due to pricing, run out of disk space and needs some more space to breath. This extension can be done without a downtime in a few steps.

Example

After increasing the physical volume using the providers interface, for this example by 10 GB, one should figure out the detailed volume structure. The best command for this is lsblk, of course you might just have it in the top of your head.

$ lsblk
NAME                                          MAJ:MIN RM  SIZE RO TYPE  MOUNTPOINT
sdb                                             8:16   0   60G  0 disk  
└─luks-scsi-0HC_Volume_73845743               253:3    0   50G  0 crypt
  └─centos_srvXX_data-docker-real             253:5    0   47G  0 lvm   
    └─centos_srvXX_data-docker                253:4    0   47G  0 lvm   /var/srv

Hint: Unneeded data, like /dev/sda, have been removed for readability from the lsblk output.

For this example, as you can see, the physical volume already has a size of 60GB while the LUKS volume only has 50GB.

To change that you need to run cryptsetup resize as root on that volume:

# cryptsetup resize luks-scsi-0HC_Volume_73845743

There is no resulting output. But the LUKS volume size will have expanded to the full volume size.

The next step is to increase the LVM “Physical Volume” (PV) size, by using pvresize:

# pvresize /dev/mapper/luks-scsi-0HC_Volume_73845743
  Physical volume "/dev/mapper/luks-scsi-0HC_Volume_73845743" changed
  1 physical volume(s) resized or updated / 0 physical volume(s) not resized

After this is done, you can increase the “Local Volume” (LV) size by using lvresize. In this example the LV is called docker and the Volume Group (VG) is called centos_srvXX_data but they could have any name.

# lvresize --size +10G /dev/centos_srvXX_data/docker
  Size of logical volume centos_srvXX_data/docker changed from 47.00 GiB (12032 extents) to 57.00 GiB (14592 extents).
  Logical volume centos_srvXX_data/docker successfully resized.

Now the logical volume, means the block storage volume on the system, is expanded by 10GB. Only leaves one task: the expansion of the filesystem.

This example will use XFS, but it works with most modern filesystems using similar filesystem management tools.

# xfs_growfs /dev/centos_srvXX_data/docker
meta-data=/dev/mapper/centos_srvXX_data-docker isize=512    agcount=10, agsize=1310720 blks
         =                       sectsz=512   attr=2, projid32bit=1
         =                       crc=1        finobt=0 spinodes=0
data     =                       bsize=4096   blocks=12320768, imaxpct=25
         =                       sunit=0      swidth=0 blks
naming   =version 2              bsize=4096   ascii-ci=0 ftype=1
log      =internal               bsize=4096   blocks=2560, version=2
         =                       sectsz=512   sunit=0 blks, lazy-count=1
realtime =none                   extsz=4096   blocks=0, rtextents=0

And that’s basically it.

Summary

To sum all commands up again:

# cryptsetup resize <LUKS volume name>
# pvresize /dev/mapper/<LUKS volume name>
# lvresize --size +<increased size> /dev/<VG name>/<LV name>
# xfs_growfs /dev/<VG name>/<LV name>

No downtime or reboot required, it all works on the fly. And given that basically all commands are called resize, it should be easy to memorize. If not, then you have this article to remind yourself.