Home

Debian/Ubuntu Tips & Tricks

Debuntu.org: .deb packages, Unix/Linux Tutorials and Articles.

User login

Get Firefox


Tips


How-To: Install Ubuntu on LVM partitions.

LVM (Logical Volume Manager) is a great piece of software which allow you to deal with Logical Volumes. Using LVM along with ext3 filesystem, you are allowed to extend the size of your logical drives which is pretty handy when running out of space.

Distributions like Fedora, Suse and Debian have a LVM aware installer. Unfortunately, at the time this article was written, Ubuntu does not offer such settings with the Desktop Install CD.

This article will cover how to create LVM partitions and how-to generate your partitions from this LVM volume.

LVM offer better flexibility when it comes to storage management. Using traditional partitioning system, when a partition becomes too small, the only solution left is to:

  • create a bigger partition
  • Copy the datas from the previous partition to the newly created one
  • Change the mounting information of the partition
  • Umount the old one, and mount the new one

LVM allow you to have better control of your storage. Excerpt from http://en.wikipedia.org/wiki/Logical_Volume_Manager_(Linux):

The LVM can:

* Resize volume groups online by absorbing new physical volumes (PV) or ejecting existing ones.
* Resize logical volumes online by concatenating extents onto them or truncating extents from them.
* Create read-only snapshots of logical volumes (LVM1).
* Create read-write snapshots of logical volumes (LVM2).
* Stripe whole or parts of logical volumes across multiple PVs, in a fashion similar to RAID0.
* Move online logical volumes between PVs.
* Split or merge volume groups in situ (as long as no logical volumes span the split). This can be useful when migrating whole logical volumes to or from offline storage.

It cannot:

* Mirror whole or parts of logical volumes, in a fashion similar to RAID1 or RAID5 mirroring of logical volumes. For this, it is recommended, that one use the Linux software RAID driver to mirror the underlying PVs to achieve redundancy.

In this tutorial, we will install Ubuntu on an LVM filesystem. Because Ubuntu does not support this out of the box, we will have to create and format our partitions using fdisk. Once the partitions will be created, we will start the installer and point it to the right partition.

Because we are going to modify the hard-drive layout, be aware that you might delete datas. Please make sure you have backed-up all important datas.

1. Requirements:

In order to be able to do this tutorial, you will need the following:

  • Ubuntu Desktop install liveCD
  • An internet connection or lvm2 package
  • Free disk space on your hard-drive

2. Preparing the environment:

2.1. Installing LVM

Boot up your computer using Ubuntu liveCD. Once logged in, open a terminal and install LVM:

$ sudo apt-get install lvm2

If you have a lvm package copy on a removeable device, type:

$ sudo dpkg -i /path/to/lvm2.deb

Finally, lod the kernel module dm-mod.

$ sudo modprobe dm-mod

Now, we need to set up our Hard drive.

2.2. Partitioning the disk:

One of the requirements of Linux on LVM is that /boot has to be on a traditional partition. Because of this requirement, we will set up a separate /boot partition and a big LVM partition covering the rest of the hard drive. This LVM partition will then be splitted in a /, /home and swap partitions. Some extra space might be left on the LVM partition allowing you to grow your existing partitions if needed be.

At this stage, I consider that you have space available on your disk, if not, please use fdisk or gparted and delete some partitions, when done, refresh the kernel partition table by typing:

$ sudo partprobe

Well, let's create a boot partition of 100M and all the available space into a LVM partition:

$ sudo fdisk /dev/sda

The number of cylinders for this disk is set to 9729.
There is nothing wrong with that, but this is larger than 1024,
and could in certain setups cause problems with:
1) software that runs at boot time (e.g., old versions of LILO)
2) booting and partitioning software from other OSs
(e.g., DOS FDISK, OS/2 FDISK)

Command (m for help): n
Command action
e extended
p primary partition (1-4)
p
Partition number (1-4): 2
First cylinder (1033-9729, default 1033):
Using default value 1033
Last cylinder or +size or +sizeM or +sizeK (1033-9729, default 9729): +100M

Command (m for help): n
Command action
e extended
p primary partition (1-4)
p
Partition number (1-4): 3
First cylinder (1058-9729, default 1058):
Using default value 1058
Last cylinder or +size or +sizeM or +sizeK (1058-9729, default 9729):
Using default value 9729

Command (m for help):t
Partition number (1-4): 3
Hex code (type L to list codes): 8e

Command (m for help):w
The partition table has been altered!

Calling ioctl() to re-read partition table.

WARNING: Re-reading the partition table failed with error 16: Device or resource busy.
The kernel still uses the old table.
The new table will be used at the next reboot.
Syncing disks.

$ sudo partprobe

OK, now we have our boot partition and our big LVM partition. We need to format /boot to ext3 filesystem, create our physical volume on /dev/sda3, create a Volume Group and Logical Volumes for /, /home and the swap partition. Finally format those logical volumes and install Ubuntu over those volumes.

$ sudo mke2fs -j /dev/sda2
$ sudo pvcreate /dev/sda3 #create a physical volume on /dev/sda3
Physical volume "/dev/sda3" successfully created
$ sudo vgcreate lvmvolume /dev/sda3 #create a volume group named lvmvolume using /dev/sda3 physical volume
Volume group "lvmvolume" successfully created
$ sudo lvcreate -n root -L 5G lvmvolume #create a logical volume of 5G named "root" in lvmvolume
Logical volume "root" created
$ sudo lvcreate -n home -L 10G lvmvolume #create a logical volume of 10G named "home" in lvmvolume
Logical volume "home" created
$ sudo lvcreate -n swap -L 2G lvmvolume #create a logical volume of 2G named "swap" in lvmvolume
Logical volume "swap" created
$ sudo mkfs -j /dev/lvmvolume/root -L root #format root as ext3
$ sudo mkfs -j /dev/lvmvolume/home -L home #format home as ext3
$ sudo mkswap -L swap /dev/lvmvolume/swap #format swap as swap filesystem, labelled swap

Here we are, no start the installer and when it comes to partitioning, select "manual" and choose the partitions /dev/sda1 (change this according to your layout) , /dev/lvmvolume/{root,home,swap} for /boot, /, /home and swap.

If the partitions are presented like: /dev/dm-X, type ls -l /dev/mapper/, this will help you tracking your volumes.

3. Finalizing the install

Before you reboot your computer, you need to install lvm2 package into your newly created system. To do this, you will chroot into you new ubuntu sytem and install the package in that environment.

As per wayupnorth comment, make sure all your future partition are mounted under /target, for example, you will need to issue:
# mount /dev/sdaX /target/boot
# mount /dev/lvmvolume/usr /target/usr
...

Open a terminal and type:

$ sudo chroot /target
# apt-get update
# apt-get install lvm2

That's it, you can now reboot your system and boot on your Ubuntu system using LVM.


Migrating from Centos

Hi - I want to migrate from an existing Redhat based system (Centos 5.1).

When I installed Centos it installed LVM as default, and partitioned accordingly.

I tried booting off the Kubuntu 7.01 Live DVD. It didn't see my disk, now I read your article I see how I could make it see it, so thank you.

What I don't know is what I would need to do differently to your instructions.

It looks like you assume that the disk is new - what do I need to do differently to be able to add Ubuntu as a GRUB boot option?

Thanks,
Martin.


Created partitions


lvcreate -n KubuntuOS -L 15G VolGroup00
lvdisplay
--- Logical volume ---
LV Name /dev/VolGroup00/CentosOS
VG Name VolGroup00
LV Write Access read/write
LV Status available
# open 1
LV Size 97.66 GB
Current LE 3125
Segments 1
Allocation inherit
Read ahead sectors 0
Block device 253:0
--- Logical volume ---
LV Name /dev/VolGroup00/VMs
VG Name VolGroup00
LV Write Access read/write
LV Status available
# open 0
LV Size 275.00 GB
Current LE 8800
Segments 1
Allocation inherit
Read ahead sectors 0
Block device 253:1
--- Logical volume ---
LV Name /dev/VolGroup00/LogVol01Swap
VG Name VolGroup00
LV Write Access read/write
LV Status available
# open 1
LV Size 1.94 GB
Current LE 62
Segments 1
Allocation inherit
Read ahead sectors 0
Block device 253:2
--- Logical volume ---
LV Name /dev/VolGroup00/KubuntuOS
VG Name VolGroup00
LV Write Access read/write
LV Status available
# open 0
LV Size 15.00 GB
Current LE 480
Segments 1
Allocation inherit
Read ahead sectors 0
Block device 253:3


Coincidence?

Man, oh man, Martin. I'm glad I found this. And you only posted this TODAY? Wow.

I too am migrating from Centos and am having the exact same problems figuring out what to do with LVM. My story is a little different from yours, though.

My last computer crashed (a homebrew model) and I went out and bought a new Dell Inspiron 530. I was greatly saddened to see that Centos wouldn't run and nobody could help me, although some tried. The current Centos kernel just doesn't support the hardware, I think. But that's okay, I'm really liking Ubuntu so far. My hard disk, however, was not damaged and there is a lot of data I want to move over. I am willing to copy all of it to some subdirectory on the Ubuntu drive temporarily and then move it over to its final place, piece by piece, as needed, and then remove the Centos disk and keep it as a spare.

So my question is a little different. How can I SEE the existing LVM subpartitions under the main partition? Then, once I've mounted the LVM group, I'll be able to see the subpartions and know what to copy.

Thanks in advance.


Have you loaded LVM?

I don't really know. But, once you load LVM can't you see the old disk in Ubuntu?

I agree that its a pain that Ubuntu doesn't include LVM as default yet. Perhaps in the next release.


Okay, let's get down to specifics

$ sudo vgcreate VolGroupCentOS /dev/sdb2
  Physical volume '/dev/sdb2' is already in volume group 'VolGroup00'
  Unable to add physical volume '/dev/sdb2' to volume group 'VolGroupCentOS'.

So a volume group already exists. Okay, Lets try to use it.

$ sudo lvm
lvm> vgdisplay
  --- Volume group ---
  VG Name               VolGroup00
  System ID
  Format                lvm2
  Metadata Areas        1
  Metadata Sequence No  3
  VG Access             read/write
  VG Status             resizable
  MAX LV                0
  Cur LV                2
  Open LV               0
  Max PV                0
  Cur PV                1
  Act PV                1
  VG Size               297.97 GB
  PE Size               32.00 MB
  Total PE              9535
  Alloc PE / Size       9535 / 297.97 GB
  Free  PE / Size       0 / 0
  VG UUID               3KTvtT-9D20-52lQ-Sg2j-As1I-N29o-D5i0go

Great. LVM sees it.
lvm> lvdisplay
  --- Logical volume ---
  LV Name                /dev/VolGroup00/LogVol00
  VG Name                VolGroup00
  LV UUID                RYjWsO-mYpv-Tul9-2YI7-38iw-Fvkt-oeK2bg
  LV Write Access        read/write
  LV Status              NOT available
  LV Size                296.03 GB
  Current LE             9473
  Segments               1
  Allocation             inherit
  Read ahead sectors     0
  --- Logical volume ---
  LV Name                /dev/VolGroup00/LogVol01
  VG Name                VolGroup00
  LV UUID                r4JZO6-Sjk5-Y2dW-fqzq-y3Vt-UnY4-ARx9bQ
  LV Write Access        read/write
  LV Status              NOT available
  LV Size                1.94 GB
  Current LE             62
  Segments               1
  Allocation             inherit
  Read ahead sectors     0
lvm> quit
  Exiting.

OK, my Volume Groups and logical volumes are visible to lvm. Let's mount em.

$ sudo mount /dev/VolGroup00/LogVol01 /old
mount: special device /dev/VolGroup00/LogVol01 does not exist

Huh, so it's visible to lvm but not to the file system?

$ ls /dev/VolGroup00
ls: /dev/VolGroup00: No such file or directory

Yup, that's right. If I was creating this from new, all the steps above for creating partitions would have taken care of this, I think. What corresponding steps need to be done to associate the existing LVM volume groups and volumes with the file system in a non-destructive way?


/dev/mapper

Hi,

You want to check under /dev/mapper, as on Martin system:
/dev/mapper/VolGroup00-CentosOS
Which is similar to /dev/VolGroup00/CentOS

Debuntu


Thanks!!!!!!!!!!!!!

That did the trick, that was the piece I was missing. Thanks so much!


No Problem, and thanks

No Problem, and thanks stevecoh1 and mrjcleaver for sharing your experience, I am sure, it will help lots of other people.
Debuntu


I've loaded LVM

deleted by author


reusing the same partitions?

Hi Martin,

In the partitioning your disk section, you might either:
- leave the layout as is, if you intend to overwrite your CentOS system, and then select each partition individually in the Installer partition editor window.
- Or, if you want to keep your old system, I am assuming that you already have Group and Logical Volumes set, therefore, you will need to create a new logical volumes:
$ sudo lvcreate -n home -L 10G lvmvolume
and format them
sudo mkfs -j /dev/lvmvolume/home -L home

Assuming that your group is called lvmvolume, if not, the command:
vgdisplay, lvdisplay .... will be handy
Debuntu


Reducing an existing partition to make space.

Thanks Chantra.

So, here's my breakdown of my 500gb disk. If I recall, I pretty much accepted the defaults. Most of my data is on an external NAS box, so what is local is mainly files for the various operating systems in VMware.

I don't want to eliminate Centos yet, so I need to dual boot.

# cat /etc/fstab (edited)

/dev/VolGroup00/CentosOS / ext3 defaults 1 1
/dev/VolGroup00/VMs /home/vms ext3 defaults 1 2
LABEL=/boot /boot ext3 defaults 1 2
/dev/VolGroup00/LogVol01Swap swap swap defaults 0 0
//nasbox/mrjcleaver /home/mrjcleaver/a-files/ cifs user,uid=mrjcleaver,gid=u
sers,rw,suid,noperm,credentials=/etc/nas-creds.txt 0 0

VolGroup00 (default name for Centos)

VG Size 465.66 GB
PE Size 32.00 MB
Total PE 14901
Alloc PE / Size 14901 / 465.66 GB
Free PE / Size 0 / 0
--- Logical volume --- (This is my Centos 5.1 Operating System install)
LV Name /dev/VolGroup00/CentosOS
LV Size 97.66 GB
--- Logical volume --- (Much of the local data is Virtual machines in VMware)
LV Name /dev/VolGroup00/VMs
LV Size 366.06 GB
--- Logical volume --- (Swap)
LV Name /dev/VolGroup00/LogVol01Swap
LV Size 1.94 GB

# df

Filesystem 1K-blocks Used Available Use% Mounted on
/dev/mapper/VolGroup00-CentosOS
99190192 53059344 41010848 57% /
/dev/mapper/VolGroup00-VMs
371822104 63703888 288926000 19% /home/vms

As you can see, the VolGroup00/VMs is bigger than I need and fills up the rest of the volume group.

I want to reduce this, by say, 100MB and create a /dev/VolGroup00/UbuntuOS lv. I imagine I can share the swap drive as only one or other of the physical OS's will run at any one time.

It occurred to me that I can first resize VolGroup00-VMs and then reduce it.
http://usr-share-man.org/man8/lvreduce.html says "ensure that any filesystem on the volume is resized before running lvreduce" but http://usr-share-man.org/man8/lvresize.html says "ensure that any filesystem on the volume is shrunk first so that the extents that are to be removed are not in use" without saying *how* to shrink it.

Would you mind clarifying 1) whether I can share swap 2) how to make room for my UbuntuOS LV without risking the data in my VolGroup00/VMs LV?

Thanks a lot. This is really helping!

Cheers,
Martin.


Found the How To!

http://tldp.org/HOWTO/LVM-HOWTO/reducelv.html


Logical volumes can be reduced in size as well as increased. However, it is very important to remember to reduce the size of the file system or whatever is residing in the volume before shrinking the volume itself, otherwise you risk losing data.


1. ext2
If you are using LVM 1 with ext2 as the file system then you can use the e2fsadm command mentioned earlier to take care of both the file system and volume resizing as follows:


# umount /home
# e2fsadm -L-1G /dev/myvg/homevol
# mount /home


Warning LVM 2 Caveat
There is currently no e2fsadm equivalent for LVM 2 and the e2fsadm that ships with LVM 1 does not work with LVM 2.


If you prefer to do this manually you must know the new size of the volume in blocks and use the following commands:


# umount /home
# resize2fs /dev/myvg/homevol 524288
# lvreduce -L-1G /dev/myvg/homevol
# mount /home

Further https://www.redhat.com/archives/linux-lvm/2006-August/msg00081.html says what to do if you can't unmount the volume you need to shrink.

Silly really. These instructions should be in the man page.


Making it real

Those instructions don't actually tell you what the figures should be. lvm2 unlike lvm1 doesn't give you tools. The consequences of getting the numbers wrong are dire (http://www.redhat.com/archives/nahant-list/2006-January/msg00065.html)

Either the superblock or the partition table is likely to be corrupt!
    Can I recover from this error ?
Don't know if you're going to be able to recover.
and (http://thr3ads.net/fedora-xen/2007/11/164429-xen-and-lvm)
I'm having no success reducing the files system. After reducing, I can
restart it ("kernel panic, not syncing")

http://rrfx.net/taxonomy_menu/1/2 has the clearest instructions and relates my concern, which are anchored in the notion that block sizes can vary.

Increasing the size of a LV (Logical Volume) is somewhat trivial. Reducing the size isn't. Its important to not make the LV smaller than the filesystem it contains. There is usually confusion as to what exactly '1G' is. So not wanting to trust that the option '70G' is the same for both resize2fs and lvreduce, I opted to:
    * resize the filesystem such that its much smaller than the desired LV size
    * resize the LV to the desired size (must always be greater/equal the filesystem size!)
    * use 'resize2fs' with no size to automatically fill the resized LV

and to make sure, I found another author who used the same method:
http://www.justlinux.com/forum/showpost.php?s=3bf49090a2ccb8e4532b2af1d7fa75ae&p=848903&postcount=4

In short, here's the actual commands I performed (I'm still running these but will save now, and will edit as procedure progresses):
0) Shutdown vmware (/etc/init.d/vmware stop)
1) umount /home/VMs
2) Commented it out from /etc/fstab
3) Back up entire 300GB or so to an external drive. (~4 hours?)
4) e2fsck -f /dev/VolGroup00/VMs (1 hour)
5) Reclaim 100GB: resize2fs /dev/VolGroup00/VMs 270G (~10:29-1:18pm)

I worried that I would need to shrink to the highest USED block but BMR from #lvm told me there is "no requirement to shrink to the highest used block either - since resize2fs is always offline for shrinking it can actually do block relocations (unlike online grow, which can't)"

In fact, BMR told me that I'll just get an error if there is not space:

Resizing the filesystem on /dev/loop0 to 61440 (1k) blocks.
resize2fs: No space left on device while trying to resize /dev/loop0

And that when resize2fs finishes it will tell you the number of the highest extent, with a message like:

MartinCleaver, it'll exit with a message like "file system on /dev/blah is now $x blocks long", iirc

So, when I saw dumpe2fs -h /dev/VolGroup00/VMs I saw Block count: 95,961,088 and Free blocks: 77,311,025, Block size: 4096, which to me makes me think 18,555,230 used * 4,096/1,024 = 74,604,920 bytes or (guessing) about 70GiB and 4,096*77,311,025 / 1024 = 316,665,958,400 bytes free, = ~ 310GiB free from a 95,961,088 * 4096 / 1024 = 383,844,352 = ~ 370GiB drive

tune2fs reported in blocks too, which means nothing outputs in human grokable numbers that I can also feed to resize2fs. There are no tools to do this for you, so I by hand 1) formulated the equation and 2) punched the numbers into a calculator.

resize2fs 1.39 (29-May-2006)
Resizing the filesystem on /dev/VolGroup00/VMs to 70778880 (4k) blocks.
The filesystem on /dev/VolGroup00/VMs is now 70778880 blocks long.

So, 70,778,880 * 4096/1024 = 283,115,520 bytes; / (1024x1024) = 270GiB; all is good.

I've just done resize2fs to reduced size of a filesystem in a partition and now I need to reduce the partition itself. I need to make sure that I don't lvreduce it less than the length of the filesystem (which is now 70778880 (4k) blocks.)

My concern next was that I knew that the LV has to be slightly bigger, but it was not clear how much is slightly? 5G? 1G? 10G?. After waiting an inordinate amount of time (ok, about an hour.) I gave up and guessed that 5GB should be plenty.

I chose this size by guessing how much bigger than the filesystem the partition needs to be. Thing is, lvreduce works in logical extents while resize2fs works in blocks. Logical extents is not clearly explained, and no one on #LVM answered.

6) lvreduce -L275G /dev/VolGroup00/VMs (finished within 5 mins):

WARNING: Reducing active logical volume to 275.00 GB
  THIS MAY DESTROY YOUR DATA (filesystem etc.)
Do you really want to reduce VMs? [y/n]: y
  Reducing logical volume VMs to 275.00 GB
  Logical volume VMs successfully resized

According to http://linux.derkeiler.com/Mailing-Lists/Fedora/2005-06/5677.html:

After shrinking the volume using lvreduce, I then use resize2fs again without specifying a size, which causes it to make the filesystem "fit" the volume.

7) resize filesystem contents again to fit the LV envelope:

resize2fs /dev/VolGroup00/VMs
resize2fs 1.39 (29-May-2006)
Resizing the filesystem on /dev/VolGroup00/VMs to 72089600 (4k) blocks.
The filesystem on /dev/VolGroup00/VMs is now 72089600 blocks long.

After I finished this procedure Blacker47 and BMR on #LVM clarified: "MartinCleaver, reduce to (size of fs + 2*PE), I think, it would be secure. In your case 270.8G should be enough - that's over the (size of fs + 2*PE) (in fact, size of fs rounded up to next PE boundary would be OK). One can find out the PE from the volume group"

vgdisplay  /dev/VolGroup00
PE Size               32.00 MB

So by this reckoning "if 32M, then 8641 should be fine" (8MB). I used a 5GB margin, so I was fine.


Error creating Logical Volumes

This article is very usefull but I encountered a problem creating my lvm volumes.
I precise I run a Linux Mint 4.0 KDE Beta 044 (based on gutsy).
I had no problem creating my physical volume and my volume group, but I couldn't create logical volumes. I had this message :

mint@mint:~$ sudo lvcreate -L 1G -n lvroot vg00
/proc/misc: No entry for device-mapper found
Is device-mapper driver missing from kernel?
Failure to communicate with kernel device-mapper driver.
/proc/misc: No entry for device-mapper found
Is device-mapper driver missing from kernel?
Failure to communicate with kernel device-mapper driver.
Incompatible libdevmapper 1.02.20 (2007-06-15)(compat) and kernel driver
striped: Required device-mapper target(s) not detected in your kernel
lvcreate: Create a logical volume

The solution is to type "sudo modprobe dm-mod".

Bye


Thank you and an idea

Thanks, great tip! Alternate iso failed me so I needed to do my lvm install with the desktop version.

Since there might be some people that are not that familiar with chrooting it could be a good idea to mention that they also need to mount their var, usr and boot partitions inside their root partition with the fresh install. Usr and var for the apt-get command to work and boot to enable the new initrd image to be generated after installing lvm2.


hi wayupnorth, After the

hi wayupnorth,
After the files have been copied from the CD to your hard disk, /target should already have all your future system partition mounted under its hierarchy. therefore, there should be no need to manually mount them.

Debuntu


Sorry, should have elaborated

I have root, usr and var on separate partitions each. That's where the need to mount them separately came from. Just mounting the root gave apt-get: command not found -messages and an error about missing groups when chrooting. Installing lvm2 results in a new initrd image being generated so I think boot should be mounted as well. I didn't have it mounted at first and saw the new image being created in an empty boot directory and thought it won't do any good there.


Tip added

Hi wayupnorth,
Thanks, I have added a tip reminding that the partitions should be mounted.
Thanks again.
Debuntu


Brilliant howto

Thank you for this howto - I am now running Kubuntu 7.04 with LVM.

When you mention the manual partitioning, shouldn't you (for completeness) mention that /boot must be chosen as well?


Thanks Vange, you are

Thanks Vange, you are welcome.
I updated this how and added the /boot bit.
Debuntu