Extending Linux VolumeGroup

Lvm2 is widely used these days and many distros use it already as default hdd mapping.
When you have to change it like extending your system with a new hdd or virtualized with more hddspace the modification steps aren’t trivial.
I will explain the steps with an example in VirtualBox but the extending is analog as putting a new drive into the system.

Changing the virtual hdd in VBox:

VBoxManage “<path_to_vdi>” –resize 50000(<-in MB)
!!Attention when you have Snapsshots in you VBox Image already this won’t work
!!Clone actual snapshot vdi into a new System or use following on your snapshot.vdi file

So now your HDD has more space but it yet has not found its way in a partion and so not in the VolGroup and LogicalVolume.

We use fdisk to allocate the space in a new partion sd3 and add it then to the VolGroup and Volume.

——-

[root@server ~]# fdisk -l

Disk /dev/sda: 500.1 GB, 500107862016 bytes
240 heads, 63 sectors/track, 64601 cylinders
Units = cylinders of 15120 * 512 = 7741440 bytes

Device Boot Start End Blocks Id System
/dev/sda1 * 1 28 211648+ 83 Linux
/dev/sda2 29 41360 312469920 8e Linux LVM

[root@server ~]# fdisk /dev/sda

Command (m for help): p

Disk /dev/sda: 500.1 GB, 500107862016 bytes
240 heads, 63 sectors/track, 64601 cylinders
Units = cylinders of 15120 * 512 = 7741440 bytes

Device Boot Start End Blocks Id System
/dev/sda1 * 1 28 211648+ 83 Linux
/dev/sda2 29 41360 312469920 8e Linux LVM

Command (m for help): n
Command action
e extended
p primary partition (1-4)
p (!!!!!!!!!!)
Partition number (1-4): 3
First cylinder (41361-64601, default 41361):
Using default value 41361
Last cylinder or +size or +sizeM or +sizeK (41361-64601, default 64601):
Using default value 64601

Command (m for help): t
Partition number (1-4): 3
Hex code (type L to list codes): 8e
Changed system type of partition 3 to 8e (Linux LVM)
Command (m for help): p

Disk /dev/sda: 500.1 GB, 500107862016 bytes
240 heads, 63 sectors/track, 64601 cylinders
Units = cylinders of 15120 * 512 = 7741440 bytes

Device Boot Start End Blocks Id System
/dev/sda1 * 1 28 211648+ 83 Linux
/dev/sda2 29 41360 312469920 8e Linux LVM
/dev/sda3 41361 64601 175701960 8e Linux LVM

Command (m for help): w
The partition table has been altered!

Calling ioctl() to re-read partition table.

WARNING: Re-reading the partition table failed with error 16: Device or resource busy.
The kernel still uses the old table.
The new table will be used at the next reboot.
Syncing disks.

Reboot
after reboot comes the volumegroup creation and extension

lvm

lvm> lvmdiskscan
/dev/ramdisk [ 16.00 MB]
/dev/root [ 296.03 GB]
/dev/ram [ 16.00 MB]
/dev/sda1 [ 206.69 MB]
/dev/VolGroup00/LogVol01 [ 1.94 GB]
/dev/ram2 [ 16.00 MB]
/dev/sda2 [ 297.99 GB] LVM physical volume
/dev/ram3 [ 16.00 MB]
/dev/sda3 [ 167.56 GB]
/dev/ram4 [ 16.00 MB]
/dev/ram5 [ 16.00 MB]
/dev/ram6 [ 16.00 MB]
/dev/ram7 [ 16.00 MB]
/dev/ram8 [ 16.00 MB]
/dev/ram9 [ 16.00 MB]
/dev/ram10 [ 16.00 MB]
/dev/ram11 [ 16.00 MB]
/dev/ram12 [ 16.00 MB]
/dev/ram13 [ 16.00 MB]
/dev/ram14 [ 16.00 MB]
/dev/ram15 [ 16.00 MB]
/dev/sdb1 [ 186.31 GB]
3 disks
18 partitions
0 LVM physical volume whole disks
1 LVM physical volume

lvm> pvcreate /dev/sda3
Physical volume “/dev/sda3″ successfully created
lvm> vgextend VolGroup00 /dev/sda3
Volume group “VolGroup00″ successfully extended
lvm> vgs
VG #PV #LV #SN Attr VSize VFree
VolGroup00 2 2 0 wz–n- 465.50G 167.53G
lvm> pvscan
PV /dev/sda2 VG VolGroup00 lvm2 [297.97 GB / 0 free]
PV /dev/sda3 VG VolGroup00 lvm2 [167.53 GB / 167.53 GB free]
Total: 2 [465.50 GB] / in use: 2 [465.50 GB] / in no VG: 0 [0 ]
lvm> vgdisplay
— Volume group —
VG Name VolGroup00
System ID
Format lvm2
Metadata Areas 2
Metadata Sequence No 4
VG Access read/write
VG Status resizable
MAX LV 0
Cur LV 2
Open LV 2
Max PV 0
Cur PV 2
Act PV 2
VG Size 465.50 GB
PE Size 32.00 MB
Total PE 14896
Alloc PE / Size 9535 / 297.97 GB
Free PE / Size 5361 / 167.53 GB
VG UUID NsVbvv-A2Sn-xxiP-E92w-wbbI-Dsbz-pERnZS

lvm> lvextend -L463.50G /dev/VolGroup00/LogVol00
Rounding up size to full physical extent 463.22 GB
Extending logical volume LogVol00 to 463.22 GB
Logical volume LogVol00 successfully resized

——-

Now we have to extend the filesystem to the new sizes.

umount /dev/mapper/VolGroup_root
e2fsck -f /dev/mapper/VolGroup_root
resize2fs /dev/mapper/VolGroup_root
mount -t ext2 /dev/mapper/VolGroup_root /

Great we are done..

Backlinks:
https://www.virtualbox.org/ticket/9103
http://community.spiceworks.com/how_to/show/2768-extending-lvm-partition-with-free-space-from-the-same-hdd-or-with-a-new-partition-from-a-new-hdd-on-centos

Mock Data Generation

Sometimes you need mockdata to upload to a DB or to test on an altenative datastore. For that Mockaroo.com works perfectly as you can define the expected field values in ranges and so generate a matching DataSet very fast. I used it to feed my elasticsearch instance for testqueries against a db schema layout that needed to be designed.  The datavalues can be choosen in a wide variety of know formats and standard values. (e.g. date, currency, country) Output formats are json, sql, csv and many more.

Need Flat Icons ?

If you need flat design icons or graphics for your app or site, check out :

http://flaticons.net

All is free for every use. You can customize interactively color, padding, framing and background. Huge library. Great ..

Manually grep Android SDK Manager Jars

The Android SDK Manager seems in various versions buggy. On Windows I recognized parts of the extra packages are not displayed in the download list. On Linux I recognized not the latest packages seems to be available. Although all displayed is based on the

http://dl-ssl.google.com/android/repository/repository.xml

http://dl-ssl.google.com/android/repository/addon.xml

repo files the SDKManager doesn’t update correctly.  Advices on the net always hint to install the latest version. But there exists a manually method too.

Easily lookup the target filename by directing your browser either for SDK packages to repository.xml or for addon/extra packages to addon.xml.

<sdk:url>support_r09.zip</sdk:url>

Then construct your target file url by simply merging both

http://dl-ssl.google.com/android/repository/support_r09.zip

Drag’n Drop in Android 4 ICS+

Create a custom ListView and add a Listener via a following method.
Add a ClipData Object to identify the selected row.

/**
* Setup what to do when we drag list items
*/
public void setupDragDrop() {
this.setOnItemLongClickListener(new OnItemLongClickListener() {
public boolean onItemLongClick(AdapterView arg0, View v,
int position, long arg3) {
ClipData data = ClipData.newPlainText("procedure", position
+ "");
v.startDrag(data, new DragShadowBuilder(v), null, 0);
return true;
}
});
CustomDragListener mDragListener = new CustomDragListener();
this.setOnDragListener(mDragListener);

Create the CustomDragListener to decorate the behaviour of your Drag’n Drop.
For simplicity we will just handle ACTION_DROP event.
Use ListView’s pointToPosition() function to determine target position id.

protected class CustomDragListener implements OnDragListener {
public boolean onDrag(View v, DragEvent event) {
final int action = event.getAction();
switch (action) {
case DragEvent.ACTION_DRAG_ENTERED:
return false;
case DragEvent.ACTION_DRAG_EXITED:
return true;
case DragEvent.ACTION_DRAG_STARTED:
return true;
case DragEvent.ACTION_DRAG_LOCATION:
return false;
case DragEvent.ACTION_DROP:
v.setBackgroundColor(Color.TRANSPARENT);
int newPosition = pointToPosition((int) (event.getX()),
(int) event.getY());
if (newPosition != ListView.INVALID_POSITION)
return processDrop(event, newPosition);
else
return false;
default:
return true;
}
}

Unwrap old position from ClipData and call helper function to update UI.

private boolean processDrop(DragEvent event, int newPosition) {
ClipData data = event.getClipData();
if (data != null) {
if (data.getItemCount() > 0) {
Item item = data.getItemAt(0);
int value = Integer.parseInt(item.getText()+"");
updateViewsAfterDropComplete(value, newPosition);
return true;
}
}
return false;
}

Update UI and Model if binding should be processed further.

private void updateViewsAfterDropComplete(int oldIndex, int index) {
ArrayList<MyListEntry> actual = myListStructure.getEntries();
MyListEntry entry = actual.get(oldIndex);
actual.remove(oldIndex);
actual.removeAll(Collections.singleton(null));
actual.add(index, entry);
myListStructure.setEntries(actual);
db.updateMyListStructure(myListStructure);
repaintUi();
}

Get dm-raid booting with lvm2 and grub (debian)

The use of the so called fakeraid or softraid is often not adviced. I don’t want to advice it also as I didn’t have to rescue my System yet. But there are possibilities. But from the performance view it doesn’t behaves bad on my system with around 150MB/s in raid1 < in contrast to> my expensive 3ware 9550SXU hwraid-system with 70MB/s.

Some specialaties are to notice during install (with debian squeeze 6.04 installer):

Since the physical volume name is provided randomly by the dmraid driver (according to your chipset), your partitions are mapped with somthing like that:

nvidia_cffbdeda <- the physical device

nvidia_cffbdeda1 <- partition 1

nvidia_cffbdeda2 <-partiton 2

 

You have to install grub afterwards again by rescue cd or chrooted into installation as during install it will fail and the system won’t be bootable the first time. Install it into MBR which means the nvidia_cffbdeda <- the physical device with the following modification:

apt-get purge grub2 grub-pc grub-common
rm -R /boot/grub
apt-get install grub-pc
update-grub
grub-install /dev/mapper/nvidia_cffbdeda
grub-install –recheck /dev/mapper/nvidia_cffbdeda

check this link for a complete grub reinstall guide: here

The installer usually creates an own partition for /boot which will be linked in the root filesystem (nvidia_cffbdeda1). This is partition1. As you installed with lvm the whole volume group resides in partition2. There you have your different volumes for /root /data /swap. All are in partition2.

 

fstab problem (partition naming problem):

A problem I regognized was that the installer declares the dmraid-partitions internally in fstab as

nvidia_cffbdedap1 insteadof nvidia_cffbdeda1

and

nvidia_cffbdedap2 insteadof nvidia_cffbdeda2

It is a new naming rule which seems not having found it’s way to all components dmraid and lvm2 yet.

 

So even if grub was twice ran correctly it can’t find the /boot partition nvidia_cffbdeda1 as fstab refers to nvidia_cffbdedap1. Changing there to the correct value resulted in a correct boot.

 

Finally we can say:

1) the naming of your virtual dm-raid partitions can be found via

ls /dev/mapper/

change fstab values according to this naming.

2) grub is aware of dmraid and lvm2 partitions and can initialize them before boot which means that you don’t have to create an own /boot partition (which I have read quite often). You can install everything in your root filesystem and let grub boot from there although the installer does it differently.

Some links:

http://pve.proxmox.com/wiki/Install_Proxmox_VE_on_Debian_Squeeze

http://ubuntuforums.org/archive/index.php/t-1338445.html

http://forum.proxmox.com/archive/index.php/t-7569.html

 

 

dmraid, lvm2 commands to know

A brief commad list to handle your partition/volumes :

dmraid:

dmraid -s [show active dmraid set]

dmraid -l [list available driver]

dmraid -ay [activate dmraid set]

lvm:

lvdisplay –v [Show current logical volumes]

umount /dev/VolumeGroupName/data [unmount logical volume 'data'

lvremove /dev/VolumeGroupNam/data [ensure you backup anything there first to another location or server]

pvdisplay –v [show current physical volumes]

pvcreate /dev/sdxxx [create new physical volume for use. This is your new HDD or array. If you are unsure the exact device, look under /dev dir. You can also use fdisk -l to help]

pvdisplay –v [Your new physical volume should now be visable]

vgdisplay –v [display size and config of volume group]

vgextend /dev/VolumeGroupName /dev/sdxxx [This will extend the volume group by the new physical volume you have added above]

lvcreate –n /dev/VolumeGroupName/data –L sizeG [This will create your new /data store in the volume group Note: The G after size is a value in Gigabytes]

lvdisplay –v [You should now see your new data store with the larger size, this can be >2TB]

mkfs –T ext3 /dev/VolumeGroupName/data [This will format your new store ready for use]