SIP debug stuff

August 15th, 2007 No comments

If you’re debugging SIP, here are a few tools can make all the difference:

SIPp on Sourceforge,
SIPp config sample at: SIPfoundry,
ngrep,
callflow produces nice diagrams and sip_scenario is a similar product producing HTML.
And of course Wireshark (formerly Ethereal) and tcpdump.

A couple of  tips when using sip_scenario:

tcpdump -s 0 -w pk.cap
-s … don’t truncate packets to default length of 96
-w … write output to pk.cap

sip_scenario 192.168.1.1:5060:1 -g:30 pk.cap
192.168.1.1:5060:1 … output ip addr 192.168.1.1 port 5060 in column 1
-g:30 … set column width to 30 (chars  wide?)

move the generated html files to a folder on your webserver and voila!

For more detailed help and samples: sip_scenario -syntax++

Categories: Uncategorized Tags:

chkconfig for Debian

August 10th, 2007 No comments

And here I’ve been wasting time installing chkconfig for Debian …

update-rc.d service_name defaults 20 90

Check man update-rc.d for details.

Categories: Uncategorized Tags:

ntop on Ubuntu Feisty

August 10th, 2007 1 comment

Not sure how but my Fawn automatically starts ntop on reboot. Either this is the default when you install the package or I’ve simply forgotten about creating links in /etc/rc3.d.

However, while everything looks nice, the network load info was missing. Also, restarting ntop (or rebooting) would lose all stats.

A quick look around turned up ‘rrd’ or Round Robin Database which is installed with rrdtool and then activating the plugin of the same name in ntop. But this lead to an error indicating that it couldn’t create/access files in /var/lib/ntop/rrd. Since ntop is launched at boot under control of user ‘ntop’, the solution was to simply give ownership of /var/lib/ntop/rrd to ntop.

Categories: Uncategorized Tags:

Useful Network Tools

July 7th, 2007 No comments

Cain & Abel

Wireshark (formerly Ethereal and the non-gui Tethereal)

Nessus

Categories: Uncategorized Tags:

Virtualization with XEN and Debian Etch

July 2nd, 2007 No comments

Copied from Andreas Gohr:

If you do a lot of very different projects like we do here at CosmoCode you often need very different environment to test new technologies without interfering with other setups. But using new servers all the time isn’t feasible at all.

The solution to this is virtualization. It allows you to run multiple completely independednt systems on the same hardware. We decided to go this way and bought I nice big server to do so. For virtualization we use the open source software XEN.

In the following I will give you some details on how to setup logical volume management (LVM), a XEN “server” (dom-0) and a XEN “client” using Debian’s soon-to-be-stable release “Etch”.

Before we continue make sure you have the LVM tools installed:

#> apt-get install lvm2

Getting LVM running is quite simple: partition your harddisk as usual for the host (master) system and create an unformatted partition for management by LVM. For example I created a 1GB swap disk at sda1, a single 10GB root partition at sda2 for the master system and put the rest into sda3.

The next step is to prepare the partition for LVM (think of it like creating a LVM fileystem). This is done with pvcreate.

#> pvcreate /dev/sda3

To use this physical device it needs to be added to a volume group first. Volume groups can span multiple physical devices, but we have only one here. The first parameter to vgcreate is the name of the group. I used the name “vserver” here.

#> vgcreate vserver /dev/sda3

Finally we could create a new virtual device using space from the volume group. This is not needed right now because it will be done later in the xen setup, but for completeness sake I will mention it here. Just use give the size in megabytes, the name of the new device and the volume group name to lvcreate. Then createa filesystem as usual and mount your device:

#> lvcreate -L1500 -ntestdevice vserver
#> mkfs.ext3 /dev/vserver/testdevice
#> mount /dev/vserver/testdevice /mnt

Okay, now we are ready to install the XEN stuff. Fortunately all needed software is in Debian etch already:

#> apt-get install xen-linux-system-2.6.18-3-xen-686 xen-tools
xen-docs-3.0 libc6-xen file xen-hypervisor-3.0.3-1-i386-pae
xen-linux-system-2.6.18-3-xen-vserver-686 bridge-utils

This will install the xend daemon and kernels for the master and the clients. Before the reboot, a small change in /etc/xen/xend-config.sxp should be done.

Replace

(network-script network-dummy)

with

(network-script ‘network-bridge netdev=eth0’)

Now reboot and check uname -a – it should display the XEN enhancement:

Linux hydra 2.6.18-3-xen-686 #1 SMP Mon Dec 4 20:48:20 UTC 2006 i686 GNU/Linux

Now it’s time to create a new client system (called domain in XEN lingua). This is easily done with the xen-create-image script, you should configure some defaults in /etc/xen-tools/xen-tools.conf first. Here are my settings:

lvm = vserver
debootstrap = 1
size = 10Gb
memory = 256Mb
swap = 512Mb
fs = ext3
dist = edge
image = sparse
kernel = /boot/vmlinuz-2.6.18-3-xen-vserver-686
initrd = /boot/initrd.img-2.6.18-3-xen-vserver-686
mirror = https://ftp.de.debian.org/debian/

Make sure you configured the LVM volume group and the kernel/initrd images correctly here.

Now install your new vserver by calling

#> xen-create-image –hostname vserver1.mydomain

The script will download the needed packages, install them in newly created logical devices and does create an initial config file in /etc/xen/vserver1.mydomain.cfg

To boot the system use the mighty xm command which is used to control all the domains.

#> xm create /etc/xen/vserver1.mydomain.cfg

You can now connect a virtual console to the just booted system with

#> xm console vserver1.mydomain

Use this to set up a root password and other intitial things, to leave the virtual console use CTRL-]. Note this is not a full terminal, log into your new xen domain with SSH to run fancy stuff like ncurses interfaces.

Okay, that’s it. This is the very simplest and basic setup, but should get you going.

Categories: Uncategorized Tags:

LVM Recovery Tale

June 30th, 2007 1 comment
LVM recovery tale quoted from Gnomedia Codeworks:

Over the weekend I had the worrying experience of losing my LVM settings and potentially all my data! A quick search on the web showed a confusing set of information, much of it for older versions of LVM and therefore rather suspect.

Well, I recovered all my data and it was really quite simple, so I’ve written up what I did in the hope that someone else, in a similar situation, will find it useful. It’s a scary thing, losing the whole hard disk and knowing that, in reality, its all there.

First the situation

I’ve got a small /boot partition as ext2, and a larger one for the root directory (2Gb, also ext2). The rest of the hard disks (nearly 120Gb) are assigned to a volume group called, descriptively, system (which is SuSE’s idea of a default name).
More accurately, they were supposed to be assigned to it. At first I had just added 60Gb to the volume group, it was my first use of LVM and I was hedging my bets. After 6 months of trouble free operation I decided to add another 60Gb of disk, which I did 3 months ago. Except that, although the physical and volume group managers all agreed that the volume group had 120Gb, the logical volume manager insisted that there was only 60Gb. I’d used Yast2 to create and add the volumes.
I tried every combination of commands I could think of to get the logical volume manager to recognize the additional space but it wouldn’t.
At the time, I was busy, so forgot about it, then last week I realised that I wanted to use the space so settled down to do something about it.

The problem

So, it seemed the best solution would be to remove the second partition that I had added (/dev/hdd1) from the physical volume manager and then add it back.
It wasn’t recognised so wouldn’t be missed, right?
Wrong!
pvremove /dev/hdd1 removed the label from /dev/hdd1 but also from /dev/hda7 (which was the original partition and full of data).
pvscan and pvs reported no physical volumes on the disk.
vgscan and vgs couldn’t find any volume groups.
lvscan and lvs were non-starters obviously.

The rather surreal thing was, the whole system kept on running quite nicely, X Server and KDE desktop and all, but I knew that as soon as I rebooted the system would be toast.

First I tried adding the partition back to the volume group system, but the system couldn’t find the ‘system’ group. I tried creating the physical volume again (pvcreate) but that told me that the volume already existed. It became clear that I would need to reboot and hope that the system sorted itself out, flushed the disks, resynced, whatever.

The solution

After rebooting the system wouldn’t come up, which is kind of what I had expected so I had to reboot from the SuSE Rescue disks. So now I had to think about how to recreate the physical volumes, volume group and logical volume and do it with the data intact. (I have daily backups but the thought of restoring the whole system, applications and data, was not too exciting, especially as I knew all the data was there and intact. With a ‘regular’ hard disk partition that had got lost I could scan the disk for potential disk partitions and restore them. But that wouldn’t work with LVM.

On a search through various sites, I found one that mentioned the importance of saving a copy of the volume group parameters to a file using vgcfgbackup. This file could then be used to restore the parameters later, assuming that the underlying physical structure hadn’t changed. Well, the physical layout hadn’t changed but unfortunately I hadn’t created a backup of the volume group parameters (the ‘descriptor area’ to use the technical term) so that didn’t seem to hopeful. I poked around in the /etc directory (I still had the ‘/’ partition remember, as that was on its own ext2 partition) and noticed that there was a /etc/lvm/backup/ directory and a /etc/lvm/archive/ directory. Further investigation and I found that these are automatically created by LVM whenever changes are made to the system.

Unfortunately, all the messing around I had done had created a non-working version of the system file and the archive files didn’t seem to be recent enough. But, I remembered that I had a backup of the system files (going back 6 months in fact) and so I dug out a copy of the /etc/lvm/backup/sysem file and used that.

Here is what I did: First find out the old UID’s of the partitions, this is in the /etc/lvm/backup/system file. They are quite long; make sure you get the UID for the physical volumes.
$pvcreate -u sdSD-2343-SD939-adIda2 /dev/hda6
$pvcreate -u dk33kd-929293nd-adfja298a /dev/hdd1
$vgcreate -v system /dev/hda7 /dev/hdd1
$vgcfgrestore -f /etc/lvm/backup/system system

and lo!, all data present and correct!

In fact, I just rebooted the system and was back where I had started with the additional benefit of an extra 60Gb of disk space, because now I had the extra partition properly included.

[Note: in the lines using pvcreate… above I could have used:
$ pvcreate -restorefile /etc/lvm/backup/system
to automatically find the ID’s but I hadn’t realized that at the time. Without the UID’s then the vgcfgrestore will not find the physical volumes that it needs to recreate the volume group.]

The lesson

Don’t panic!
Keep a safe copy of your /etc/lvm/ files!
Make sure that you have a Rescue disk that understands the LVM system!

Apart from the above disaster, which seems to have sorted itself out very easily, I have had no trouble with the LVM system. At first I was worried that if there was a failure it would lose everything. There is something very comforting about a simple ext2 (or FAT) partition in that I know it can just be hacked at the bit level and rebuilt. Something like LVM, which is logical volumes on top of volume groups on top of physical volumes is impossible to rebuild ‘by hand’ so I’m learning to trust technology a bit.

End quote

Categories: Uncategorized Tags:

Mount from Image with Multiple Partitions

June 30th, 2007 No comments

Quoted from Jeffrey Fernandez:

If you have played around with virtualisation technologies, specifically Xen, then you may have experienced creating a disk image. To create a disk image on Linux, I execute the following command:

dd if=/dev/zero of=OpenSuse-10.2.img bs=1024k count=0 seek=5000

This would have created for me an image which is approximately 5GB in size. And then I format it to an ext3 file-system with:

mkfs.ext3 OpenSuse-10.2.img

If I now run a file command against this image, it should tell me what type of file-system it is.

file OpenSuse-10.2.imgOpenSuse-10.2.img: Linux rev 1.0 ext3 filesystem data (large files)

Thats all fine when there is a single partition for your disk image. Recently I happened to encounter a problem whereby there was more than a single partition on an image.

Enter “Xen Virtual Machine Management”

I recently have been playing with the “Xen Virtual Machine Management” which is available in OpenSuse 10.2, and it totally rocks. It automates the process of creating you the configuration file for your Xen VM along with the disk images and the final VM without having you to type anything on the command line. Not that I don’t like the command line but I was just pointing our that OpenSuse developers have done a great job with the Xen Manager.

After helping you create the necessary configuration, you can start installing your VM through a little xterm window. This is not one of those “bootstrap” scripts working in the background, but a full OpenSuse Operating System install, just like how you would do when you install an Operating System on your desktop, except this is in text mode.

Now because you are creating a full Operating system installation for your VM, you can basically go through the installation process and create your partitions for your different mount points, select the required packages to be installed and so on. So in the end you end up with a disk image with multiple partitions in it. This is where I encountered the problem. Normally when you have a single partition as show in our earlier example, you can mount the image through a loop-back device like shown below:

mount -o loop OpenSuse-10.2.img mnt

But since the image which gets created during the installation process of the VM, has multiple partittions, a simple mount won’t work. This is what happens when I try to mount an image (hda is the image) which has multiple partitions:

mount -o loop hda mntmount: you must specify the filesystem type

And if I run the same file command on this image file, the following is the output:

hda: x86 boot sector; partition 1: ID=0x83, active, starthead 0, startsector 1060290, 16787925 sectors; partition 2: ID=0x83, starthead 254, startsector 17848215, 3116610 sectors

As you can see within the output, its lists two partitions for the image. We can look at the detailed listing by executing:

fdisk -lu hda

Which outputs:

You must set cylinders.
You can do this from the extra functions menu.

Disk hda: 0 MB, 0 bytes
255 heads, 63 sectors/track, 0 cylinders, total 0 sectors
Units = sectors of 1 * 512 = 512 bytes

Device Boot      Start         End      Blocks   Id  System
  hda1   *     1060290    17848214     8393962+  83  Linux
Partition 1 has different physical/logical endings:
     phys=(1023, 254, 63) logical=(1110, 254, 63)
  hda2        17848215    20964824     1558305   83  Linux
Partition 2 has different physical/logical beginnings (non-Linux?):
     phys=(1023, 254, 63) logical=(1111, 0, 1)
Partition 2 has different physical/logical endings:
     phys=(1023, 254, 63) logical=(1304, 254, 63)

The “u” flag within the fdisk command gives the partition tables sizes in sectors instead of cylinders. This is important for us as we will be using that information for some calculation. Also important to us is the “Bytes/sector” values

Calculate the Offset in “Bytes”

Now in order to mount the two partitions from the image file, we need to know the starting Byte from which each partition starts. To calculate this offset, we mulitply the (start_sector * sector_byte_size). So for the Partition 1 we have (1060290 * 512) = 542868480 and for Partition 2 we have (17848215 * 512) = 9138286080. I pretty well know that the Partition 1 was the root partition and Partition 2 was the Home partition so I will mount them into appropriate folder names.

Now we can use those “Start Byte” values to mount those partitions in two seperate locations:

mount -o loop,offset=542868480 hda rootmount/
mount -o loop,offset=9138286080 hda homemount/

Categories: Uncategorized Tags:

Disk Cloning and Backup

June 29th, 2007 No comments

The old standby to clone a partition:
dd if=/dev/hda1 of=/output/file
Apparently a newer tool, dcfldd, does a better job.

Much smaller though not an “image” copy:
tar zcvpf backup.tgz --exclude=dev --exclude=sys --exclude=proc --directory=/ .
And if you don’t want to create the intermediate backup file:
tar zcp --exclude=dev --exclude=sys --exclude=proc --exclude=/this-folder --directory=/ . | tar zx
Note: /sys, /proc and /dev are rebuilt as linux boots and in the no-backup version, you must exclude the destination folder (or the copy never ends).

Categories: Uncategorized Tags:

Migrate CentOS-4.4 to Xen

June 23rd, 2007 No comments

Boot Knoppix
In a root shell window:

vgchange -ay							# activate LVM

e2fsck -f /dev/vg00/lv00

resize2fs /dev/vg00/lv00 9G					# resize ext3 fs

lvreduce /dev/vg00/lv00 -L9G					# resize logical vol

Reboot host OS

mount xen2.neoxo.com:/share /mnt/share				# mount nfs share

lvcreate -s -n backup -L9G /dev/vg00/lv00			# create lvm snapshot

mount /dev/vg00/backup /mnt/backup

tar zcvpf /mnt/share/app2-backup.tgz -directory=/mnt/backup .	# tar lv backup to nfs

On Xen host

lvcreate -n app2.swap -L2G vg0

mkswap /dev/vg0/app2.swap

lvcreate -n app2 -L9G vg0

mkfs.ext3 /dev/vg0/app2

mount /dev/vg0/app2 /mnt/app2

cd /mnt/app2

tar zxvf /share/app2.tgz

In the VM do the following

Copy /boot/* from another CentOS-4.4 VM

rename/remove /etc/sysconfig/hwconf

change entries in fstab to use scsi (sda) instead of lvm

To prepare /etc/xen config files

Copy and modify a working config, or read the Xen docs

To reboot VM automatically

Create a link to the appropriate /etc/xen/xxx.conf in /etc/xen/auto/
Categories: Uncategorized Tags:

A Mac-like Rails Environment

June 23rd, 2007 No comments

While I use eclipse and the single-click Ruby with Rails under Windows this may be more to your liking:

https://garbageburrito.com/blog/entry/391/a-macesque-rails-development-environment-on-windows

Categories: Uncategorized Tags: