Converting a virtual disk image: VDI or VMDK to an ISO you can distribute | TurnKey GNU/Linux
You are here
Category: All
Converting a virtual disk image: VDI or VMDK to an ISO you can distribute
Blog Tags:
virtualbox
convert
vmware
Converting a virtual disk image: VDI or VMDK to an ISO you can distribute
Alon Swartz
- Thu, 2010/02/11 - 09:21 -
19 comments
Why would anyone in their right mind want to convert a VM into an ISO?
Good question, the answer for Conor Fox (who was the inspiration for this post - thanks Conor!) was to distribute his customized TurnKey PostgreSQL image so others could use it.
Distributing an ISO as opposed to a VM image allows it to be installed on any virtualization platform, as well as on bare metal, with the added bonus of running live.
I suppose that's a good enough reason, so lets get to it.
Convert VM disk to raw image and mount it
First we need to get qemu-img, a tool bundled with
qemu
(KVM's virtualization backend) to convert the VM disk to a raw image, and
TKLPatch
, the TurnKey customization mechanism to package the ISO.
If you are not using a TurnKey installation, see the
TKLPatch installation notes
apt-get install qemu
apt-get install tklpatch
I'll show how to convert a VMWare VMDK image into raw disk format. If you are using a different virtualization platform such as Virtualbox, see
this post
on converting a VDI to a raw image.
qemu-img convert -f vmdk turnkey-core.vmdk -O raw turnkey-core.raw
Next, mount the raw disk as a loopback device.
mkdir turnkey-core.mount
mount -o loop turnkey-core.raw turnkey-core.mount
GOTCHA 1
: If your VM has partitions, it's a little tricker. You'll need to setup the loop device, partition mappings and finally mount the rootfs partition. You will need kpartx to setup the mappings.
loopdev=$(losetup -s -f turnkey-core.raw)
apt-get install kpartx
kpartx -a $loopdev
# p1 refers to the first partition (rootfs)
mkdir turnkey-core.mount
mount /dev/mapper/$(basename $loopdev)p1 turnkey-core.mount
Extract root filesystem and tweak for ISO configuration
Now, make a copy of the root filesystem and unmount the loopback.
mkdir turnkey-core.rootfs
rsync -a -t -r -S -I turnkey-core.mount/ turnkey-core.rootfs
umount -d turnkey-core.mount
# If your VM had partitions (GOTCHA 1):
kpartx -d $loopdev
losetup -d $loopdev
Because the VM is an installed system as opposed to the ISO, the file system table needs to be updated.
cat>turnkey-core.rootfs/etc/fstab<
tmpfs /tmp tmpfs nosuid,nodev 0 0
EOF
GOTCHA 2:
If your VM uses a kernel optimized for virtualization (like the one included in the TurnKey VM builds), you need to replace it with a generic kernel, and also remove vmware-tools if installed.
tklpatch-chroot turnkey-core.rootfs
# inside the chroot
apt-get update
apt-get install linux-image-generic
dpkg --purge $(dpkg-query --showformat='${Package}\n' -W 'vmware-tools*')
dpkg --purge $(dpkg-query --showformat='${Package}\n' -W '*-virtual')
exit
Generate the ISO
Finally, prepare the cdroot and generate the ISO.
tklpatch-prepare-cdroot turnkey-core.rootfs/
tklpatch-geniso turnkey-core.cdroot/
Thats it!
Bonus: By default the ISO will boot automatically. If you want to include the
TurnKey bootsplash and bootmenu
, extract the cdroot from a TurnKey ISO and tell tklpatch-prepare-cdroot to use it as a template.
tklpatch-extractiso turnkey-core.iso
tklpatch-prepare-cdroot turnkey-core.rootfs/ turnkey-core.cdroot/
tklpatch-geniso turnkey-core.cdroot/
Ever needed to package a VM as a distributable ISO?
Post a comment
Comments
Virtual kernel does not include squashfs
Alon Swartz
- Tue, 2010/02/16 - 17:39
The virtual kernel does not include squashfs support, so the resulting ISO won't be able to unpack the root filesystem, and the boot will fail.
Also, an ISO can be installed to bare-metal, so it doesn't make much sense to include the optimized virtual kernel even if it was possible (to be complete, it is possible with a couple of workarounds but not worth the effort, and possible side-effects).
reply
Process should be very similar
Alon Swartz
- Sat, 2010/02/20 - 19:50
I suppose (without thinking about it too much) that the process should be very similar, minus the VM conversion.
In general it should go something like this:
Mount the secondary drive/partition, and copy out the root filesystem
Tweak as needed for ISO configuration
Generate the ISO
I would be interested to know if you try this, and what the outcome is. Keep us updated.
reply
As the tutorial says...
Jeremy Davis
- Tue, 2011/02/15 - 12:49
Once you have created the .raw file you then mount it (like you would an .iso) and copy out the contents (which should only be ~1.7GB - ie your original data size). Your resulting .iso should be around that size too. If not then I would assume there is something funny/wrong going on.
reply
No sorry, got no idea.
Jeremy Davis
- Sat, 2011/04/02 - 02:10
Perhaps you could try RemasterSys in the meantime? Although TBH I haven't used it with a server (but it's been quite useful for desktop use).
PS Did you document your iFolder progress? It'd be great if you could share as I'd love to create an iFolder TKLPatch and then perhaps an official TKL iFolder appliance might be released.
reply
Excellent, thanks.
Jeremy Davis
- Thu, 2011/04/07 - 04:18
I will investigate further. When I get a chance I will look at perhaps uploading it to a PPA. I'll post if/when I get to that.
reply
The ISO file you created includes free space
Jeremy Davis
- Sun, 2011/05/22 - 08:33
That's why it's larger than your original VDI. And actually it's a RAW file (which is a raw harddrive image). If you follow the tutorial it is only halfway through the process. To complete it you need to mount the image you have created and copy out the files, then make an ISO of the files only (that doesn't include the free space). This should bring your ISO back down to a similar (or perhaps smaller) filesize as your original VDI.
reply
Try this:
Jeremy Davis
- Tue, 2011/06/07 - 15:06
Just guessing but try:
tklpatch-prepare-cdroot Mikrotik
(No trailing slash). The clue for me was the error message (unless it's just a typo):
cp: omitting directory `Mikrotik
//
boot/'
Although must admit that the tutorial does include the trailing slash so not sure if that's it.
reply
Hmmm beyond me...
Jeremy Davis
- Tue, 2011/06/07 - 16:56
This tutorial works with TKL and I would imagine any Ubuntu or Ubuntu based distro. It may even also work on Debian - but perhaps not...
A quick scan of the second link you provided suggests that that particular version is a mishmash of 2 old versions of Debian but I couldn't quite work out what the go was with the new version. The fact that searching/scanning the website, forum and wiki resulted in no idea on what the OS is based on or what sort of licence it's under doesn't bode well (although perhaps I'm wrong). It seems quite closed and as such I'd suspect that the only way you will be able to do what you are hoping to is with the help and/or support of the makers/community. Have you posted on their forums?
But TBH unless there is a compelling reason, personally I'd be looking about for an opensource product that does the same thing thgat does support hacking etc. I haven't tested any but
Wikipedia has a list of starters
. No doubt there is probably more if you have a good google. My 2c anyway...
reply
True regarding the opensource
Andreas
- Tue, 2011/06/07 - 23:07
True regarding the opensource firewalls distros.
Except I've been through ALL of THEM. :) Seriously, I'm running on Zentyal at my office at the moment, but I've been through Smoothwall, pfSense, ClarkConnect, IpCop, IPFire, etc etc etc.
//disclaimer - I am not affiliated with Mikrotik in any way whatsoever. In fact, they would probably look down on what I am trying to do by creating a LiveCD.
Mikrotik is absolutly the best of the best. Its a bit funky to get started with, ( I was actually planning on running pfSense before I read a single post from a random comment on the pfSense forums that said "Try Mikrotik, once you learn it, you will never go back." I spent last weekend learning it, and without a doubt, it frakking rocks.
Just to get started, a web interface and windows interface for the firewall.. Why would you need the windows interface? Well, for starters, it allows you to connect to the firewall to reconfigure it regardless of your machines IP settings. It connects via MAC address, TCP/IP4 or TCP/IP6 as well as autoscans the local network so a freshly installed firewall is accessible without even touching the console.
The 24 hour time limit on the demo is a bummer but you can backup your config, reformat , and reinstall your config to continue using it or learning it. As well as their licensing is confusing and a bit expensive for SOHO usage.. but considering you can buy some of the RouterBoards including the license for less than the licensing alone, I'm not sure where I stand on that.
I highly reccomend it if you do any networking at all, or if you just want something fun to play with.
reply
I don't think this is going to work for you
Jeremy Davis
- Wed, 2011/06/08 - 16:26
AFAIK unless the original distro supports running live then this will not work for you to create a live distro.
I'm sure creating a live version of the router OS is possible, but not without some serious hacking. The TKL devs have hacked the default Ubuntu Server OS so it can run live so unlike default Ubuntu Server TKL can run live already. I imagine you would need to do similar hacking first to allow this distro to run live. How you would actually go about that is well beyond me and may also be outside the terms of the Mikrotik licence.
As you've possibly guessed from my previous posts I'm a strong advocate of the open source model. But I also try to be respectful of the relevant licencing. If you like Mikrotik RouterOS that much I suggest you consult with them on what your options are and what it will cost you to do what you want.
reply
obviously it depends on the size of the image
Jeremy Davis
- Tue, 2011/06/14 - 06:35
But if it's a big image, yes it will take a while. A good baseline timeframe to consider is how long would it take to copy the file? Then add a bit (depending on hardware). You can see if it's still doing something via a new terminal window. I don't remember the commands off the top of my head but if you google "check progress qemu-img convert commandline linux" or something similar. I haven't actually done it with qemu-img but I am assuming it would be similar to checking progress of dd.
reply
Don't think you'll have any joy with Win7
Jeremy Davis
- Sat, 2011/12/24 - 03:09
vfat should be a recognised format and depending on what distro you are using, you may need to instal ntfs drivers before it will recognise NTFS FS.
Regardless of that though, because of the nature of how Windows works, I don't think you'll have any joy using this method anyway. Even if you do manage to convert your VM to an ISO I imagine that you'll find that trying to run it on anything that doesn't closely resemble the virtual hardware of your VM will result in a BSOD.
I'm guessing you are trying to create a Win7 ISO with preinstalled apps and/or customised config? I don't know enough about Win7 to give you any really good suggestions, but if you are looking to create a preconfigured image then the MS tool sysprep is probably your best bet and then create an image of your syspreped system. If you hunt about online I'm sure you'll find tons of info detailing the best course of action.
reply
No worries
Jeremy Davis
- Sat, 2011/12/24 - 04:31
Another afterthought may be vLite. AFAIK it is the successor to nLite which was a quite cool way of creating a customised XP install CD (vLite is for Vista & 7 IIRC).
I have successfully been using a combination of sysprep and a FOG server to create and deploy Win7 images to multiple machines. It may be overkill (or simply not suitable) for your purposes but it works really well for mine! Possibly worth a look...?
reply
If you are using TKL then it should be installable
Jeremy Davis
- Wed, 2012/04/04 - 02:03
And if not (but still something Linux) it should be possible, although to be honest I have no idea how you would go about that. I would suggest that you post on the forums/mailing list for whatever distro you're using and ask there. That way hopefully you will get some answers relevant to your scenario.
reply
That's definitely one way
Jeremy Davis
- Sat, 2013/03/02 - 05:18
Although I haven't tried it on a server OS. Also I didn't realise that it was back up and running (IIRC it died some time ago, but obviously got revived somewhere along the line) so thanks for that info.
reply
This blog post is very old
Jeremy Davis
- Fri, 2014/02/07 - 23:16
I suspect the issue is that TKL now uses LVM by default. Unfortunately I am not sure on a workaround.
reply
Is the OS Debian or a derivative (e.g. Ubuntu)?
Jeremy Davis
- Wed, 2015/04/22 - 11:31
If so then that would be the problem...
Otherwise that's really weird! According to
Debian
dpkg-divert is provided by the dpkg package. It is a fundamental part of Debian (and it's derivative's) package management system and should certainly be installed!
reply
These instructions are really old...
Jeremy Davis
- Thu, 2016/07/28 - 03:00
These instructions are really old so may need some updating. IMO they could certainly be improved!
Anyway, it's clear that for whatever reason your image isn't mounting properly. Does it have partitions? To see what is going on, try this:
kpartx -l turnkey-core.raw
If it is partitioned you should get something like this:
# kpartx -l turnkey-core.raw
loop0p1 : 0 9765 /dev/loop0 1
loop0p2 : 0 26624 /dev/loop0 12288
loop deleted : /dev/loop0
You can then use kpartx directly to mount the image as a loopback:
kpartx -a turnkey-core.raw
The partitions will be as per the output when the -l switch was used. I.e. /dev/mapper/loop0p1 will be the first partition and /dev/mapper/loop0p2 is the second. So you then use mount to mount /dev/mapper/loop0p1 to turnkey-core.mount. When you are done, umount turnkey-core.mount and delete the loopback device with kpartx:
kpartx -d turnkey-core.raw
Actually I just noticed that you are trying to mount /dev/mapper/loop0p
(lower case 'L') whereas it should be /dev/mapper/loop0p
(number '1'). Perhaps that's your only issue?
reply
I doubt it...
Jeremy Davis
- Thu, 2021/07/01 - 04:49
This is pretty dated now and was Linux specific. I'm not even sure that it would still work with Linux, let alone an alternate OS. So I doubt that it would work with Mac OSX, but you could try and see how you go?!
reply
Pages
Add new comment
Apps
Specials
Web development
Framework
Stack
IT Infrastructure
Content management
Blogging
Ecommerce
Education
Wiki
Media
Business management
CRM
ERP
Invoicing
Messaging
Email
Forum
Chat
Issue tracking
Project management
Database
NoSQL
Developer tools
Help
Forums
Support
General
Development
Documentation
Security and News Announcements
Low-traffic newsletter: up to one email a month.
Previous issues
Categories
development
news
appliances
community
debian
release
hub
stable
iso
security
cloud
ec2
aws
proxmox
lxc
ubuntu
tkldev
tips
v16.x
drupal
More tags
Recent posts
Free up disk space
4th Aug, 2024
Python PEP 668 - working with "externally managed environment"
29th Jul, 2024
v18.0 Stable Release #6 - 10 Newly Updated ISOs, Hub Builds & Proxmox/LXC builds
17th Apr, 2024
v18.0 Stable Release #5 - 20 Updated ISOs & Hub Builds - Proxmox/LXC builds
12th Feb, 2024
v18.0 Stable Release #4 - 10 Updated ISOs & Hub Builds - Proxmox/LXC builds
5th Feb, 2024
1 of 63
next ›
Archive
August 2024
(1)
July 2024
(1)
April 2024
(1)
February 2024
(2)
November 2023
(1)
October 2023
(1)
September 2023
(1)
July 2023
(1)
April 2023
(1)
March 2023
(1)
December 2022
(1)
November 2022
(2)
Pages
Recent comments
V19?
TurnKey LXC templates are available via the Proxmox UI
Non-Aws platforms
Thanks for that!
the command for adding the
mtoolshub
Great post
randomness