[author: Alex]
I discovered Bitnami a while back, but never took the time to try out their virtual appliances.
I recently stumbled on them again and decided to open up the Bitnami ownCloud virtual appliance, to compare it with a Jidoteki ownCloud virtual appliance I created just for testing. Here’s our post about a RPi ownCloud we built a while back. It’s almost identical to the x86_64 version.
Comparing the VMs
1. The first thing I noticed were the file sizes:
Bitnami: 448MB compressed, 1.5GB uncompressed (on disk)
Jidoteki: 90MB compressed, 141MB uncompressed (on disk)
That’s a 10x difference in disk usage for a VM that does the exact same thing.
2. Something interesting about the disks:
Bitnami: 1 x 17GB disk (sda1) w/ext4
Jidoteki: 1 x 8GB disk (sda1) w/ext2, 1 x 250GB disk (sdb1) w/ext4 + LVM
I thought this Bitnami disk setup was weird, since ownCloud can easily require a lot of disk space for storing its data - why doesn’t the the VM ship with a 2nd disk by default? Jidoteki includes a 2nd disk setup with LVM, and automatically expands on boot when the user attaches a 3rd, 4th, 5th, or even 6th disk. Of course that is entirely configurable.
3. Network adapters
Bitnami: PCNet32 (AMD PCnet-PCI II) in Bridged mode
Jidoteki: E1000 (Intel Pro/1000 MT Desktop) in NAT mode
I guess this doesn’t matter so much, since the user can change it themselves, but the pcnet32 adapter is a 10mbit adapter, compared with the e1000 which is 1gbit. Bridged mode by default will automatically make the VM accessible on the local network - before it has a chance to be configured. I think that’s a security risk, albeit a rather small one, at the expense of convenience - which is something I usually frown upon. We set ours to NAT by default, which limits access to the user’s local computer - default “secured” (I know, I know, NAT is not security..).
4. VM import and boot times were also quite interesting:
Bitnami: 5mins import, 5mins boot
Jidoteki: 30seconds import, 30seconds boot
Import and boot were done with VirtualBox on my MacBook Air. Both will likely be much faster using VMware, or a faster computer. I’m not sure what the Bitnami appliance was doing behind the scenes during the boot process, since all output is suppressed early in the boot process.
Verdict: Jidoteki VMs are better
Ideas for Bitnami:
1) There is almost no output from the moment it boots until the login screen with the Bitnami banner. This made me question whether the boot was working or not.
2) The WRITE SAME error message is common for virtual machine disks. It’s not harmful, but scary. We suppress it with this code early in the boot process:
# Suppress WRITE SAME messages
disks=`find /sys | grep max_write_same_blocks`for i in $disks; do
[ -f “$i” ] && echo 0 > “$i”
done
3) Replace GRUB with Syslinux for faster boot.
4) Setup LVM by default, so the disk can be expanded in the future. LVM root is sometimes challenging, and quite difficult to recover, so perhaps follow our approach instead ;)
Comparing the VM contents
To compare the contents, I mounted the VMDK disk on a Linux system with the help of qemu-nbd.
1. Too many files
Bitnami: 67123 files and directories
Jidoteki: 11 files and directories
No wonder the Bitnami OVA is so big, it has way too much stuff! ;)
Haha ok, so we cheated here. In fact the rootfs on the Jidoteki OVA contains compressed squashfs packages (TCZ extensions) which hold all the files for specific applications. This follows the TinyCore Linux architecture, which allows us to have only a few files on disk representing the entire OS, libraries and applications.
2. Open Source licenses
Bitnami: 77 files found in /opt/bitnami/licenses/, 0 NOTICE file
Jidoteki: 8 files found in /usr/share/doc/License/, 1 NOTICE file
Bitnami does a remarkable job of including all licenses for every Open Source software used in the appliance. Unfortunately, there’s no indication on where to obtain the sources. For the record, that’s not good enough for GPL compliance, and it’s useless for people who want to modify the software or to see what patches were applied. The Jidoteki appliances include a NOTICE file in the License directory, with a URL to download all the individual sources, patches, and build scripts, as well as an ISO containing all the sources. Only the GPL requires that, but we do it for all the Open Source software we include.
3. Namespacing and separation
Bitnami: All changes namespaced to /opt/bitnami
Jidoteki: All changes namespaced to /opt/jidoteki, except some things
Jidoteki scatters OS customizations all over the place and it’s a bit of a mess, but there’s one important difference: All OS customizations are contained in the rootfs. The base OS (corepure64) is completely unmodified except for the kernel modules which were updated. This is quite different from Bitnami, which is a full OS installation. That means on the Bitnami appliance it’s impossible to determine what parts of the OS were customized, and what shipped by default with the OS.
4. OS File and directory sizes
Bitnami: ~800MB full OS, full documentation, full kernel modules and headers, unstripped binaries, perl and python libs (probably required for Ubuntu).
Jidoteki: ~12MB bare minimum OS, no documentation, optimized VM kernel modules, no headers, stripped binaries, no perl or python libriaries.
The OS used in Jidoteki appliances (TinyCore Linux for now…) is ridiculously small and lightweight. They made a great effort to cut out the cruft and leave us with a fully functioning system. The Bitnami OS seems to be completely un-optimized and accounts for half the disk space used in the appliance. That becomes an issue when you want to run multiple appliances side-by-side.
5. App Files and directory sizes
Bitnami: ~636MB for Bitnami, the ownCloud app, and its dependencies
Jidoteki: ~125MB for Jidoteki, the ownCloud app, and its dependencies
Bitnami does something interesting here: they include a common directory of shared libraries, locales, and other files which use up 100MB, and which are already included with the OS. It seems they duplicated those files to maintain control of shared libraries independently of the base OS. They also include the full Apache and PHP applications, which also use nearly 100MB. Jidoteki includes a stripped down version of Nginx and PHP instead, which only require 8MB.
Verdict: Jidoteki VMs are better
Ideas for Bitnami:
1) Put more effort into trimming down the base OS. All those libraries and binaries are typically not needed, or are only needed for one specific system. Ideally, you could switch to a smaller base OS which already has everything stripped down - but that’s a lot of work. It’s typically easier to refactor than rewrite ;)
2) Consider open sourcing and distributing the actual sources of your appliances, and separating the Bitnami-specific files from the default Ubuntu files. I understand they are probably all just unmodified base files/packages from Ubuntu, but that doesn’t help me know what changed in the OS you provided.
Comparing security and updates
It seems Bitnami does a great job of keeping on top of security issues. Their changelogs appear to be automatically generated, but they are frequent and quite well up-to-date, so that’s good.
1. Updates to the base OS
Bitnami: looks like a simple apt-get upgrade combined with a Bitnami-specific upgrade process
Jidoteki: Jidoteki-specific offline upgrade process
The Bitnami upgrade process seems quite simple, but a lot is left in the hands of the user. The appliance itself must be maintained by the user, requires internet for updates, and can disrupt ownCloud if something breaks. With Jidoteki, we handle the maintenance of the entire OS, along with kernel and CVE security updates. A user only needs to upload our update package to the appliance, and it will automatically update the files on the boot disk. The process is atomic and almost guaranteed to work. A reboot is required to boot the OVA with the latest version.
2. Updates to ownCloud
Bitnami: lots of manual steps and DB migration stuff (occ upgrade)
Jidoteki: automatic with auto occ upgrade on reboot
We automated the entire ownCloud app upgrade process, including the DB migrations. This was necessary because ownCloud does things a bit differently. Bitnami leaves it to you with a set of outdated instructions to follow. Fingers crossed if you’re successful on the first try ;)
3. Internet updates
Bitnami: yes
Jidoteki: no
We aim for our appliances to run behind the firewall, and in fact they don’t even require internet at all. You can literally plug in a USB key with an update package, and then use that to perform an upgrade of a Jidoteki appliance. Of course we have a method of performing online internet updates as well, but that’s not the default. Bitnami defaults to online/internet updates because it’s more convenient. Again I think that’s not the best approach, but probably necessary considering their market.
4. “The user can fudge the appliance factor”
Bitnami: oh yes
Jidoteki: nope
With Bitnami, the user has full root access to the appliance, and can pretty much do anything - including break it. Sure it’s the user’s fault if they do, but it significantly increases their support workload from people who have no idea what they’re doing. Jidoteki targets enterprise installations. We make sure to run the OS in memory only, and to have a read-only system to prevent “accidental” changes. This is by design in TinyCore Linux, which is another reason we use that as the base OS.
Verdict: Jidoteki VMs are better
Ideas for Bitnami:
1) Consider a read-only OS to prevent users from making unexpected changes. Some parts of the system can be read/write, but that should be reserved for configuration files and “data”, not libraries and binaries. If someone wants to install their own version of Apache, they should do it on their own Linux installation.
2) When updates are not atomic, bad things can happen. It seems like a repeat of the previous point, but if the user makes a small system change, and applies an update that fails, they could lose all their data - which is a terrible situation.
Comparing system management
This is the last point, thanks for sticking through to the end.
1. Network settings management through the console
Bitnami: login as bitnami, you’re on your own
Jidoteki: console gui to configure basic network settings
When people first boot an appliance, the first thing they will need to do is configure the network settings, perhaps either change the IP or specify different DNS servers. Jidoteki appliances always have a simple console boot gui to change those settings - which doesn’t require a login or typing obscure commands.
2. App and other settings management
Bitnami: login as bitnami, use ctlscript.sh or other obscure commands
Jidoteki: web Dashboard and REST API for system management, or obscure commands haha
Bitnami appliances don’t include a Management Dashboard, which is unfortunate. They do provide phpMyAdmin and SSH, but everything else seems to be command-line based. Jidoteki includes a user-friendly web Dashboard and a dev-friendly REST API for system management. It’s a bit limited in what it can do, but the basics are there (seeing service status, appliance usage graphs, etc). It’s sufficient for a first-run and for basic maintenance.
Verdict: Well you’ve probably figured it out by now
Conclusion
I didn’t mean this post to knock on Bitnami. Their company seems like it has a really great culture, and I love that they are supporting Open Source and simplifying access to Open Source Software. Their contributions have been valuable so far, but they need to improve in certain areas regarding their virtual machines.
Jidoteki is focused on creating rock-solid virtual appliances destined for the enterprise. We focus exclusively on doing things the correct way, with security and privacy as a top priority. Our solution is available for businesses who want to distribute their software on-premises using the best and simplest approach, along with the ability to automate appliance builds using continuous integration systems such as Jenkins.
Feel free to contact us if you’re looking to build and distribute an on-prem virtual appliance. We’ll be more than happy to discuss your requirements.