Inside Jidometa: Loose coupling of customer data

[Read part 3 of Inside Jidometa]

One of the major differences between Jidoteki appliances, and most other setups, is that we isolate customer data and ensure it’s not tightly coupled to the system.

image

Looking at the graph above, we can see the customer data is stored on disk 2, while the OS/apps/kernel are stored on disk 1.

Issues with tight coupling

Adding disks to a virtual appliance is quite trivial. Moving them to another appliance is just as simple. We know this, so we made a plan to support it.

In a typical non-Jidoteki appliance, databases, log files, application settings and data are stored on the same disk. This tight coupling introduces problems which are almost impossible to workaround once deployed:

Our disk setup

Jidoteki appliances use a minimum of two disks. The first disk is quite small (8GB sparse disk) and used exclusively for storing the system and custom settings (ex: network settings). The small OS disk means importing an OVA for the first time is very quick. The second disk is sized based on our customers’ requirements, but typically range between 10GB and 2TB (sparse disk), depending on the application. In any case, the customer can opt to not import the second disk, or simply delete it and re-create it with whatever size they want.

The best part about disk 2 is we deploy it using LVM, which allows anyone to attach a third, fourth.. eighth disk (max 8 disks) of virtually any size. If a deployed appliance uses 2TB and needs to increase to 42TB, an additional disk can be added and the appliance will automatically grow the LVM disk.

Other benefits to loose coupling

By dedicating the entire disk to the customer data, we can provide full permissions to the appliance admin (the customer) to login via SSH and modify any file(s) on that disk. The customer controls their data and remains free to alter it how they want (at their own risk, of course). We do this in our own on-premises virtual appliance, Jidoteki Meta (Jidometa), and guide our customers in doing it this way as well.

Another benefit is having the ability to detach the disk from the appliance, deploy a new appliance on a different (physical?) server, and then re-attach the disk. It’s not as fancy as live migration, but moving disks is effortless compared to transferring data with rsync or some other wild migration scheme.

Of course, if it’s that easy to move data, that means it’s just as easy to backup the data, or create a fault-tolerant setup for disaster recovery scenarios.

Finally, as mentioned in a previous blog post, dedicating the second disk to customer data means it can also be replaced by network storage, such as NFS, AoE, iSCSI, or NBD. I can’t imagine the logistics (and coding nightmare) of having to support network storage when the application was designed for storing on the same disk. With Jidoteki, we simply mount the network storage to /data, instead of mounting the local disk 2 to /data. No software changes required. Easy.

Moving forward

As always, we’re happy to discuss your requirements if you plan on providing an on-prem appliance to enterprise customers. We’re familiar with enterprise requirements, and try our best to provide the most rock-solid on-premises virtual appliances out there. In a few days we’ll be celebrating 5 years of doing this stuff. It really is our specialty, and we’d love to help more companies go on-prem, the right way.

Feel free to contact us, or signup for our new monthly newsletter to stay up-to-date with what we’re doing.