2023-06-25 20:18:31 +00:00
|
|
|
---
|
|
|
|
title: "LXD: Containers for Human Beings"
|
|
|
|
subtitle: "Docker's great and all, but I prefer the workflow of interacting with VMs"
|
2023-08-16 19:34:57 +00:00
|
|
|
date: 2023-08-11T16:30:00-04:00
|
2023-06-25 20:18:31 +00:00
|
|
|
categories:
|
2023-08-16 19:34:57 +00:00
|
|
|
- Technology
|
2023-06-25 20:18:31 +00:00
|
|
|
tags:
|
2023-08-16 19:34:57 +00:00
|
|
|
- Sysadmin
|
|
|
|
- Containers
|
|
|
|
- VMs
|
|
|
|
- Docker
|
|
|
|
- LXD
|
2023-06-25 20:18:31 +00:00
|
|
|
draft: true
|
|
|
|
rss_only: false
|
|
|
|
cover: ./cover.png
|
|
|
|
---
|
|
|
|
|
2023-08-16 19:34:57 +00:00
|
|
|
This is a blog post version of a talk I presented at both Ubuntu Summit 2022 and
|
2023-06-25 20:18:31 +00:00
|
|
|
SouthEast LinuxFest 2023. The first was not recorded, but the second was and is
|
2023-08-18 02:04:07 +00:00
|
|
|
on [SELF's PeerTube instance.][selfpeertube] I apologise for the terrible audio,
|
2023-08-24 00:38:21 +00:00
|
|
|
but there's unfortunately nothing I can do about that. If you're already
|
|
|
|
intimately familiar with the core concepts of VMs or containers, I would suggest
|
|
|
|
skipping those respective sections. If you're vaguely familiar with either, I
|
|
|
|
would recommend reading them because I do go a little bit in-depth.
|
2023-06-25 20:18:31 +00:00
|
|
|
|
|
|
|
[selfpeertube]: https://peertube.linuxrocks.online/w/hjiTPHVwGz4hy9n3cUL1mq?start=1m
|
|
|
|
|
2023-08-16 19:34:57 +00:00
|
|
|
{{< adm type="warn" >}}
|
|
|
|
|
|
|
|
**Note:** Canonical has decided to [pull LXD out][lxd] from under the Linux
|
|
|
|
Containers entity and instead continue development under the Canonical brand.
|
2023-08-24 00:38:21 +00:00
|
|
|
The majority of the LXD creators and developers have congregated around a fork
|
|
|
|
called [Incus.][inc] I'll be keeping a close eye on the project and intend to
|
|
|
|
migrate as soon as there's an installable release.
|
2023-08-16 19:34:57 +00:00
|
|
|
|
|
|
|
[lxd]: https://linuxcontainers.org/lxd/
|
|
|
|
[inc]: https://linuxcontainers.org/incus/
|
|
|
|
|
|
|
|
{{< /adm >}}
|
|
|
|
|
2023-06-25 20:18:31 +00:00
|
|
|
## The benefits of VMs and containers
|
|
|
|
|
2023-08-24 00:38:21 +00:00
|
|
|
- **Isolation:** you don't want to allow an attacker to infiltrate your email
|
|
|
|
server through your web application; the two should be completely separate
|
|
|
|
from each other and VMs/containers provide strong isolation guarantees.
|
2023-06-25 20:18:31 +00:00
|
|
|
- **Flexibility:** <abbr title="Virtual Machines">VMs</abbr> and containers only
|
2023-08-18 02:04:07 +00:00
|
|
|
use the resources they've been given. If you tell the VM it has 200 MBs of
|
|
|
|
RAM, it's going to make do with 200 MBs of RAM and the kernel's <abbr
|
|
|
|
title="Out Of Memory">OOM</abbr> killer is going to have a fun time 🤠
|
2023-06-25 20:18:31 +00:00
|
|
|
- **Portability:** once set up and configured, VMs and containers can mostly be
|
2023-08-24 00:38:21 +00:00
|
|
|
treated as black boxes; as long as the surrounding environment of the new host
|
|
|
|
is similar to the previous in terms of communication (proxies, web servers,
|
|
|
|
etc.), they can just be picked up and dropped between various hosts as
|
|
|
|
necessary.
|
2023-08-18 02:04:07 +00:00
|
|
|
- **Density:** applications are usually much lighter than the systems they're
|
|
|
|
running on, so it makes sense to run many applications on one system. VMs and
|
|
|
|
containers facilitate that without sacrificing security.
|
2023-08-24 00:38:21 +00:00
|
|
|
- **Cleanliness:** VMs and containers are applications in black boxes. When
|
|
|
|
you're done with the box, you can just throw it away and most everything
|
|
|
|
related to the application is gone.
|
2023-07-18 17:21:24 +00:00
|
|
|
|
|
|
|
## Virtual machines
|
|
|
|
|
2023-08-24 00:38:21 +00:00
|
|
|
As the name suggests, Virtual Machines are all virtual; a hypervisor creates
|
|
|
|
virtual disks for storage, virtual <abbr title="Central Processing
|
|
|
|
Units">CPUs</abbr>, virtual <abbr title="Network Interface Cards">NICs</abbr>,
|
|
|
|
virtual <abbr title="Random Access Memory">RAM</abbr>, etc. On top of the
|
|
|
|
virtualised hardware, you have your kernel. This is what facilitates
|
|
|
|
communication between the operating system and the (virtual) hardware. Above
|
|
|
|
that is the operating system and all your applications.
|
|
|
|
|
|
|
|
At this point, the stack is quite large; VMs aren't exactly lightweight, and
|
|
|
|
this impacts how densely you can pack the host.
|
|
|
|
|
|
|
|
I mentioned a "hypervisor" a minute ago. I've explained what hypervisors in
|
|
|
|
general do, but there are actually two different kinds of hypervisor. They're
|
|
|
|
creatively named **Type 1** and **Type 2**.
|
|
|
|
|
|
|
|
### Type 1 hypervisors
|
|
|
|
|
|
|
|
These run directly in the host kernel without an intermediary OS. A good example
|
|
|
|
would be [KVM,][kvm] a **VM** hypervisor than runs in the **K**ernel. Type 1
|
|
|
|
hypervisors can communicate directly with the host's hardware to allocate RAM,
|
|
|
|
issue instructions to the CPU, etc.
|
|
|
|
|
|
|
|
[debian]: https://debian.org
|
|
|
|
[kvm]: https://www.linux-kvm.org
|
|
|
|
[vb]: https://www.virtualbox.org/
|
|
|
|
|
2023-08-18 02:04:07 +00:00
|
|
|
```kroki {type=d2,d2theme=flagship-terrastruct,d2sketch=true}
|
2023-08-24 00:38:21 +00:00
|
|
|
hk: Host kernel
|
2023-08-24 01:51:27 +00:00
|
|
|
hk.h: Type 1 hypervisor
|
|
|
|
hk.h.k1: Guest kernel
|
|
|
|
hk.h.k2: Guest kernel
|
|
|
|
hk.h.k3: Guest kernel
|
|
|
|
hk.h.k1.os1: Guest OS
|
|
|
|
hk.h.k2.os2: Guest OS
|
|
|
|
hk.h.k3.os3: Guest OS
|
|
|
|
hk.h.k1.os1.app1: Many apps
|
|
|
|
hk.h.k2.os2.app2: Many apps
|
|
|
|
hk.h.k3.os3.app3: Many apps
|
2023-08-24 00:38:21 +00:00
|
|
|
```
|
|
|
|
|
|
|
|
### Type 2 hypervisors
|
|
|
|
|
|
|
|
These run in userspace as an application, like [VirtualBox.][vb] Type 2
|
|
|
|
hypervisors have to first go through the operating system, adding an additional
|
|
|
|
layer to the stack.
|
|
|
|
|
|
|
|
```kroki {type=d2,d2theme=flagship-terrastruct,d2sketch=true}
|
|
|
|
hk: Host kernel
|
2023-08-24 01:51:27 +00:00
|
|
|
hk.os: Host OS
|
|
|
|
hk.os.h: Type 2 hypervisor
|
|
|
|
hk.os.h.k1: Guest kernel
|
|
|
|
hk.os.h.k2: Guest kernel
|
|
|
|
hk.os.h.k3: Guest kernel
|
|
|
|
hk.os.h.k1.os1: Guest OS
|
|
|
|
hk.os.h.k2.os2: Guest OS
|
|
|
|
hk.os.h.k3.os3: Guest OS
|
|
|
|
hk.os.h.k1.os1.app1: Many apps
|
|
|
|
hk.os.h.k2.os2.app2: Many apps
|
|
|
|
hk.os.h.k3.os3.app3: Many apps
|
2023-07-18 17:21:24 +00:00
|
|
|
```
|
|
|
|
|
|
|
|
## Containers
|
|
|
|
|
2023-08-24 01:51:27 +00:00
|
|
|
As most people know them right now, containers are exclusive to Linux.[^1] This is
|
|
|
|
because they use namespaces and cgroups to achieve isolation.
|
2023-08-18 02:04:07 +00:00
|
|
|
|
2023-08-24 01:51:27 +00:00
|
|
|
- **[Linux namespaces]** partition kernel resources like process IDs, hostnames,
|
|
|
|
user IDs, directory hierarchies, network access, etc.
|
|
|
|
- **[Cgroups]** limit, track, and isolate the hardware resource use of a set of
|
|
|
|
processes
|
2023-08-18 02:04:07 +00:00
|
|
|
|
2023-08-24 01:51:27 +00:00
|
|
|
[Linux namespaces]: https://en.wikipedia.org/wiki/Linux_namespaces
|
|
|
|
[Cgroups]: https://en.wikipedia.org/wiki/Cgroups
|
|
|
|
|
|
|
|
### Application containers
|
|
|
|
|
|
|
|
```kroki {type=d2,d2theme=flagship-terrastruct,d2sketch=true}
|
|
|
|
Host kernel.Container runtime.c1: Container
|
|
|
|
Host kernel.Container runtime.c2: Container
|
|
|
|
Host kernel.Container runtime.c3: Container
|
|
|
|
|
|
|
|
Host kernel.Container runtime.c1.One app
|
|
|
|
Host kernel.Container runtime.c2.Few apps
|
|
|
|
Host kernel.Container runtime.c3.Full OS.Many apps
|
2023-08-18 02:04:07 +00:00
|
|
|
```
|
|
|
|
|
2023-08-24 01:51:27 +00:00
|
|
|
### System containers
|
|
|
|
|
2023-08-18 02:04:07 +00:00
|
|
|
```kroki {type=d2,d2theme=flagship-terrastruct,d2sketch=true}
|
2023-08-24 01:51:27 +00:00
|
|
|
hk: Host kernel
|
|
|
|
hk.c1: Container
|
|
|
|
hk.c2: Container
|
|
|
|
hk.c3: Container
|
|
|
|
hk.c1.os1: Full OS
|
|
|
|
hk.c2.os2: Full OS
|
|
|
|
hk.c3.os3: Full OS
|
|
|
|
hk.c1.os1.app1: Many apps
|
|
|
|
hk.c2.os2.app2: Many apps
|
|
|
|
hk.c3.os3.app3: Many apps
|
2023-07-18 17:21:24 +00:00
|
|
|
```
|
|
|
|
|
2023-08-24 00:38:21 +00:00
|
|
|
## When to use VMs
|
2023-07-18 17:21:24 +00:00
|
|
|
|
|
|
|
- Virtualising esoteric hardware
|
|
|
|
- Virtualising non-Linux operating systems (Windows, macOS)
|
|
|
|
- Completely isolating processes from one another with a decades-old, battle-tested technique
|
|
|
|
|
2023-08-16 19:34:57 +00:00
|
|
|
{{< adm type="note" >}}
|
|
|
|
See Drew DeVault's blog post [_In praise of qemu_](https://earl.run/rmBs) for a great use of VMs
|
|
|
|
{{< /adm >}}
|
2023-07-18 17:21:24 +00:00
|
|
|
|
2023-08-24 00:38:21 +00:00
|
|
|
### When you use application containers
|
2023-07-18 17:21:24 +00:00
|
|
|
|
|
|
|
- Microservices
|
|
|
|
- Extremely reproducible builds
|
|
|
|
- (NixOS.org would likely be a better fit though)
|
|
|
|
- Dead-set on using cloud platforms with extreme scaling capabilities (AWS, GCP, etc.)
|
|
|
|
- When the app you want to run is _only_ distributed as a Docker container and
|
|
|
|
the maintainers adamantly refuse to support any other deployment method
|
|
|
|
- (Docker does run in LXD 😉)
|
|
|
|
|
|
|
|
### System containers
|
|
|
|
|
|
|
|
- Anything not listed above 👍
|
|
|
|
|
|
|
|
## Crash course to LXD
|
|
|
|
|
2023-08-24 00:38:21 +00:00
|
|
|
### Installation
|
|
|
|
|
|
|
|
{{< adm type="note" >}}
|
|
|
|
|
|
|
|
**Note:** the instructions below say to install LXD using [Snap.][snap] I
|
2023-08-24 01:51:27 +00:00
|
|
|
personally dislike Snap, but LXD is a Canonical product and they're doing their
|
|
|
|
best to prmote it as much as possible. One of the first things the Incus project
|
|
|
|
did was [rip out Snap support,][rsnap] so it will eventually be installable as a
|
|
|
|
proper native package.
|
2023-08-24 00:38:21 +00:00
|
|
|
|
|
|
|
[snap]: https://en.wikipedia.org/wiki/Snap_(software)
|
|
|
|
[rsnap]: https://github.com/lxc/incus/compare/9579f65cd0f215ecd847e8c1cea2ebe96c56be4a...3f64077a80e028bb92b491d42037124e9734d4c7
|
|
|
|
|
|
|
|
{{< /adm >}}
|
|
|
|
|
2023-07-18 17:21:24 +00:00
|
|
|
1. Install snap following [Canonical's tutorial](https://earl.run/ZvUK)
|
2023-08-24 00:38:21 +00:00
|
|
|
- LXD is natively packaged for Arch and Alpine, but configuration can be a
|
|
|
|
massive headache.
|
2023-07-18 17:21:24 +00:00
|
|
|
2. `sudo snap install lxd`
|
|
|
|
3. `lxd init`
|
|
|
|
4. `lxc image copy images:debian/11 local: --alias deb-11`
|
|
|
|
5. `lxc launch deb-11 container-name`
|
|
|
|
6. `lxc shell container-name`
|
2023-08-24 00:38:21 +00:00
|
|
|
|
|
|
|
### Usage
|
|
|
|
|
|
|
|
{install my URL shortener}
|
2023-08-24 01:51:27 +00:00
|
|
|
|
|
|
|
[^1]: Docker containers on Windows and macOS actually run in a Linux VM.
|