santisbonIn this first installment of the series on building a cloud native homelab, we go over selecting your hardware.
The emphasis here is on cost optimization so we're not getting a cluster board, a bunch of compute modules (general purpose or machine learning), servers, a rack mount, or anything like that. We'll start out with one or two single-board computers (SBCs) that we can set up easily without a big capital expense. This will allow us to:
- Deploy new nodes to our homelab in an automated fashion, just like on a public cloud with cloud-init.
- Set up a Kubernetes cluster and deploy workloads to it using cloud native technologies for container orchestration, storage, and more.
- Interact with the real world through sensors, cameras, motors, and other hardware if we choose to.
The most popular SBC is Raspberry Pi but finding one during a shortage may be hard. Fortunately at the time of this writing the shortage is over and you can now easily find one at a reasonable price. There are cheaper options, though, and you might be tempted to buy something like the Orange Pi. Let's go over the differences to show exactly what you're getting with each.
TL;DR Get the Raspberry Pi if you want a cloud native homelab
| Raspberry Pi 4B | Orange Pi 3B | |
|---|---|---|
| Origin | United Kingdom | China |
| RAM | 8GB LPDDR4-3200 SDRAM | 8GB LPDDR4/4x |
| Storage | Micro-SD card slot | Micro SD slot. Supports an eMMC module: 16GB - 256GB. SPI Flash: 16MB/32MB. M.2 M-KEY slot for SATA3 or PCIe 2.0 NVMe SSD. |
| Price (USD) | $75 + shipping | $45 + shipping |
| OS | Raspberry Pi OS, Ubuntu, RetroPie, Recalbox, OctoPi, Home Assistant, others | Orange Pi OS [1], Ubuntu, Debian, Android, others |
| Official images from Ubuntu and Debian? | Yes | No. Custom Ubuntu and Debian images provided by Orange Pi. |
| cloud-init included? | Yes (at least on Ubuntu) | No |
| Default Ubuntu package repo | ubuntu.com | huaweicloud.com |
| Linux kernel release [4] | 5.15.0-1037-raspi (kernel extra modules available) | 5.10.160-rockchip-rk356x (kernel extra modules not available) |
| CPU | Broadcom BCM2711 Quad core Cortex-A72 (ARM v8) 64-bit SoC 1.8GHz | Rockchip RK3566 Quad-core Cortex-A55 (ARM v8) 64-bit SoC 1.8GHz |
| Graphics Processing Unit (GPU) | Integrated | ARM Mali G52 2EE (Bitfrost) [2] |
| Neural Processing Unit (NPU) | - | Rockchip NPU AI accelerator with 0.8Tops@INT8 performance. RKNN model type supports conversion of Caffe / TensorFlow / TFLite / ONNX / PyTorch / Keras / Darknet models |
| Vision Processing Unit (VPU) | - | Yes |
| Graphics & Video | OpenGL ES 3.1, Vulkan 1.0. H.265 (4K 60fps decode). H.264 (1080p 60fps decode, 1080p 30fps encode). |
GPU OpenGL ES 3.2/2.0/1.1, Vulkan 1.1, OpenCL 2.0. Embedded high performance 2D acceleration hardware. VPU H.265/H.264/VP9 (4K 60fps decode). H.265 (1080p 100fps encode). H.264 (1080p 60fps encode). |
| Ethernet | Gigabit | Gigabit |
| Wireless | Wi-Fi 5 (802.11ac) Bluetooth 5.0, BLE |
Wi-Fi 5 (802.11ac) Bluetooth 5.0, BLE Wired antenna (not shown). Useful when building into a metal housing. |
| USB | 2 USB 3.0 ports 2 USB 2.0 ports |
1 USB 3.0 HOST 2 USB 2.0 HOST 1 USB 2.0 Device or HOST mode |
| General Purpose Input/Output (GPIO) | 40 pin GPIO header | 40 pin header supporting interface types: GPIO, UART, I2C, SPI, PWM |
| Video output | 2 × micro-HDMI up to 4K 60fps. 1 × 2-lane MIPI DSI display port. |
1 × HDMI TX 2.0 up to 4K 60fps. 1 × 2-lane MIPI DSI display port. eDP1.3 |
| Camera | 1 × 2-lane MIPI CSI camera port | 1 × 2-lane MIPI CSI camera port |
| Audio | 4-pole stereo audio and composite video port | 3.5mm headphone jack audio in/out |
| Power | 5V DC via USB-C (minimum 3A) [3]. 5V DC via GPIO header (minimum 3A) [3] Power over Ethernet (PoE) enabled (requires separate PoE HAT). |
5V DC via USB-C (3A) |
| Button | - | 1 MaskROM key 1 RESET key 1 POWER key |
| Fan | Use the GPIO pins (3.3V or 5V) 2.54mm pin pitch | 2-Pin 1.25mm pin pitch 5V fan connector |
| Real-time clock (RTC) | Use the GPIO pins (3.3V) 2.54mm pin pitch | 2-Pin 1.25mm pin pitch battery backup connector |
| Dimensions | 85mm x 56mm x 11mm 46g |
85mm x 56mm x 17mm 49g |
[1] Orange Pi OS doesn't support the GPU, NPU, or VPU. But Debian, Ubuntu, and Android support them.
[2] arm.com, Rockchip GitHub
[3] A good quality 2.5A power supply can be used if downstream USB peripherals consume less than 500mA in total.
[4] Linux kernel extra modules nvme_fabrics and nvme_tcp are required for cloud native storage like OpenEBS.
- You'll also need a 5V 3A power supply for each board if you're not setting them up with Power over Ethernet (PoE).
- You don't need a mouse and keyboard for the boards because you'll manage them over SSH from your laptop. Even if you installed a graphical desktop, you could still access it remotely with a VNC viewer.
- If you're not setting them up with Wi-Fi, you'll need an ethernet cable (CAT 5e or better) for each board.
- For many applications you can use the Pi by itself without heat sinks or a fan and monitor the temperature in millidegree Celsius with
cat /sys/class/thermal/thermal_zone*/temp. But we're going to be running some resource-intensive software like Kubernetes. For our cloud native homelab I recommend getting heat sinks, a fan, and a case to house everything from the start. The Raspberry Pi ARM cores and GPU will start throttling when they reach around 80°C. - Speaking of cases, I recommend getting a cluster case. It's way cheaper than a cluster board and it houses your Pis in a stackable way so it's great for expanding your homelab in the future. This one from the UK is particularly nice and made with Perspex (a high-quality acrylic) but there are other options on Amazon with faster shipping.
- For storing large amounts of data you'll need external storage like an SSD drive. I recommend a SATA SSD although if you already have an NVMe SSD you can use that as well, just know that your board won't take full advantage of its speeds. This one is a great option along with this cable. Just make sure the cable/adapter has an ASMedia chipset.
With Raspberry Pi it's easy to automate the setup with cloud-init. You're also able to have cloud native replicated storage with OpenEBS thanks to the extra kernel modules for NVMe over Fabrics. If you want your homelab to be cloud native, Raspberry Pi is the way to go.
For Orange Pi, cloud-init is not included. On top of that, the images provided by Orange Pi can't be mounted on macOS so you can't inject the user-data anyway (and you have to use the images they provide, others won't work). That means you'll have to set it up manually. There's also the issue of not being able to use cloud native storage due to a lack of extra modules for that kernel release. Having said that, Orange Pi's AI capabilities might be a good addition to your homelab if you need a low-cost AI-accelerated IoT device and don't need any cloud native features like an automated setup or replicated storage.
If you have a 32-bit SBC like the Raspberry Pi 2, keep in mind that it can't run some Kubernetes distributions like MicroK8s so whatever SBCs you choose make sure they're 64-bit. If you have a Pi 2 it's great as a dedicated retro gaming console with Recalbox.
In the next part of this series, we'll set up our nodes as a Kubernetes cluster.
