Air-Gapped Kubernetes Environment with Nexus-Based Repository (Ubuntu 24.04)

In an air-gapped environment, installing a Kubernetes cluster requires a reliable internal repository that can serve packages, container images, and Helm charts without internet access.
This guide describes how to install Nexus Repository and an Apache2 web server on the Master Node, and how to use them to supply APT packages, Docker images, and Helm charts for building a Kubernetes cluster.

⚠️ Although running Nexus on a dedicated server is recommended for production, this document assumes Nexus is installed directly on the Master Node to reduce costs.

---

1. Architecture Overview

Master Node

• Nexus Repository (APT, Docker Registry, Helm Repo)
• Apache2 web server (provides kubeadm, kubelet, kubectl binaries)
• Kubespray execution and cluster provisioning

Worker Nodes

• Pull packages and images from the Master Node Nexus
• Fetch Kubernetes binaries from the Master Node Apache2
• Apache2 → Port 80
• Nexus → Port 8081 (APT/Helm), Port 5000 (Docker Registry)

---

2. Prerequisites

Required Files

• kube-system.tar.gz (Kubernetes images exported via ctr image export)
• docker-images/ directory (additional required images)
• nexus-3.xx-unix.tar.gz
• Java 8 or later (OpenJDK 17 recommended; included with Ubuntu 24.04)

Environment

• Master Node: 192.168.100.10 (example)
• Worker Nodes: 192.168.100.11, 192.168.100.12, …
• OS: Ubuntu 24.04 (noble)

---

3. Extract Kubernetes Image Bundle (Master)

tar xvzf ~/kube-system.tar.gz

---

4. Nexus Installation & Startup (Master)

Extract

tar xvzf nexus-3.74.0-05-unix.tar.gz -C /opt/
ln -s /opt/nexus-3.74.0-05 /opt/nexus

Register as systemd service
/etc/systemd/system/nexus.service:

[Unit]
Description=Nexus Repository
After=network.target

[Service]
Type=forking
ExecStart=/opt/nexus/bin/nexus start
ExecStop=/opt/nexus/bin/nexus stop
User=nexus
Restart=on-abort

[Install]
WantedBy=multi-user.target

systemctl enable nexus
systemctl start nexus
systemctl status nexus

• Nexus Web UI: http://192.168.100.10:8081

---

5. Nexus Repository Configuration

• APT Repository → ubuntu24-repo (Hosted)
• Docker Registry → docker-registry (Hosted, HTTP Port 5000)
• Helm Repository → helm-charts (Hosted)

⚠️ Configure Docker on all Worker Nodes: /etc/docker/daemon.json

{
  "insecure-registries": ["192.168.100.10:5000"]
}

systemctl restart docker

---

6. Apache2 Installation (Master)

apt install apache2 -y
systemctl enable apache2
systemctl start apache2

• Place kubeadm, kubelet, and kubectl binaries under /var/www/html/.
• In Kubespray, set download_url to point to Apache2 (http://192.168.100.10/).

---

7. Kubespray Configuration (Master)

Inventory

vi ~/kubespray/inventory/mycluster/inventory.ini

Download URLs
Edit roles/download/defaults/main.yml:

kubeadm_download_url: "http://192.168.100.10/kubeadm"
kubectl_download_url: "http://192.168.100.10/kubectl"
kubelet_download_url: "http://192.168.100.10/kubelet"

---

8. Distribute Docker Images (Master → Worker)

scp -r ~/docker-images user@192.168.100.11:/home/user/
scp -r ~/docker-images user@192.168.100.12:/home/user/

---

9. APT Repository Configuration (All Nodes)

Ubuntu 24.04 (noble) → /etc/apt/sources.list.d/nexus.sources

Types: deb
URIs: http://192.168.100.10:8081/repository/ubuntu24-repo/
Suites: noble noble-updates noble-backports
Components: main universe restricted multiverse
Trusted: yes

Types: deb
URIs: http://192.168.100.10:8081/repository/ubuntu24-repo/
Suites: noble-security
Components: main universe restricted multiverse
Trusted: yes

⚠️ In air-gapped setups, use Trusted: yes to bypass GPG validation.

---

10. Install Python3 & Ansible (Master)

apt update
apt install python3 python3.12-venv -y

cd ~/kubespray
python3 -m venv myenv
source myenv/bin/activate
pip install -U -r requirements.txt

---

11. Kubernetes Cluster Installation

Distribute SSH Keys

ssh-keygen -t rsa
ssh-copy-id user@192.168.100.11
ssh-copy-id user@192.168.100.12

Run Ansible Playbook

ansible-playbook -i inventory/mycluster/inventory.ini \
  --become --become-user=root cluster.yml -K

Handle Image Errors

sudo ctr image import ~/docker-images/kube-system.tar

---

12. Helm Repository Setup (Master)

Register Repo

helm repo add internal-helm http://192.168.100.10:8081/repository/helm-charts/

Upload Chart

helm package mychart/
curl -u admin:password \
  --upload-file mychart-0.1.0.tgz \
  http://192.168.100.10:8081/repository/helm-charts/

Update

helm repo update

---

13. Extending Nexus Usage

• APT/YUM → OS and system tools
• Docker Registry → Kubernetes container images
• Helm Repo → Add-ons (monitoring, logging, ingress, etc.)
• Maven/NPM/PyPI → Internal development artifacts

---

Summary

In an air-gapped Kubernetes environment, installing Nexus directly on the Master Node allows centralized management of APT packages, Docker images, and Helm charts.

• Apache2 → Provides Kubernetes binaries
• Nexus → Central repository (APT, Docker, Helm)
• Kubespray → Automates cluster provisioning

For stable operations, ensure proper configuration of insecure-registries, Trusted: yes, and download_url. These settings are critical for a successful offline Kubernetes installation.

🛠 마지막 수정일: 2025.09.26