Skip to content

Commit 4cd9af6

Browse files
prometherionprometherion
committed
docs: kamaji-ingress-addon guide
Signed-off-by: Dario Tranchitella <[email protected]>
1 parent cae3c60 commit 4cd9af6

File tree

3 files changed

+217
-0
lines changed

3 files changed

+217
-0
lines changed

‎docs/content/enterprise-addons/index.md

+20
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,20 @@
1+
# Enterprise Addons
2+
3+
This document contains the documentation of the available Kamaji addons.
4+
5+
## What is a Kamaji Addon
6+
7+
A Kamaji Addon is a separate component installed in the same Kamaji cluster.
8+
It offers an additional set of features required for enterprise-grade usage.
9+
10+
The developed Kamaji addons are closed-source available and work with the upstream Kamaji edition.
11+
12+
## Distribution of Addons
13+
14+
The Kamaji Addons are available behind an active [subscription license](https://clastix.io/support/).
15+
16+
Once a subscription is activated, the [CLASTIX](https://clastix.io) team will automate the push of required OCI (container images) and Helm Chart artefacts to the customer's OCI-compatible repository.
17+
18+
## Available addons
19+
20+
- [Ingress Addon](/enterprise-addons/ingress): expose Tenant Control Planes behind an Ingress Controller to reduce the amount of required LoadBalancer services

‎docs/content/enterprise-addons/ingress.md

+194
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,194 @@
1+
# Ingress Addon
2+
3+
A Kubernetes API Server could be announced to users in several ways.
4+
The most preferred way is leveraging on Balancers with their dedicated IP.
5+
6+
However, IPv4 addresses could be limited and scarce in availability, as well as expensive when running in the Cloud.
7+
A possible optimisation could be implementing an Ingress Controller which routes traffic to Kubernetes API Servers on a host-routing basis.
8+
9+
Despite this solution sounding optimal for end users, it brings some challenges from the worker nodes' standpoint.
10+
11+
## Challenges
12+
13+
Internally deployed applications that need to interact with the Kubernetes API Server will leverage on the `kubernetes` endpoint in the `default` namespace:
14+
every request sent to the `https://kubernetes.default.svc` endpoint will be forwarded to the Kubernetes API Server.
15+
16+
The routing put in place by the Kubernetes CNI is based on the L4, meaning that all the requests will be forwarded to the Ingress Controller with no `Host` header,
17+
making impossible a routing based on the FQDN.
18+
19+
## Solution
20+
21+
The `kamaji-addon-ingress` is an addon that will expose the Tenant Control Plane behind an Ingress Controller.
22+
It's responsible for creating an `Ingress` object with the required HTTP rules, as well as the annotations needed for the TLS/SSL passthrough.
23+
24+
Following is the list of supported Ingress Controllers:
25+
26+
- [HAProxy Technologies Kubernetes Ingress](https://github.com/haproxytech/kubernetes-ingress)
27+
28+
> Active subscribers can request additional Ingress Controller flavours
29+
30+
## How to enable the Addon
31+
32+
The addon can be enabled for all Tenant Control Plane instances annotated with the key `kamaji.clastix.io/ingress.domain`.
33+
The value must be the expected suffix domain of generated resources.
34+
35+
```yaml
36+
apiVersion: kamaji.clastix.io/v1alpha1
37+
kind: TenantControlPlane
38+
metadata:
39+
annotations:
40+
kamaji.clastix.io/ingress.domain: clastix.cloud # the expected kamaji-addon-ingress label
41+
creationTimestamp: "2024-10-03T13:51:29Z"
42+
finalizers:
43+
- finalizer.kamaji.clastix.io
44+
- cloud.clastix.io/paralus
45+
- finalizer.kamaji.clastix.io/soot
46+
generation: 4
47+
labels:
48+
clastix.cloud/namespace: default
49+
clastix.cloud/user: apezzuto-1003
50+
platform: capsule
51+
name: tenant-00
52+
namespace: apezzuto
53+
resourceVersion: "409361036"
54+
uid: 592ee7b8-cd07-48cf-b754-76f370c3f87c
55+
```
56+
57+
Once a Tenant Control Plane has been annotated with this key, the addon will generate the following `Ingress` object.
58+
59+
```yaml
60+
apiVersion: networking.k8s.io/v1
61+
kind: Ingress
62+
metadata:
63+
annotations:
64+
haproxy.org/ssl-passthrough: "true"
65+
creationTimestamp: "2024-10-17T09:27:36Z"
66+
generation: 2
67+
name: 592ee7b8-cd07-48cf-b754-76f370c3f87c
68+
namespace: apezzuto
69+
ownerReferences:
70+
- apiVersion: kamaji.clastix.io/v1alpha1
71+
blockOwnerDeletion: true
72+
controller: true
73+
kind: TenantControlPlane
74+
name: tenant-00
75+
uid: 592ee7b8-cd07-48cf-b754-76f370c3f87c
76+
resourceVersion: "408133404"
77+
uid: 7986d8b5-795b-4207-9b5c-9e6b29b1d140
78+
spec:
79+
ingressClassName: haproxy
80+
rules:
81+
- host: apezzuto-tenant-00.k8s.clastix.cloud
82+
http:
83+
paths:
84+
- backend:
85+
service:
86+
name: tenant-00
87+
port:
88+
number: 6443
89+
path: /
90+
pathType: Prefix
91+
- host: apezzuto-tenant-00.konnectivity.clastix.cloud
92+
http:
93+
paths:
94+
- backend:
95+
service:
96+
name: tenant-00
97+
port:
98+
number: 8132
99+
path: /
100+
pathType: Prefix
101+
```
102+
103+
The pattern for the generated hosts is the following:
104+
`${tcp.namespace}-${tcp.name}.{k8s|konnectivity}.${ADDON_ANNOTATION_VALUE}`
105+
106+
> Please, notice the `konnectivity` rule will be created only if the `konnectivity` addon has been enabled.
107+
108+
## Infrastructure requirements
109+
110+
For Tenant Control Plane objects leveraging on this addon, the following changes must be implemented.
111+
112+
### Ingress Controller
113+
114+
The Ingress Controller must be deployed to listen for `https` connection on the default port `443`:
115+
if you have different requirements, please, engage with the CLASTIX team.
116+
117+
### DNS resolution
118+
119+
The following zones must be configured properly according to your DNS provider:
120+
121+
```
122+
*.konnectivity.clastix.cloud A <YOUR_INGRESS_CONTROLLER_IP>
123+
*.k8s.clastix.cloud A <YOUR_INGRESS_CONTROLLER_IP>
124+
```
125+
126+
### Certificate SANs
127+
128+
```yaml
129+
networkProfile:
130+
certSANs:
131+
- apezzuto-tenant-00.k8s.clastix.cloud
132+
- apezzuto-tenant-00.konnectivity.clastix.cloud
133+
dnsServiceIPs:
134+
- 10.96.0.10
135+
podCidr: 10.244.0.0/16
136+
port: 6443
137+
serviceCidr: 10.96.0.0/16
138+
```
139+
140+
### Service type and Ingress
141+
142+
The Kubernetes API Server can be exposed using a `ClusterIP`, rather than a Load Balancer.
143+
144+
```yaml
145+
spec:
146+
controlPlane:
147+
service:
148+
serviceType: ClusterIP
149+
ingress:
150+
hostname: apezzuto-tenant-00.k8s.clastix.cloud:443
151+
ingressClassName: unhandled
152+
```
153+
154+
The `ingressClassName` value must match a non-handled `IngressClass` object,
155+
the addon will take care of generating the correct object.
156+
157+
> Nota Bene: the `hostname` must absolutely point to the 443 port
158+
159+
### Kubernetes components extra Arguments
160+
161+
The Kubernetes API Server must start with the following flag:
162+
163+
```yaml
164+
spec:
165+
controlPlane:
166+
deployment:
167+
extraArgs:
168+
apiServer:
169+
- --endpoint-reconciler-type=none
170+
```
171+
172+
The `kamaji-addon-ingress` will be responsible for populating the `kubernetes` EndpointSlice object in the Tenant cluster.
173+
174+
If you're running with `konnectivity`, also this extra argument must be enforced:
175+
176+
```yaml
177+
spec:
178+
addons:
179+
konnectivity:
180+
agent:
181+
extraArgs:
182+
- --proxy-server-host=apezzuto-tenant-00.konnectivity.clastix.cloud
183+
- --proxy-server-port=443
184+
```
185+
186+
## Air-gapped environments
187+
188+
The `kamaji-addon-ingress` works with a deployed component in the Tenant Cluster based on the container image `docker.io/clastix/tcp-proxy:latest`.
189+
190+
The same image can be replaced by customising the Addon Helm value upon installation:
191+
192+
```
193+
--set options.tcpProxyImage=private.repository.tld/tcp-proxy:latest
194+
```

‎docs/mkdocs.yml

+3
Original file line numberDiff line numberDiff line change
@@ -77,4 +77,7 @@ nav:
7777
- reference/versioning.md
7878
- reference/api.md
7979
- 'Telemetry': telemetry.md
80+
- 'Enterprise Addons':
81+
- enterprise-addons/index.md
82+
- enterprise-addons/ingress.md
8083
- 'Contribute': contribute.md

0 commit comments

Comments
 (0)