Skip to content

Commit 809d0de

Browse files
prometherionprometherion
committed
docs: kamaji-ingress-addon guide
Signed-off-by: Dario Tranchitella <[email protected]>
1 parent cae3c60 commit 809d0de

File tree

3 files changed

+199
-0
lines changed

3 files changed

+199
-0
lines changed

‎docs/content/enterprise-addons/index.md

+20
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,20 @@
1+
# Enterprise Addons
2+
3+
This document contains the documentation of the available Kamaji addons.
4+
5+
## What is a Kamaji Addon
6+
7+
A Kamaji Addon is a separate component installed in the same Kamaji cluster.
8+
It offers an additional set of features required for enterprise-grade usage.
9+
10+
The developed Kamaji addons are closed-source available and work with the upstream Kamaji edition.
11+
12+
## Distribution of Addons
13+
14+
The Kamaji Addons are available behind an active [subscription license](https://clastix.io/support/).
15+
16+
Once a subscription is activated, the [CLASTIX](https://clastix.io) team will automate the push of required OCI (container images) and Helm Chart artefacts to the customer's OCI-compatible repository.
17+
18+
## Available addons
19+
20+
- [Ingress Addon](/enterprise-addons/ingress): expose Tenant Control Planes behind an Ingress Controller to reduce the amount of required LoadBalancer services

‎docs/content/enterprise-addons/ingress.md

+176
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,176 @@
1+
# Ingress Addon
2+
3+
A Kubernetes API Server could be announced to users in several ways.
4+
The most preferred way is leveraging on Load Balancers with their dedicated IP.
5+
6+
However, IPv4 addresses could be limited and scarce in availability, as well as expensive for public ones when running in the Cloud.
7+
A possible optimisation could be implementing an Ingress Controller which routes traffic to Kubernetes API Servers on a host-routing basis.
8+
9+
Despite this solution sounding optimal for end users, it brings some challenges from the worker nodes' standpoint.
10+
11+
## Challenges
12+
13+
Internally deployed applications that need to interact with the Kubernetes API Server will leverage on the `kubernetes` endpoint in the `default` namespace:
14+
every request sent to the `https://kubernetes.default.svc` endpoint will be forwarded to the Kubernetes API Server.
15+
16+
The routing put in place by the Kubernetes CNI is based on the L4, meaning that all the requests will be forwarded to the Ingress Controller with no `Host` header,
17+
making impossible a routing based on the FQDN.
18+
19+
## Solution
20+
21+
The `kamaji-addon-ingress` is an addon that will expose the Tenant Control Plane behind an Ingress Controller.
22+
It's responsible for creating an `Ingress` object with the required HTTP rules, as well as the annotations needed for the TLS/SSL passthrough.
23+
24+
Following is the list of supported Ingress Controllers:
25+
26+
- [HAProxy Technologies Kubernetes Ingress](https://github.com/haproxytech/kubernetes-ingress)
27+
28+
> Active subscribers can request additional Ingress Controller flavours
29+
30+
## How to enable the Addon
31+
32+
Annotate the Tenant Control Plane instances with the key `kamaji.clastix.io/ingress.domain` and the domain suffix domain value:
33+
34+
```shell
35+
kubectl annotate tenantcontrolplane $NAME kamaji.clastix.io/ingress.domain=$SUFFIX_DOMAIN
36+
```
37+
38+
The value must be the expected suffix domain of generated resources.
39+
40+
```yaml
41+
apiVersion: kamaji.clastix.io/v1alpha1
42+
kind: TenantControlPlane
43+
metadata:
44+
annotations:
45+
kamaji.clastix.io/ingress.domain: clastix.cloud # the expected kamaji-addon-ingress label
46+
name: tenant-00
47+
namespace: apezzuto
48+
```
49+
50+
Once a Tenant Control Plane has been annotated with this key, the addon will generate the following `Ingress` object.
51+
52+
```yaml
53+
apiVersion: networking.k8s.io/v1
54+
kind: Ingress
55+
metadata:
56+
annotations:
57+
haproxy.org/ssl-passthrough: "true"
58+
name: 592ee7b8-cd07-48cf-b754-76f370c3f87c
59+
namespace: apezzuto
60+
spec:
61+
ingressClassName: haproxy
62+
rules:
63+
- host: apezzuto-tenant-00.k8s.clastix.cloud
64+
http:
65+
paths:
66+
- backend:
67+
service:
68+
name: tenant-00
69+
port:
70+
number: 6443
71+
path: /
72+
pathType: Prefix
73+
- host: apezzuto-tenant-00.konnectivity.clastix.cloud
74+
http:
75+
paths:
76+
- backend:
77+
service:
78+
name: tenant-00
79+
port:
80+
number: 8132
81+
path: /
82+
pathType: Prefix
83+
```
84+
85+
The pattern for the generated hosts is the following:
86+
`${tcp.namespace}-${tcp.name}.{k8s|konnectivity}.${ADDON_ANNOTATION_VALUE}`
87+
88+
> Please, notice the `konnectivity` rule will be created only if the `konnectivity` addon has been enabled.
89+
90+
## Infrastructure requirements
91+
92+
For Tenant Control Plane objects leveraging on this addon, the following changes must be implemented.
93+
94+
### Ingress Controller
95+
96+
The Ingress Controller must be deployed to listen for `https` connection on the default port `443`:
97+
if you have different requirements, please, engage with the CLASTIX team.
98+
99+
### DNS resolution
100+
101+
The following zones must be configured properly according to your DNS provider:
102+
103+
```
104+
*.konnectivity.clastix.cloud A <YOUR_INGRESS_CONTROLLER_IP>
105+
*.k8s.clastix.cloud A <YOUR_INGRESS_CONTROLLER_IP>
106+
```
107+
108+
### Certificate SANs
109+
110+
```yaml
111+
networkProfile:
112+
certSANs:
113+
- apezzuto-tenant-00.k8s.clastix.cloud
114+
- apezzuto-tenant-00.konnectivity.clastix.cloud
115+
dnsServiceIPs:
116+
- 10.96.0.10
117+
podCidr: 10.244.0.0/16
118+
port: 6443
119+
serviceCidr: 10.96.0.0/16
120+
```
121+
122+
### Service type and Ingress
123+
124+
The Kubernetes API Server can be exposed using a `ClusterIP`, rather than a Load Balancer.
125+
126+
```yaml
127+
spec:
128+
controlPlane:
129+
service:
130+
serviceType: ClusterIP
131+
ingress:
132+
hostname: apezzuto-tenant-00.k8s.clastix.cloud:443
133+
ingressClassName: unhandled
134+
```
135+
136+
The `ingressClassName` value must match a non-handled `IngressClass` object,
137+
the addon will take care of generating the correct object.
138+
139+
> Nota Bene: the `hostname` must absolutely point to the 443 port
140+
141+
### Kubernetes components extra Arguments
142+
143+
The Kubernetes API Server must start with the following flag:
144+
145+
```yaml
146+
spec:
147+
controlPlane:
148+
deployment:
149+
extraArgs:
150+
apiServer:
151+
- --endpoint-reconciler-type=none
152+
```
153+
154+
The `kamaji-addon-ingress` will be responsible for populating the `kubernetes` EndpointSlice object in the Tenant cluster.
155+
156+
If you're running with `konnectivity`, also this extra argument must be enforced:
157+
158+
```yaml
159+
spec:
160+
addons:
161+
konnectivity:
162+
agent:
163+
extraArgs:
164+
- --proxy-server-host=apezzuto-tenant-00.konnectivity.clastix.cloud
165+
- --proxy-server-port=443
166+
```
167+
168+
## Air-gapped environments
169+
170+
The `kamaji-addon-ingress` works with a deployed component in the Tenant Cluster based on the container image `docker.io/clastix/tcp-proxy:latest`.
171+
172+
The same image can be replaced by customising the Addon Helm value upon installation:
173+
174+
```
175+
--set options.tcpProxyImage=private.repository.tld/tcp-proxy:latest
176+
```

‎docs/mkdocs.yml

+3
Original file line numberDiff line numberDiff line change
@@ -77,4 +77,7 @@ nav:
7777
- reference/versioning.md
7878
- reference/api.md
7979
- 'Telemetry': telemetry.md
80+
- 'Enterprise Addons':
81+
- enterprise-addons/index.md
82+
- enterprise-addons/ingress.md
8083
- 'Contribute': contribute.md

0 commit comments

Comments
 (0)