Enhancement: Webhook is not accessible on EKS with Calico

[dipeti]

Hi,

When using EKS with only Calico CNI the Kubernetes API server on the control plane (managed by AWS) cannot reach webhooks that use a service pointing to pods on Calico pod network.
The visible error is the following:

  Error: Internal error occurred: failed calling webhook "validate-image-cache.kubefledged.io": Post "https://kubefledged-operator-kube-fledged-webhook-server.kubefledged.svc:3443/validate-image-cache?timeout=1s": Address is not allowed

In order to make the webhook pod accessible (on Calico) from the control plane (not on Calico) I need to expose the service port on the hosting node. hostNetwork: true

Can you please add the option of configuring the hostNetwork on the webhook server's deployment manifest?

#helm-charts/kubefledged/templates/deployment-webhook-server.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: {{ include "kubefledged.fullname" . }}-webhook-server
  labels:
    {{- include "kubefledged.labels" . | nindent 4 }}
spec:
  replicas: {{ .Values.replicaCount }}
  selector:
    matchLabels:
      {{- include "kubefledged.selectorLabels" . | nindent 6 }}-webhook-server
  template:
    metadata:
      labels:
        {{- include "kubefledged.selectorLabels" . | nindent 8 }}-webhook-server
    spec:
#SOMETHING ALONG THESE LINES 
{{- if .Values.args.webhookServerHostNetwork }}
      hostNetwork: true
{{- end }}

source: https://medium.com/@denisstortisilva/kubernetes-eks-calico-and-custom-admission-webhooks-a2956b49bd0d

[senthilrch]

Thanks @dipeti I'll have this implemented in v0.9.0 release