Troubleshooting the On-Prem Kubernetes Deployment
This section covers common issues you may encounter when deploying ESET PRIVATE Scanning Solutions on‑prem in a Kubernetes environment and provides basic steps to diagnose and resolve them.
1.Deployment or Helm-related issues
Helm install fails or hangs
•Verify that the Helm version matches what the chart expects (e.g., Helm 3.x).
•Ensure you are using the correct Helm chart and values‑ecscn.yaml was prepared correctly.
•Check for syntax errors or missing values by running helm template . -f values‑ecscn.yaml first, then helm install once the manifest renders cleanly.
The chart uses outdated or incorrect image tags
•Confirm that scanner.version, loadbalancer.version, and other image tags in the values file match the latest versions available in the ESET Container Registry (review README file for parameters configuration).
•If you encounter ImagePullBackOff or ErrImagePull, make sure:
oThe registry URL is correct (ecscnprod.azurecr.io).
oDocker credentials are properly configured in the secret referenced by image.pullSecret (for example, ecscn‑registry‑auth).
oThe node has outbound HTTPS access to ecscnprod.azurecr.io.
2.Network and Load Balancer issues
Load Balancer IPs do not respond
•Ensure the IPs you reserved for the Scanner and Grafana LoadBalancer services are not in use by other devices and are reachable from the host running Scanner Agent.
•If the IPs are not reachable, check:
oARP responses on the node (for example, arping from the host running Scanner Agent).
oNode’s firewall or host‑based security rules allow traffic on the LoadBalancer IP and port 50052.
Scanner or Grafana services are stuck in Pending or no IP appears
•Ensure you are using a supported CNI that implements Kubernetes LoadBalancer semantics (e.g., Cilium, Calico, etc.).
•Check that the loadbalancer.service.type parameter is set to LoadBalancer in the Helm values.
•Review events with kubectl get events -A and kubectl describe service <service‑name> for any CNI or IP‑pool‑related errors.
3.Certificate and TLS issues
Scanner or LoadBalancer pods fail due to certificate problems
•If you use cert‑manager (enabled via env.certificate.certManager.enabled), verify that:
oThe cert‑manager CRDs and controller are installed and healthy.
oThe Issuer or ClusterIssuer (env.certificate.certManager.issuer.name) exists and is correctly configured.
oThe DNS names and IP addresses specified in env.certificate.certManager.dnsNames and env.certificate.certManager.ipAddresses match the LoadBalancer endpoint.
•If you supply certificates manually, ensure the env.certificate.ca, env.certificate.crt, and env.certificate.key values are properly formatted and base64‑encoded where required, and that the env.certificate.secretName secret is created before the pods start.
4.Scanner Agent and connectivity issues
Scanner Agent cannot reach the Cloud Scanner LoadBalancer
•Confirm that the host VM (running Scanner Agent) or container can reach the LoadBalancer IP and port (for example, 10.1.222.198:50052) via telnet.
•Ensure the samples directory (/home/samples or similar) is correctly mounted and accessible inside the Scanner Agent container.
•If the agent runs but produces no scans or returns timeout errors, verify:
oThe scanner_agent image version is up to date and you’re running the latest version available.
oThe gRPC endpoint address and port in the scanner_agent command line match the Load Balancer configuration.
5.Resource and policy‑related problems
Pods stuck in Pending or crash loop
•Check resource limits and requests defined in scanner.resources, loadbalancer.resources, and related parameters.
•If the cluster is under‑provisioned, Kubernetes may not schedule the pods or they may be evicted. Increase node capacity or relax requests/limits selectively for testing.
•Review kubectl describe pod <pod‑name> and kubectl logs <pod‑name> for container‑level errors (e.g., forbidden network calls, missing certificates, or license‑related messages).
NetworkPolicy or CNI‑related connectivity drops
•If env.networkPolicy.enabled is set to true, ensure the env.networkPolicy.ingressCIDRs allow traffic from the host running the Scanner Agent.
•Check the status and logs of your CNI and any LoadBalancer controller (e.g., kubectl get pods -n kube‑system, controller‑specific status commands) for policy violations, IP‑pool exhaustion, or broken connectivity.
If you still cannot resolve an issue, collect the following artifacts and contact ESET support:
•Output from kubectl get pods,svc,events,secrets -n <namespace>.
•Relevant Helm values file (with secrets redacted).
•Logs from Scanner, Load Balancer, and Scanner Agent containers.
•Any error messages or events related to image pull, network policy, or Load Balancer IP assignment.