Component IP Change Impact and Remediation
IP or hostname changes in infrastructure components can affect Apinizer connection points, certificates, and cluster discovery. This page consolidates what breaks when a change is made, whether Apinizer Workers keep running, and the recommended remediation order.
Before starting IP or hostname changes, plan a maintenance window if possible. Pay attention to replica set and Elasticsearch cluster fault tolerance ((n-1)/2).
Impact Summary
The Apinizer Worker column shows whether existing Worker pods continue to process API traffic. Even when Workers keep running, Manager access, log writes, or a new publish (re-publish) may still be affected.
| Component / Scenario | What breaks | Apinizer Worker | Remediation section |
|---|---|---|---|
| Kubernetes Control Plane IP | API server access, kubeconfig, apiserver certificates | Runs | Control Plane IP change |
| Kubernetes → Virtual IP migration | Control-plane access, join endpoint, HA configuration | Stops | Virtual IP migration |
| Kubernetes Worker node IP | Node NotReady, NodePort / access address | Stops on the affected node | Worker node IP change |
| MongoDB node IP | Replica set membership, Apinizer MongoDB connection | Stops | MongoDB IP change |
| MongoDB hostname | Replica set identity, auth conflicts | Stops | MongoDB hostname change |
| Elasticsearch node IP | Cluster discovery, log writes, Kibana access | Runs | Elasticsearch IP change |
| Apinizer access and connections | Manager / Gateway access, log connector, secret URI | Runs | Apinizer-side updates |
Recommended Remediation Order
- Backup — Critical configuration files (move them with
mv; delete only after verification) - Data layer — MongoDB, then Elasticsearch (Apinizer logs depend on these layers)
- Kubernetes — Control-plane, VIP migration if needed, worker nodes
- Apinizer — MongoDB secret / URI, Elasticsearch Connection, Gateway Access URL
- Verification — Verification checklist
Kubernetes Control Plane IP Change
When the control-plane server IP address changes, Kubernetes configuration files and apiserver certificates must be updated. Running Apinizer Worker pods continue to process API traffic; new scheduling or pod restarts are not possible while the API server is unreachable.
Configuration files
Update IP addresses to the new value in the following files:
/etc/kubernetes/admin.conf
/etc/kubernetes/controller-manager.conf
/etc/kubernetes/kubelet.conf
/etc/kubernetes/scheduler.conf
/etc/kubernetes/manifests/etcd.yaml
/etc/kubernetes/manifests/kube-apiserver.yaml
Certificate renewal
Move existing apiserver certificates aside instead of deleting them, then regenerate:
cd /etc/kubernetes/pki
sudo mkdir -p /etc/kubernetes/pki.backup
sudo mv apiserver.crt apiserver.key apiserver-kubelet-client.crt apiserver-kubelet-client.key /etc/kubernetes/pki.backup/
sudo kubeadm init phase certs apiserver-kubelet-client
sudo kubeadm init phase certs apiserver
sudo systemctl restart kubelet
Some warnings during certificate creation can be ignored. If issues persist, see Kubernetes Certificate Check and Renewal.
Kubeconfig update
Copy the updated config for cluster access:
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
After cluster access and node status are verified, delete the backups:
sudo rm -rf /etc/kubernetes/pki.backup
Kubernetes Virtual IP Migration
In environments with multiple control-plane nodes, the cluster is reconfigured to place control-plane access behind a Virtual IP (VIP) or load balancer. This scenario differs from a single control-plane IP change; it is for HA migration. Apinizer Workers do not run during this operation.
Example environment: control-plane-1, control-plane-2, control-plane-3, worker-1, worker-2 — cluster started with kubeadm init on control-plane-1.
Define a Virtual IP on the load balancer; all control-plane access must be routed to this VIP on port 6443.
If you are not using a load balancer, see Kubernetes Setup — High Availability for Keepalived and HAProxy steps to create a virtual IP.
Removing nodes from the cluster
Run the following on the other control-plane nodes (control-plane-2, control-plane-3) except control-plane-1, and on the worker nodes (worker-1, worker-2):
sudo kubeadm reset
From control-plane-1, remove the other control-plane and worker nodes from the cluster:
kubectl delete node control-plane-2
kubectl delete node control-plane-3
kubectl delete node worker-1
kubectl delete node worker-2
In this scenario only control-plane-1 remains in the cluster. The other control-plane and worker nodes rejoin through the VIP.
Tasks on the control-plane-1 server
sudo systemctl stop kubelet
sudo systemctl stop containerd
Move files instead of deleting them. Also move certificates that must not be reused into the backup directory:
sudo mv -f /etc/kubernetes /etc/kubernetes.backup
sudo mv -f /var/lib/kubelet /var/lib/kubelet.backup
sudo mv -f ~/.kube/config ~/.kube/config.backup
sudo mkdir -p /etc/kubernetes/pki
sudo cp -r /etc/kubernetes.backup/pki /etc/kubernetes
sudo mkdir -p /etc/kubernetes.backup/removed-certs/etcd
sudo mv /etc/kubernetes/pki/apiserver.* /etc/kubernetes.backup/removed-certs/
sudo mv /etc/kubernetes/pki/etcd/peer.* /etc/kubernetes.backup/removed-certs/etcd/
sudo systemctl start containerd
sudo kubeadm init --pod-network-cidr "10.244.0.0/16" --control-plane-endpoint <VIRTUAL_IP> --upload-certs --ignore-preflight-errors=DirAvailable--var-lib-etcd
Note the kubeadm join commands; they will be used for worker and additional control-plane nodes.
Tasks on worker nodes
sudo systemctl stop kubelet
sudo systemctl stop containerd
sudo mv -f /etc/kubernetes /etc/kubernetes.backup
sudo mv -f /var/lib/kubelet /var/lib/kubelet.backup
sudo systemctl start containerd
sudo systemctl start kubelet
Run the kubeadm worker join command from control-plane-1 on the worker nodes.
Tasks on the other control-plane nodes
Run the kubeadm control-plane join command on the control-plane-2 and control-plane-3 servers.
Cluster status check
kubectl cluster-info
kubectl get node -o wide
Deleting backup files
After verification succeeds, delete the backups on all affected nodes in one step:
sudo rm -rf /etc/kubernetes.backup
sudo rm -rf /var/lib/kubelet.backup
sudo rm -f ~/.kube/config.backup
Kubernetes Worker Node IP Change
When a worker node IP changes, the node usually enters NotReady state and NodePort / external access addresses are affected. Only Apinizer Workers on the node whose IP changed stop; Workers on other nodes keep running.
What breaks
- Node registration may retain the old IP
- Manager or Gateway access via NodePort (
http://<WORKER_IP>:32080, etc.) is interrupted - If Apinizer Gateway Runtime Access URL is tied to node IP, client access breaks
Remediation
- Complete the IP change at the operating system level on the worker node
- If needed, remove and re-add the node to the cluster:
# On the control-plane node
kubectl delete node <WORKER_NODE_NAME>
sudo kubeadm token create --print-join-command
# On the worker node
sudo kubeadm reset
sudo kubeadm join <CONTROL_PLANE_OR_VIP_IP>:6443 --token <TOKEN> --discovery-token-ca-cert-hash sha256:<HASH>
- If using load balancer or DNS, update backend pool / records
- Check Gateway Access URL and access addresses in Apinizer-side updates
MongoDB IP Change
MongoDB replica set member host fields are tied to IP addresses. When IP changes, Apinizer Manager, Worker, and Cache pods cannot connect to MongoDB; Workers stop.
Take a MongoDB backup before the operation. Pay attention to replica set fault tolerance ((n-1)/2).
sudo mongodump --host localhost --port=25080 --username=apinizer --password=<PASSWORD> -d apinizerdb --authenticationDatabase=admin --gzip --archive=/home/apinizer/mongodump/apinizer-backup-<DATE>.archive
Replica set update
Connect to MongoDB:
mongosh mongodb://<NEW_MONGO_IP>:25080 --authenticationDatabase "admin" -u "apinizer" -p
Update the host of the changed member (adjust the index in members array for the affected node):
cfg = rs.conf()
cfg.members[0].host = "<NEW_MONGO_IP>:25080"
rs.reconfig(cfg, { force: true })
Check all replica set members:
rs.status()
Apinizer connection
After MongoDB IP change, apply mongo secret and URI steps in Apinizer-side updates.
MongoDB Hostname Change
Hostname change differs from IP change; replica set identity is defined by hostname. Renaming a node directly can cause conflicts. Workers stop because replica set identity is broken.
Connect to the primary node and take a backup. Pay attention to replica set fault tolerance ((n-1)/2).
mongosh mongodb://localhost:25080 --authenticationDatabase "admin" -u "apinizer" -p
sudo mongodump --host localhost --port=25080 --username=apinizer --password=<PASSWORD> -d apinizerdb --authenticationDatabase=admin --gzip --archive=/home/apinizer/mongodump/apinizer-backup-<DATE>.archive
mongosh mongodb://localhost:25080 --authenticationDatabase "admin" -u "apinizer" -p
rs.status()
# If primary, step down first with rs.stepDown()
rs.remove("<OLD_HOSTNAME>")
On the affected server:
sudo systemctl stop mongod
sudo hostnamectl set-hostname <NEW_HOSTNAME>
sudo reboot
Update old hostname entries in /etc/hosts if present.
sudo systemctl start mongod
# From primary node
rs.add("<NEW_HOSTNAME>")
rs.status()
Elasticsearch IP Change
When an Elasticsearch node IP changes, cluster discovery and the transport layer are affected. Apinizer API traffic logs are written to this cluster via the Elasticsearch connector. Workers continue to process API traffic; log writes break.
What breaks
- Node may drop from cluster (
discovery.seed_hosts,network.host) - Apinizer Elasticsearch Connection host list shows old IP
- Kibana / log search access is interrupted
elasticsearch.yml update
On the affected node, update IP-bound fields in elasticsearch.yml:
node.name: "<NEW_NODE_IP>"
network.host: "<NEW_NODE_IP>"
cluster.initial_master_nodes: ["<NEW_NODE_IP>"]
discovery.seed_hosts: ["<NEW_NODE_IP>"]
In multi-node clusters, update all master/data node IPs consistently in discovery.seed_hosts and cluster.initial_master_nodes lists.
sudo systemctl restart elasticsearch
Check cluster health:
curl -u elastic:<PASSWORD> -k "https://<NEW_NODE_IP>:9200/_cluster/health?pretty"
Apinizer Elasticsearch Connection
Management Console → Connection Management → Elasticsearch connector → update Host & Port fields with the new IP. See Elasticsearch Connection Management for details.
Apinizer-Side Updates
After infrastructure IP changes are complete, Apinizer connection points must be updated. If Worker pods are up, API traffic continues; Manager, logs, and client access break when the secret, connector, or access URL still points to the old value.
MongoDB connection secret (Kubernetes)
Manager, Worker, and Cache deployments use the MongoDB secret. The connection string must include the new IPs. Update the secret directly with kubectl — do not create a YAML file; kubectl base64-encodes the values automatically:
kubectl create secret generic mongo-db-credentials \
-n <NAMESPACE> \
--from-literal=dbUrl="mongodb://<MONGO_USER>:<MONGO_PASSWORD>@<MONGO1_IP>:25080,<MONGO2_IP>:25080,<MONGO3_IP>:25080/?authSource=admin&replicaSet=apinizer-replicaset" \
--from-literal=dbName="<MONGO_DBNAME>" \
--dry-run=client -o yaml | kubectl apply -f -
Update the secret in every namespace that uses it. Then restart the pods:
kubectl rollout restart deployment/<MANAGER_DEPLOYMENT> -n <MANAGER_NAMESPACE>
kubectl rollout restart deployment/<WORKER_DEPLOYMENT> -n <WORKER_NAMESPACE>
kubectl rollout restart deployment/<CACHE_DEPLOYMENT> -n <CACHE_NAMESPACE>
The secret name (mongo-db-credentials) must match the module mongo.secretName value; the key names dbUrl and dbName are fixed. For secret creation steps during installation, see Kubernetes Setup.
Gateway Runtime Access URL
Management Console → Server Management → Gateway Runtimes → select environment → update Access URL according to new load balancer DNS or node IP / NodePort configuration.
After the change, re-publish the environment. See Gateway Runtimes for details.
Manager access address
If Manager NodePort or Ingress address changed, update the user access URL (e.g. http://<NEW_WORKER_IP>:32080).
Elasticsearch connector
If API traffic logs are written to Elasticsearch, update connector host list and run connection test.
hostAliases (optional)
If backend DNS resolution is IP-based, update the hostAliases block in deployment manifests with the new IP.
Verification Checklist
| Check | Command / action | Expected |
|---|---|---|
| Kubernetes nodes | kubectl get node -o wide | All nodes Ready |
| Cluster access | kubectl cluster-info | API server reachable via new IP/VIP |
| MongoDB replica set | rs.status() | All members PRIMARY / SECONDARY, healthy |
| Elasticsearch | GET _cluster/health | status: green or yellow (acceptable) |
| Manager pod | kubectl get pods -n <MANAGER_NS> | Pod Running / Ready |
| Worker pod | kubectl get pods -n <WORKER_NS> | Pod Running / Ready |
| Manager UI | Browser to Manager URL | Login screen loads |
| Gateway access | Health/version via Gateway Access URL | Response received |
| Log writes | Kibana or ES index check | New log records arriving |