Ana içeriğe geç

Database Connection Issues

Problem Symptoms​

Warning

Database connection issues usually manifest themselves with the following symptoms:

  • Timeout errors in API requests
  • Connection refused or Connection timeout errors
  • Database queries failing
  • High number of connection pool errors
  • Pods not being able to connect to database

Problem Causes​

Database connection issues can usually be caused by the following factors:

  • Network Connection Issues: Pods not being able to access database
  • Connection Pool Exhaustion: Reaching maximum connection count
  • Database Server Issues: Database server not responding
  • Wrong Connection Settings: Incorrect host, port, or credentials
  • Firewall Rules: Network security rules blocking connection
  • DNS Resolution Issues: Database host name not being resolved

Detection Methods​

1. Log Analysis​

Check Log Files

Search for database connection errors in log files:

kubectl logs <pod-name> | grep -i "connection"
kubectl logs <pod-name> | grep -i "database"
kubectl logs <pod-name> | grep -i "timeout"

2. Connection Test​

Connect to Pod

Test database connection from inside pod:

# Connecting to pod
kubectl exec -it <pod-name> -- /bin/bash

# Testing database connection
telnet <database-host> <database-port>

3. Connection Pool Metrics​

Info

Monitor connection pool metrics:

  • Active connection count
  • Pending connection count
  • Connection timeout count
  • Connection pool usage percentage

Solution Recommendations​

1. Checking Connection Settings​

Check database connection settings:

  • Host: Is database server address correct?
  • Port: Is port number correct?
  • Database Name: Is database name correct?
  • Username/Password: Are credentials correct?
  • Connection String: Is connection string format correct?

2. Optimizing MongoDB Connection Pool Settings​

Info

Apinizer stores its configuration in MongoDB. There is no JDBC or Hikari pool, so spring.datasource.hikari.* settings have no effect. Set the pool through MongoDB connection string options in the SPRING_DATA_MONGODB_URI environment variable.

SPRING_DATA_MONGODB_URI="mongodb://apinizer:<password>@mongo1:25080,mongo2:25080,mongo3:25080/?authSource=admin&replicaSet=apinizer-replicaset&maxPoolSize=100&minPoolSize=10&maxIdleTimeMS=600000&waitQueueTimeoutMS=5000&serverSelectionTimeoutMS=30000"
OptionMeaning
maxPoolSizeMaximum number of connections per MongoDB server
minPoolSizeMinimum number of idle connections kept open
maxIdleTimeMSTime after which an idle connection is closed
waitQueueTimeoutMSMaximum time a request waits for a free connection
serverSelectionTimeoutMSMaximum time to find an available MongoDB server
Note

Which component reads these options:

  • Manager and Integration: Spring Boot builds the MongoDB client from the URI. All options above apply.
  • Worker and the Cache MapStore client: The code sets the pool size and the socket timeouts after reading the URI. The URI values for maxPoolSize, minPoolSize, waitQueueTimeoutMS, connectTimeoutMS and socketTimeoutMS are overridden. The fixed values are a maximum of 50 connections, a minimum of 5 (Worker) or 2 (Cache), a 5 second wait time, a 10 second connect timeout and a 30 second read timeout. Other options such as maxIdleTimeMS and serverSelectionTimeoutMS still apply.

Change the variable on the Deployment, then restart the pods.

3. Checking Network Connection​

Check network connection:

# DNS resolution test
nslookup <database-host>

# Port accessibility test
nc -zv <database-host> <database-port>

# Network policy check
kubectl get networkpolicies

4. Checking Database Server Status​

Check database server status:

  • Is database server running?
  • Are there sufficient resources (CPU, RAM)?
  • Has maximum connection limit been reached?
  • Are there errors in database logs?

5. Checking Firewall Rules​

Check firewall rules:

  • Do pods have access to database port?
  • Are network policies blocking connection?
  • Are security group rules correct?

6. Resolving DNS Resolution Issues​

Resolve DNS resolution issues:

  • Are CoreDNS pods running?
  • Are DNS settings correct?
  • Is service discovery working properly?

Preventive Measures​

1. Connection Pool Monitoring​

  • Regularly monitor connection pool metrics
  • Get early warning by setting up alerts
  • Optimize connection pool usage

2. Health Checks​

  • Configure database health checks
  • Set up liveness and readiness probes
  • Set up automatic failover mechanisms

3. Connection Retry Mechanism​

  • Add connection retry mechanism
  • Use exponential backoff
  • Determine maximum retry count
En son tarihinde güncellendi