Skip to main content

​
Problem Symptoms

Database connection issues usually manifest themselves with the following symptoms:
  • Timeout errors in API requests
  • Connection refused or Connection timeout errors
  • Database queries failing
  • High number of connection pool errors
  • Pods not being able to connect to database

​
Problem Causes

Database connection issues can usually be caused by the following factors:
  • Network Connection Issues: Pods not being able to access database
  • Connection Pool Exhaustion: Reaching maximum connection count
  • Database Server Issues: Database server not responding
  • Wrong Connection Settings: Incorrect host, port, or credentials
  • Firewall Rules: Network security rules blocking connection
  • DNS Resolution Issues: Database host name not being resolved

​
Detection Methods

​
1. Log Analysis

1

Check Log Files

Search for database connection errors in log files:
kubectl logs <pod-name> | grep -i "connection"
kubectl logs <pod-name> | grep -i "database"
kubectl logs <pod-name> | grep -i "timeout"

​
2. Connection Test

1

Connect to Pod

Test database connection from inside pod:
# Connecting to pod
kubectl exec -it <pod-name> -- /bin/bash

# Testing database connection
telnet <database-host> <database-port>

​
3. Connection Pool Metrics

Monitor connection pool metrics:
  • Active connection count
  • Pending connection count
  • Connection timeout count
  • Connection pool usage percentage

​
Solution Recommendations

​
1. Checking Connection Settings

Check database connection settings:
  • Host: Is database server address correct?
  • Port: Is port number correct?
  • Database Name: Is database name correct?
  • Username/Password: Are credentials correct?
  • Connection String: Is connection string format correct?

​
2. Optimizing Connection Pool Settings

Optimize connection pool settings:
# application.yml or environment variables
spring:
  datasource:
    hikari:
      maximum-pool-size: 20
      minimum-idle: 5
      connection-timeout: 30000
      idle-timeout: 600000
      max-lifetime: 1800000
Recommendations:
  • maximum-pool-size: Set maximum connection count
  • minimum-idle: Determine minimum idle connection count
  • connection-timeout: Set connection timeout duration
  • idle-timeout: Determine idle connections closing time

​
3. Checking Network Connection

Check network connection: 
# DNS resolution test
nslookup <database-host>

# Port accessibility test
nc -zv <database-host> <database-port>

# Network policy check
kubectl get networkpolicies

​
4. Checking Database Server Status

Check database server status:
  • Is database server running?
  • Are there sufficient resources (CPU, RAM)?
  • Has maximum connection limit been reached?
  • Are there errors in database logs?

​
5. Checking Firewall Rules

Check firewall rules:
  • Do pods have access to database port?
  • Are network policies blocking connection?
  • Are security group rules correct?

​
6. Resolving DNS Resolution Issues

Resolve DNS resolution issues:
  • Are CoreDNS pods running?
  • Are DNS settings correct?
  • Is service discovery working properly?

​
Preventive Measures

​
1. Connection Pool Monitoring

  • Regularly monitor connection pool metrics
  • Get early warning by setting up alerts
  • Optimize connection pool usage

​
2. Health Checks

  • Configure database health checks
  • Set up liveness and readiness probes
  • Set up automatic failover mechanisms

​
3. Connection Retry Mechanism

  • Add connection retry mechanism
  • Use exponential backoff
  • Determine maximum retry count