How to Fix HikariPool-1 Connection is not available 30000ms

How to Fix the HikariPool-1 Connection is not available 30000ms Timeout

A 30-Second Diagnosis to Silence the 3 A.M. Alerts

HikariPool-1 - Connection is not available, request timed out after 30000ms

The moment this log hits Slack, it is almost always a sudden traffic spike or a connection that was never returned. The first perspective to lock in is this:

The database is not dead — your application's connection pool is exhausted.

If too many clients / Too many connections on the DB server means "the DB hit its limit," this error means "there is no connection left to lend from your application's pool." After waiting 30 seconds (the default connectionTimeout), nobody returned a connection, so the request gave up. Spring Boot 3.x uses HikariCP as the default DataSource, so the settings in this post apply with no extra dependency.

This post keeps theory to a minimum and focuses on immediate recovery in this order: read the logs → branch by cause → copy-paste config → verify.

1. Read the Error Log Precisely: Anatomy of One Pool-Stats Line

HikariCP usually prints a stats line like this right before the timeout. About 90% of the diagnosis lives in that single line.

HikariPool-1 - Pool stats (total=10, active=10, idle=0, waiting=12)

• total: total connections currently held by the pool
• active: connections currently in use (borrowed)
• idle: idle connections (available to lend)
• waiting: threads blocked waiting for a connection

Pattern reading table

If active=max and waiting>0, treat it as exhaustion and move into the five-cause branch.

2. Branching Across the Five Causes: Diagnosis and Config per Cause

① Connection leak (not returned)

Symptoms: active sticks at max and never returns to idle over time. Does not recover even after traffic drops. Diagnosis: Enable leakDetectionThreshold and look for "Apparent connection leak detected" in the logs (see section 4).

② Pool size too small

Symptoms: Fine most of the time; waiting spikes only at peak. Connections are returned normally. Diagnosis: In APM/metrics, check whether active is constantly pinned at max.

③ Slow queries and long transactions

Symptoms: active=max because each connection is held for a long time. Evidence in DB CPU / slow logs. Diagnosis (MySQL):

SHOW PROCESSLIST;
SHOW STATUS LIKE 'Threads_connected';

Diagnosis (PostgreSQL) — find sessions sitting idle with an open transaction:

SELECT pid, state, now() - xact_start AS duration, query
FROM pg_stat_activity
WHERE state = 'idle in transaction'
ORDER BY duration DESC;

④ DB max_connections limit

Symptoms: The pool wants to grow, but the DB rejects new connections. Classic microservice failure: number of services × pool size exceeds the DB limit. Diagnosis (PostgreSQL): Compare SHOW max_connections; with the current connection count.

⑤ Network/firewall idle timeout

Symptoms: Intermittent timeout only on the first request during a quiet night. The pool mistakes a dead connection for a live one. Fix: Set maxLifetime shorter than the DB/firewall timeout (diagram in section 5).

Copy-paste application.yml

spring:
  datasource:
    hikari:
      # 동시에 빌려줄 수 있는 최대 커넥션. 풀 사이즈 공식으로 산정(아래 참고)
      maximum-pool-size: 20
      # 항상 유지할 최소 유휴 커넥션. 보통 max와 같게 두면 풀 출렁임이 줄어듦
      minimum-idle: 20
      # 커넥션을 못 받을 때 대기 시간(ms). 30초는 너무 길다 → 빠르게 실패시켜 스레드 회수
      connection-timeout: 10000
      # 유휴 커넥션 정리 시간(ms). minimum-idle == max면 무의미
      idle-timeout: 600000
      # 커넥션 최대 수명(ms). 반드시 DB wait_timeout/방화벽보다 짧게! (예: 30분)
      max-lifetime: 1800000
      # 미반납 의심 임계(ms). 운영에서 30초 설정해 누수 스택트레이스 확보
      leak-detection-threshold: 30000

Practical tip: Leaving connection-timeout at the default 30 seconds means that during an outage, threads get stuck for 30 seconds each and Tomcat workers freeze as a whole. I prefer 10 seconds so we fail fast and retry. An immediate error plus retry is better for users than a 30-second blank screen.

3. Pool Size Formula

Blindly setting maximum-pool-size: 100 is the worst prescription. Growing the pool dumps concurrent queries onto the DB, and the DB dies first. The HikariCP wiki recommends this starting point:

connections = (core_count * 2) + effective_spindle_count

• core_count: CPU cores on the DB server
• effective_spindle_count: number of disks (for SSD/cloud, usually approximate as small or 0–1)

For an 8-core DB, roughly 8*2 + 1 = 17 is a reasonable starting point. In a microservice environment, you have to multiply again. 10 services × pool of 20 × 5 autoscaled instances = 1,000 connections hitting one DB. When container autoscaling adds instances, the aggregate pool size explodes with them, so manage pool size against the total, not "per instance."

4. Catching the Leak Culprit: leakDetectionThreshold

With leak-detection-threshold: 30000 enabled, HikariCP prints a stack trace for any connection not returned within 30 seconds.

Apparent connection leak detected:
  java.lang.Exception
    at com.example.order.OrderService.process(OrderService.java:42)
    ...

The top of that stack, OrderService.java:42, is the code that borrowed a connection and never returned it.

Before / After

Case A — missing try-with-resources

// Before: 예외 발생 시 close()가 호출되지 않아 누수
Connection conn = dataSource.getConnection();
PreparedStatement ps = conn.prepareStatement(sql);
ps.executeQuery(); // 여기서 예외 나면 conn 영영 안 돌아옴

// After: try-with-resources로 자동 반납 보장
try (Connection conn = dataSource.getConnection();
     PreparedStatement ps = conn.prepareStatement(sql)) {
    ps.executeQuery();
}

Case B — @Transactional scope too wide

// Before: 외부 API 호출이 트랜잭션 안에 있어 커넥션을 수초간 점유
@Transactional
public void order() {
    save(entity);
    slowExternalApiCall(); // 3초 동안 DB 커넥션 붙잡고 대기
}

// After: DB 작업만 트랜잭션으로 좁히고, 외부 호출은 밖으로
public void order() {
    saveInTx(entity);      // @Transactional 메서드
    slowExternalApiCall(); // 커넥션을 이미 반납한 상태
}

If you use JPA/Spring, most leaks come from two patterns: a manual getConnection(), or a transaction whose scope is too wide.

5. Diagram: maxLifetime vs. DB Idle Timeout Collision

This is the core of cause ⑤. The DB wait_timeout or a firewall silently killed an idle connection, but the pool still believes that connection is alive and lends it out. → The borrower fails immediately on a dead connection → retries pile up into timeouts.

방화벽 idle cut: 600초
DB wait_timeout : 28800초(MySQL 기본 8시간)
HikariCP maxLifetime: 1800초  ← 가장 짧게!

[OK]  풀이 1800초마다 스스로 커넥션을 폐기·재생성 → 죽은 커넥션을 빌려주지 않음
[BAD] maxLifetime > 방화벽/DB timeout → 풀이 죽은 커넥션을 살아있다고 착각

Rule: maxLifetime must always be shorter than the smaller of DB wait_timeout and the firewall idle timeout. Typically set it to 80% or less of that value.

Conclusion: Recurrence-Prevention Checklist

• Is leak-detection-threshold enabled in production (early leak warning)
• Was maximum-pool-size sized from the formula (do not just crank it up)
• Is maxLifetime < DB wait_timeout / firewall idle timeout
• Are pool stats (active/idle/waiting) collected as metrics with alerts
• Does the microservice aggregate (services × pool × instances) stay under DB max_connections

FAQ

Q. Will increasing connection-timeout (30 seconds) even more fix it? A. No. Stretching the timeout only delays the error; the exhausted pool stays exhausted. Waiting threads stay blocked even longer and overall latency gets worse. You have to fix the cause (leak, size, or slow queries).

Q. Is a bigger pool always safer? A. It is dangerous. Start from (core_count * 2) + spindle. Growing the pool floods the DB with concurrent queries so the DB dies first, and in microservices it multiplies by the number of services and exceeds max_connections.

Q. How do I tell a leak from a slow query? A. If active does not drop even after traffic falls, it is a leak. If active only pins at max during peaks and recovers otherwise, it is slow queries or an undersized pool. Combining leak-detection-threshold logs with pg_stat_activity (idle in transaction) makes it definitive.