Fixing certbot Renewal Failures: Cause-by-Cause Troubleshooting for certificate expired

Fixing certbot Renewal Failures: A Cause-by-Cause Troubleshooting Guide for certificate expired

If a Your certificate (...) will expire in 0 days email landed in your inbox at dawn, open a terminal before you pour coffee. There is only one command to run before the site goes down.

sudo certbot renew --dry-run

This command simulates the renewal process without touching the live certificate. The error it prints is exactly why a real renewal would fail. This article maps those messages to causes and gives you copy-paste commands to fix them.

Note: if renewal succeeded but clients (curl, Java apps, etc.) still fail certificate trust, this is not the article you want—read the 'unable to get local issuer certificate' post instead. This article covers server-side certificate renewal; that one covers client-side CA chain verification. The symptoms look similar; the causes are completely different.

Step 1: Reproduce the failure with dry-run and read the logs

First, check the current certificate status and expiry date.

sudo certbot certificates

Look at Expiry Date and VALID: N days in the output. If expiry is imminent, run --dry-run immediately and inspect the logs on failure.

# Detailed log of the most recent attempt
sudo tail -n 50 /var/log/letsencrypt/letsencrypt.log

# Auto-renewal timer logs (snap install; for apt use -u certbot.service)
sudo journalctl -u snap.certbot.renew.service --since "1 day ago"

Recent certbot often ships as a snap by default, so the binary path is frequently /snap/bin/certbot, and ECDSA is now the default certificate type. The log location (/var/log/letsencrypt/) is unchanged, so start there. Grab the one key line from the log and jump straight to the matching cause below.

Narrow down the cause by the message shown in the browser or in certbot renew output.

Separately from the renewal result, make a habit of checking the certificate actually being served right now. If the file is new but the remote result shows the old expiry date, the web server hasn't reloaded.

# Expiry date of the file
sudo openssl x509 -enddate -noout -in /etc/letsencrypt/live/example.com/fullchain.pem
# Validity of the certificate actually being served
echo | openssl s_client -connect example.com:443 -servername example.com 2>/dev/null | openssl x509 -noout -dates

Step 2: Cause-by-cause playbook (copy-paste commands)

Cause 1: Port 80 blocked (HTTP-01 challenge)

If you see this in the logs, it is a firewall/security group problem.

Timeout during connect (likely firewall problem)

HTTP-01 challenge requires Let's Encrypt servers to reach your port 80 from the outside. If that is blocked, renewal always fails.

# Who is listening on port 80
sudo ss -tlnp | grep :80

# Allow through the OS firewall
sudo ufw allow 80/tcp

# On cloud (AWS/GCP/Oracle), also confirm security group / network ACL inbound 80 is open

On cloud instances especially, opening the OS firewall still times out if security-group inbound is closed. Check both.

Cause 2: nginx/apache plugin conflict

Here's how to choose a plugin.

--standalone binds port 80 itself, so it conflicts if nginx already owns port 80. In that case, use hooks to stop and start briefly.

sudo certbot certonly --standalone \
  --pre-hook "systemctl stop nginx" \
  --post-hook "systemctl start nginx" \
  -d example.com

Cause 2-1: webroot path mismatch (404)

If the path you pass with --webroot -w differs from the web server's real document root, Let's Encrypt can't find the challenge file and you get a 404.

sudo nginx -T 2>/dev/null | grep -E '^\s*(server_name|root)'      # nginx
sudo apachectl -S; grep -R "DocumentRoot" /etc/apache2/ /etc/httpd/ 2>/dev/null   # apache
# Check with a test file that it's actually visible from outside
sudo mkdir -p /var/www/html/.well-known/acme-challenge
echo ok | sudo tee /var/www/html/.well-known/acme-challenge/test
curl -i http://example.com/.well-known/acme-challenge/test

The plugins (--nginx/--apache) handle installation and reload, but if you only obtained the certificate with webroot/standalone, you must set up the reload yourself.

Cause 3: DNS-01 validation failure

Wildcard (*.example.com) certificates require DNS-01. It fails when the TXT record has not propagated.

# Confirm _acme-challenge TXT record has propagated
dig TXT _acme-challenge.example.com +short

If you use a DNS plugin, credential file permissions are a common trap.

# Cloudflare example: credentials file must be mode 600
chmod 600 ~/.secrets/cloudflare.ini

sudo certbot certonly \
  --dns-cloudflare \
  --dns-cloudflare-credentials ~/.secrets/cloudflare.ini \
  -d "*.example.com" -d example.com

Cause 4: Rate limit exceeded

If you see this in the logs, you have hit the limit.

too many certificates already issued

Let's Encrypt allows 50 certificates per registered domain per week and 5 duplicate certificates per week. To avoid burning the quota with repeated failures, always test on staging first.

sudo certbot certonly --staging -d example.com

Staging has a separate quota, so you can test freely. Once it works, drop --staging and run for real. If the limit is already hit, you typically wait up to a week for it to reset—so the real lesson is: don't burn quota on tests.

Cause 5: Expired because the timer/cron never ran

This is the most common "silent death" pattern: the command works, but auto-renewal never ran, so the cert expired.

The timer name depends on how certbot was installed.

# List certbot-related timers regardless of install method
systemctl list-timers --all | grep -i certbot

# snap example: status and recent run logs
systemctl status snap.certbot.renew.timer
sudo journalctl -u snap.certbot.renew.service --since "7 days ago"

# If you use cron
crontab -l; ls /etc/cron.d/ | grep -i certbot

If the timer is inactive or missing from the list, auto-renewal has stopped. Turn it back on using the name from the table (sudo systemctl enable --now snap.certbot.renew.timer for snap, certbot.timer for apt).

If you use cron directly, cron's PATH may not include /snap/bin, so use the absolute path. Installing certbot via both snap and apt runs different binaries and timers and causes confusion, so keep only one (check with which -a certbot).

# crontab -e (absolute path for a snap install)
0 3 * * * /snap/bin/certbot renew --quiet --deploy-hook "systemctl reload nginx"

Step 3: Prevent recurrence — automate nginx reload with a deploy-hook

The trap I see most in production is this one: renewal succeeded, but the browser still says "expired." The cause is simple: nginx is still holding the old certificate in memory because it never reloaded.

Attach a deploy-hook so nginx reloads only on successful renewal.

sudo certbot renew --deploy-hook "systemctl reload nginx"

Once a renewal with --deploy-hook succeeds, certbot saves the value as renew_hook under [renewalparams] in /etc/letsencrypt/renewal/<domain>.conf and keeps using it on later renewals. If you edit the file by hand, put it inside the [renewalparams] section.

# /etc/letsencrypt/renewal/example.com.conf
[renewalparams]
renew_hook = systemctl reload nginx

To apply it to every certificate, the simplest way is an executable script in the deploy hook directory.

sudo tee /etc/letsencrypt/renewal-hooks/deploy/reload-nginx.sh >/dev/null <<'EOF'
#!/bin/sh
systemctl reload nginx
EOF
sudo chmod +x /etc/letsencrypt/renewal-hooks/deploy/reload-nginx.sh

With short-lived ACME certificates (6-day lifetime) and OCSP stapling being retired, auto-renewal reliability matters even more going forward. The shorter the renewal cycle, the faster a missed hook becomes an outage—treat deploy-hook as mandatory, not optional.

Step 4: Emergency issuance on D-1 (expires tomorrow)

If auto-renewal is broken and the cert expires tomorrow, force a new issuance with minimal downtime.

sudo certbot certonly --standalone \
  --pre-hook "systemctl stop nginx" \
  --post-hook "systemctl start nginx" \
  -d example.com --force-renewal

--force-renewal issues a new certificate regardless of remaining validity. nginx downtime is usually a few seconds, so aim for a low-traffic window—or, if you can, issue with --webroot for zero downtime.

Conclusion: Copy-paste checklist

1. Run certbot renew --dry-run and capture the error message
2. Match keywords in /var/log/letsencrypt/letsencrypt.log
3. timeout → port 80 / security group; "too many certificates" → rate limit / staging
4. DNS-01 → dig TXT _acme-challenge..., credential file mode 600
5. Check auto-renewal with systemctl list-timers --all | grep -i certbot (snap: snap.certbot.renew.timer)
6. Automate nginx reload with a deploy-hook

One more time: if renewal clearly succeeded but clients still fail trust, that is not a server-side renewal problem—it is client-side CA chain verification. See the 'unable to get local issuer certificate' article.

FAQ

Q. dry-run succeeds but the real renewal fails. Why? A. dry-run uses the staging server, so it is not subject to production rate limits. If only the real renewal fails, you most likely hit the production rate limit (too many certificates already issued). Check the logs and wait for the window to reset.

Q. Renewal succeeded but the browser still shows expired. A. nginx/apache is still serving the old cert from memory. Manually run systemctl reload nginx, and register the reload with --deploy-hook or a script in /etc/letsencrypt/renewal-hooks/deploy/ to prevent it next time.

Q. Only one of several domains fails to renew. A. That domain's DNS A record likely points at a different server, or the port-80 validation path is blocked. Check the failing domain individually with dig and curl -I http://<domain>/.well-known/acme-challenge/test.