Skip to content

Tuning Netdata Web Log Alerts

To stop Netdata's web_log_1m_redirects alert from firing on normal HTTP-to-HTTPS redirect traffic, edit /etc/netdata/health.d/web_log.conf and raise the redirect threshold to 80%, then reload with netdatacli reload-health. The default threshold is too sensitive for any server that forces HTTPS — automated traffic hits port 80, gets a 301, and Netdata flags it as a WARNING even though nothing is wrong.

The Short Answer

sudo /etc/netdata/edit-config health.d/web_log.conf

Change the warn line in the web_log_1m_redirects template to:

warn: ($web_log_1m_requests > 120) ? ($this > (($status >= $WARNING ) ? ( 1 ) : ( 80 )) ) : ( 0 )

Then reload:

netdatacli reload-health

Background

Production nodes forcing HTTPS see a lot of 301s. The default Netdata threshold is too sensitive for sites with a high bot-to-human ratio — it was designed for environments where redirects are unexpected, not standard operating procedure.

The tuned logic warns only when there are more than 120 requests/minute AND redirects exceed 80% of traffic (dropping to 1% once already in WARNING state to prevent flapping).

Steps

  1. Identify what's actually generating the redirects
# Check status code distribution
awk '{print $9}' /var/log/apache2/access.log | sort | uniq -c | sort -nr

# Top redirected URLs
awk '$9 == "301" {print $7}' /var/log/apache2/access.log | sort | uniq -c | sort -nr | head -n 10
  1. Open the Netdata health config
sudo /etc/netdata/edit-config health.d/web_log.conf
  1. Find the web_log_1m_redirects template and update the warn line
warn: ($web_log_1m_requests > 120) ? ($this > (($status >= $WARNING ) ? ( 1 ) : ( 80 )) ) : ( 0 )
  1. Reload without restarting the service
netdatacli reload-health
  1. Verify the alert cleared
curl -s http://localhost:19999/api/v1/alarms?all | grep -A 15 "web_log_1m_redirec"

Gotchas & Notes

  • Ubuntu/Debian only: The edit-config path and apache2 log location are Debian-specific. On Fedora/RHEL the log is at /var/log/httpd/access_log.
  • The 1% recovery threshold is intentional: Without it, the alert will flap between WARNING and CLEAR constantly on busy sites. The hysteresis keeps it stable once triggered.
  • Adjust the 120 req/min floor to your traffic: Low-traffic sites may need a lower threshold; high-traffic sites may need higher.

See Also

  • [[Netdata service monitoring]]