Netdata Community

How to troubleshoot poor performance in the WebUI

Hi everyone,

Quick question. How can we troubleshoot slow chart loads / performance issues in the WebUI? For instance, quite often when the webUI loads it will either take a while for the page to initially render or if we scroll to a specific chart down the page that chart takes a while to render.

Here’s a screenshot of the developer tools:

If I do the comparable cURL command with timing enabled:

time curl 'http://vault-dbsa102:19999/api/v1/data?chart=vault.fetch_times&_=1638542499749&format=json&points=382&group=average&gtime=0&options=ms%7Cflip%7Cjsonwrap%7Cnonzero&after=-420' \
  -H 'Connection: keep-alive' \
  -H 'Pragma: no-cache' \
  -H 'Accept: application/json, text/plain, */*' \
  -H 'Cache-Control: no-cache, no-store' \
  -H 'User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.45 Safari/537.36' \
  -H 'Sec-GPC: 1' \
  -H 'Referer: http://vault-dbsa102:19999/' \
  -H 'Accept-Language: en-US,en;q=0.9' \
  -H 'Cookie: ph_mqkwGT0JNFqO-zX2t0mW6Tec9yooaVu7xCBlXtHnt5Y_posthog=%7B%22distinct_id%22%3A%2217d480d8ba9930-074bb2e7cc14f4-1e396452-1fa400-17d480d8baaaa9%22%2C%22%24device_id%22%3A%2217d480d8ba9930-074bb2e7cc14f4-1e396452-1fa400-17d480d8baaaa9%22%7D' \
  --compressed

I see that it takes 16 seconds

0.01s user 0.01s system 0% cpu 16.031 total

Here’s the global/web config

$ curl -s http://localhost:19999/netdata.conf
# netdata configuration
#
# You can download the latest version of this file, using:
#
#  wget -O /etc/netdata/netdata.conf http://localhost:19999/netdata.conf
# or
#  curl -o /etc/netdata/netdata.conf http://localhost:19999/netdata.conf
#
# You can uncomment and change any of the options below.
# The value shown in the commented settings, is the default value.
#

# global netdata configuration

[global]
	config directory = /etc/netdata/conf.d
	history = 43932
	page cache size = 64
	dbengine multihost disk space = 512
	memory mode = save
	errors to trigger flood protection = 1000
	update every = 5
	# run as user = netdata
	# glibc malloc arena max for plugins = 1
	# glibc malloc arena max for netdata = 1
	# hostname = vault-dbsa102.prod.ostk.com
	# stock config directory = /usr/lib64/netdata/conf.d
	# log directory = /var/log/netdata
	# web files directory = /usr/share/netdata/web
	# cache directory = /var/cache/netdata
	# lib directory = /var/lib/netdata
	# home directory = /var/log/netdata
	# lock directory = /var/lib/netdata/lock
	# plugins directory = "/usr/libexec/netdata/plugins.d" "/etc/netdata/custom-plugins.d"
	# dbengine disk space = 256
	# host access prefix =
	# memory deduplication (ksm) = yes
	# TZ environment variable = :/etc/localtime
	# timezone = America/Denver
	# debug flags = 0x0000000000000000
	# debug log = /var/log/netdata/debug.log
	# error log = /var/log/netdata/error.log
	# access log = /var/log/netdata/access.log
	# facility log = daemon
	# errors flood protection period = 1200
	# OOM score = 1000
	# process scheduling policy = idle
	# pthread stack size = 8388608
	# cleanup obsolete charts after seconds = 3600
	# gap when lost iterations above = 1
	# cleanup orphan hosts after seconds = 3600
	# delete obsolete charts files = yes
	# delete orphan hosts files = yes
	# enable zero metrics = no
	# dbengine extent pages = 64

[web]
	web files owner = netdata
	web files group = netdata
	bind to = 0.0.0.0
	allow management from = localhost dbaautomation*
	# ssl key = /etc/netdata/conf.d/ssl/key.pem
	# ssl certificate = /etc/netdata/conf.d/ssl/cert.pem
	# tls version = 1.3
	# tls ciphers = none
	# ses max window = 15
	# des max window = 15
	# mode = static-threaded
	# listen backlog = 4096
	# default port = 19999
	# disconnect idle clients after seconds = 60
	# timeout for first request = 60
	# accept a streaming request every seconds = 0
	# respect do not track policy = no
	# x-frame-options response header =
	# allow connections from = localhost *
	# allow connections by dns = heuristic
	# allow dashboard from = localhost *
	# allow dashboard by dns = heuristic
	# allow badges from = *
	# allow badges by dns = heuristic
	# allow streaming from = *
	# allow streaming by dns = heuristic
	# allow netdata.conf from = localhost fd* 10.* 192.168.* 172.16.* 172.17.* 172.18.* 172.19.* 172.20.* 172.21.* 172.22.* 172.23.* 172.24.* 172.25.* 172.26.* 172.27.* 172.28.* 172.29.* 172.30.* 172.31.* UNKNOWN
	# allow netdata.conf by dns = no
	# allow management by dns = heuristic
	# enable gzip compression = yes
	# gzip compression strategy = default
	# gzip compression level = 3
	# web server threads = 1
	# web server max sockets = 256
	# custom dashboard_info.js =

Netdata version

netdata -v
netdata v1.31.0

Hi, @mjtice.

Can you try the same command but on localhost? So we can exclude network problems.

In addition, go the “Netdata” section (from the right navbar or scroll to the bottom) for the charts that show the Netdata web server’s request rate and latency. It may be serving a lot more requests than you think.

Such requests are normally handled in a few milliseconds, so there’s definitely something bad happening on your node.

So… I’m not sure why I didn’t bother checking from the localhost or even within the same data center but it looks like this slowness is related to my VPN connection. I had a coworker validate with the same result.

I’ll see if my networking team can investigate this. Thanks for your help, guys!