Dashboard CPU usage gets stuck at 100% after a few hours of running

Universal Dashboard
1

I’m on 2.9.0, server CPU is an Intel Xeon Silver 4215

How do you typically troubleshoot resource usage? I have a dashboard that needs to be killed and restarted 3-4 times a day because it sends the server’s CPU usage to 100% and holds there.

Is this a known issue? Are there things I can look out for that typically cause UD to get stuck like this? The dashboard isn’t particularly heavy or intensive, nor is it very computationally complex. I have some UDCharts, UDGrids, UDinputs and UDTables, as well as some invoke-sqlcmd2 calls. The server usually hangs out around 15-20% CPU utilization but after a few hours of running this one dashboard it goes through the roof.

If this is a known issue, are there any tips on ways to automate restarting this dashboard? I’m in kind of an uncomfortable situation because I convinced executive leadership to move some processes and procedures to a dashboard I wrote and now I’m having to keep an eye on the server to restart the DB every couple of hours

2

Depends how you hosting your dashboard, like I use IIS so I can stop and start my dashboard say every morning at 04:00am when no-one would be using it, using a powershell scheduled job. I know Adam Driscoll released some thing for visual studio code, and I downloaded it, and it showed you the amount of endpoints you had running…this kind of made me not use endpoints as much as I was. I been reading this forum long enough and have seen other posts on this type of issue.
At a guess it is going to be one of your endpoints you have re-occuring every X amount of time, that add-on for visual studio should show this…You also need to make sure you give all your endpoints -id parameter to identify the one that is causing the issue…I hope this gives you a start in troubleshooting it

1 Like
3

Appreciate your response, as always.

I just have the dashboard running in a 'shell window on our app server. I think IIS hosting is going to be the way to go though

4

You may want to also consider collecting some information with a tool like DebugDiag: https://www.toptal.com/dot-net/hunting-high-cpu-usage-in-dot-net

Although it might not point to the exact script or endpoint, it may point to whether this is an issue with UD itself of something that you script is exhibiting.

1 Like
5

If anyone comes here looking for a way to automate killing the process and restarting of the dashboard without hosting in IIS, here’s what I did:

I put these two lines at the beginning of the PS script (you would replace OMNIBOARD with the name of your dashboard):

gps powershell|?{$_.mainWindowTitle -eq ‘OMNIBOARD’}|spps
$host.ui.RawUI.WindowTitle = “OMNIBOARD”

I then used task scheduler to run the script every 2 hours (with a -noexit flag to keep the console window open, of course). The task will find the running dashboard, kill it, and spawn a new one. This should hopefully eliminate the possibility of CPU hitting 100 after 3-4 hours.

Obviously the ideal solution is to host in IIS, but that would probably take a full day to set up and I can’t spare that full day at the moment :eyes:

1 Like
6

I confirmed that it was the 48 cache variables and 7 endpoints I was using that was causing this problem. I removed every single one of them and I haven’t had a single problem all day

1 Like