Dashboard Stopped Trigger - All dashboards are stopping every minute

PowerShell Universal
1 
Product: PowerShell Universal
Version: 3.7.5

I’m seeing the dashboard stopped trigger repeating for every dashboard, every minute, for periods of time - a few hours. Then it will stop being triggered for a period of time, and then start again.

This means I am flooded with email alerts for dashboards stopping (that are still running when checked).

Any idea what is causing the dashboard stopped trigger to be fired every minute?

Example Log

01/31/2023 10:18:18 [LIVE] The dashboard “DeployVM” has stopped.
01/31/2023 10:18:38 [LIVE] The dashboard “Teams User Automation” has stopped.
01/31/2023 10:18:41 [LIVE] The dashboard “Schedule Call Diverts” has stopped.
01/31/2023 10:18:42 [LIVE] The dashboard “Call Routing” has stopped.
01/31/2023 10:18:45 [LIVE] The dashboard “Caller ID” has stopped.
01/31/2023 10:18:48 [LIVE] The dashboard “test” has stopped.
01/31/2023 10:19:10 [LIVE] The dashboard “Teams Announcements” has stopped.
01/31/2023 10:19:12 [LIVE] The dashboard “Teams Call Diverts” has stopped.
01/31/2023 10:19:15 [LIVE] The dashboard “DeployVM” has stopped.
01/31/2023 10:19:32 [LIVE] The dashboard “Teams User Automation” has stopped.
01/31/2023 10:19:35 [LIVE] The dashboard “Schedule Call Diverts” has stopped.
01/31/2023 10:19:36 [LIVE] The dashboard “Call Routing” has stopped.
01/31/2023 10:19:38 [LIVE] The dashboard “Caller ID” has stopped.
01/31/2023 10:19:40 [LIVE] The dashboard “test” has stopped.
01/31/2023 10:20:50 [LIVE] The dashboard “Teams Announcements” has stopped.
01/31/2023 10:20:55 [LIVE] The dashboard “Teams Call Diverts” has stopped.
01/31/2023 10:20:58 [LIVE] The dashboard “DeployVM” has stopped.
01/31/2023 10:21:15 [LIVE] The dashboard “Teams User Automation” has stopped.
01/31/2023 10:21:18 [LIVE] The dashboard “Schedule Call Diverts” has stopped.
01/31/2023 10:21:18 [LIVE] The dashboard “Call Routing” has stopped.
01/31/2023 10:21:21 [LIVE] The dashboard “Caller ID” has stopped.
01/31/2023 10:21:23 [LIVE] The dashboard “test” has stopped.
01/31/2023 10:21:39 [LIVE] The dashboard “Teams Announcements” has stopped.
01/31/2023 10:21:43 [LIVE] The dashboard “Teams Call Diverts” has stopped.
01/31/2023 10:21:48 [LIVE] The dashboard “DeployVM” has stopped.
01/31/2023 10:22:05 [LIVE] The dashboard “Teams User Automation” has stopped.
01/31/2023 10:22:12 [LIVE] The dashboard “Schedule Call Diverts” has stopped.
01/31/2023 10:22:14 [LIVE] The dashboard “Call Routing” has stopped.
01/31/2023 10:22:22 [LIVE] The dashboard “Caller ID” has stopped.
01/31/2023 10:22:26 [LIVE] The dashboard “test” has stopped.
01/31/2023 10:23:23 [LIVE] The dashboard “Teams Announcements” has stopped.
01/31/2023 10:23:26 [LIVE] The dashboard “Teams Call Diverts” has stopped.
01/31/2023 10:23:34 [LIVE] The dashboard “DeployVM” has stopped.
01/31/2023 10:23:48 [LIVE] The dashboard “Teams User Automation” has stopped.
01/31/2023 10:23:52 [LIVE] The dashboard “Schedule Call Diverts” has stopped.
01/31/2023 10:23:54 [LIVE] The dashboard “Call Routing” has stopped.
01/31/2023 10:23:57 [LIVE] The dashboard “Caller ID” has stopped.
01/31/2023 10:24:01 [LIVE] The dashboard “test” has stopped.
01/31/2023 10:24:17 [LIVE] The dashboard “Teams Announcements” has stopped.
01/31/2023 10:24:20 [LIVE] The dashboard “Teams Call Diverts” has stopped.
01/31/2023 10:24:22 [LIVE] The dashboard “DeployVM” has stopped.
01/31/2023 10:24:37 [LIVE] The dashboard “Teams User Automation” has stopped.
01/31/2023 10:24:39 [LIVE] The dashboard “Schedule Call Diverts” has stopped.
01/31/2023 10:24:40 [LIVE] The dashboard “Call Routing” has stopped.
01/31/2023 10:24:43 [LIVE] The dashboard “Caller ID” has stopped.
01/31/2023 10:24:45 [LIVE] The dashboard “test” has stopped.
01/31/2023 10:28:18 [LIVE] The dashboard “Teams Announcements” has stopped.
01/31/2023 10:28:21 [LIVE] The dashboard “Teams Call Diverts” has stopped.
01/31/2023 10:28:23 [LIVE] The dashboard “DeployVM” has stopped.
01/31/2023 10:28:36 [LIVE] The dashboard “Teams User Automation” has stopped.
01/31/2023 10:28:38 [LIVE] The dashboard “Schedule Call Diverts” has stopped.
01/31/2023 10:28:39 [LIVE] The dashboard “Call Routing” has stopped.
01/31/2023 10:28:41 [LIVE] The dashboard “Caller ID” has stopped.
01/31/2023 10:28:43 [LIVE] The dashboard “test” has stopped.
01/31/2023 10:28:58 [LIVE] The dashboard “Teams Announcements” has stopped.
01/31/2023 10:29:01 [LIVE] The dashboard “Teams Call Diverts” has stopped.
01/31/2023 10:29:03 [LIVE] The dashboard “DeployVM” has stopped.
01/31/2023 10:29:19 [LIVE] The dashboard “Teams User Automation” has stopped.
01/31/2023 10:29:22 [LIVE] The dashboard “Schedule Call Diverts” has stopped.
01/31/2023 10:29:22 [LIVE] The dashboard “Call Routing” has stopped.
01/31/2023 10:29:25 [LIVE] The dashboard “Caller ID” has stopped.
01/31/2023 10:29:27 [LIVE] The dashboard “test” has stopped.
01/31/2023 10:41:11 [LIVE] The dashboard “Teams Announcements” has stopped.
01/31/2023 10:41:21 [LIVE] The dashboard “Teams Call Diverts” has stopped.
01/31/2023 10:41:27 [LIVE] The dashboard “DeployVM” has stopped.
01/31/2023 10:41:49 [LIVE] The dashboard “Teams User Automation” has stopped.
01/31/2023 10:41:51 [LIVE] The dashboard “Schedule Call Diverts” has stopped.
01/31/2023 10:41:52 [LIVE] The dashboard “Call Routing” has stopped.
01/31/2023 10:41:54 [LIVE] The dashboard “Caller ID” has stopped.
01/31/2023 10:41:56 [LIVE] The dashboard “test” has stopped.
01/31/2023 10:42:15 [LIVE] The dashboard “Teams Announcements” has stopped.
01/31/2023 10:42:17 [LIVE] The dashboard “Teams Call Diverts” has stopped.
01/31/2023 10:42:19 [LIVE] The dashboard “DeployVM” has stopped.
01/31/2023 10:42:35 [LIVE] The dashboard “Teams User Automation” has stopped.
01/31/2023 10:42:37 [LIVE] The dashboard “Schedule Call Diverts” has stopped.
01/31/2023 10:42:39 [LIVE] The dashboard “Call Routing” has stopped.
01/31/2023 10:42:44 [LIVE] The dashboard “Caller ID” has stopped.
01/31/2023 10:42:47 [LIVE] The dashboard “test” has stopped.
01/31/2023 10:43:48 [LIVE] The dashboard “Teams Announcements” has stopped.
01/31/2023 10:43:51 [LIVE] The dashboard “Teams Call Diverts” has stopped.
01/31/2023 10:43:54 [LIVE] The dashboard “DeployVM” has stopped.
01/31/2023 10:44:13 [LIVE] The dashboard “Teams User Automation” has stopped.
01/31/2023 10:44:17 [LIVE] The dashboard “Schedule Call Diverts” has stopped.
01/31/2023 10:44:18 [LIVE] The dashboard “Call Routing” has stopped.
01/31/2023 10:44:22 [LIVE] The dashboard “Caller ID” has stopped.
01/31/2023 10:44:26 [LIVE] The dashboard “test” has stopped.
01/31/2023 10:44:51 [LIVE] The dashboard “Teams Announcements” has stopped.
01/31/2023 10:44:54 [LIVE] The dashboard “Teams Call Diverts” has stopped.
01/31/2023 10:44:57 [LIVE] The dashboard “DeployVM” has stopped.
01/31/2023 10:45:17 [LIVE] The dashboard “Teams User Automation” has stopped.
01/31/2023 10:45:20 [LIVE] The dashboard “Schedule Call Diverts” has stopped.
01/31/2023 10:45:21 [LIVE] The dashboard “Call Routing” has stopped.
01/31/2023 10:45:24 [LIVE] The dashboard “Caller ID” has stopped.
01/31/2023 10:45:26 [LIVE] The dashboard “test” has stopped.
01/31/2023 10:46:17 [LIVE] The dashboard “Teams Announcements” has stopped.
01/31/2023 10:46:19 [LIVE] The dashboard “Teams Call Diverts” has stopped.
01/31/2023 10:46:21 [LIVE] The dashboard “DeployVM” has stopped.
01/31/2023 10:46:34 [LIVE] The dashboard “Teams User Automation” has stopped.
01/31/2023 10:46:37 [LIVE] The dashboard “Schedule Call Diverts” has stopped.
01/31/2023 10:46:37 [LIVE] The dashboard “Call Routing” has stopped.
01/31/2023 10:46:40 [LIVE] The dashboard “Caller ID” has stopped.
01/31/2023 10:46:42 [LIVE] The dashboard “test” has stopped.
01/31/2023 10:46:56 [LIVE] The dashboard “Teams Announcements” has stopped.
01/31/2023 10:47:00 [LIVE] The dashboard “Teams Call Diverts” has stopped.
01/31/2023 10:47:02 [LIVE] The dashboard “DeployVM” has stopped.
01/31/2023 10:47:16 [LIVE] The dashboard “Teams User Automation” has stopped.
01/31/2023 10:47:18 [LIVE] The dashboard “Schedule Call Diverts” has stopped.
01/31/2023 10:47:19 [LIVE] The dashboard “Call Routing” has stopped.
01/31/2023 10:47:23 [LIVE] The dashboard “Caller ID” has stopped.
01/31/2023 10:47:25 [LIVE] The dashboard “test” has stopped.
02/01/2023 10:27:12 [LIVE] The dashboard “Teams Announcements” has stopped.
02/01/2023 10:27:15 [LIVE] The dashboard “Teams Call Diverts” has stopped.
02/01/2023 10:27:17 [LIVE] The dashboard “DeployVM” has stopped.
02/01/2023 10:27:31 [LIVE] The dashboard “Teams User Automation” has stopped.
02/01/2023 10:27:34 [LIVE] The dashboard “Schedule Call Diverts” has stopped.
02/01/2023 10:27:35 [LIVE] The dashboard “Call Routing” has stopped.
02/01/2023 10:27:38 [LIVE] The dashboard “Caller ID” has stopped.
02/01/2023 10:27:41 [LIVE] The dashboard “test” has stopped.
02/01/2023 10:27:57 [LIVE] The dashboard “Teams Announcements” has stopped.
02/01/2023 10:27:59 [LIVE] The dashboard “Teams Call Diverts” has stopped.
02/01/2023 10:28:01 [LIVE] The dashboard “DeployVM” has stopped.
02/01/2023 10:28:17 [LIVE] The dashboard “Teams User Automation” has stopped.
02/01/2023 10:28:20 [LIVE] The dashboard “Schedule Call Diverts” has stopped.
02/01/2023 10:28:21 [LIVE] The dashboard “Call Routing” has stopped.
02/01/2023 10:28:24 [LIVE] The dashboard “Caller ID” has stopped.
02/01/2023 10:28:26 [LIVE] The dashboard “test” has stopped.
02/01/2023 10:33:16 [LIVE] The dashboard “Teams Announcements” has stopped.
02/01/2023 10:33:17 [LIVE] The dashboard “Teams Call Diverts” has stopped.
02/01/2023 10:33:20 [LIVE] The dashboard “DeployVM” has stopped.
02/01/2023 10:33:37 [LIVE] The dashboard “Teams User Automation” has stopped.
02/01/2023 10:33:43 [LIVE] The dashboard “Schedule Call Diverts” has stopped.
02/01/2023 10:33:45 [LIVE] The dashboard “Call Routing” has stopped.
02/01/2023 10:33:49 [LIVE] The dashboard “Caller ID” has stopped.
02/01/2023 10:33:52 [LIVE] The dashboard “test” has stopped.
02/01/2023 10:34:16 [LIVE] The dashboard “Teams Announcements” has stopped.
02/01/2023 10:34:23 [LIVE] The dashboard “Teams Call Diverts” has stopped.
02/01/2023 10:34:26 [LIVE] The dashboard “DeployVM” has stopped.
02/01/2023 10:34:40 [LIVE] The dashboard “Teams User Automation” has stopped.
02/01/2023 10:34:44 [LIVE] The dashboard “Schedule Call Diverts” has stopped.
02/01/2023 10:34:45 [LIVE] The dashboard “Call Routing” has stopped.
02/01/2023 10:34:49 [LIVE] The dashboard “Caller ID” has stopped.
02/01/2023 10:34:52 [LIVE] The dashboard “test” has stopped.
02/02/2023 11:51:37 [LIVE] The dashboard “Caller ID” has stopped.

Thanks,
Finn

2

Stop is also triggered during a restart so it could be some of the following reasons:

It could be:

Variables being updated by something (Set-PSUVariable)

Dashboards restart when variables are set because we need to recreate the environment. This is an inefficiency we should resolve but could be the root cause.

Environments are being updated

Same reason as above but not as common to update this all the time.

Files are changing in the dashboard folder

Some file is changing in the dashboard folder causing the configuration system to reload the dashboard. I’ve seen this happen when users log to the dashboard folder.

Recycle schedule is set on the environment

The dashboard processes will restart based on the recycle schedule to free up resources

Max Virtual Memory is set on the environment

The dashboard processes are reaching max virtual memory and restarting.

3

Thanks Adam, this might well explain the behaviour!

4

It appears that if any change is made to any file in the “Repository” folder (or subfolder), all dashboards are reloaded. Is this the expected behaviour?

I was under the impression that the “Auto Deploy” flag had to be true to reload a dashboard if it’s file changed?

5

UPDATE: Since version 3.7.10 the dashboards are not reloading quite as much.
Changes to 1 dashboard file no longer causes all of the dashboards to reload.
Changes to 1 script file no longer cause all of the dashboards to reload.
Changes to a variable will cause all dashboards to reload.

The Auto-Deploy flag is working as expected again.
An individual dashboard will reload if the file is changed and the Auto-Deploy flag is true.
Dashboards will not reload if the file is changed and Auto-Deploy is set to false.

Also the debugger environment is now working correctly, the debug sessions were ending due to the dashboards restarting so much.

Thank you @adam for the bug fixes!

Thanks,
Finn

1 Like