Trouble starting a dashboard, HTTP Error 500

Hello, we’ve run into this problem before but I can’t recall what the fix was. Has anyone else experienced this?

When starting UD, the server seems to start normally, no errors. But when trying to browse to the dashboard I immediately receive an error 500. This is true for our custom dashboard as well as an empty dashboard.

With verbose logging enabled we can see the following when sending a request to the server:

[Error] Microsoft.AspNetCore.DataProtection.KeyManagement.KeyRingProvider An error occurred while reading the key ring.

Screenshot:

Hi

did you solve this issue? i have the exact same issue.
Or does anyone else have a solution?

regards Nathan

Did anyone find a solution to this? I now randomly have this problem after UD working for many months locally it now won’t run any dashboards.

[Error] Microsoft.AspNetCore.DataProtection.KeyManagement.KeyRingProvider An error occurred while reading the key ring.

Bit of an old thread but this might help someone if they have this in the future. I had the same happen again on my PC. After spending a day trying various things I think this is to do with having Powershell Universal on your machine (even if you have uninstalled it). After uninstalling it I had to go through and remove all the files/folders from appdata and program data then running v2.9 of UD locally worked!

Back to the drawing board. The problem has returned and I have no idea why, it just started randomly happening again.

@adam would you have any pointers as to why this might happen? This only seems to happen when running locally - If I run it in an Azure webapp I haven’t had this problem. It’s the case for any dashboard, I can create a new blank dashboard and it will behave the same.

Previously I have used an Azure VM for ‘local’ development and when this happened every few months I would just rebuild a new VM and the problem disappears but this got quite expensive so I’m now developing on my PC so I need to get to the bottom of this issue.

Here’s a snippet of the log file where it shows the error:

15:40:01 [Info] Microsoft.AspNetCore.Hosting.Internal.WebHost Request starting HTTP/1.1 GET http://localhost:10000/  
15:40:01 [Info] Microsoft.AspNetCore.DataProtection.KeyManagement.XmlKeyManager Creating key {3b046cf4-ca4a-42da-b5b2-85c02ab53d63} with creation date 2021-08-17 14:40:01Z, activation date 2021-08-17 15:12:53Z, and expiration date 2021-11-15 14:40:01Z.
15:40:01 [Error] Microsoft.AspNetCore.DataProtection.KeyManagement.KeyRingProvider An error occurred while reading the key ring.
15:40:01 [Error] Microsoft.AspNetCore.Server.Kestrel Connection id "0HMB1HBAM9DLC", Request id "0HMB1HBAM9DLC:00000001": An unhandled exception was thrown by the application.
15:40:01 [Info] Microsoft.AspNetCore.Hosting.Internal.WebHost Request finished in 28.2374ms 500 
15:40:21 [Debug] Quartz.Core.QuartzSchedulerThread Batch acquisition of 0 triggers

It looks like the default key management is stored in app data: Data Protection key management and lifetime in ASP.NET Core | Microsoft Docs

It also looks like keys expire after 90 days: Data Protection key management and lifetime in ASP.NET Core | Microsoft Docs

From the sound of the article, it should generate new keys after those 90 days.

I’m wondering if you can clear the key persistence to avoid having to rebuild the machines. You could back up the folder described in that article and then attempt to start the dashboard after removing it.

wow thanks for such a quick reply!
Yes, does look like it’s related to that. Looking at the log it’s trying to create the key which expires after 90 days (which makes sense why the issue returns after 3 months). I backed up and removed the DataProtection-Keys folder and it does seem to recreate it but the folder remains empty when I try to re-load the dashboard and I get the same error.

Hmmm. Bummer. I wonder if it’s trying to access something else that needs to be cleared. You might be able to use procmon to examine the files\registry keys PS might be trying to access.

UD doens’t setup any data protection stuff so everything is default.

I can’t see anything useful in procmon (although I’m not discounting that I could have missed something as it’s a minefield of logs).

I can see that it’s querying that directory for *.xml and the result is “no such file”. It doesn’t appear to attempt to write to that location though which I would expect it to do if it tries to create a new key.

I also tried setting full control over HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\ASP.NET on administrators, local service, network service and system which is essentially what this script does
aspnetcore/Provision-AutoGenKeys.ps1 at main · dotnet/aspnetcore · GitHub which I found from this article:
windows - What is the default registry location that is used for the registry key ring for ASP.NET Core Data Protection system - Server Fault
That is slightly different as it’s for IIS app pools which we aren’t using but thought I’d manually do what that script does in case it was using the registry and had a permission issue, but unfortunately that didn’t work.

I also tried setting the:

<system.applicationHost><applicationPools><applicationPoolDefaults><processModel>

setProfileEnvironment="false"

as well as

setProfileEnvironment="true"

in the applicationHost.config file. But again, to no avail!

I’m a bit stumped!

Man. That’s very weird. It certainly seems like it must be storing something somewhere if reinstalling the OS works.

I’ll poke around in the ASP.NET Core source to see if anything pops out that you can try.

I spun up a new VM and recorded with perfmon to see what happens when it works, it does attempt to write to the appdata location and creates the .xml key file. Knowing the expected behaviour I went back to my PC and filtered perfmon by that location. I reinstalled the PS UD module, removed the DataProtection-Keys folder and ran a new blank dashboard - For some reason that bit seems to be key, running my existing app still didn’t work until I did this. After running a new blank dashboard it recreated the DataProtection-Keys folder and was able to write the .xml file. I re-loaded my app and it now works fine.

What is strange though is I tried to re-test the theory by removing the DataProtection-Keys folder and running my app and it is now able to create the .xml file every time.

It would appear that the combination of reinstalling the UD module and running a blank dashboard allowed it to do something which now allows the process to work as it should do again. Seems odd and I have no idea why this worked but it did for me! If it happens again in 3 months when the key expires I’ll do the same again and update this thread with results.

1 Like

WOAH… this article was spot on… in my case all I had to do was clear-out the DataProtection-Keys folder contents restart the server and WALLLAAHH!!! Dashboard started again :wink:

Just a quick update as the problem returned after 3 months when the key expired. I found this time that running the dashboard still wasn’t re-creating the key after clearing out and reinstalling the module but what I did find was that if I ran the dashboard in PS ISE instead of VS Code it was able to create it. I’m wondering if the module re-install isn’t necessary and after clearing the data protection keys folder a dashboard just needs to be spun up once in ISE to create the key file.