Univeral won't load. Stuck at Syncronizing with Git

PowerShell Universal
1 
Product: PowerShell Universal

While resetting up my push only git sync it seemed to get stuck syncronizing. Nothing was happening in the repo, and I didn’t see anything happening on the server, so I restarted the service. I am now stuck at this loading screen.

image

This keeps repeating over and over in the logs

2023-06-14 08:30:08.943 -04:00 [INF] Request starting HTTP/2 GET https://URL.DOMAIN.LOCAL/api/v1/alive - -
2023-06-14 08:30:08.946 -04:00 [INF] Executing endpoint 'UniversalAutomation.AliveController.Get (Universal.Server)'
2023-06-14 08:30:08.946 -04:00 [INF] Route matched with {action = "Get", controller = "Alive"}. Executing controller action with signature Microsoft.AspNetCore.Mvc.IActionResult Get() on controller UniversalAutomation.AliveController (Universal.Server).
2023-06-14 08:30:08.947 -04:00 [INF] Executing OkObjectResult, writing value of type '<>f__AnonymousType1`3[[System.Boolean, System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e],[System.Boolean, System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e],[System.String, System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]]'.
2023-06-14 08:30:08.947 -04:00 [INF] Executed action UniversalAutomation.AliveController.Get (Universal.Server) in 0.9706ms
2023-06-14 08:30:08.947 -04:00 [INF] Executed endpoint 'UniversalAutomation.AliveController.Get (Universal.Server)'
2023-06-14 08:30:08.947 -04:00 [INF] Request finished HTTP/2 GET https://URL.DOMAIN.LOCAL/api/v1/alive - - - 200 75 application/json;+charset=utf-8 4.6630ms

So I seem to be stuck with a non working environment. @adam is there a way to wipe out the git settings without access to the UI?

2

Restarted the entire server now getting this repeating in the logs

2023-06-14 09:01:49.346 -04:00 [INF] AuthenticationScheme: Cookies was challenged.
2023-06-14 09:01:49.346 -04:00 [INF] AuthenticationScheme: Bearer was challenged.
2023-06-14 09:01:50.304 -04:00 [INF] Executing OkObjectResult, writing value of type '<>f__AnonymousType1`3[[System.Boolean, System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e],[System.Boolean, System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e],[System.String, System.Private.CoreLib, Version=6.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]]'.
2023-06-14 09:01:50.363 -04:00 [INF] Authorization failed. These requirements were not met:
DenyAnonymousAuthorizationRequirement: Requires an authenticated user.
DenyAnonymousAuthorizationRequirement: Requires an authenticated user.
3

You’d have to remove the git settings in the database if you don’t have access to the UI.

In terms of the log, I’d check to see if there is another log file with more information. Sometimes it ends up writing to two logs.

4

The other log file was just showing groom job info

I ended up restoring the whole thing from last nights backup and it’s working now, even with a git repo configured that doesn’t exist anymore.

5

Not sure what exactly is going on at the “Synchronizing with Git” part (other than the obvious), but it might be a good idea to have some sort of timeout there.

6

I agree. I opened an issue for that here: Git Sync should time out on startup · Issue #2466 · ironmansoftware/issues · GitHub

I have seen git sync hang before if using the external git client and it pops up a credential dialog.

EDIT: It looks like we do have a timeout for the external git client now of 60 seconds.

7

I am using the external client. The built in client doesn’t like my gitlab instance for some reason.

Was the timeout put in place in 4.0.0? If so I wouldn’t have seen that since I rolled back to a previous version.

That’s what I get for screwing around :joy:

8

The timeout was included 8 months ago so it’s in all recent versions of 3.x

You would also see an error in the log if it was timing out.

if (!process.WaitForExit(60000))
{
    _logger.LogError("Failed waiting for a git command after 60 seconds. Stopping the git process.");
    process.Kill();
    throw new Exception("Failed waiting for a git command after 60 seconds.");
}
9

So it was something else. It sat at that page for 30+ minutes.

10

Ok. If you ever have the bandwidth to try again, a memory dump would be helpful as it would indicate exactly where it is hanging.

11

I’ll see if I can reproduce the issue in my test environment.