Continuous errors in log, PSU crashing, & missing schedules

PowerShell Universal
#1 
Product: PowerShell Universal
Version: 1.5.8
Type: MSI Install

Hi,

I’m continuously seeing the below errors in the logs for requests on different resources:

I noticed the below error as well:

These errors are littered in the log.

I’ve seen PSU start throwing 500s for every resource due to the above errors, and the PowerShellUniversal service either crashes, or is still running and needs to be kicked.

This morning, I’ve noticed that some previously setup schedules are missing from Automation, and an old deleted schedule is suddenly back.

I’d like some help with this… we’re developing a few tools for production and the recent unreliability of the platform is a bit unsettling.

#2

Crashed again this weekend - this was the error thrown in the log before crashing:

2021-01-24T20:32:21.2155679-05:00  [INF] Error executing job 24199: Operations that change non-concurrent collections must have exclusive access. A concurrent update was performed on this collection and corrupted its state. The collection's state is no longer correct.    at System.Collections.Generic.Dictionary`2.Remove(TKey key)
   at LiteDB.CacheService.SetDirty(BasePage page)
   at LiteDB.PageService.NewPage[T](BasePage prevPage)
   at LiteDB.DataService.Insert(CollectionPage col, Byte[] data)
   at LiteDB.LiteEngine.InsertDocument(CollectionPage col, BsonDocument doc, BsonType autoId)
   at LiteDB.LiteEngine.<>c__DisplayClass22_0.<Insert>b__0(CollectionPage col)
   at LiteDB.LiteEngine.Transaction[T](String collection, Boolean addIfNotExists, Func`2 action)
   at LiteDB.LiteEngine.Insert(String collection, BsonDocument doc, BsonType autoId)
   at LiteDB.LiteFileStream.WriteChunks()
   at LiteDB.LiteFileStream.Flush()
   at LiteDB.LiteFileStream.Dispose(Boolean disposing)
   at System.IO.Stream.Close()
   at LiteDB.LiteStorage.Upload(String id, String filename, Stream stream)
   at UniversalAutomation.FileStorage`1.Insert(T item) in D:\a\universal\universal\src\UniversalAutomation.LiteDB\LiteDBDatabase.cs:line 385
   at UniversalAutomation.ExecutionService.TryExecute(Job job, Schedule schedule) in D:\a\universal\universal\src\UniversalAutomation\Services\ExecutionService.cs:line 335
   at UniversalAutomation.ExecutionService.Execute(Job job, Schedule schedule) in D:\a\universal\universal\src\UniversalAutomation\Services\ExecutionService.cs:line 181 System.InvalidOperationException (f7e53d70)

Lined up with the timestamp of the service terminating unexpectedly in the event log:

unnamed-2
unnamed-21361×312 14.5 KB

We’re looking to pinpoint what is causing this to happen…

#3

Please open a support case for this: support@ironmansoftware.com

We are actively looking into this issue already,