Set-PSUCache ResourceExhausted Using Large DataSet

PowerShell Universal
1 
Product: Powershell Universal
Version: 1.5.14

Issue Reported at: Set-PSUCache ResourceExhausted when setting large data · Issue #74 · ironmansoftware/issues · GitHub

Describe the bug
When setting the value of a PSUCache entry to a value larger than 4MB(?).

To Reproduce
Steps to reproduce the behavior:

  1. In a UA script, Create a $variable storing greater than 4MB of data.
  2. Follow this with Set-PSUCache -Key AnyName -Value $variable 
$Variable = #Some 4MB or larger dataset
Set-PSUCache -Key Any -Value $Variable

Expected behavior
Successful storage of dataset in PSUCache entry

Version

Version Number: 1.5.14
Operating System:

Microsoft Windows Server 2019 Datacenter
10.0.17763 Build 17763

Additional context
I had run a process to collect system information from ~50 hosts. This included the output from:

{
        $Features = Get-WindowsFeature | Select-Object -Property DisplayName, InstallState -ErrorAction SilentlyContinue 
        $OptionalFeatures = Get-WindowsOptionalFeature -Online | Select-Object -Property FeatureName, State | Sort-Object FeatureName -ErrorAction SilentlyContinue
        $WmiSoftware = Get-CimInstance -Class CIM_Product | Select-Object -Property Name, Vendor, Version, Caption -ErrorAction SilentlyContinue
        $ComputerInfo = Get-ComputerInfo -ErrorAction SilentlyContinue 
        [PSCustomObject]@{
            Features         = $Features
            OptionalFeatures = $OptionalFeatures
            WmiSoftware      = $WmiSoftware
            ComputerInfo     = $ComputerInfo
        }   
    }

I assume this is converted to serialized data using the PSSerializer, however, I have not yet tested pre-compressing the data.

Logs

PS>TerminatingError(Set-PSUCache): "Status(StatusCode=ResourceExhausted, Detail="Received message larger than max (17130338 vs. 4194304)")"

Set-PSUCache: C:\PathTo\MyScript.ps1:182
Line |
 182 |          Set-PSUCache -Key PsuCacheKey -Value $SomeValue
     |          ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
     | Status(StatusCode=ResourceExhausted, Detail="Received message larger than max (17130338 vs. 4194304)")

None of the logs have any additional info at that time. I checked all the logs (which are set to log at the Debug level) including Event Logs, and there is nothing of note at the time the error occurred. just:

2021-03-18T18:17:23.0337284-05:00  [DBG] Job 119919 status Failed. (544534e5)
2021-03-18T18:17:23.0337497-05:00  [DBG] Job 119919 error . (534902b2)
2021-03-18T18:17:24.0908395-05:00  [INF] Job 119919 completed (551badf3)
2

This looks like a limitation in the gRPC protocol message size. We can likely increase this limit. Thanks for opening the issue.

1 Like
3

Awesome. Glad to see it is a (presumably) simple solution.

Absolutely! Now that I know about the centralized issue tracker, I’ll make sure to use it!

1 Like
4

Hi @adam, sorry to bring up an older thread, but it looks like we’re still getting hitting this 4MB limit. We tried on version 1.5.16 and 1.5.20.

5

Hey @RamonMA, we’ve increased this limit to 32GB in 2.1.0. I can roll this change back into a 1.5.x build and get it out.

6

Gotcha, much appreciated.