'$Session:' scoped variables leaking memory?

[BH_Brian]

3.8.9 on windows server 2022

It seems like at least $Session: scoped variables may not be getting cleaned up when a session ends…I’m not sure how to prove this but I’ve noticed that over time the ‘powershell.exe’ proces spawned by Universal Server continues to grow in memory used till it gets up around 1-2 GB and then my dashboard stops loading. I don’t get any error messages, just nothing loads. If I restart Universal Server then the memory use goes back down and everything works for about a week until another restart is needed.

It could also be that I’m not doing something right to clean up these variables or to properly destroy/clean up a user session.

[BH_Brian]

doing some testing the same leak seems to apply to ‘Page:’ scope as well. The document says these are both custom providers so perhaps the error exists in common/shared code between the two.

[adam]

Can you check what your sessions tab looks like? It will have a list of active sessions on the server.

Sessions and pages time out with the same frequency (default is 25 minutes).

[BH_Brian]

This is our pre-prod instance. It has been sitting idle since I created the original post reporting the issue.
Memory usage, which I can see increase when I open a session to the dashboard hasn’t decreased since then.

I’ll try to come up with some demo code today. The only thing we ‘might’ be doing differently than when you test is that the dashboard has user authentication enabled…not sure if/how that might impact the session cleanup.

[BH_Brian]

Here is a little sample code that should create a noticable increase in memory utilization that can be easily seen if it is later released:

    New-UDButton -id 'Test' -text 'Test' -Icon (New-UDIcon -Icon 'broadcast_tower' -Size 4x) -OnClick{
        set-udelement -Id 'Test' -Properties @{disabled = $true}
        $junk = Get-Process
        $Session:MemoryHog = [System.Collections.ArrayList]::new()
        $Deepcopy = [System.Management.Automation.PSSerializer]::Serialize($junk)
        For($i=0; $i -lt 1000; $i++)
        {
            $Session:MemoryHog.add(([System.Management.Automation.PSSerializer]::Deserialize($deepcopy)))
        }
        set-udelement -Id 'Test' -Properties @{disabled = $false}
    }

So far it seems that:

1. Using an authenticated or unauthenticated sessions doesn’t make a difference
2. Using the ‘Integrated’ environment or a custom environment doesn’t make a difference
3. ‘Page:’ scope seems to suffer from the same issue.

even if a session doesn’t time out (for example the user ends it by navigating away or closing the tab/browser) the session should still get cleaned up correct?

[adam]

I can certainly reproduce a memory leak when doing this. It actually runs for a really, really long time and I didn’t wait for it to finish. I reduced it to 100 iterations and it still jumped the memory by a noticeable amount. This is expected with such a large set of data.

I then set my session timeout for my dashboards to 1 minute and reproduce the issue. I closed the browser and waited the time out. My session was removed but the memory wasn’t immediately reclaimed. The reason this happens is due to .NET garbage collection. Unless there is more memory pressure it won’t reclaim the memory right away in case it needs to reuse that to allocate more objects. After about 5 minutes I saw the memory drop back down to about where it was before clicking the button.

Somethings I would recommend on your end:

1. Verify that sessions are expiring properly. As mentioned, they will time out after 25 minutes.
2. Consider reducing the session timeout to expire sessions more quickly.
3. Evaluate what you are storing in session and page scope. It may be just that too much data is being stored per session.
4. Ensure that the server has enough memory to process the number of users accessing the dashboard.

To answer you question about sessions:

Sessions do not get remove immediately after a user closes the page. This is because the user could open a new page and then start to use the session again and want to work with the same data.

Pages also do not get removed when the user closes the page. This is because it’s very hard to detect whether this was actually a user action. For example, if a user is using a mobile device and locks the phone, the connection to the server will be lost. If the user unlocks the phone, the page doesn’t actually reload and the web socket is just reconnected. The page would no longer work at this point.

Another issue is that desktop browsers now force tabs to sleep if they are not used to conserve resources. This can cause the same issue.

If you are still having memory issues and want to provide a memory dump, I’m happy to review it.

[turnitoff_onagain]

How do we reduce the session timeout when using the built in web server? I checked my sessions and there are a few dozen sitting as disconnected, some are 12+ hours old.

Some users have multiple disconnected sessions for the same dashboard. I am on 3.8.8

[adam]

You shouldn’t have inactive sessions over 25 minutes unless you’ve changed the session timeout.

The session time out value is in dashboards.ps1. You can set with the -SessionTimeout parameter of New-PSUDashboard.

Either your session timeout job isn’t running for some reason or it isn’t scheduled properly. Restarting the dashboard should clear the sessions. I ran through this in my lab to make sure it was happening properly and it seems to be in my small use-case.

What environment are you running this dashboard in?

[turnitoff_onagain]

No special timeouts have been set in dashboards.ps1

The dashboard is running in powershell 5.1. I can try running in 7.x in our mirrored dev dashboard for a while to see if any of the pages have issues in the environment.

How long after restarting the dashboard should the sessions clear?

I’m seeing old sessions of my own in the admin console as well

[adam]

Ah. Those are just a historical list of sessions. If you go look in Dashboards\ Dashboard \ Session, you will see active sessions.