UD Tech Discussion - Moving to ASP.NET Core 3.1 and .NET Core 3.1

Universal Dashboard
1

We’ve spent a lot of time rearchitecting the React\Client side in UDv3. This brings tons of new controls but we still have tech debt on the backend. I just want to jot this down so people understand where we are headed post-UDv3. This topic covers architectural changes that will happen likely in the v3.1 UD time frame rather than with v3.0.

The Problem(s)

Universal Dashboard is built on ASP.NET Core 2.1 and .NET Core 2.1. It uses the Kestrel web server and is hosted directly in PowerShell. This means that all the assemblies that UD requires are loaded directly into PowerShell.

Assembly Conflicts

From my perspective, this is the most annoying thing.

When you try to load UD in the VS Code extension or try to use the Az module, you may run into problems because both of those use the same .NET assemblies but different versions as UD. This causes UD to fail to load. :sob:

Version Lock-In

UD needs to support Windows PowerShell v5.1, which is built on the .NET Framework. New versions of ASP.NET Core no longer support .NET Framework. We can’t upgrade past version 2.1 so we are losing out on so many cool features (Blazor?), security fixes, and bug fixes (WebSocket reconnects!). Later next year, ASP.NET Core v2.1 will no longer be supported by Microsoft. :sob:

The Solution

The Universal Dashboard web server will become it’s own executable. It will still be shipped in the module but won’t be loaded directly into PowerShell.

Architecture

Configuration 1

One idea is to have Start-UDDashboard start the UD.exe process. UD then connects back to PS.exe to execute everything.

PowerShell.exe        |     UniversalDashboard.exe        |   Client Browser

Start-UDDashboard ->  |     Starts Web Server             |          
Runs PowerShell       |     <- Processes User Request     | < - Requests a page 
Return result         |     Cache request if necessary -> | Renders result

There will be a smaller performance hit so we hope to introduce some request caching and performance improvements to resolve this issue.

This means that you could still use PowerShell v5.1 and we can upgrade UD and avoid assembly loading conflicts.

Configuration 2

This won’t resolve the assembly conflict issue but may be fine for some users.

If you’re using PowerShell v7, we’ll be able to execute the PowerShell completely in the UD process (I think). This means that running PSv7 will be faster since we can host the runtime directly in UDv3.

PowerShell.exe          |     UniversalDashboard.exe        |       Client Browser

Start-UDDashboard ->    |     Starts Web Server             |          
                        |     Processes User Request        | < - Requests a page 
                        |     Runs PowerShell Script        |
                        |     Cache request if necessary -> |    Renders result

Configuration 3

One other configuration that I’m considering is having a per-session PowerShell.exe\Pwsh.exe instance. This means that whenever a user connects to UD, we spin up a new PowerShell.exe\Pwsh.exe instance. All the interaction that is done by the user happens in that process and then when that session is terminated, the process goes away. This cleans up any resources that the user allocated. This also means that sessions don’t conflict with each other at all.

This obviously would use the most resources and be slower because we need to spin up a new PowerShell instance but we might consider it as a feature for users that desire this type of isolation.

It also breaks the caching story because we now have separate processes so using the $Cache variable would require an external, distributed cache. This is actually sitting in the issue backlog.

Status

I’ve only played with this a little bit. I’ve managed to upgrade UDv3 to .NET Core 3.1 and ASP.NET Core 3.1. It “just works” but getting the separation between PowerShell execution and the webserver is complicated since everything is tightly coupled.

I just wanted to provide a forward-looking tech discussion on this so others understand the problem and the solution that will be required.

2

Mm… Let me ask you (all) a question, is it really necessary to continue supporting Powershell 5.1/.NET Framework once .NET 5.0 is released? Even if you are using Powershell 5.1 in a complete company environment (like we do) it is still possible to install and use Powershell 7 on the UD Server (OR to install .NET 5.0 on the UD Server). It would simply be another requirement for UniversalDashboard.
But I might be missing something?

Otherwise, VSCode runs perfectly with Powershell 7 and UniversalDashboard 2.9.0 on my machine (With AZ module). I can also load UniversalDashboard in VSCode without any problems. What exactly was/is the problem? Maybe I can help.

3

Dropping .NET Framework\v5.1 support would be a dream but from my understanding is that most people are relying on it. Not sure about people’s ability to run PSv7\7.1 in their environments.

Here’s the discussion on VS Code: Cant start dashboard from Visual Code =====> PowerShell Integrated Console <=====

The problem with the Az module discussion is here: Community version conflicts with Azure 'Az' module

The problem is that this could really crop up with any assembly. I’ve also seen mixing and matching of other modules with UD causing this as well.

Honestly, it’s a bit of a problem with PowerShell in general. Any module could potentially conflict with another module. UD just takes advantage of some pretty common assemblies which seem to cause problems more often.

4

I’d love to hear thoughts on PowerShell v5.1 support from the community.

@AlonGvili @augustin.ziegler @psDevUK @wsl2001 @BoSen29 @OpsEng @mylabonline

I know I asked this question before and it was mostly a resounding response that everyone was using v5.1. That said, what’s stopping the use of PSv7? Is it corporate policies? Scripts that don’t work in PSv7? UD requires a lot of config already so I wonder if taking a dependency on PSv7 would really be that big of a deal for most.

5

Response from Twitter as well. It seems like module support is certainly an issue.

6

This i what I am thinking about (hope this will help).

I love the progress of Powershell with v7 but to be honest do not use it in production for the following reasons:

  • there is no reason to but the effort in order to first test every script (nevertheless if it would work without a change)
  • another problem still is the native module compatibility (I know there are module wrappers)

The architecture for all my .Net API projects are completely decopeled from the PS module and the API using a REST API … but this was currently only the case because these are backend API (no front end) … which makes the architecture much easier.

It is a big step to decommissioning PS5 but can be a good one (faster than many other products)

My preferred option would be the first one, as it is resolving all the problems and still support PS5 … and as far that I understood is more separating PS and the API… what would result in an cleaner code (… I guess :wink: )

7

Okay, i understand the problem…

I also use Powershell 7 only in test and development environments and on the servers running UniversalDashboard. Therefore I can understand that it is not yet used everywhere. But is there anything that prevents you from using Powershell 7 only for UD?

Is the WindowsComaptiblity Module still working on powershell 7? If yes, SOME modules could possibly be running on pwsh 7.
The official microsoft compatiblity list is only listing the official modules, mh…

8

Personally i would not mind upgrading my production environment to v7, as it is in GA now.

I’m not currently using any modules that cannot either be ported or remade, and might be in an unique situation regarding this.

The config #1 gets my vote for now, and possible notify that the v5 support is EOL in 2021 ish?

9

@adam

perhaps Victor Ortiz can try the newest release of the Sharepoint powershell module: https://www.powershellgallery.com/packages/Microsoft.Online.SharePoint.PowerShell/16.0.19927.12000
This is working for me.
I had used this one for a while to load the sharepoint CSOM libraries directly: https://www.powershellgallery.com/packages/SharePointOnline.CSOM/1.0.4/Content/SharePointOnline.CSOM.psm1

(i have no twitter account ;-))

10

@adam
i vote for dropping support on powershell v5.1 for future UD, if anybody still need to use poweshell 5.1 they can use UD 2.9 or 3.0 but i think UD 3.1 should address latest and greatest technology out there.

1 Like
11

@wsl2001 what is you preferred option? 1,2 or 3 (or something else)

I totally agree to use the newest technology (.Net Core 3.1, Blazor, …) but PS7 is in my opinion to early for v3.1.
And I don’t see the downsides for option 1 to include PS5 (and maybe has to choose between using PS5 or PS7)

12

Wow :open_mouth: good read there…from my point nothing stopping me upgrading the dashboard PC to powershell 7. If it needs to be done to run UD I would be happy to make the change. But the thing that sold your software to me was I didn’t have to make any installs to make it work so could potentially put people off investing the time to look at it…? Configuration 2 would be my choice

13

@augustin.ziegler
I vote for option 2, using PS7 will be the ultimate goal for future UD, also this will make all of us here in this community gain more experience using PS7 and makes UD performance a lot better.

UD version 2 and 3 already supported PS5.1 and there is plenty of features already included for people using PS5.1, i think its time to start shifting the gear towards future technology’s support and easy implementing in UD.

14

I agree on your technology way of thinking but another reason (not the module compatibility) it the “assembly loading” issue that would be resolved in option 1 (and not in option 2).

This means the Azure module and I think some others are not working.
That means a huge downside for the UD cloud future :wink:

15

you know when i stand on this, i think moving to asp.net core 3.1 will give us alot of values ,
specially the streaming option for endpoints

16

Isn’t option 1 also on 3.1 and will give this opportunity? Or do I miss some UD background?

17

moving to 3.1 will give this option

18

Moving to .NET core 3.1 would be great, leveraging those enhancements and features like Blazor!

And Configuration 2 would be my vote - I will accept any updating to enhance performance. The use of PS7 should be in the eyes of most PowerShell users anyway with some of the new PS7 features like Pipeline parallelization and error handling.

and to your point @augustin.ziegler about Azure modules - We don’t use modules we talk directly to the REST APIs using UD/custom modules. The problem I have with MSFT modules (and most for that matter) is dependencies and frequent breaking changes. Using REST and building your own allows more flexibility when something changes. And now that UD supports OpenId with Access Token we can pass that puppy right into the Header.

function Get-AzureRegions {
    param(
        [Parameter()]
        [String]$AccessToken,
        [Parameter()]
        [String]$SubscriptionId
    )
    $AuthHeader = @{'Authorization' = "Bearer $($AccessToken)"}
    $return = (Invoke-RestMethod -Method GET -Headers $AuthHeader -Uri "https://management.azure.com/subscriptions/$($SubscriptionId)/locations?api-version=2019-11-01")
    return $return
}
19

@adam maybe for config 1 we can start using the pipe option so the data transfer size will be small and we can do cool things with the pipe options, i hope you remember our talk on this.

20

I’m toying the pipes you suggested along with some other transport methods. One nice thing we could do is leverage PS Remoting over named pipes so that we could easily connect to external PS processes. The other thing to consider is leveraging gRPC for ASP.NET Core since if we move to ASP.NET Core v3 we can build gRPC services.

Once we get past the v3 release, I’ll do some performance testing to see the best way to do something like this.

As for option which option is picked in the end, it will probably depend on some of these performance tests as well as how complicated it makes UD running the PS in a separate process.

This is a FANTASTIC feedback though. I really appreciate every one chipping in.

2 Likes