V3.0.0-beta7 - API Troubleshooting

[neo]

Hi @adam I am seing some strange things with authentication on API’s. If i have a POST API with simple code in:

New-PSUEndpoint -Url "/api/v1/TestApi" -Method @('POST') -Endpoint {
  param ($InputVar)
  $InputVar
} -Environment "Integrated"

Invoke-RestMethod https:// -URL- /api/v1/TestApi -Method POST, or run through PSU i get a HTML output:

If i then change this to a GET, or -Method @(‘GET’, ‘POST’) and call with GET i get the results back.

Next i add authentication on:

I get a 500, this is using an Administrator token (you can see what happens if i dont pass a key though i get unauthorized):

Thanks!

[neo]

Further to this, i have noticed that ‘Invoke_PsuScript’ is also failing in 3.0.0-beta7:

If i have an API (no auth, GET) with the following code:

Invoke-PSUScript -Script 'TestScript.ps1' -Integrated
i get the following error:

Invoke-RestMethod: An error occurred while saving the entity changes. See the inner exception for details.
at , : line 2
at , : line 1

If i change this to:

 Get-PSUScript -Name 'TestScript.ps1' -Integrated | Get-PSUJob -OrderDirection Descending -First 1
Get-PSUJobPipelineOutput -Job $Job
Get-PSUJobOutput -Job $Job

This works fine.

Any thoughts?

[adam]

Can you try on the latest nightly build of v3? I don’t see this issue.

[neo]

Unfortuently i still see the same…

If i try this through PowerShell i get a response:

If i then add in ‘GET’ it works in the GUI and PowerShell:

I then add authentication in, with API key and it doesnt work through either, i get the error:
Invoke-RestMethod: Response status code does not indicate success: 500 (Internal Server Error).

This is using an API key (Token) copied directly from the Application Tokens. This is the same if i use HTTPS or HTTP direcly to the server on :5000

[adam]

Ah. I see the first issue with POST. I didn’t try it in the UI but I definitely see the HTML returned. I’ll open an issue for that.

As for the 500 error, I don’t see that one. Can you please check your log file for some more information on what might be causing that?

[neo]

Nothing hits the log…

Is there another log for this?

Cheers!

[adam]

I’d look in the PSU server log. The 500 error probably means we’re throwing an unhandled exception and we should have a pretty good stack trace in there.

Settings \ General \ Diagnostics

[neo]

Auth turned off:

[neo]

2022-05-25 15:14:35.402 +00:00 [ERR] An unhandled exception has occurred while executing the request.
System.InvalidOperationException: The LINQ expression ‘DbSet()
.Where(i => i.Name.Equals(
value: __identityName_0,
comparisonType: OrdinalIgnoreCase))’ could not be translated. Additional information: Translation of the ‘string.Equals’ overload with a ‘StringComparison’ parameter is not supported. See Collations and case sensitivity - EF Core | Microsoft Learn for more information. Either rewrite the query in a form that can be translated, or switch to client evaluation explicitly by inserting a call to ‘AsEnumerable’, ‘AsAsyncEnumerable’, ‘ToList’, or ‘ToListAsync’. See Client vs. Server Evaluation - EF Core | Microsoft Learn for more information.
at Microsoft.EntityFrameworkCore.Query.QueryableMethodTranslatingExpressionVisitor.g__CheckTranslated|15_0(ShapedQueryExpression translated, <>c__DisplayClass15_0& )
at Microsoft.EntityFrameworkCore.Query.QueryableMethodTranslatingExpressionVisitor.VisitMethodCall(MethodCallExpression methodCallExpression)
at Microsoft.EntityFrameworkCore.Query.QueryableMethodTranslatingExpressionVisitor.VisitMethodCall(MethodCallExpression methodCallExpression)
at Microsoft.EntityFrameworkCore.Query.QueryCompilationContext.CreateQueryExecutor[TResult](Expression query)
at Microsoft.EntityFrameworkCore.Storage.Database.CompileQuery[TResult](Expression query, Boolean async)
at Microsoft.EntityFrameworkCore.Query.Internal.QueryCompiler.CompileQueryCore[TResult](IDatabase database, Expression query, IModel model, Boolean async)
at Microsoft.EntityFrameworkCore.Query.Internal.QueryCompiler.<>c__DisplayClass9_01.<Execute>b__0() at Microsoft.EntityFrameworkCore.Query.Internal.CompiledQueryCache.GetOrAddQuery[TResult](Object cacheKey, Func1 compiler)
at Microsoft.EntityFrameworkCore.Query.Internal.QueryCompiler.Execute[TResult](Expression query)
at System.Linq.Queryable.FirstOrDefault[TSource](IQueryable1 source) at PowerShellUniversal.SQL.SqlTable1.FirstOrDefault() in D:\a\universal\universal\src\PowerShellUniversal.SQL\Database.cs:line 463
at Universal.Server.Services.UniversalAuthorizationService.AuthorizedAsync(HttpContext context, String roles, Boolean supportChallenge) in D:\a\universal\universal\src\Universal.Server\Services\Authentication\UniversalAuthorizationService.cs:line 115
at Universal.Server.Services.ApiService.ExecuteAsync(HttpContext httpContext) in D:\a\universal\universal\src\Universal.Server\Services\API\ApiService.cs:line 187
at Universal.Server.Middleware.RoutingMiddleware.Invoke(HttpContext httpContext, IPolicyEvaluator policyEvaluator) in D:\a\universal\universal\src\Universal.Server\Middleware\RoutingMiddleware.cs:line 95
at Swashbuckle.AspNetCore.SwaggerUI.SwaggerUIMiddleware.Invoke(HttpContext httpContext)
at Swashbuckle.AspNetCore.Swagger.SwaggerMiddleware.Invoke(HttpContext httpContext, ISwaggerProvider swaggerProvider)
at Universal.Server.Middleware.SwaggerAuthenticationMiddleware.InvokeAsync(HttpContext context, RequestDelegate next) in D:\a\universal\universal\src\Universal.Server\Middleware\SwaggerAuthMiddleware.cs:line 42
at Microsoft.AspNetCore.Builder.UseMiddlewareExtensions.<>c__DisplayClass6_1.<b__1>d.MoveNext()
— End of stack trace from previous location —
at AspNetCoreRateLimit.RateLimitMiddleware`1.Invoke(HttpContext context) in D:\a\universal\universal\src\AspNetCoreRateLimit\Middleware\RateLimitMiddleware.cs:line 109
at Microsoft.AspNetCore.Session.SessionMiddleware.Invoke(HttpContext context)
at Microsoft.AspNetCore.Session.SessionMiddleware.Invoke(HttpContext context)
at Microsoft.AspNetCore.Diagnostics.ExceptionHandlerMiddleware.g__Awaited|6_0(ExceptionHandlerMiddleware middleware, HttpContext context, Task task)

[adam]

Perfect. That’s exactly what I need. I’ll take a look and get this fixed.

[neo]

Great, thanks!

[adam]

Both these issues will be fixed in tonight’s nightly build.

[neo]

That seems to have fixed the issue with POST API’s, they now work through authentication, however there are more issues…

Scripts dont run anymore, they sit in ‘queued’ status:

In hangfire, i see an error:

Error Details:

Failed
Can not change the state to ‘Enqueued’: target method was not found.

Newtonsoft.Json.JsonSerializationException
Error setting value to ‘Roles’ on ‘PowerShellUniversal.Identity’.

Newtonsoft.Json.JsonSerializationException: Error setting value to ‘Roles’ on ‘PowerShellUniversal.Identity’.
—> System.ArgumentNullException: Value cannot be null. (Parameter ‘value’)
at System.String.Join(String separator, String value)
at PowerShellUniversal.Identity.set_Roles(String value) in D:\a\universal\universal\src\PowerShellUniversal\Models\Identity.cs:line 28
at Newtonsoft.Json.Serialization.ExpressionValueProvider.SetValue(Object target, Object value)
— End of inner exception stack trace —
at Newtonsoft.Json.Serialization.ExpressionValueProvider.SetValue(Object target, Object value)
at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.EndProcessProperty(Object newObject, JsonReader reader, JsonObjectContract contract, Int32 initialDepth, JsonProperty property, PropertyPresence presence, Boolean setDefaultValue)
at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.PopulateObject(Object newObject, JsonReader reader, JsonObjectContract contract, JsonProperty member, String id)
at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateObject(JsonReader reader, Type objectType, JsonContract contract, JsonProperty member, JsonContainerContract containerContract, JsonProperty containerMember, Object existingValue)
at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateValueInternal(JsonReader reader, Type objectType, JsonContract contract, JsonProperty member, JsonContainerContract containerContract, JsonProperty containerMember, Object existingValue)
at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.SetPropertyValue(JsonProperty property, JsonConverter propertyConverter, JsonContainerContract containerContract, JsonProperty containerProperty, JsonReader reader, Object target)
at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.PopulateObject(Object newObject, JsonReader reader, JsonObjectContract contract, JsonProperty member, String id)
at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateObject(JsonReader reader, Type objectType, JsonContract contract, JsonProperty member, JsonContainerContract containerContract, JsonProperty containerMember, Object existingValue)
at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateValueInternal(JsonReader reader, Type objectType, JsonContract contract, JsonProperty member, JsonContainerContract containerContract, JsonProperty containerMember, Object existingValue)
at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.Deserialize(JsonReader reader, Type objectType, Boolean checkAdditionalContent)
at Newtonsoft.Json.JsonSerializer.DeserializeInternal(JsonReader reader, Type objectType)
at Newtonsoft.Json.JsonConvert.DeserializeObject(String value, Type type, JsonSerializerSettings settings)
at Hangfire.Common.SerializationHelper.Deserialize(String value, Type type, SerializationOption option)
at Hangfire.Storage.InvocationData.DeserializeArgument(String argument, Type type)
— End of stack trace from previous location —
at Hangfire.Storage.InvocationData.DeserializeArgument(String argument, Type type)
at Hangfire.Storage.InvocationData.DeserializeArguments(MethodInfo methodInfo, String arguments)
at Hangfire.Storage.InvocationData.DeserializeJob()

I have cleaned down the entire build, deleted the SQL db and recreated, renamed .universal folder so its a clean install. I then created one Script called 'TestScript.ps1"

This has content of “Hello”, if i run it, it sits as queued as above - this is a bug but its not stopping me testing other areas like API’s. I then have created an API to trigger the ‘TestScript.ps1’

I then trigger this script (no auth) and get:

I do see the error in PSU - Endpoints - Log:

For some reason i do not have a trace in the logs for this 400 latest error in the logs.

[neo]

The code in the API ‘PsuScript’ is:
Invoke-PSUScript -Script "TestScript.ps1" -Integrated

If i change this to “hello” only, it works fine.

[adam]

This should have been caught in our integration tests. Let me verify that everything is running under test correctly. I’ll also update the nightly build page to include test results.

[neo]

Im getting a lot closer now. The idea i have is to use an API to run a script, wait for the script to complete, provide output, then if success sync a UD component on a specific node. This way i can control and ensure each node is correct.

Using API:
SUCCESS:
Invoke-PsuScript -Script 'MyScript.ps1' -ComputerName localhost -AppToken $Key -Wait
Sync-PSUComponent -Id "MyDynamic" -Integrated

FAILURE (both lines fail independently of eachother):
Invoke-PsuScript -Script 'MyScript.ps1' -Integrated
Sync-PSUComponent -Id "MyDynamic" -ComputerName localhost -AppToken $Key

Using Script (MyScript1.ps1):
SUCCESS:
Sync-PSUComponent -Id "MyDynamic" -Integrated

FAILURE:
Sync-PSUComponent -Id "MyDynamic" -ComputerName localhost -AppToken $Key

I have a working method now targetting the script on a certain node, and the Sync-PsuComponent runs integrated, but thought you might want to test these scenarios