We have our first 2 scripts running in psu on a schedule and with runas accounts. We find that they often fail due to error:
Error executing job: Did not receive port from client process.
In the log we see the same
[INF] Error executing job 11781: Did not receive port from client process. at UniversalAutomation.Services.GrpcExecutionService.ExecuteScript(Job job, ExecutionEnvironment environment, ExecuteScriptRequest request, ExecutionCallback executionCallback) in C:\actions-runner_work\universal\universal\src\Universal.Server\Services\Automation\GrpcExecutionService.cs:line 87
at UniversalAutomation.ExecutionService.ExecutePowerShell(Script script, IEnumerable`1 parameters, Job job, Schedule schedule, ExecutionCallback callback) in C:\actions-runner_work\universal\universal\src\Universal.Server\Services\Automation\ExecutionService.cs:line 719
at UniversalAutomation.ExecutionService.TryExecute(Job job, Schedule schedule) in C:\actions-runner_work\universal\universal\src\Universal.Server\Services\Automation\ExecutionService.cs:line 368
at UniversalAutomation.ExecutionService.Execute(Job job, Schedule schedule) in C:\actions-runner_work\universal\universal\src\Universal.Server\Services\Automation\ExecutionService.cs:line 215 System.Exception
Sometimes the scripts run without issue but very often they fail with above error.
We followed the recommendations in the documentation about this and changed the value for JobHandshakeTimeout in appsettings
This hasn’t improved the situation. Anything else we can do or check to investigate this issue?
PSU is installed on Server 2019 running in a vm. No antivirus currently, 8gb of ram and 2 vcpu. Not sure but that seems enough to handle 2 small scripts. If needed we can give the server more resources but I’m not sure if that is the issue here.
This shouldn’t be a performance problem with such little allocated on the server.
I would recommend double checking CPU and memory utilization to ensure that it’s at optimal levels. I know that we had another user that had a lot of other things running on the PSU server and it caused this error to occur.
Are you using Windows PowerShell or PowerShell 7 for the scripts?
The server is dedicated for Universal. Nothing else runs on it.
CPU is hovering around a couple percent (2-3%) occasionally going up to 10% caused by the Universal.Server process. Memory is stable at 24%.
We have enabled debugging and we see the “grooming” job runs every minute which is default. We have set that now to 60min to see if it makes a change. Also the GIT sync runs very frequently (every second or so?) but we left that as is.
We are using powershell 5.1 (the default) at the moment and have this environment specifically selected for the scripts. The runas accounts are defined in the schedules.
Is there anything we can do or check to figure this out? Thank you for all the info and help.
I have been away for awhile. I have noticed that almost none of the jobs have run in the meantime… always the same issue.: Did not receive port from client process.
Below what I see in the logs
Job 11799 started.
2023-02-12 01:00:09.572 +01:00 [DBG] Job 11799 bound to port 60358.
2023-02-12 01:00:25.046 +01:00 [INF] Error executing job 11799: Did not receive port from client process. at UniversalAutomation.Services.GrpcExecutionService.ExecuteScript(Job job, ExecutionEnvironment environment, ExecuteScriptRequest request, ExecutionCallback executionCallback) in C:\actions-runner_work\universal\universal\src\Universal.Server\Services\Automation\GrpcExecutionService.cs:line 87
at UniversalAutomation.ExecutionService.ExecutePowerShell(Script script, IEnumerable`1 parameters, Job job, Schedule schedule, ExecutionCallback callback) in C:\actions-runner_work\universal\universal\src\Universal.Server\Services\Automation\ExecutionService.cs:line 719
at UniversalAutomation.ExecutionService.TryExecute(Job job, Schedule schedule) in C:\actions-runner_work\universal\universal\src\Universal.Server\Services\Automation\ExecutionService.cs:line 368
at UniversalAutomation.ExecutionService.Execute(Job job, Schedule schedule) in C:\actions-runner_work\universal\universal\src\Universal.Server\Services\Automation\ExecutionService.cs:line 215 System.Exception
I previously had this issue, identical server specs to yours. Though, we often saw spikes of the CPU.
We increased RAM to 16GB, and CPU to 4 vCPU. This made a dramatic difference, and we rarely ever see client port errors anymore. We only ever ran 1-2 scripts at a time, and this cleared the issue right up.
Thank you very much for your reply. I have followed your recommendation and will monitor the situation.
Hopefully it clears the issue like in your case. I’ll report back as soon as I know more.
Thanks again! Much appreciated.