Product: PowerShell Universal
Version: 3
Seems we have multiple jobs not firing. We have jobs run daily at midnight / cron jobs at specific times, and over the last 5 days nothing has fired. It sees the next run time, but seems to just skip it and then list the next interval.
Let me know what information would be helpful for diagnosis, or i can email / open a ticket.
How are you hosting PSU?
I’d also suggest checking out the hangfire dashboard to see if there’s an failures listed there: Hangfire - PowerShell Universal
Hello Adam,
We are hosting PSU in IIS. The settings in IIS for 20 min timeouts and always running are set.
I would verify that something isn’t recycling the app pool. Although always running is set, we’ve definitely seen times when IIS still can recycle the app pool and it fails to restart the schedules.
I have reset the setting and did a iisreset. Will monitor for a day or two to update / verify my findings. I appreciate the quick reply
Hello Adam,
I have verified all settings look to be correct in the IIS, and that the app Pool is recycling at the appropriate time. The jobs that are scheduled to run daily though are simply not firing and constantly going to the next days interval as expected to run, but nothing happens. You can see recycling happening here. The most recent ones were me recycling the app pool for testing. Let me know what logs or other information may be helpful.
While I haven’t looked at app pool recycling, which I’m setting up event logs for right now, the behavior you described is exactly what is happening on my system. It’s not happening to all of my jobs either. I have a weekly one on Weds at 9am that fires off fine.
Yeah. I’m not exactly sure where to go with it either because it acts like everything is fine, and then does absolutely nothing. I’ve tried removing the schedules / scripts / jobs and re introducing but it’s still the same result.
I suspect that the recycling is causing the app pool to restart but the PSU process isn’t actually starting. Because of this, the schedule jobs won’t run. Once you access the site, it may seem like it takes a while the first time and that’s because the process is actually starting up.
I would try to avoid recycling all together, if possible. We’ve added uptime information in the 3.3 release so we’ll be able to say for sure whether this assumption is the case.
Hello Adam
I’m going to be rebooting the server today with App Pool recycling off, and all settings in IIS following the documentation. I’ll be able to update by tomorrow. If this is not fixed by doing as such what are the recommended next steps for diagnosis? Without the scheduled jobs kicking off that’s a big damper in operations.
Let me know how it goes. Recycling is really the enemy of scheduled jobs since IIS doesn’t always seem to start the PSU process until it’s accessed.
I’ve heard of some users setting up a scheduled task that just calls Invoke-RestMethod on the PSU web site to ensure that IIS keeps the site up and running. We actually have an endpoint that returns a 200 OK when the web site is running and doesn’t do anything else.
Invoke-RestMethod http://localhost/api/v1/alive
IIS can be a real pain to get configured correctly.
I didn’t get a chance to do it yesterday so I manually had to run the scripts today, but this morning, after disabling the app pool recycling, I did reboot the server. We have jobs scheduled for tomorrow so should be able to update with that tomorrow some time.
If the jobs continue to not run though, what are the options we have for further diagnosis to determine root cause?
I think our next step is to verify that this is actually the recycling causing the problem. If you see this go away with this change, then we know but if it is still happening we should look at the hangfire dashboard to verify that the schedule is correctly ending up in the queue.
I may not be on the correct version stated, but I checked this morning and no jobs kicked off. I did reboot the machine and open up the browser / go to PSU yesterday and then shut the browser.
This is what I see under General
Where else could I be checking to help further diagnose?
Hello Adam,
Just following up on this thread. This is really hindering a migration project to move to PSU. Any advice would be great.
You are using version 2.x and won’t have some of the same functionality. Can you please open a support case by emailing support@ironmansoftware.com?
I just wanted to followup and say mine was app pool recycling. I’ve got the settings sorted and my jobs finally ran this morning.
Thanks!
I guess I spoke a bit too soon. Seems that if the server isn’t accessed regularly the hangfire stuff is suspending. I will have to look at doing the scheduled task option it looks like in order to keep it going for the days we don’t login to access the site directly. This is a bit of a nuisance, but will be fine overall. If this is addressed in the new release, i’ll be getting it installed in the next day or so.
We updated the IIS docs with some new settings that seem to have helped folks out: https://docs.powershelluniversal.com/config/hosting/hosting-iis#configuration-for-jobs
Primarily changing the app pool to .NET 4.0 and setting preload enabled on the website seem to have helped.
Turning on preload is causing issues for me, causes authorization errors after the session times out.
