Looks like in the App Config you can add ASPNETCORE_FORWARDEDHEADERS_ENABLED = true and it will forward the headers or whatever correctly to the container.
Found this article that helped me.
Forwarded Headers Middleware Updates in .NET Core 3.0 preview 6 | ASP.NET Blog (microsoft.com)