SqlException: Database 'PSUDatabase' already exists. Choose a different database name

I haven’t. I’ve been hosting in Azure with a docker setup to an Azure SQL instance and it works fine.

You could try to use the included SQL files to create the database and then set RunMigrations to false in the compose environment section to avoid having PSU attempt to create the database.