Found the weirdest workaround that makes the games work.
First I have to run it in the default TM, then make exceptions for certain games. Where they can either be run in high performance, or in the 7800 XT depending.
It looks like the launch is checked with the default, then future appliance is done with the exception. From there, in the game settings, I can adjust even further.