Switch debug mode USB/COM

Is it possible to switch debug mode in software?

No, only using a pin.

But the combination soft / hardware is working just fine :wink:
Use a GPIO output to flip a switch on the mode pin and reboot from software, as long if your PCB is powered you stay in the same mode.