There’s a protection diode on the Panda II. What you’re seeing is the drop from that diode. You can short the diode (make sure you know what you’re doing, and how this will affect your design and the protection the diode offers), or power from an external 5V source.