It can be powered on through the paper clip, the PS. Running but not powering the MB nor the CPU.You only see the burnt PCB, you cannot see the microscopic circuits inside the chips on that PCB. Those circuits are meant to be powered by 1-3 volts, microwaves can induce thousands of volts. Thousands of volts scramble the circuits inside chips and turn them into blocks of metal. Plugging that scrambled GPU into your motherboard was no different than plugging a fork into it. The motherboard is fried now and cannot tell the PSU to turn on (or the PSU thinks the motherboard is also a fork and won't turn on to protect itself).
eWaste the entire system and buy a new one, nothing in there is trustworthy.