Do I win a prize?
It turns on, stable boot everytime now, with or without a cartridge.
It's a bad solder joint somewhere, pressing firmly on the chips did nothing earlier, but I did it again anyway after I'd removed the board from the case, powered it up and it's 'fine' now.
One problem though, the spring under + is gone, so I have to somewhat type upside down, but I did make a little basic program nonetheless.
Think I need another C64C to finish this one.
Edit: and the bad connection is definately to do with the memory or controller - first boot I had 1Kb free - pressed again - reboot and now it's reliably 38911 free.