Perhaps instead of waking him up suddenly by lifting his house (I try not to disturb nests) just move the substrate in another part of the cage, jingle a box of treats, and move items. So that he gradually gets woken up by disturbances in his cage, and not a big scary hand disturbing his slumber?
Try and put yourself in the "shoes" of Charlie. What would be very scary to you when everything in the world is new - and there are giant creatures (who are baldy in the fur department and look very strange). If you stick to a routine of when you wake him up (7pm on the dot, or whenever) then he'll more than likely change his sleeping pattern. But you have to be consistent and persistent.
