Hello,
Welcome to HC! I'm sorry to hear this little one has has a rough start but it sounds like things will improve for him from now.
There will be nothing wrong you cleaning his cage at all! Is the cage suitable first off? If you have a ball you could put him in there while cleaning or put a towel in the bath and some toys for him to explore. If neither of those are available a secure plastic storage box or similar (with ventilation of course) while you clean the cage would be ok to hold him for a while.
Depending how dirty the cage is just hot water to soak for a while will be fine. If it is really bad then a soak in bleach and put out in the sun to dry will help. Throw away any badly soiled wooden toys.
Let us know how things are
