Firstlywell done for adoptingthis little chappy, and secondly no, its not too late at all.
Hands are quite a scarey thing for hamster, especially anything swooping from above as to them its like a bird!
To get him out of the cage I would use his ball or a box or tub, to scoop him up, then take him to a safe place - many of us use an empty bath tub as they cannot escape! nd just gently tip the box on to your leg and let him crawl out. Let him run about all overyou, getting him used to your smell and your voice too. Take some treats with you and offer him these so that he realises that hands are not all bad, and bring nice things! Do this once a day or twice if you can manage for about 10mins at a time. When you are ready to can practive stroking him on his back and then picking him up. When you do this pick him up scooping with both your hands cupped around his sides and then let him go straight away - gradually increse the amount of time that he his held for - eg count to 3 then 5 then longer untill he is happy being picked up.
There are some fatastic tips on the wiki that Moderator and breeder Soffle has written - her tips are very good and she has delt with lots of baby hams and problem hamsters too.
http://www.hamstercentral.com/wiki/Taming_tips