You can buy some gardening gloves or thick gloves that may give you confidence to pick him up.
Also, if he hates being picking up and you don't want to, you can buy a plastic gardening scoop - if I place that in the cage with me holding the handle prammy runs into it ready to come out.
By the way, if you get that, make sure you place your hand at the end after he climbs in so he can't jump out. Kind of cup the hamster. Prammy's never jumped out of the scoop but I make sure she can't!