I've definitely seen this, especially with younger and nervous hams. I think it's more of a confidence issue. It takes time to build up trust between you and the hamster, so that they can grow accustomed to being picked up by a hand.
We can't forget that they are a prey animal, and a hand swooping down from above them is very much like the advance of a predator. Encouraging your hamster to understand that being picked up is not a threat is the biggest part of taming.
He will have come to understand that when the ball is presented, it is an opportunity for him to come out and exercise and is of no threat to him. Your hand is a different matter.
It takes time, but with regular routine handling, this problem should disappear. Some hams like to take it slow, and gradually building up that trust gently will help him grow to see the pick up as a positive thing.