Looks like grain beetles or biscuit beetles. These bugs can't live on your hamster - if they were living on your hamster he'd be crawling with them and that just doesn't happen. Sadly they will be from the food.
If it's a new bag of food then maybe get it from somewhere else next time - it may be very old stock or stored in bad conditions. To kill any bugs or bug eggs in pet food that may hatch out, the food needs to be frozen (not put in the fridge - that won't work). Must be in the freezer. If you get a new big from a different place it may be fine anyway without freezing. But I always freeze hamster food for a week now anyway to prevent such things!
So you've done another full cage clean. No more bugs since? Or just this one? I think it's really important to get on top of the food situation so he doesn't need to have any more full cage cleans.
I can't see any point taking your hamster to the vet as they'll just say clean the cage and replace the food. The bugs aren't from your hamster. I know it makes you creep thinking about it!
The reason they will be near him is because he is warm and sits on his hoard! Or the room is warm and the bugs are hatching out in the food and then crawling elsewhere.
So get another new bag of food. Not sure which food you're using but Harry Hamster is a safe bet and contains everything needed. Remove any food left in the cage (ie his hoard) and put new food there to replace the hoard. And new food in his bowl, then put the bag of food in the freezer (if you have a freezer) or the freezer compartment of the fridge? You can take some out each day to put in his food bowl. (Leave an hour for it to defrost before putting it in the bowl). But leave the bag in the freezer for a week. After that it will be fine.
It's always the food
Unless you can see these bugs anywhere else in your house - but if there are bugs in the pet cage, it's the food!
Hope you've got on top of it now after this recent clean. The rest is time and patience with your hamster as he gets to know you. He'll still be a bit scared and skitty.
This includes all food in my book - including treat sticks! The only thing I don't freeze is sunflower seeds and pumpkin seeds I buy as treats as I get human grade ones. But any hamster treats, snacks, treat sticks, food - freeze it. To prevent bugs! It is very very common with all pet foods.