It is only a few days and yes you need patience and time
They actually need approx 2 full weeks to fully settle into a cage after a change of environment and during that time will gradually make habits and scent mark and start to feel at home. It is best not to clean anything for that two weeks or change anything.
That doesn't mean you can't start taming but you need to wait till the hamster shows signs of coming out or coming to the bars independently.
Taming really starts with familiarising yourself with them, so talk to him gently through the bars as much as you can - so he gets familiar with your voice and presence. You could offer the odd treat through the bars as well. But putting your hand in the cage with treats on can scare them off.
So my rule of thumb is - if they come to you at the cage bars, and show signs of wanting to come out in that two weeks then yes get them out for a taming session (let him walk into a tube or ball eg to lift him out then put that down in a large box eg and carry that to the safe enclosure for taming).
If not then wait. After two weeks I would tempt them out with a piece of cucumber in the end of a tube.
But after two weeks the change can be very noticeable and they can be out and about more and more confident.
Your hammy is still a baby and will get scared easily, so it's about building trust and familiarity.