It is perfectly fine to leave it in the cage. My Almond adored his sand bath. The only time I would remove a sand bath is if the hammy has a problem with it. Sometimes a hammy may have dry skin issues that the sand exacerbates. Just keep an eye on your hammy and see how your little guys do. It's easy enough to remove the bath if a problem arises. The photos below are my little Campbell's hybrid, who has since passed away. The little ones are adorable, aren't they?