I think he may of just been a bit startled.
As he is new, it's normal for them to be a bit jumpy, especially if you move something in his cage as he is still getting used to his surroundings, new scents and of course, you! He may of never had much human interaction before, so expect him to be a little bit nervous until his taming is underway.
Don't worry though, he should be better once settled in and more used to you! Little baby steps