Quick google got me this discussion from a few years ago which covers the topic fairly well.
German minimum cage size
DOUBLE EDIT: Trigger warning below for those of us who maybe don't like to think of how hamsters who make up scientific experiments might come to the end of their lives.
Google also gives me this
swiss study which is freely available, disclaimer I do not know if
Animal Welfare is a well regarded publication. EDIT: This link doesn't seem to work but the name of the study is "Behaviour of golden hamsters (Mesocricetus auratus) kept in four different cage sizes" with authors K Fischer, SG Gebhardt-Henrich and A Steiger. The PDF does come up when I search "Hamster cage size stress study"
I did a quick search on Elsevier Science Direct for other free articles but no very promising summaries and I don't have my academic access anymore anyway. I think the Fischer study above seems to be the one most people refer to.