From what I understand, the almonds we buy in grocery stores are safe for hamsters, and we would never eat the ones that are unsafe for them because they are poisonous to humans as well. I still avoid them though, cause it eases my mind.
Here is an interesting article:
Certain Almonds are Highly Poisonous