How do a group of Spies get intimidated by 3 giants? What things intimidate and scare us? What factors trigger our primal emotions?

What is the cause of the Spies’ intimidation, as opposed to Joshua and Caleb’s confidence? Is this just a question of personality?