Why are albino people portrayed as evil in the books and movies?

It's a quick shortcut - people's instinct is to be wary of someone who looks very different to them.