I never pictured him as furry either, you would think James & Co. would have teased him about it if he was. Oops! Logic!

Or we would have heard comments in Harry's narrative voice or something, yeah. I've wanted to use this defense sometimes when I see people saying odd things like Snape smelled bad, or whatever. Surely, if he did, we would have heard about it at some point from Harry, Sirius, or James. (For that matter, I'm not sure when or if the word "ugly" is actually used. We get a lot of harping on individual features, but that specific word? I don't know. It's one of my many snipe hunts I need to get around to.)