I would go with what was said below but I thought it was more like "Oscar knew since five funny looks ago that they weren't staring at him because he was cool. It was because they thought he was a weirdo." Or something like that. Because when people give funny looks, it's usually because they're judging ya, and not in the "" way. As in the "hide ya kids from the *weirdo*" way.
It means literally what it says, I think. Oscar knew. He's known since five funny looks ago that he really wants to bang whoever is giving him the funny looks.