I'd say just try letting him down lightly a few times by telling him you just can't feel the same for him. And if it gets worse and he continues, you'll just have to be more firm with him.
I'm not sure what to tell you other than that. Sorry.
Oh I so want the answer to this question - I am having the exact same problem except he does have my number because my friend gave it to him. I do not owe my friend!!!
I'm not sure what to tell you other than that. Sorry.