You need a break from him to clear your head. It's obvious you really care for him, but until you separate yourself from the situation you will not be able to decide if you can pursue a relationship with him knowing what you know. You may be able to forgive and forget (with time), but you may not be able to. And you shouldn't torture yourself with him while you're trying to figure it out.