Do I try to talk to him again or just leave it?

I have hooked up with James afew times, the last time we hooked up was at the end of May. It was kept a very quiet secret for about two months. About a month ago I went for lunch with my friend Tasha, she just guessed me and James were hooking up and I didn't deny it. She promised she would tell no one. Then over the last two days it all blew up. Ben is Tasha's boyfriend and he is also somewhat friends with James. Tasha obviously told Ben. Ben then told other people and soon James discovered what people were saying. James then confronted me on facebook last night. He said that he has heard that I have been telling people; I could tell he was mad. I didn't know what to say so all I said was some lame comment all he said was k and stopped replying back. The thing that hurts me most is me and James were just starting to talk again and things were going well and now its all screwed up. What do I do? Do I try talking to him letting him know the truth or do I just leave it? He is a very stubborn guy; I wouldn't say he is mean but he isn't the nicest guy..