I am a female Cancerian, although I can not speak for males I wouldn't mind answering your question from my point of view as a cancer.

The answer is that when we find someone we can connect with, we can't help but try to get to know them more and have them know us. We feel empathic towards those who we are close to so you were right about him not meaning harm in what he was saying, I think that he wants you to experience the joy of a relationship and wants you to be happy either way (with or without him).

If you are interested in him then take the initiative! Most Cancerians are too shy to tell someone that we like them first and so we passively give the people we like nudges.