Was i being rude?............?

No. If he was making you uncomfortable with how forward he was being then you have every right to say what you said. Don't go stay with him and when he's done trippin out, go have coffee and leave it at that.