Ohhh this is me too!
my family and close friends know, but co-workers and some friends don't.
I always hear remarks when they talk like "ohh I think he's gay", not about me someone else who I know actually is gay as one it's obvious and two I've seen some evidence, then they make a weird face or some other remark...while I'm there, it's awkward at times haha.
Another time I was on holiday with a sort of friend and they said to me "Ewww look at those two gays". Of course I had already seen them as it was on a beach, these two guys with muscled tanned bodies wearing only white speedos were under the outdoor shower and kissing
at that time I just said "eewww....yeah..." (but was secretly wanting to join HAHA)
Shame I didn't have the courage to say something like "do you have something against gay people?" at times i still don't have that courage hence why I'm still half in the closet like you OP :\
Hopefully it will get easier for us