Fear of people standing over/next to me while I am sitting down

As a 6'3 male, I just can't stand and fear when, a person/s is standing around me or next to me while I am sitting down.

I immediately feel flustered, hot and bothered and have to stand up.

Its weird I know, but I think because I'm used to towering over people because of my height, I'm not used to it, and it freaks me out.