It's happened to me: the wind blew my jacket open outside the Acme, just as a soccer mom was walking by and looking in my direction.
I acted like nothing was wrong, calmly closed (and zipped) my jacket, and went on into the Acme. There was no police response, so I guess I pulled it off.
Bottom line: act like you belong. Wherever you are, whatever you're doing, you have a right to be there, and you've earned the Right to Bear Arms. If you look shameful or scared, people will react badly. If you don't, they won't.