If you are outside, you can spray around your shoulders with a personal spray from a chemist ect, so they do not pester you.
If at home, make sure all windows have fly screens on them.
Make sure also...and this is the key! that anyone going out the door, or coming in is quick.
No leaving it open even a few seconds...they will get in.
If you don't have a screen door you can buy portable ones now, that attach to your out side door, and cost very little and easy to use.