I used to when I went every weekend to Bondi Beach, however now I realise I am not invincible and there is no way I would do this any more.
The flags are there for a reason, and having watched that TV show where they rescue people who dont swim in flags, I would not do this again.