I actually do mostly peel them despite knowing that the goodness in near the surface in most vegetables.
Most places say do not peel mushrooms, so I bought a special mushroom brush.
However, depending on how dirty they are I still tend to peel them as I always think they have soil which has had chemicals in it and on it so I feel safer somehow by peeling them.
Next time I have them, I will just dry brush them, which is recommended and see if they taste different.