A large number of efficient statistical methods have been designed for a frequent case when the distributions are normal (Gaussian). In practice, many probability distributions are not normal. In this case, Gaussian-based techniques cannot directly applied. In many cases, however, we can apply these techniques indirectly -- by first applying an appropriate transformation to the original variables, after which their distribution becomes close to normal. Empirical analysis of different transformations has shown that the most successful are the power transformations X → Xh and their modifications. In this paper, we provide a symmetry-based explanation for this empirical success.