Mean is a minimiser of norm: it minimizes the loss of penalizing your
'prediction' of (many instances of) a random quantity. You can assume that the
instances will be revealed after you have made the prediction.
If your prediction is over/larger by you will be penalized by .
If your prediction is lower by
then also the penalty is . This makes mean symmetric. It punishes
overestimates the same way as underestimates.
Now, if you were to be punished
by absolute value as opposed to then median would be your best
Lets denote the error by if the error is an over-estimate and
if its under. Both and are non-negative. Now if the penalties were to
be that would have led to the different quantiles depending on
the values of . Note introduces the asymmetry.
If you were to do introduce a similar asymmetric treatment of and
that would have given rise to expectiles.