While thinking about something else, I noticed recently the following result, which is certainly not new:

Let be a compact topological group [ADDITIONAL ASSUMPTION pointed out by Y. Choi: connected, Lie group], and let be a finite-dimensional irreducible unitary continuous representation of on a vector space . Then the natural representation of on decomposes as a direct sum of one-dimensional characters if and only if is of dimension .

One direction is clear: if has dimension one, then is simply the trivial one-dimensional representation. For the converse, here is an argument with character theory.

As a first step, note that if (of dimension , say) has this property, then in fact decomposes as a direct sum of *distinct* one-dimensional characters: indeed, the multiplicity of a character in is the same as

where is the probability Haar measure on , and since

we get

by the orthogonality relations of characters. (Algebraically, this is just an application of Schur’s lemma).

Thus if we decompose into irreducible representations, we get

where the are distinct one-dimensional characters. We then know by orthogonality that

Now the last-integral is bounded by

(since ). Comparing, this means that there must be equality throughout in this estimate, which in turn implies that for all . Since is unitary of size , this implies that is scalar for all , and since it is assumed to be irreducible, it is in fact one-dimensional.

I see two interesting points in this argument: (1) is there a purely algebraic proof of the last part? I haven’t thought very hard about this yet, but it would be nice to have one; (2) the appearance of the fourth moment of is nicely reminiscent of the Larsen alternative (see Section 6.3 of my notes on representation theory, for instance…)

no doubt I am missing something, but why can’t rho of g sometimes have trace zero? (This calculation reminds me of one I had to do a while ago with L^1 norms of characters)

You’re right that I was a bit too fast — the argument goes through if the group is a connected Lie group (the function |Tr(rho(g))|^2 is continuous and the equality implies that its value is either 0 or d^2 at any point, and so it must be constant, and equal to d^2 since it is a non-zero representation), but it might not work otherwise. (I’ll look for counterexamples in finite groups…)

Indeed, if one takes a dihedral group of order 8, the unique 2-dimensional representation has |tr(rho)|^2 taking values 0 and 4, and it is the direct sum of the four one-dimensional characters! Thanks for catching this…

I think this is an algebraic version of what you wrote down in the last step, although I can’t see why it is completely parallel:

Let $\chi(g) = Tr (\rho(g))$. Now $\pi(g)$ acts on End(V). Consider a weighted average of these maps: $$\int_G |\chi(g)|^2 \pi(g) dg$$

This is an endomorphism of End(V) that projects onto the subspace fixed by all $|\chi(g)|^2 g$. Thus its trace is the dimension of the fixed subspace, which has dimension at most $d^2$.

On the other hand, the trace of this operator is also $\int_G |\chi(g)|^4 dg$, which is $d^2$ as you have argued.

So every endomorphism of End(V) is fixed by $|\chi(g)|^2 g$. This implies that for any $v \in V$,

$|\chi (g)|^2 g \cdot T(v) = T(gv)$

Take $g$ to be the identity, we see that $|\chi(1)|^2 = 1$, meaning that $\rho$ is dimension 1 to start with.