1. Introduction

In general, the reflection group is generated by a set of involutions in  \(\mathbb R^n\) , which can be embedded into an orthogonal group. By representation theory, a faithful map  \(\rho\) , from the reflection group  \(W\)  to  \(GL(\mathbb R^n)\)  represents Coxeter Group. In 1934, H.S.M. Coxeter, in his publication "Discrete Groups Generated by Reflections", made a massive progress on such groups satisfying certain properties. And those groups are so called Coxeter group. Coxeter groups generalize the nature of reflections and symmetry in space. And nowadays, they have many applications in mathematics, particularly in algebraic groups, semi-simple Lie algebras and more. Furthermore, as a broader demonstration of symmetry and reflection, Orthogonal groups can be regarded as an extended branch of both Coxeter groups and all reflection groups. Observing the group action of orthogonal groups on harmonic polynomials, we uncover relationships between the space of harmonic polynomials  \(\mathcal{H}\)  and Orthogonal groups  \(O(n)\) , leading us to delve into the concepts about the representation of Orthogonal groups with respect to  \(\mathcal{H}\) . In this paper, we are going to delve deeper on how Coxeter Group defined in order to demonstrate its representation in term of reflection and symmetry, discuss its applications on orthogonal groups with group actions, and then get in touch with some invariant theory at the end.

2. Representation of Finite Coxeter Groups

2.1. Characterisation of Coxeter groups

Let  \(A=\{a_1, ..., a_n\}\)  be a set. Let  \(F(A)\)  be the free group generated by  \(A\) . To give a detailed definition of Coxeter group, we need to introduce group presentation.

Definition: Let  \(A\)  and  \(F(A)\)  be the definition above. Let  \(R\)  be a set of relations between elements in  \(A\) . The presentation  \(\langle A|R\rangle\)  is defined as the quotient group  \(F(A)/N\) , where  \(N\)  is the smallest normal subgroup of  \(F(A)\)  that contains each elements of  \(R\) . [1](Mukherjee, 2021, p.2)

This definition raises  \(Theorem\ 1.2.4\)  in [2]. In this case, we consider A as a set of symbols or words under multiplication operation, called  \(concatenation\) , by maintaining only reduced words through removing all forbidden occurrences, and empty word 1 is the identity element of A.

Theorem: Assume that  \(\phi:Aarrow G\)  is a map from the alphabet A to the group G. Then  \(\phi\)  can be uniquely extended to a group homomorphism  \(\phi:F(A)arrow G\) . Let  \(\langle A|R\rangle\)  be a group presentation such that  \(\phi(w_1)=\phi(w_2)\)  is satisfied for each expression  \(w_1=w_2\)  in R. Then  \(\phi\)  induces to a homomorphism of groups  \(\langle A|R\ranglearrow G\) .[1](Mukherjee, 2021, p.2)

Proof: Let  \(\phi:Aarrow G\)  s.t.  \(\phi(a^{-1})=\phi(a)^{-1}\)  for all  \(a\in A\) .
We will show  \(\phi:F(A)arrow G\)  is a group homomorphism first.
Let  \(v\in F(A)\)  such that  \(v\)  is a reduced word  \(a_1a_2...a_n\)  for a specific  \(a_i\in A\cup A^{-1}\) .
Set  \(\phi(v)=\prod_{i=1}^n \phi(a_i)\) .
Suppose we take another reduced word  \(w\in F(A)\)  such that  \(w=b_1b_2...b_m\)  where there exists a maximal  \(r\in\mathbb Z_{\geq 0}\)  such that  \(a_{n-k}b_{k+1}=1\)  for  \(1\leq k\leq r-1\) .
Therefore, by the operation under concatenation,

Hence,  \(\phi\)  preserves group operation in  \(F(A)\)  as a group homomorphism.
Moreover, as  \(w_1=w_2\)  for all  \(w_1, w_2\in F(A)\) , we have  \(w_1w_2^{-1}=e\) , identity of  \(F(A)\) .
Equivalently,  \(\phi(w_1w_2^{-1})=\phi(e)=e\)  so that  \(w_1w_2^{-1}\)  belongs to Ker  \(\phi\) .
Hence, the normal subgroup generated by  \(w_1w_2^{-1}\)  as  \(\langle w_1w_2^{-1}\rangle=\langle e\rangle\)  naturally implies that  \(F(A)/\langle w_1w_2^{-1}\rangle\)  is equivalent to  \(F(A)/\langle e\rangle\)  as  \(F(A)\)  itself.
Therefore, by the universal property of the free group with Ker  \(\phi\)  from expression in R,  \(\langle A|R\rangle\)  which logically equivalent as  \(F(A)\)  is isomorphic to a group G.

Accordingly, if G is isomorphic to the group  \(\langle A|R\rangle\) , then this group G has a group presentation  \(\langle A|R\rangle\) . From here, we are able to use the concepts introduced to define  \(Coxeter\ Groups\) .

Definition: Let  \(M=(m_{ij})_{1\leq i, j\leq n}\)  be a symmetric  \(n\times n\)  matrix with entries from  \(\mathbb N\cup\{\infty\}\)  such that  \(m_{ii}=1\)  for all  \(1\leq i\leq n\)  and  \(m_{ij}\gt 1\)  whenever  \(i\neq j\) . The  \(Coxeter\ Group\)  of type M is the group  \(W(M)=\{(s_1, ..., s_n)|(s_is_j)^{m_{ij}}=1|1\leq i, j\leq n, m_{ij}\lt \infty\}\)  where we can also denote  \(\{s_1, ..., s_n\}\)  as a set of generators  \(S\) , and  \(W\)  as group satisfying the relations described in group presentation.
By above, we called the pair  \((W, S)\)  as  \(Coxeter\ System\)  of type  \(M\) . [1](Mukherjee, 2021, p.2)

Additionally, there is another concept would help us to understand Coxeter matrix  \(M\)  with edges and vertices as labelled graph as stated in [1].

Definition: In  \(M\) , we say that  \(i\)  is adjacent to  \(j\) , or  \(i\sim j\) , if  \(m_{ij}\geq 3\) . If  \(m_{ij}\gt 3\) , we label the edge  \(\{i, j\}\)  with the value of  \(m_{ij}\) . Then, a  \(connected\ component\)  of  \(M\)  is a maximal subset of  \(J\)  of  \([n]=\{1, ..., n\}\)  such that  \(m_{jk}=2\)  for each  \(j\in J\)  and  \(k\in [n]\setminus J\) , where maximal defined as no other vertex can be added to it. In the graph M, this means that  \(j\)  and  \(k\)  have no edge between them.
If  \(M\)  has a single connected component, it is called  \(connected\)  or  \(irreducible\) . A Coxeter group  \(W\)  over a Coxeter diagram  \(M\)  is called  \(irreducible\)  if  \(M\)  is connected.[1](Mukherjee, 2021, p.2)

We could see more examples of visualization of the Coxeter diagram in  \(Example\ 1.3\)  in [3].

2.2. Length function on Coxeter groups

Consider the case where  \(w\in W(M)\)  can be represented by more than one reduced word, we call a word of minimal length the  \(minimal\ expression\) . The  \(length\)  of  \(w\)  is defined by the length of such minimal expression, denoted by  \(l(w)\) . Consider that  \(w\)  is an element of  \(W(M)\)  and can be expressed by more than one reduced word. In such a case, we refer to the word with the shortest length as the  \(minimal\ expression\) . The  \(length\)  of  \(w\)  is determined by the length of this minimal expression and is symbolized by  \(l(w)\) .

Lemma: "Let  \((W,S)\)  be a  \(Coxeter\ system\) . Then  \(l(sw)=l(w)\pm1\)  for  \(s\in S\)  and  \(w\in W\) ."[1](Mukherjee, 2021, p.3)

Proof: First we have that  \(l(w)-1=l(s(sw))-1\leq l(sw)\leq l(w)+1\)  Let  \((W,S)\)  be a Coxeter system. Construct a group homomorphism  \(sgn: Warrow \{\pm 1\}\) , determined by  \(sgn(s)=-1\)  for each  \(s\in S\) . Then  \(sgn(w)=(-1)^{l(w)}\) , which implies  \((-1)^{l(sw)}=sgn(sw)=sgn(s)sgn(w)=(-1)^{l(w)+1}\) . It follows that  \(l(sw)=l(w)\pm 1\) .

Definition: Choose a subset  \(T\subseteq S\) .  \(w\in W\)  is called left T-reduced if  \(l(tw)\gt l(w)\)  for all  \(t\in T\) . The set of all left  \(T\) -reduced elements of  \(W\)  is denoted by  \(^TW\) . Analogously, for  \(K\subseteq W\) , we can define right K-reduced and  \(W^K\) . [1](Mukherjee, 2021, p.3)

Next we give a decomposition law for length function, it helps us to decompose a element in a Coxeter group according to the decomposition of length.

Lemma: Let  \((W,S)\)  be a Coxeter system. For each  \(w\in W\)  and  \(T\subseteq S\) , there exists  \(u\in \langle T\rangle\)  and  \(v\in ^TW\)  such that  \(w=uv\)  and  \(l(w)=l(u)+l(v)\) . Also we can decompose  \(w\)  according to the right  \(K\) -reduced set.[1](Mukherjee, 2021, p.3)

Proof: Consider the subset  \(D\)  of  \(\langle T\rangle\times W\)  consisting of all pairs  \((u,v)\)  with  \(w=uv\)  and  \(l(w)=l(u)+l(v)\) . Since this set is nonempty, we can find an element of  \(D\)  with l(u) maximal. Suppose  \(v\)  is not left  \(T\) -reduced, which means there exists  \(t\in T\)  such that  \(l(tv)\lt l(v)\) . Then  \(v=tv'\)  for some  \(v'\in W\)  with  \(l(v)=l(v')+1\) . Since  \(w=utv'\) ,  \(l(w)\leq l(u)+1=l(tu)\) . Therefore, we find  \((ut,v')\in D\)  with  \(l(ut)\gt l(u)\) , a contradiction to the maximal hypothesis. Then we conclude that  \(v\in ^TW\) , satisfying the condition above.

In order to be consistent with the notion in later parts, we label these subgroups  \(\langle T \rangle\)  as follows.

Definition: For any subset  \(T\subseteq S\) , define  \(W_T\)  to be the subgroup of  \(W\)  generated by all  \(t\in T\) . We call such subgroups  \(parabolic\ subgroups\) .

For the convenience, we label a length function  \(l_T\)  relative to the generating set of  \(T\) . It is easy to get  \(l(w)\leq l_T(w)\)  for all  \(w\in W_T\) .

2.3. Reflection representation of Coxeter groups

In this part, we will focus on the geometric representations of Coxeter groups, define an abstract notion of reflections in vector space. Our main goal is to construct the reflection representations of Coxeter groups and inscribe the irreducibility of such representations. To begin with, we now define  \(linear\ representation\)  of a group.

Definition: A  \(linear\ representation\)  of a group  \(G\)  is a group homomorphism  \(\rho: Garrow GL(V)\) , where  \(V\)  is a vector space.[1](Mukherjee, 2021, p.3)

Meanwhile, we also need to specify  \(reflections\)  over a real vector space, say  \(V\) .

Definition: A  \(reflection\)  on a real vector space  \(V\)  is a linear transformation on  \(V\)  fix a subspace of  \(V\)  of codimension 1 called its  \(mirror\)  and having a nontrivial eigenvector with eigenvalue  \(-1\) , called a  \(root\)  of the reflection. [1](Mukherjee, 2021, p.3)

To further demonstrate a detailed construction of such reflections without loss of generality, we use the  \(lemma\ 4.3\)  from [2] to extend our understanding of reflection for all abstract cases.

Lemma: Let  \(\phi:Varrow\mathbb R\)  be a nonzero linear form on the real vector space  \(V\) ,  \(a\in V\setminus \{0\}\) . Then the following hold:  \((i)\)  The map  \(r_{a, \phi}:Varrow V\)  defined by  \(r_{a,\phi}(v)=v-\phi(v)a\)  is a reflection if and only if  \(\phi(a)=2\) .  \((ii)\)  Every reflection on  \(V\)  can be written in this way.
[1](Mukherjee, 2021, p.3)

Proof:  \((i)\)  ( \(\Leftarrow\) ) Suppose  \(\phi(a)=2\)  in advance.We will show  \(r_{a, \phi}\)  as stated is a reflection by its properties of  \(mirror\)  and  \(root\) .

Hence,  \(a\)  is a nontrivial eigenvector with eigenvalue -1 in  \(r_{a, \phi}\) .
Then, let  \(bx+y\in V\) . Therefore,  \(r_{a, \phi}\)  preserves linear operations in addition and multiplication.
Then,  \(\phi\)  is surjective since for an arbitrary  \(r\in\mathbb R\) , there exists  \(r=r\phi(\frac{a}{2})=\phi(\frac{r}{2}a)\)  such that  \(\phi(\frac{a}{2})=\frac{1}{2}\cdot2=1\) .
Furthermore, because of subjectivity of  \(\phi\) , Im  \(\phi\)  is  \(\mathbb R\) . Then, by first isomorphic theorem with  \(\phi\) ,  \(V/\) Ker  \(\phi\cong\)  Im  \(\phi=\mathbb R\) . In other words,  \(dim V/\) Ker  \(\phi=dim\ V-dim\ \) Ker  \(\phi=dim\ \mathbb R=1\) .
Then, since  \(r_{a, \phi}(x)=x-\phi(x)a=x\)  for all  \(x\in\)  Ker  \(\phi\) ,  \(r_{a, \phi}\)  fixes a subgroup of  \(V\)  of codimension 1.
In sum, by showing  \(r_{a, \phi}\)  is a linear transformation on V with mirror and roots, we know it is a reflection.
 \((\Rightarrow)\)  Suppose  \(r_{a, \phi}:Varrow V\)  as stated is reflection. Then, we are showing  \(\phi(a)=2\) 
Let  \(a'\)  be a root of the reflection,  \(r_{a, \phi}\) . We have As  \(\frac{2}{\phi(a')}\)  is a scalar multiplication in  \(\mathbb R\) ,  \(a\)  is also a root. Therefore,  \((ii)\)  Let  \(r:Varrow V\)  be reflection with root  \(a\)  fixing a subspace  \(U\)  which has basis  \(e_1, ..., e_{n-1}\)  and vector space  \(V\)  has basis  \(e_1, ..., e_n\)  so that the codimension is 1.
Let  \(a\)  be  \(e_n\)  and set  \(v=\Sigma_{i=1}^n k_i e_i\)  as a vector in  \(V\)  for  \(k_i\in\mathbb R\) . We would observes the structure of  \(r\)  that Then, define  \(\phi:Varrow\mathbb R\)  by  \(\phi(v)=2k_n\)  for  \(v\in V\) , we can write  \(r_a(v)=v-\phi(v)a\) .
Suppose for sake of contradiction, there is another way to write reflection,  \(r'\) , then it must express  \(a\)  by the linear combination of the basis of  \(U\)  such that  \(r'_a(a)=r'_a(\Sigma_{i=1}^{n-1}k_ie_i)=\Sigma_{i=1}^{n-1}k_ie_i=a\) . This is a contradiction by its property of mirror.
In conclusion, all reflection must be written in this way.

Considering a vector space  \(V\)  over  \(\mathbb R\)  with a basis  \(\{e_i\}\) , we impose a geometry on  \(V\)  in such a way that the 'angle' between  \(e_i\)  and  \(e_j\)  will be compatible with  \((m_{ij})\)  in a Coxeter system, just like the geometry of dihedral group. Then we define a symmetric bilinear form  \(B\)  on  \(V\)  as follows:

Definition: Let  \((W,M)\)  be a Coxeter system with  \(M=(m_{ij})\) . Let  \(V\)  be a vector space with basis  \(\{e_i\}\) . Define the symmetric bilinear form  \(B\)  on  \(V\)  by \[B(e_i,e_j)=-2cos\frac{\pi}{m_{ij}}\] The quadratic form  \(Q\)  is given by  \(Q(x)=\frac{1}{2}B(x,x)\) . [1](Mukherjee, 2021, p.4)

Next we want to construct the representation of Coxeter group with such defined reflections in the vector space  \(V\) . Define  \(\phi_i(x)=B(e_i,x)\) .

Proposition: There is a unique homomorphism  \(\rho: Warrow GL(V)\)  sending  \(w\)  to  \(\rho_w\) , where  \(\rho_w(x)=x-B(x,e_i)e_i\) , and the group  \(\rho(W)\)  preserves the form  \(B\)  on  \(V\) . Moreover, for each pair  \(s_i,s_j\in S\) , the order of  \(s_is_j\)  in  \(W\)  is precisely  \(m_{ij}\) . [1](Mukherjee, 2021, p.4,5)

Proof: We can easily see that  \(\rho_w\)  preserves the form  \(B\) , which means  \(B(\rho_w\alpha,\rho_w\beta)=B(\alpha,\beta)\)  for all  \(\alpha,\beta \in V\) . To get such a homomorphism from  \(W\)  onto this linear group, we just need to show that \[(\rho_{s_i}\rho_{s_j})^{m_{ij}}=1\] for  \(i\neq j\) . If we consider the subspace  \(V_{ij}=span\{e_i,e_j\}\) , by calculation on  \(B\) , we can prove that the restriction of  \(B\)  to  \(V_{ij}\)  is positive semidefinite and nondegenerate when  \(m\lt \infty\) . See more details in [4]. Then  \(V\)  can be decomposed into the orthogonal direct sum of  \(V_{ij}\)  and its orthogonal complement, which is also fixed by  \(\rho_{s_i}\)  and  \(\rho_{s_j}\) . For the case  \(m=\infty\) , we can also compute the infinite order of  \(\rho_{s_i}\rho_{s_j}\) . The result follows.

Such a homomorphism  \(\rho\)  is called the  \(geometric\ representation\)  of  \(W\) , also the  \(reflection\ representation\)  of  \(W\) .

Remark: There may be other ways to construct the representation of  \(W\)  as a group generated by other forms of reflection, like acting in a hyperbolic space. See [4] and [5].

Definition: A linear representation  \(\rho: Garrow GL(V)\)  is called  \(irreducible\)  if there is no linear subspace of  \(V\)  invariant under  \(\rho(G)\)  except for  \(\{0\}\)  and  \(V\)  itself. The linear representation is called  \(absolutely\ irreducible\)  if it is irreducible and is still irreducible when extending the scalars to  \(\mathbb C\) . [1](Mukherjee, 2021, p.6)

Then we will give the characterization of irreducible reflection representation.

Definition: "The  \(radical\)  of  \(B\)  is given by  \(Rad(B)=\{x\in V|B(x,y)=0\ for\ all\ y\in V\}\) ." [1](Mukherjee, 2021, p.6)

Proposition: Let  \((W,M)\)  be a Coxeter system, then the following statements are equivalent.  \((i)\)  The reflection representation  \(\rho\)  of  \(W\)  is irreducible.  \((ii)\)  The reflection representation  \(\rho\)  of  \(W\)  is absolutely irreducible.  \((iii)\)   \(Rad(B)=\{0\}\) .
[1](Mukherjee, 2021, p.7)

Proof: For  \((i)\Longleftrightarrow (ii)\) , we claim that for an irreducible Coxeter group  \((W,M)\) , any proper invariant subspace of  \(V\)  with respect to the relection representation  \(\rho\)  of  \(W\)  on  \(V\)  is contained in  \(Rad(B)\) . See  \(Lemma\ 4.10\)  in [1]. Since  \((iii)\Rightarrow (ii)\)  is obvious, we only need to show that  \((ii)\Rightarrow (iii)\) . The claim above can be extended the vector space  \(V\)  over  \(\mathbb C\) . Suppose the reflection representation of  \(W\)  is irreducible, then  \(Rad(B)=\{0\}\) , which also holds in  \(\mathbb C\) . So any proper invariant subspace must be trivial. Then the reflection representation of  \(W\)  is absolutely irreducible.

2.4. Reflection representation is faithful

We want to show that the reflection representation  \(\rho\)  is faithful, which means different elements are represented by different linear mappings, i.e.  \(\rho\)  is injective. Then Ker  \(\rho=0\) , which implies  \(W(M)\cong \)  Im  \(\rho\) .

Definition: The  \(root\ system\ \Phi\)  of  \(W\)  is defined by the collection of all vectors  \(\rho_w(e_i)\) . Since  \(W\)  preserves the form  \(B\)  on  \(V\) , these are unit vectors. For any root  \(\alpha=\sum k_ie_i\) ,  \(k_i\in \mathbb R\) ,call  \(\alpha\)   \(positive\) (resp.  \(negative\) ) and write  \(\alpha\gt 0\)  (resp.  \(\alpha\lt 0\) ) if all  \(k_i\geq 0\)  (resp.  \(k_i\leq 0\) ).

Theorem: Let  \(w \in W\)  and  \(s_i\in S\) . If  \(l(ws_i)\gt l(w)\) , then  \(\rho_w(e_i)\gt 0\) . If  \(l(ws_i)\lt l(w)\) , then  \(\rho_w(e_i)\lt 0\) .

Proof: We only need to prove the first part since the second follows from the first. We prove by induction on  \(l(w)\) . For the case  \(l(w)=0\) ,  \(w=1\) ,  \(\rho_w(e_i)=e_i\gt 0\) . For  \(l(w)\gt 0\) , we can find  \(s_j\in S\)  such that  \(l(ws_j)=l(w)-1\) . Since  \(l(ws_i)\gt l(w)\) . We have that  \(s_i\neq s_j\) , i.e.  \(i\neq j\) . Let  \(I=\{s_i,s_j\}\) , then  \(W_I\)  is a dihedral subgroup of  \(W\) . By lemma 3.3, we can decompose  \(w\in \)  based on  \(I\) . Then  \(w=vv_I\)  with  \(l(w)=l(v)+l_I(v_I)\) ,  \(v\in W^I, v_I\in W_I\) . We just need to find the action of  \(\rho_v\)  and  \(\rho_{v_I}\)  on roots. By properties of the length function, we can easily prove that  \(l(vs_i)\gt l(v)\) . See Thm 5.4 in [4]. By induction,  \(\rho_v(e_i)\gt 0\) . Also we can prove  \(l(ve_j)\gt l(v)\) ,  \(\rho_v(e_j)\gt 0\) . All we need to show is that  \(\rho_{v_I}\)  maps  \(e_i\)  to a nonnegative linear combination of  \(e_i\)  and  \(e_j\) . Also see  \(Theorem\ 5.4\)  in [4].

Corollary: The representation  \(\rho: Warrow GL(V)\)  is faithful.

Proof: Let  \(w\in \) Ker  \(\rho\) , if  \(w\neq 1\) , there exists  \(s\in S\)  such that  \(l(ws)\lt l(w)\) . Since  \(w\in \) Ker  \(\rho\) ,  \(\rho_w(e_i)=e_i\gt 0\) , but the theorem states that  \(\rho_w(e_i)\lt 0\) , which is a contradiction.

Corollary: "If  \((W, S)\)  is a Coxeter system and  \(J\)  a subset of S, the subgroup of  \(\langle L\rangle\)  of  \(W\)  is a Coxeter group with Coxeter system  \((\langle J\rangle, J)\) ." [1](Mukherjee, 2021, p.10)

Proof: See proofs by  \(Theorem\ 5.5\ (iii)\)  and  \(Lamma\ 5.4\)  in [1].

2.5. Classification of finite Coxeter groups

For the classification of Coxeter group, we will present the fundamental theorem below. We can classify the finite Coxeter groups by compute the quadratic form  \(Q_M\) . According to  \(Definition\ 2.4\) , the concept of connected components of  \(M\)  introduces  \(decomposition\ law\)  for Coxeter Group, referring to  \(Proposition\ 2.2.5\)  in [2].

Proposition: "Let  \(W\)  be a Coxeter Group of type  \(M\)  and let  \(J_1, ..., J_t\)  be a partition of the vertex set of the labelled graph  \(M\)  into connected components. Then  \(W(M)\cong W(J_1)\times W(J_2)\times...\times W(J_t)\) ."[1](Mukherjee, 2021, p.2)

Proof: By induction on numbers of components of Coxeter group, we only need to prove that  \(W(A\bigsqcup B)\cong W(A)\times W(B)\)  for two connected components  \(A\)  and  \(B\) . See 6.1 in [4].

Consequently, it is suffices to classify a Coxeter group  \(W(M)\)  by identify the classification of each decomposition  \(W(J_1), ..., W(J_t)\) . Then according to definition of each partition  \(J_i\)  where  \(1\leq i\leq t\)  as connected components, we are able to determine the irreducibility of  \(W(J_i)\)  by  \(Definition\ 2.4\) . Besides,  \(group\ action-variant\ bilinear\ form\) , which defined in the following lemma, would help us bridging the connections for later on classification.

Lemma: "Let  \(\rho: Garrow GL(V)\)  be a linear representation of a finite group  \(G\)  on a finite-dimensional real vector space  \(V\) . Then,  \((i)\)  There is a positive-definite  \(G\) -invariant bilinear form on  \(V\) .  \((ii)\)  Moreover,  \(\rho\)  is absolutely irreducible.  \((iii)\)  If each linear map  \(Varrow V\)  commuting with  \(G\)  is multiplication by a scalar, then the form  \(\kappa\)  is the unique  \(G\) -invariant bilinear form on  \(V\)  up to scalar multiples."
[1](Mukherjee, 2021, p.11)

Proof:  \((i)\)  For any positive definite symmetric bilinear form  \(\kappa\)  on  \(V\) , let  \(\overline{\kappa}\)  be  \(G\) -invariant: \[\overline{\kappa}=\sum_{g\in G}\kappa(\rho(g)\alpha,\rho(g)\beta)\] where  \(\alpha,\beta \in V\) .  \((ii)\)  Decompose  \(V\)  into the direct sum of any subspace and its orthogonal complement relative to the positive definite form  \(\overline{\kappa}\)  in  \((i)\) . Also the orthogonal complement of a  \(G\) -invariant space is also  \(G\) -invariant.  \((iii)\)  Any nondegenerate form develops an isomorphism between  \(V\)  and  \(V^*\) . With the  \(G\) -invariant form, this becomes an isomorphism of  \(G\) -modules. Suppose  \(\kappa\)  and  \(\kappa'\)  are nondegenerate symmetric  \(G\) -invariant bilinear forms on  \(V\) .Composing the isomorphism defined by  \(\kappa\)  with the inverse of that defined in  \(\kappa'\)  gives a  \(G\) -module isomorphism of  \(V\)  onto  \(V\) . Since this is just a scalar,  \(\kappa\)  and  \(\kappa'\)  are proportional.

With positive-definite symmetric bilinear form  \(\kappa\)  on  \(V\) , we would apply the following theorem to assure the Coxeter group of type  \(M\)  is finite.

Theorem: "For any Coxeter system  \((W,S)\)  of type  \(M=(m_{ij})\)  such that  \(W\)  is irreducible, then the following are equivalent:  \((i)\)   \(W\)  is finite.  \((ii)\)  The reflection representation  \(\rho: Warrow GL(V)\)  is irreducible.  \((iii)\)  The quadratic form  \(Q_M\)  is positive definite." [1](Mukherjee, 2021, p.11)

Proof:  \((i)\Rightarrow (ii)\)  Use Lemma 6.2, we can construct a positive-definite bilinear form  \(\kappa\)  that is invariant under  \(w\) . Suppose that  \(E\)  is a proper nontrivial invariant subspace of  \(V\) , then we can decompose  \(V=D\bigoplus E\)  and show that  \(D\)  is also invariant under  \(W\) , which means  \(B=0\)  in  \(V\) , a contradiction to the fact that  \(B(e_i,e_i)=2\) . Hence  \(\rho\)  is irreducible.  \((ii)\Rightarrow (iii)\)  Since  \(B\)  is a  \(W\) -invariant bilinear form on  \(V\) . By Prop 4.8,  \(\rho\)  is absolutely irreducible. Then by Lemma 6.2 B is a scalar multiple of a positive-definite bilinear form. Since  \(B(e_i,e_i)\gt 0\) , the scalar has to be positive, which means  \(B\)  is positive-definite.  \((iii)\Rightarrow (i)\)  We want to use the fact that  \(\rho(W)\)  is a discrete subgroup of  \(GL(V)\)  with specific topology based on the construction of the dual representation of  \(\rho\) . See 6.2 in [4]. We can embed  \(\rho(W)\)  into the orthogonal group  \(O(n)\) , which is compact. Since a discrete subgroup of a compact Hausdorff group is closed, hence finite,  \(W\cong \rho(W)\)  is also finite.

Theorem: All irreducible finite Coxeter groups are classified with respect to corresponding Coxeter diagrams below.

Table 1: Coxeter-Dynkin Diagram of irreducible finite Coxeter Groups from [1]

name	diagram
\(A_n\)   \((n\geq 1)\)	\begin{dynkinDiagram}[text style/.style={scale=0.8,black}, edge length=1.2cm,labels={1,2,n-1,n}, mark=o]A{} \end{dynkinDiagram}
\(B_n=C_n\)   \((n\geq 3)\)	\begin{dynkinDiagram}[text style/.style={scale=0.8,black}, edge length=1.2cm,labels={1,2,n-2,n-1,n}, mark=o]B{} \end{dynkinDiagram}
\(D_n\)   \((n\geq 4)\)	\begin{dynkinDiagram}[text style/.style={scale=0.8,black}, edge length=1.2cm,labels={1,2,n-3,n-2,n-1,n}, ,label directions={,,,right,,},mark=o]D{} \end{dynkinDiagram}
\(E_6\)	\begin{dynkinDiagram}[text style/.style={scale=0.8,black}, edge length=1.2cm,label, mark=o]E6 \end{dynkinDiagram}
\(E_7\)	\begin{dynkinDiagram}[text style/.style={scale=0.8,black}, edge length=1.2cm,label, mark=o]E7 \end{dynkinDiagram}
\(E_8\)	\begin{dynkinDiagram}[text style/.style={scale=0.8,black}, edge length=1.2cm,label, mark=o]E8 \end{dynkinDiagram}
\(F_4\)	\begin{dynkinDiagram}[text style/.style={scale=0.8,black}, edge length=1.2cm,label, mark=o]F4 \end{dynkinDiagram}
\(G_2\)	\begin{dynkinDiagram}[text style/.style={scale=0.8,black}, edge length=1.2cm,label, mark=o]G2 \end{dynkinDiagram}
\(H_3\)	\begin{dynkinDiagram}[text style/.style={scale=0.8,black}, edge length=1.2cm,label, mark=o]H3 \end{dynkinDiagram}
\(H_4\)	\begin{dynkinDiagram}[text style/.style={scale=0.8,black}, edge length=1.2cm,label, mark=o]H4 \end{dynkinDiagram}
\(I_2^{(m)}\)   \((m\geq 3)\)	\begin{dynkinDiagram}[edge length=1.2cm, gonality=m, mark=o]I{} \end{dynkinDiagram}

Proof: ( \(\Leftarrow\) ) Using Thm 6.3, we only need to check  \(Q_M\)  is positive-definite for each  \(M\)  in diagrams above. Then  \(W(M)\)  is finite. ( \(\Rightarrow\) ) By calculation on  \(Q_M\)  of certain exhausting cases. We can determine some conditions that the Coxeter diagrams must hold. The only connected diagrams satisfying these conditions are exactly those in diagrams above. See more details in [1] Thm 6.3.

3. Representation of Orthogonal Groups

Since the abstract Coxeter group is embedded in the orthogonal group  \(O(n)\)  by a chosen bilinear form on V, we will discover some properties of the representation of  \(O(n)\) . To begin with, if we consider all the polynomials in  \(n\)  variables, then the Laplace operator is invariant under the group action which  \(O(n)\)  acts on those polynomials. Then we can construct a representation of  \(O(n)\)  via the space of harmonic polynomials, denoted as  \(\mathcal{H}(n)\)  with  \(n\)  variables.

3.1. Decomposition of  \(\mathcal{H}(2)\)  and irreducible representations of  \(O(2)\) 

As a starting point, let's observe some concepts and examples to understand both.

Definition: A  \(graded\ ring\)  is a ring that is decomposed into a direct sum \[R=\bigoplus_{n=0}^\infty R_n=R_0\oplus R_1\oplus R_2\oplus\cdots\] of additive groups, such that \[R_mR_n\subseteq R_{m+n}\] for all nonnegative integers  \(m\)  and  \(n\) . A nonzero element of  \(R_n\)  is said to be  \(homogeneous\ of\ degree\ n\) . An algebra  \(A\)  over a ring  \(R\)  is a  \(graded\ algebra\)  if it is a graded ring.

Remark: The polynomial ring is a graded algebra. The homogeneous elements of degree  \(n\)  are exactly the homogeneous polynomials of degree  \(n\) .

Hence we can decompose the polynomial ring  \(R[X]\)  into the subspaces of homogeneous polynomials  \(R_n[X]\)  of fixed degree  \(n\) .\[R[X]=\bigoplus_{n=0}^\infty R_n[X]=R_0[X]\oplus R_1[X]\oplus R_2[X]\oplus\cdots\] Considering the harmonic polynomials ring  \(\mathcal{H}\)  which is also a graded algebra, we find that each subspace has a fixed finite dimension, which we will prove later. First, we will focus on  \(\mathcal{H}(2)\)  which by Definition 3.1, \[\mathcal{H}(2)= \mathcal{H}_0(2)\oplus \mathcal{H}_1(2)\oplus \mathcal{H}_2(2)\oplus\cdots\]

Table 2: Chart of basis of  \(\mathcal{H}_m(2)\)  for all  \(m\geq 0\) 

\(\mathcal{H}_m(2)\)	Span \(\{\) Basis \(\}\)
\(\mathcal{H}_0(2)\)	\(\mathbb C\)
\(\mathcal{H}_1(2)\)	Span  \(\{x, y\}\)
\(\mathcal{H}_2(2)\)	Span  \(\{x^2-y^2, xy\}\)
\(\mathcal{H}_3(2)\)	Span  \(\{x^3-3y^2x, y^3-3x^2y\}\)
\(\mathcal{H}_4(2)\)	Span  \(\{x^4+y^4-6x^2y^2, x^3y-xy^3\}\)
...	...
\(\mathcal{H}_m(2)\)	Span  \(\{Re\ z^m, Im\ z^m\}\)
...	...

Computing the basis of each decomposition terms, we observes the following shown in the table above. For  \(\mathcal{H}_0(2)\) ,\[\mathcal{H}_0(2)=\{P(x, y)=c|c\in\mathbb C\}=\mathbb C\] Hence,  \(\mathcal{H}_0(2)\)  has dimension 1. For  \(\mathcal{H}_1(2)\) , \[\mathcal{H}_1(2)=\{P(x, y)=ax+by|a, b\in\mathbb C\}=Span \{x, y\}\] Hence,  \(\mathcal{H}_1(2)\)  has dimension 2. For  \(\mathcal{H}_2(2)\) , \[\mathcal{H}_2(2)=\{P(x, y)=ax^2+by^2+cxy|a, b, c\in\mathbb C\}=Span \{x^2-y^2, xy\}\] Hence,  \(\mathcal{H}_2(2)\)  has dimension 2. Continuing on computing basis, in fact,  \(\mathcal{H}_m(2)=span\{Re\ z^m, Im\ z^m\}\)  where  \(z=x+iy\) . Hence,  \(\mathcal{H}_m(2)\)  also has dimension 2. In sum, the decomposition of  \(\mathcal{H}(2)\)  shows  \(\mathcal{H}_0(2)\)  has dimension  \(1\)  and all other decomposition terms has dimension  \(2\) . Next we will discuss the irreducible representations of  \(O(2)\) . We conclude that  \(O(2)\)  has only three types of irreducible representations. \begin{claim}  \((i)\)  The trivial representation  \(1\)  \[G\stackrel{1}\longrightarrow 1\]  \((ii)\)  The projection/determinant representation  \(1_{det}\)  \[O(2)\cong \mathbb{Z}_2\ltimes SO(2)\stackrel{1_{det}}\longrightarrow \mathbb{Z}\]  \((iii)\)  A representation  \(2_q\)  \[O(2)\cong\mathbb{Z}_2\ltimes U(1)\stackrel{2_q}\longrightarrow GL_2(\mathbb{C})\] given by \[(d,e^{i\theta}\stackrel{2_q}\longrightarrow \sigma_1^{\frac{1-d}{2}}e^{iq\theta\sigma_3})\] where  \(q \in \mathbb{N}\) .  \(\sigma_1=\begin{pmatrix} 0 \& 1
1 \& 0
\end{pmatrix}\) ,  \(\sigma_2=\begin{pmatrix} 0 \& -i
i \& 0
\end{pmatrix}\) ,  \(\sigma_3=\begin{pmatrix} 1 \& 0
0 \& -1
\end{pmatrix}\) . \end{claim} If we let  \(Ad\)  be the adjoint representation of  \(O(2)\)  in its Lie algebra. We could show that it is equivalent to one of the representations in  \((i),(ii)\)  or  \((iii)\) . See details in [6] and Section 11.1 of [7]. In conclusion, except for the trivial representation, we have a 1-dim irreducible representation of determinant and other 2-dim irreducible representations related to the charge conjugation [6]. These are exactly the dimensions of the subspace  \(H_m(2)\)  of  \(H(2)\)  for  \(m\in\mathbb{N}\) . We will conclude such relations in the general case later. Moreover, since  \(O(2)=SO(2)\cup O^-(2)\)  where  \(SO(2)=\{A\in M_2(\mathbb R)|det(A)=1\}\)  and  \(O^-(2)=\{B\in M_2(\mathbb R)|det(B)=-1\}\) , we could observe the group action of  \(O(2)\)  on  \(\mathcal{H}(2)\) . Take  \(A=\begin{pmatrix} cos\theta \& -sin\theta
sin\theta \& cos\theta \end{pmatrix}\in SO(2)\)  and  \(B=\begin{pmatrix} -cos\theta \& sin\theta
sin\theta \& cos\theta \end{pmatrix}\in O^-(2)\) . Assume  \(\begin{pmatrix} x
y \end{pmatrix}\)  are two variables of  \(\mathcal{H}(2)\) . Then, we compute for  \(\begin{pmatrix} x'
y' \end{pmatrix}\)  after  \(O(2)\)  acts on  \(\mathcal{H}(2)\) .

Next, We are going to test some examples to see why  \(\Delta\)  is  \(O(2)\) -invariant. \begin{example} Let  \(P_m(x, y)=(x+iy)^m\)  be harmonic polynomials with two variables in  \(\mathcal{H}(2)\)  because of what we claimed about  \(\mathcal{H}_m(2)\) .
We will show  \(\begin{pmatrix} x'
y' \end{pmatrix}\)  after  \(O(2)\)  acts on  \(\mathcal{H}(2)\)  is still variables in  \(\mathcal{H}(2)\) .
for  \(P_0(x, y)\in \mathcal{H}_0(2)\) ,

for  \(P_1(x, y)\in \mathcal{H}_1(2)\) ,

for  \(P_2(x, y)\in\mathcal{H}_2(2)\) ,

\end{example} Referring to what we find about each basis of the decomposition terms of  \(\mathcal{H}(2)\)  as the table 2, we claim that  \(\Delta\)  is invariant under  \(O(2)\) . Moreover,  \(\Delta\)  is invariant under  \(O(n)\)  for the general case, which we will prove later. This gives us an idea to construct the representation of  \(O(n)\)  via the space of harmonic polynomials.

3.2. Decomposition of  \(\mathcal{H}_m\)  and irreducible representations of  \(O(n)\) 

Definition: "The  \(spherical\ harmonics\)  can be expressed as the restriction to the unit sphere  \(S^{n-1}\)  of certain polynomial functions  \(\mathbb{R}^narrow \mathbb{C}\) . Specifically, we say that a complex-valued polynomial function  \(p: \mathbb{R}^narrow \mathbb{C}\)  is  \(homogeneous\)  of degree  \(m\)  if  \(p(\lambda x)=\lambda^mp(x)\)  for all  \(\lambda\in\mathbb{R}\)  and all  \(x\in \mathbb{R}^n\) ." Adapted from [8]

Let  \(\mathcal{P}_m\)  denote the space of complex-valued homogeneous polynomials of degree  \(m\)  in n real variables. Let  \(\mathcal{H}_m\)  denote the subspace of  \(\mathcal{P}_m\)  consisting of all harmonic polynomials: \[\mathcal{H}_m:=\{p\in \mathcal{P}_m|\Delta p=0\}\] Let  \(\mathcal{SH}_m\)  denote the space of functions on the unit sphere  \(S^{n-1}\) : \[\mathcal{SH}_m:=\{f:S^{n-1}arrow \mathbb{C}| for some p\in \mathcal{H}_m, f(x)=p(x) for all x\in S^{n-1}\}\]

Definition: Take function  \(\rho\) , as a representation function of  \(G\)  applied to the algebra  \(\mathcal{P}(V)\)  of polynomial functions defined on  \(V\) , such that \[\rho(g)f(v)=f(g^{-1}v) for f\in\mathcal{P}(V) \]

The finite-dimensional spaces  \(\mathcal{P}_m(V)\)  of homogeneous polynomials of degree  \(m\)  are  \(G\) -invariant and the restriction  \(\rho_m\)  of  \(\rho\)  to  \(\mathcal{P}_m(V)\)  is a regular representation of  \(G\) . If we want to define such a representation of orthogonal group  \(O(n)\)  via the space of harmonic polynomials  \(\mathcal{H}\) , we need to ensure that the action of  \(O(n)\)  on  \(\mathcal{H}\)  is closed. Now, let's referring back to what we promised after we show some invariant properties of Laplace's operator  \(\Delta\)  when  \(O(2)\)  acts on it, we are going to prove the general case for  \(O(n)\) .

Proposition: Laplace operator  \(\Delta\)  is  \(O(n)\) -invariant.

Proof: Let  \(A=(a_{ij})_{i, j}\in O(n)\)  and define a harmonic function  \(u\)  s.t.  \(\Delta u=0\) .
We will show, if we define  \(v(x)\coloneqq u(Ax)\)  for  \(x\in\mathbb R^n\) , then  \(\Delta v=0\) .
First, by the property of  \(O(n)\) ,

Hence, we know Then, let  \(Ax=\begin{pmatrix}y_1 \& y_2 \& ... \& y_n \end{pmatrix}^T\) , we could compute first-order partial derivative  \(v_{x_l}\)  by  \(x_l\)  for  \(0\lt l\leq n\) . Next, substitute the above results into the second-order partial derivative  \(v_{x_lx_l}\) . Therefore, we get Referring back to the property of  \(O(n)\) , Therefore, due to  \(\Delta v=\Delta u=0\) ,  \(\Delta\)  is  \(O(n)\) -invariant.

Hence  \(\mathcal{H}\)  is invariant under the action of  \(O(n)\) . Let  \(G\)  be  \(O(n)\) ,  \(\mathcal{P}(V)\)  be  \(\mathcal{H}\)  in Def 3.3. Then we get the representation  \(\rho\)  via  \(\mathcal{H}\) . As we mentioned,  \(\mathcal{H}\)  can be decomposed into the direct sum of subspaces of homogeneous harmonic polynomials with various degrees. We state that each  \(\mathcal{H}_m\)  is an irreducible representation of  \(O(n)\)  of dimension  \(\tbinom{n+m-1}{n-1}-\tbinom{n+m-3}{n-1}\) (for  \(m=0 or 1\) , the second term is zero). To prove this, we will use the lemma below.

Lemma: Every homogeneous polynomials  \(p\in\mathcal{P}_m\)  can be uniquely written as \[p=p_m+|x|^2p_{m-2}+\cdots + \begin{cases} |x|^mp_0, \& m even,
|x|^{m-1}p_1, \& m odd \end{cases}\] where  \(p_j\in\mathcal{H}_j\) .

See proof in Corollary 1.8 of [9]. In particular, by induction on the dimension(Chapter IX.\textsection2.[10]), we can get dim \(\mathcal{H}_m=\tbinom{n+m-1}{n-1}-\tbinom{n+m-3}{n-1}\) .

3.3. Some Invariant Theory

Moreover, each polynomials  \(p\)  in the Euclidean space  \(\mathbb{R}^n\)  can be uniquely written as a finite sum \[p=h_0+r^2h_1+\cdots+r^{2j}h_j+\cdots\] where  \(r^2=x_1^2+\cdots+x_n^2\)  for  \(x=(x_1,...,x_n)\in R^n\)  and  \(h_j\)  are harmonic polynomials in  \(R^n\) . In other words, the space  \(\mathcal{P}\)  of  \(\mathbb{C}\) -valued polynomials on  \(R^n\)  decomposes as \[\mathcal{P}=\bigoplus_{m=0}^\infty r^{2m}\mathcal{SH}\] where  \(\mathcal{SH}=Ker(\Delta)\cap\mathcal{P}\)  is the space of spherical harmonics in  \(R^n\) . This result is known as the Fischer decomposition. The underlying symmetry is given by  \(O(n)\)  and the invariant operators  \(\Delta,r^2,h\)  generate the Lie algebra  \(\mathfrak{sl}(2)\)  where \[h=x_1
tial_{x_1}+\cdots+x_n
tial_{x_n}+\frac{n}{2}\] is the Euler operator. Furthermore, the decomposition above can be combined into the statement that the space  \(\mathcal{P}\)  of polynomials in n variables has the following decomposition into irreducible components under the joint  \(O(n)\)  and  \(\mathfrak{sl}(2)\)  actions: \[\mathcal{P}=\bigoplus_{m=0}^\infty H_m\otimes V_m\] where  \(V_m\)  is the lowest weight  \(\mathfrak{sl}(2)\) -module with lowest weight  \(m+\frac{n}{2}\) . Actually this is the reductive dual pair  \((O(n),SL(2,\mathbb{R})\)  over the real numbers. Such results about Invariant Theory and Duality can be found in 5.6 in [11]. Next, we state a theorem for the general case.

Theorem: Suppose  \(G\)  is a reductive linear algebraic group acting by a regular representation on a vector space  \(V\) . Then the algebra  \(\mathcal{P}(V)^G\)  of  \(G\) -invariant polynomials on  \(V\)  is finitely generated as a  \(\mathbb{C}\) -algebra.

We get that there always exists a finite set of basic invariants when  \(G\)  is reductive. Let  \(\{f_1,...,f_n\}\)  be generators for  \(\mathcal{P}(V)^G\) . Since  \(\mathcal{P}(V)\)  and  \(\mathcal{P}(V)^G\)  are graded algebras, relative to the usual degree of a polynomial, there is a set of basic invariants with each  \(f_i\)  homogeneous, degree  \(d_i\) . We can also prove that  \(\{d_i\}\)  is uniquely determined, see 5.1.1 in [11]. Applying to  \(O(n)\) , this is just the results we have presented. In the end, this paper emphasizes the broad applicability of the developed concepts of Coxter Group in the context of duality in general by combining them with orthogonal groups and invariant theory. The paper underscores the connection between the constructed representations and the results in invariant theory, bringing to the understanding of duality in mathematical structures.