Regarding Linear Isometries
Proof-1
This idea for this proof is inspired by my colleague, Vivek Sivaramakrishnan. Since \(R\) is \(k\times d\) with \(k< d\), the nullity of \(R\) is non-zero. That is, there is some non-zero vector \(x\in \mathbb{R}^{d}\) such that \(Rx=0\). If \(Rx=0\), then \(||Rx||^{2} =0\). However, \(||x||\neq 0\). Hence, we see that \(R\) cannot be a norm-preserving map.
Proof-2
A slightly circuitous way of proving this result by contradiction. If possible, let there exist such a norm-preserving linear map from \(\mathbb{R}^{d}\) to \(\mathbb{R}^{k}\). If \(R\) preserves norms, then for any vector \(x\in \mathbb{R}^{d}\), we have:
\[ \begin{equation*} \begin{aligned} ||Rx||^{2} & =||x||^{2} \end{aligned} \end{equation*} \]
Using the fact that \(||Rx||^{2} =( Rx)^{T}( Rx)\), we have:
\[ \begin{equation*} \begin{aligned} ||x||^{2} & =x^{T}\left( R^{T} R\right) x \end{aligned} \end{equation*} \]
Call \(M=R^{T} R\). The above equation then becomes:
\[ \begin{equation*} ||x||^{2} =x^{T} Mx,\ \forall x\in \mathbb{R}^{d} \end{equation*} \tag{1}\]
Let the rank of \(R\) be \(r\). We know that the rank of \(M\) is the same as the rank of \(R\). Therefore, \(\text{rank}(M) = r\). Importantly, we have \(r \leqslant k < d\).
Also note that \(M\) is a symmetric, positive semi-definite matrix of shape \(d\times d\). So it has a full set of eigenvalues:
\[ \lambda _{1} \geqslant \cdots \geqslant \lambda _{r} >\lambda _{r+1} =\cdots \lambda _{d} =0 \]
Consider one of the zero eigenvalues with corresponding eigenvector \(v\). Then:
\[ \begin{equation*} \begin{aligned} v^{T} Mv & =v^{T}( Mv)\\ & =v^{T}( 0v)\\ & =0 \end{aligned} \end{equation*} \]
However, from Equation 1, \(v^{T} Mv=||v||^{2}\) and \(||v||^{2}\) cannot be zero, since \(v\) is an eigenvector. That leads us to a contradiction and we are done with the proof.