SVCCA

(Raghu et al., 2017) proposed a way to compare two representations that is both invariant to affine transform and fast to compute ¹

based on canonical correlation analysis which was invariant to linear transformation.

definition

Given a dataset $X=\{x_1,\cdots ,x_m\}$ and a neuron $i$ on layer $l$, we define $z_i^l$ to be the vector of outputs on $X$, or:

$$z_i^l=(z_i^l(x_1),\cdots ,z_i^l(x_m))$$

SVCCA proceeds as following:

1. Input: takes as input two (not necessary different) sets of neurons $l_1=\{z_1^{l_1},\cdots ,z_{m_1}^{l_1}\}$ and $l_2=\{z_1^{l_2},\cdots ,z_{m_2}^{l_2}\}$

2. Step 1: Perform SVD of each subspace to get subspace $l_1^{{}^{\prime }}\subset l_1,l_2^{{}^{\prime }}\subset l_2$

3. Step 2: Compute Canonical Correlation similarity between $l_1^{{}^{\prime }},l_2^{{}^{\prime }}$, that is maximal correlations between $X,Y$ can be expressed as:

   $$\max \frac{a^T{\sum }_{XY}b}{\sqrt{a^T{\sum }_{XX}a}\sqrt{b^T{\sum }_{YY}b}}$$

   where ${\sum }_{XX},{\sum }_{XY},{\sum }_{YX},{\sum }_{YY}$ are covariance and cross-variance terms.

   By performing change of basis $\widetilde{x_1}={\sum }_{xx}^{\frac{1}{2}}a$ and $\widetilde{y_1}={\sum }_{YY}^{\frac{1}{2}}b$ and Cauchy-Schwarz we recover an eigenvalue problem:

   $$\widetilde{x_1}=\mathrm{argmax}[\frac{x^T{\sum }_{XX}^{\frac{1}{2}}{\sum }_{XY}{\sum }_{YY}^{-1}{\sum }_{YX}{\sum }_{XX}^{-\frac{1}{2}}x}{\Vert x\Vert }]$$

4. Output: aligned directions $(\widetilde{z_i^{l_1}},\widetilde{z_i^{l_2}})$ and correlations ${\rho }_i$

distributed representations

SVCCA has no preference for representations that are neuron (axed) aligned. ²

Raghu, M., Gilmer, J., Yosinski, J., & Sohl-Dickstein, J. (2017). SVCCA: Singular Vector Canonical Correlation Analysis for Deep Learning Dynamics and Interpretability. arXiv preprint arXiv:1706.05806 arxiv

Footnotes

1. means allowing comparison between different layers of network and more comparisons to be calculated than with previous methods ↩

2. Experiments were conducted with a convolutional network followed by a residual network:

   convnet: conv --> conv --> bn --> pool --> conv --> conv --> conv --> conv --> bn --> pool --> fc --> bn --> fc --> bn --> out

   resnet: conv --> (x10 c/bn/r block) --> (x10 c/bn/r block) --> (x10 c/bn/r block) --> bn --> fc --> out

   Note that SVD and CCA works with $\text{span}(z_1,\cdots ,z_m)$ instead of being axis aligned to $z_i$ directions. This is important if representations are distributed across many dimensions, which we observe in cross-branch superpositions! ↩