Random Fourier Gaussian Features

The Gaussian kernel, widely used in machine learning (like Support Vector Machines and Gaussian Processes), projects data into an infinite-dimensional feature space. Random Fourier Features approximate this continuous kernel using a finite-dimensional mapped space. By drawing random frequencies from the corresponding spectral density (a Gaussian distribution), we can compute low-dimensional explicit features whose inner products approximate the true Gaussian kernel, allowing massive scale-up without computing an exact N×N kernel matrix.

Experiment: Visualizing Kernel Approximation

Open the Random Fourier Gaussian Features module to explore how the finite-sample features approximate the exact Radial Basis Function (RBF) curve.

1. Keep the default Seed at 1, Length scale at 1, Point x at 0, and Point y at 1. Note the mismatch between the amber approximation curve and the blue exact curve.
2. Increase the Frequency pairs to 8. The amber curve more closely hugs the true kernel curve because higher dimensionality reduces the variance of the approximation.
3. Change the Seed to 7 or 42. Notice how the shape of the approximation curve changes randomly with each new set of sampled frequencies, sometimes overestimating and sometimes underestimating the exact kernel at specific points.
4. Adjust the Length scale to 0.5. The kernel curve becomes narrower, and the frequencies adapt accordingly.
5. Set Point x to 0 and Point y to 0. Observe that the exact kernel and approximation are identically 1.0 (with 0 error), as cos(0) = 1.