Gradient Reversal Domain Adaptation

Domain adaptation aims to align feature representations between a source domain (where labels are available) and a target domain (where labels are not). The Gradient Reversal approach uses an adversarial domain discriminator to distinguish source from target features. During backpropagation, the gradient from this domain classifier is reversed by multiplying it by a negative adversarial weight -λ, encouraging the feature extractor to learn representations that are invariant to the domain shift.

Experiment

Interactive module experiment: Gradient Reversal Domain Adaptation Module.

Follow these steps to observe the adversarial gradient reversal in action:

1. Observe the baseline gradients with the default values. Note the sign of the reversed total feature gradient.
2. Change the Adversarial weight to 0. The total feature gradient will no longer subtract the domain gradient, reflecting standard supervised training without domain adaptation.
3. Change the Feature weight to 0 and the Domain head weight and Adversarial weight to 2. Observe the saddle point behavior.