Gierhos et al 2024 – Towards flexible perception with Visual Memory
Setting:
- We can build a classifier, but what happens when we want to change the output labels, maybe because of distribution shift?
Basic idea:
- Let's separate the representation learning and the labeling parts of a traditional classifier
- Let's store a "Visual Memory" which consists of feature vectors and the corresponding label.
- Now, how do we classify a new image? Comb over the visual memory, find the feature vectors that are closest to the features for the new image, and take the majority label