A lot of people would see this quote as a criticism of stamp collecting. I think the quote can be interpreted in a different light.I don't know whether Rutherford ever collected stamps, but he's not necessarily saying anything unpleasant about stamp collecting.
It's fair to say that a lot of science is about taxonomy - categorising things. A biologist, for example, might want to divide plankton or lizards into their various phyla and genuses and try to find similarities between them. To a theoretical physicist and atom-smasher like Rutherford, this would be the scientific equivalent of stamp collecting. And it is quite dissimilar to the sort of theoretical physics that Rutherford found so exciting.
This wouldn't mean that taxonomy is tedious or unimportant. After all, without taxonomy Darwin would have been unable to write the Origin of the Species. So stamp collectors shouldn't feel belittled by this famous quote.
Incidentally, I'm not sure that stamp collectors have ever been responsible for making huge advances in civilisation and human knowledge. But at least they can't be blamed for creating the atom bomb.
How stamps helped George V survive the war