Book The Memory Police — Yoko Ogawa

Most of the inhabitants are oblivious to these changes, while those who remember live in fear of the Memory Police.
To the people on the island, a disappeared thing no longer has any meaning. It can be burned in the garden, thrown in the river, or handed over to the Memory Police. Soon enough, the island forgets it ever existed.
When a young novelist discovers that her editor is in danger of being taken away by the Memory Police, she desperately wants to save him. For some reason, he doesn't forget, and it's becoming increasingly difficult for him to hide his memories. Who knows what will vanish next?