To be accurate, caches both keep and delete data. In fact, deleting data is a pretty important function of most caches as the nature of a cache is a fast lookup, not an infinite store.
This is of course a glib statement, but in general cached have specific (often configurable) eviction policies, so we would expect to see the root of the cache held perpetually anyways.