Thinking about caching

I have heard it multiple times: Caching is evil! The risks and benefits of caching are obvious. The main risks using a caching strategy are to work with stale data and to waste memory. On the other hand the benefits are reducing database access and increasing performance by reading pre calculated data. The Java EE entity manager (JPA) uses a first level cache and optional a second level cache. The first level cache is related to the persistence context which means the cache is valid between a single transaction. After the transaction commits or possibly after a rollback the first level cache is gone.

The second level cache (L2-Cache) has a longer lifetime. The second level cache is independent of a particular entity manager instance and the related transactions. Using a second level cache reduces database access to gain better performance. The second level cache should be configured that the count of maintained entities are limited and that the cache is refreshed automatically after an expired timeframe. To reduce the problem of stale data put mainly entities with static data in the second level cache. Static data are system data which could be application profile data, labels of an I18n strategy or fixed GUI related data which alter seldom.

The rule of thumb is:

Caching is valuable without risk for static data which are not refreshed during program execution. Caching can be evil for frequent altered data even in a transactional context.

Caching to reduce database access is one part of the story, but caching could also lead to better performance within the application itself. Java has several collection classes which supports safer caching to avoid stale data. A good example is the WeakHashMap.  An entry in a WeakHashMap will automatically be removed when its key is no longer in ordinary use.

Since Java 5 the LinkedHashMap can be used as a LRU Cache (last recently used cache). By overriding the “removeEldestEntry” method to implement a policy for removing stale mappings automatically when new mappings are added. There are even more sophisticated caching solutions (e.g.: Ehcache) available which handle evict and refreshing better as the simple Java API implementations.

The conclusion is:  Caching could be evil when wrongly considered and valuable when considered with care. Therefore use caching with care!