Caching reboot: javax.cache & Ehcache 3

@ljacomet#ehcachedvx
Caching reboot
javax.cache & Ehcache 3

Agenda
• Why a cache?

Agenda
• Why a cache?
• JSR-107
• APIs and features
• Provider extensions

Agenda
• Why a cache?
• JSR-107
• Caching in an application
• Cache aside
• Cache through

Agenda
• Why a cache?
• JSR-107
• Cache aside
• Cache through
• Resilience strategy

Agenda
• Why a cache?
• JSR-107
• Cache aside
• Cache through
• Resilience strategy
• Extras

Who am I?
• Louis Jacomet
• Software engineer, closer to 40 than 20 and still coding!
• Engineer at Terracotta (Software AG) since 2013
• Working on Ehcache OS and EE mostly
• team manager and ﬁddling in infrastructure
• No idea how to do a (nice) UI - especially a web one
• Enjoys concurrency, thinking hard about APIs, …

Why a cache?

What were these numbers?

• Minute:
• L1$cache$access$$$$$$$$$$$$$$$0,5$s$$$One$heartbeat$
• Branch$misprediction$$$$$$$$$$5$$$s$$$Yawn$
• L2$cache$access$$$$$$$$$$$$$$$7$$$s$$$Long$yawn$
• Mutex$lock/unlock$$$$$$$$$$$$25$$$s$$$Coffee$preparation

• Minute:
• Hour:
• RAM$access$$$$$$$$$$$$$$$$$$100$$$s$$$Brushing$your$teeth$
• Compressing$1K$$$$$$$$$$$$$$$50$$$min$One$TV$show$episode

• Minute:
• Hour:
• RAM$access$$$$$$$$$$$$$$$$$$100$$$s$$$Brushing$your$teeth$
• Compressing$1K$$$$$$$$$$$$$$$50$$$min$One$TV$show$episode
• Day:
• 2KB$sent$on$1GB$network$$$$$$$5,5$h$$$Your$work$afternoon

• Week:
• Random$read$SSD$$$$$$$$$$$$$$$$$1,7$d$$$$$$$A$weekNend$
• Sequence$read$1$MB$from$RAM$$$$$2,9$d$$$$$$$A$long$weekNend$
• Round$trip$in$datacenter$$$$$$$$5,8$d$$$$$$$Holidays$
• Sequence$read$1MB$from$SSD$$$$$11,6$d$$$$$$$Two$weeks$delivery

• Week:
• Year:
• Seek$on$rotational$HDD$$$$$$$$$16,5$weeks$$$School$semester$
• Sequence$read$1MB$from$HDD$$$$$$7,8$months$$Almost$a$baby

• Week:
• Year:
• Seek$on$rotational$HDD$$$$$$$$$16,5$weeks$$$School$semester$
• Sequence$read$1MB$from$HDD$$$$$$7,8$months$$Almost$a$baby
• Decade:
• Data$roundNtrip$around$the$world$4,8$years$$PhD$thesis

So what is a cache?

So what is a cache?
• Data structure holding a temporary copy of some data

So what is a cache?
• Trade off between higher memory usage for reduced latency

So what is a cache?
• Trade off between higher memory usage for reduced latency
• Targets :
• Data which is reused
• Data which is expensive to compute or retrieve

Ehcache (3)

Ehcache (3)
• New version, fully integrated with JSR-107
• Breaks compatibility with 2.x line

Ehcache (3)
• Developed in the open
• https://github.com/ehcache/ehcache3

Ehcache (3)
• Developed in the open
• https://github.com/ehcache/ehcache3
• Public dev meeting, almost weekly, on hangouts on air
• Recordings available onYoutube

JSR-107
• Ancient JSR
• Started in 2001
• Approved a year ago (March 2014)

JSR-107
• Ancient JSR
• Started in 2001
• javax.caching API + TCK + reference implementation
• Eases integration of a cache in frameworks

JSR-107
• Ancient JSR
• Started in 2001
• javax.caching API + TCK + reference implementation
• Eases integration of a cache in frameworks
• Ongoing discussions about a version 2.0
• Async API for example

JSR-107 : Features

JSR-107 : Features
•CacheManager / Cache

JSR-107 : Features
•Expiration
•Creation/Access/Update

JSR-107 : Features
•Expiration
•Integration
•CacheLoader / Writer

JSR-107 : Features
•Expiration
•Integration
•CacheEntryListener
•Created/Updated
•Removed/Expired
•Get old value

JSR-107 : Features
•Expiration
•Integration
•Created/Updated
•Removed/Expired
•Get old value
•Entry processor

JSR-107 : Features
•Expiration
•Integration
•Created/Updated
•Removed/Expired
•Get old value
•Entry processor
•Annotations

JSR-107 : Features
•Expiration
•Integration
•Created/Updated
•Removed/Expired
•Get old value
•Entry processor
•Annotations
•MBeans, exposing
•Configuration
•Statistics

JSR-107 : Features
•Expiration
•Integration
•Created/Updated
•Removed/Expired
•Get old value
•Entry processor
•Annotations
•MBeans, exposing
•Configuration
•Statistics
•No capacity control !

@YourTwitterHandle@YourTwitterHandle@ljacomet#ehcachedvx
Demo

And my XML??

JSR-107 configuration
• Programmatic only
• Enter the world of provider extensions
• Ehcache 3 expects CacheManager URI to resolve to a
conﬁguration ﬁle
• From there, multiple scenarios are possible

public BusinessObject computeAndWin(String param1,
String param2) {
 
String key = createKey(param1, param2); 
BusinessObject cachedResult = cache.get(key); 
if (cachedResult == null) { 
cachedResult = loadAndCompute(param1, param2); 
cache.put(key, cachedResult); 
} 
return cachedResult; 
}
Cache Aside

Cache Aside : not so easy

• Requires synchronisation between
• cache
• and system of record

• cache
• JSR-107 has no locking option

• cache
• JSR-107 has no locking option
• Potential to be hairy code quite fast

Cache Aside: options

• Rely on the abstraction of your framework
• Spring Caching
• Hibernate
• …

• Rely on the abstraction of your framework
• Spring Caching
• Hibernate
• …
• Alternatives ?

public BusinessObject computeAndWin(String param1,
String param2) {
 
return cache.get(createKey(param1, param2)); 
}
Cache Through

Cache Through
Cache
Application
code
RDBMS

Cache Through

Cache Through
• get* => CacheLoader
• cache miss indicates no data available

Cache Through
• put* => CacheWriter
• Each use means writing to the system of record

Cache Through
• put* => CacheWriter
• Each use means writing to the system of record
• Constraints:APIs of CacheLoader / CacheWriter

Cache Through : JSR-107

• Peculiar behaviours

• putIfAbsent(K key,V value): boolean

• Ignores CacheLoader but not CacheWriter

• Ehcache 3 allows to tweak this behaviour through
conﬁguration

• Ehcache 3 allows to tweak this behaviour through
conﬁguration
• Behaviour consistent across atomic operations

Cache Through : Write Behind

• Asynchronous writes to the the system of record
• User thread no longer suffers from the latency of the write

• Asynchronous writes to the the system of record
• User thread no longer suffers from the latency of the write
• Introduces its own complexities
• Write queue: persistent or not?
• Mutation guarantee: once and only once? at least once?
• What about cache eviction?

Resilience strategy

Resilience strategy
• Proposition:
“A cache level error
should not be the cause for
a user level error “

Resilience strategy
• Proposition:
“A cache level error
should not be the cause for
a user level error “
P.S.This is ongoing development and subject to change

Ehcache 3

Ehcache 3
• Handle errors internally as much as possible

Ehcache 3
• In memory cache => No exceptions in the execution thread

Ehcache 3
• Default is to log errors, will be conﬁgurable / replaceable

Ehcache 3
• Advanced setups (Distributed + write behind for example)

Ehcache 3
• Force user to think about the use cases where the cache can
become inconsistent

Ehcache 3
• Force user to think about the use cases where the cache can
become inconsistent
• Will be required to conﬁgure a handler for these cases

Offheap solution

Offheap solution
• Solving Garbage Collector pauses on large heaps

Offheap solution
• Enables extreme cache scale up
• Multi TerraBytes … as long as your server allows it

Offheap solution
• Enables extreme cache scale up
• Multi TerraBytes … as long as your server allows it
• Production tested implementation
• 2.0.0 available on GitHub
• https://github.com/Terracotta-OSS/offheap-store

Clustering solution

Clustering solution
• Sharing of data across multiple JVMs

Clustering solution
• Terracotta 4.3.0 again has an OpenSource offering
• Integration with Ehcache 2.10.0

Clustering solution
• Terracotta 4.3.0 again has an OpenSource offering
• Integration with Ehcache 2.10.0
• Ehcache 3 will have clustering option in OpenSource
• Code starting to appear on GitHub

Questions ?

Caching reboot: javax.cache & Ehcache 3

More Related Content

What's hot

Similar to Caching reboot: javax.cache & Ehcache 3

Recently uploaded

Caching reboot: javax.cache & Ehcache 3