Skip to content

Commit

Permalink
Expire schemas from Avro cache based on age instead of recency
Browse files Browse the repository at this point in the history
Guava's cache performs poorly under high loaded when tracking recency. The
details are in google/guava#2408. The short version is that each access is
tracked and aggregated. If there's constant read access by more threads
than there are cores (as happens for the AvroSerde in a Presto worker),
the whole thing gets backed up and ultimately leads to a lengthy GC pause
and query timeouts.

Expiring based on age instead of recency avoids the issue.
  • Loading branch information
wagnermarkd authored and electrum committed May 12, 2017
1 parent d27ca1e commit 1d2f31e
Showing 1 changed file with 3 additions and 2 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -17,15 +17,16 @@
import com.google.common.cache.CacheBuilder;

import java.util.Set;
import java.util.concurrent.TimeUnit;

/**
* This is a thread-safe, size-bounded fork of the Hive version.
* This is a thread-safe, time-bounded fork of the Hive version.
* It also includes the correctness fix from HIVE-11288.
*/
public abstract class InstanceCache<K, V>
{
private final Cache<K, V> cache = CacheBuilder.newBuilder()
.maximumSize(100_000)
.expireAfterWrite(1, TimeUnit.MINUTES)
.build();

protected InstanceCache() {}
Expand Down

0 comments on commit 1d2f31e

Please sign in to comment.