From a4d6f7d3672ccd16f9355a56463d0dc4261e9af0 Mon Sep 17 00:00:00 2001 From: Rob Austin Date: Tue, 23 Sep 2014 15:23:57 +0100 Subject: [PATCH 01/10] Update README.md --- README.md | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/README.md b/README.md index be5077093..d148d0aea 100644 --- a/README.md +++ b/README.md @@ -60,6 +60,15 @@ Most of the time. If you compare HashMap, ConcurrentHashMap and Chronicle Map, most of the maps in your system are likely to be HashMap. This is because HashMap is lightweight and synchronized HashMap works well for lightly contended use cases. By contention I mean, how many threads on average are trying to use a Map. One reason you can't have many contended resources, is that you only have so many CPUs and they can only be accessing so many resources at once (ideally no more than one or two per thread at a time) + +#### Known Issues + +Chronicle map lets you assign a map larger than your available memory, This will however impact performance as Chronicle Map will have to page the segments to and from disk as data is written or read from the map. This feature works brilliantly on Linux but unfortunately its not supported on Windows, if you use more memory than is physically available you will experience the following error : + + + + + #### When to use ConcurrentHashMap Some of the time. From c18d16f3009c03e357f64651d910f19240e86d54 Mon Sep 17 00:00:00 2001 From: Rob Austin Date: Tue, 23 Sep 2014 15:27:11 +0100 Subject: [PATCH 02/10] Update README.md --- README.md | 11 ++++++++--- 1 file changed, 8 insertions(+), 3 deletions(-) diff --git a/README.md b/README.md index d148d0aea..ea3a773db 100644 --- a/README.md +++ b/README.md @@ -65,9 +65,14 @@ If you compare HashMap, ConcurrentHashMap and Chronicle Map, most of the maps in Chronicle map lets you assign a map larger than your available memory, This will however impact performance as Chronicle Map will have to page the segments to and from disk as data is written or read from the map. This feature works brilliantly on Linux but unfortunately its not supported on Windows, if you use more memory than is physically available you will experience the following error : - - - +```java +Java frames: (J=compiled Java code, j=interpreted, Vv=VM code) +j sun.misc.Unsafe.compareAndSwapLong(Ljava/lang/Object;JJJ)Z+0 +j net.openhft.lang.io.NativeBytes.compareAndSwapLong(JJJ)Z+13 +j net.openhft.lang.io.AbstractBytes.tryLockNanos8a(JJ)Z+12 +j net.openhft.lang.io.AbstractBytes.tryLockNanosLong(JJ)Z+41 +j net.openhft.collections.AbstractVanillaSharedHashMap$Segment.lock()V+12 +``` #### When to use ConcurrentHashMap Some of the time. From f39eabd95b90dabca438e061d19d82b55fafcf8e Mon Sep 17 00:00:00 2001 From: Rob Austin Date: Tue, 23 Sep 2014 15:28:07 +0100 Subject: [PATCH 03/10] Update README.md --- README.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index ea3a773db..3f87ffa53 100644 --- a/README.md +++ b/README.md @@ -36,7 +36,8 @@ Click here to get the [Latest Version Number](http://search.maven.org/#search%7C * [Identifier for Replication](https://github.com/OpenHFT/Chronicle-Map#identifier-for-replication) * [Bootstrapping](https://github.com/OpenHFT/Chronicle-Map#bootstrapping) * [Clustering](https://github.com/OpenHFT/Chronicle-Map#cluster) - +* [Known Issues](https://github.com/OpenHFT/Chronicle-Map#known-issues) +* #### Examples * [Replicating data between process on different servers](https://github.com/OpenHFT/Chronicle-Map#example--replicating-data-between-process-on-different-servers) From 511f195a9f5de14ecb53c6f61a6d96768b8b9d3f Mon Sep 17 00:00:00 2001 From: Rob Austin Date: Tue, 23 Sep 2014 15:30:44 +0100 Subject: [PATCH 04/10] Update README.md --- README.md | 26 +++++++++++++------------- 1 file changed, 13 insertions(+), 13 deletions(-) diff --git a/README.md b/README.md index 3f87ffa53..3799bd832 100644 --- a/README.md +++ b/README.md @@ -57,23 +57,10 @@ A Chronicle Map implements the java.util.concurrent.ConcurrentMap, so on the fac ## When to use #### When to use HashMap -Most of the time. - If you compare HashMap, ConcurrentHashMap and Chronicle Map, most of the maps in your system are likely to be HashMap. This is because HashMap is lightweight and synchronized HashMap works well for lightly contended use cases. By contention I mean, how many threads on average are trying to use a Map. One reason you can't have many contended resources, is that you only have so many CPUs and they can only be accessing so many resources at once (ideally no more than one or two per thread at a time) -#### Known Issues -Chronicle map lets you assign a map larger than your available memory, This will however impact performance as Chronicle Map will have to page the segments to and from disk as data is written or read from the map. This feature works brilliantly on Linux but unfortunately its not supported on Windows, if you use more memory than is physically available you will experience the following error : - -```java -Java frames: (J=compiled Java code, j=interpreted, Vv=VM code) -j sun.misc.Unsafe.compareAndSwapLong(Ljava/lang/Object;JJJ)Z+0 -j net.openhft.lang.io.NativeBytes.compareAndSwapLong(JJJ)Z+13 -j net.openhft.lang.io.AbstractBytes.tryLockNanos8a(JJ)Z+12 -j net.openhft.lang.io.AbstractBytes.tryLockNanosLong(JJ)Z+41 -j net.openhft.collections.AbstractVanillaSharedHashMap$Segment.lock()V+12 -``` #### When to use ConcurrentHashMap Some of the time. @@ -449,6 +436,19 @@ cluster = clusterReplicatorBuilder.create(); map = clusterReplicatorBuilder.create((short) 1, builder); ``` +#### Known Issues + +Chronicle map lets you assign a map larger than your available memory, This will however impact performance as Chronicle Map will have to page the segments to and from disk as data is written or read from the map. This feature works brilliantly on Linux but unfortunately its not supported on Windows, if you use more memory than is physically available you will experience the following error : + +```java +Java frames: (J=compiled Java code, j=interpreted, Vv=VM code) +j sun.misc.Unsafe.compareAndSwapLong(Ljava/lang/Object;JJJ)Z+0 +j net.openhft.lang.io.NativeBytes.compareAndSwapLong(JJJ)Z+13 +j net.openhft.lang.io.AbstractBytes.tryLockNanos8a(JJ)Z+12 +j net.openhft.lang.io.AbstractBytes.tryLockNanosLong(JJ)Z+41 +j net.openhft.collections.AbstractVanillaSharedHashMap$Segment.lock()V+12 +``` + # Example : Replicating data between process on different servers Lets assume that we had two server, lets call them server1 and server2, if we wished to share a map between them, this is how we could set it up From 7946ade83e3a03ea92d5b2152723e2706a0608c2 Mon Sep 17 00:00:00 2001 From: Rob Austin Date: Tue, 23 Sep 2014 18:03:53 +0100 Subject: [PATCH 05/10] Update README.md --- README.md | 1 + 1 file changed, 1 insertion(+) diff --git a/README.md b/README.md index 3799bd832..12492daf1 100644 --- a/README.md +++ b/README.md @@ -37,6 +37,7 @@ Click here to get the [Latest Version Number](http://search.maven.org/#search%7C * [Bootstrapping](https://github.com/OpenHFT/Chronicle-Map#bootstrapping) * [Clustering](https://github.com/OpenHFT/Chronicle-Map#cluster) * [Known Issues](https://github.com/OpenHFT/Chronicle-Map#known-issues) +* [Development Tasks - JIRA] (https://higherfrequencytrading.atlassian.net/browse/HCOLL) * #### Examples From bce6296868cf655af986484ff9ed4e14122a4ba8 Mon Sep 17 00:00:00 2001 From: Rob Austin Date: Tue, 23 Sep 2014 18:06:43 +0100 Subject: [PATCH 06/10] Update README.md --- README.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index 12492daf1..96b77e9c4 100644 --- a/README.md +++ b/README.md @@ -38,7 +38,8 @@ Click here to get the [Latest Version Number](http://search.maven.org/#search%7C * [Clustering](https://github.com/OpenHFT/Chronicle-Map#cluster) * [Known Issues](https://github.com/OpenHFT/Chronicle-Map#known-issues) * [Development Tasks - JIRA] (https://higherfrequencytrading.atlassian.net/browse/HCOLL) -* + + #### Examples * [Replicating data between process on different servers](https://github.com/OpenHFT/Chronicle-Map#example--replicating-data-between-process-on-different-servers) From abc56b855a1923f595c92d0f0c3564ad507513a8 Mon Sep 17 00:00:00 2001 From: Rob Austin Date: Tue, 23 Sep 2014 19:17:04 +0100 Subject: [PATCH 07/10] HCOLL-133 CloseablesManager.checkState is throwing java.lang.IllegalStateException --- .../map/AbstractChannelReplicator.java | 31 +++++++++++++++++-- .../chronicle/map/CloseablesManager.java | 6 +++- .../net/openhft/chronicle/map/Issue42.java | 12 +++++++ 3 files changed, 45 insertions(+), 4 deletions(-) diff --git a/src/main/java/net/openhft/chronicle/map/AbstractChannelReplicator.java b/src/main/java/net/openhft/chronicle/map/AbstractChannelReplicator.java index deade5e8d..dd7b30e2b 100644 --- a/src/main/java/net/openhft/chronicle/map/AbstractChannelReplicator.java +++ b/src/main/java/net/openhft/chronicle/map/AbstractChannelReplicator.java @@ -329,16 +329,39 @@ abstract class AbstractChannelReplicator implements Closeable { final Thread thread = new Thread(new Runnable() { public void run() { + SelectableChannel socketChannel = null; try { if (reconnectionInterval > 0) Thread.sleep(reconnectionInterval); - SelectableChannel socketChannel = doConnect(); - closeables.add(socketChannel); + socketChannel = doConnect(); + + try { + closeables.add(socketChannel); + } catch (IllegalStateException e) { + + // close could have be called from another thread, while we were in Thread.sleep() + // which would cause a IllegalStateException + closeQuietly(socketChannel); + return; + } + AbstractConnector.this.socketChannel = socketChannel; + } catch (Exception e) { - LOG.error("", e); + closeQuietly(socketChannel); + LOG.debug("", e); + } + } + + private void closeQuietly(SelectableChannel socketChannel) { + if (socketChannel == null) + return; + try { + socketChannel.close(); + } catch (Exception e1) { + // do nothing } } }); @@ -346,6 +369,8 @@ abstract class AbstractChannelReplicator implements Closeable { thread.setName(name); thread.setDaemon(true); thread.start(); + + } public void setSuccessfullyConnected() { diff --git a/src/main/java/net/openhft/chronicle/map/CloseablesManager.java b/src/main/java/net/openhft/chronicle/map/CloseablesManager.java index e4a655530..16eefb0c3 100644 --- a/src/main/java/net/openhft/chronicle/map/CloseablesManager.java +++ b/src/main/java/net/openhft/chronicle/map/CloseablesManager.java @@ -29,7 +29,7 @@ final class CloseablesManager implements Closeable { private static final Logger LOG = LoggerFactory.getLogger(CloseablesManager.class.getName()); private boolean isClosed = false; - private List closeables = new ArrayList(); + private final List closeables = new ArrayList<>(); private void checkState() { if (isClosed) @@ -84,4 +84,8 @@ final class CloseablesManager implements Closeable { // do nothing } } + + boolean isClosed() { + return isClosed; + } } diff --git a/src/test/java/net/openhft/chronicle/map/Issue42.java b/src/test/java/net/openhft/chronicle/map/Issue42.java index e9d03840a..f0f6ea530 100644 --- a/src/test/java/net/openhft/chronicle/map/Issue42.java +++ b/src/test/java/net/openhft/chronicle/map/Issue42.java @@ -9,8 +9,20 @@ import static net.openhft.chronicle.map.Alignment.NO_ALIGNMENT; import static net.openhft.chronicle.map.Builder.getPersistenceFile; public class Issue42 { + + private static String OS = System.getProperty("os.name").toLowerCase(); + + public static boolean isWindows() { + return (OS.indexOf("win") >= 0); + } + + @Test public void crashJVMWindowsTest() throws IOException { + + if (!isWindows()) + return; + final ChronicleMap map = ChronicleMapBuilder .of(CharSequence.class, CharSequence.class) .entryAndValueAlignment(NO_ALIGNMENT) From 10d8ad548176bbcca76ec0b1c4055bc7117d6219 Mon Sep 17 00:00:00 2001 From: Rob Austin Date: Tue, 23 Sep 2014 19:30:40 +0100 Subject: [PATCH 08/10] HCOLL-133 CloseablesManager.checkState is throwing java.lang.IllegalStateException --- .../java/net/openhft/chronicle/map/CloseablesManager.java | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/src/main/java/net/openhft/chronicle/map/CloseablesManager.java b/src/main/java/net/openhft/chronicle/map/CloseablesManager.java index 16eefb0c3..7416fe9a2 100644 --- a/src/main/java/net/openhft/chronicle/map/CloseablesManager.java +++ b/src/main/java/net/openhft/chronicle/map/CloseablesManager.java @@ -55,6 +55,8 @@ final class CloseablesManager implements Closeable { public synchronized void closeQuietly(Closeable closeable) { try { close(closeable); + } catch (IllegalStateException e) { + // this can occur if already closed ( for example closed called from another thread ) } catch (IOException e) { LOG.error("", e); } @@ -85,7 +87,7 @@ final class CloseablesManager implements Closeable { } } - boolean isClosed() { + boolean isClosed() { return isClosed; } } From 1ca7b5f1ea2d530d7f5a8cc8e7a3b4222d21e85b Mon Sep 17 00:00:00 2001 From: Rob Austin Date: Tue, 23 Sep 2014 19:31:11 +0100 Subject: [PATCH 09/10] fixed typo --- src/main/java/net/openhft/chronicle/map/CloseablesManager.java | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/main/java/net/openhft/chronicle/map/CloseablesManager.java b/src/main/java/net/openhft/chronicle/map/CloseablesManager.java index 7416fe9a2..b520e7344 100644 --- a/src/main/java/net/openhft/chronicle/map/CloseablesManager.java +++ b/src/main/java/net/openhft/chronicle/map/CloseablesManager.java @@ -56,7 +56,7 @@ final class CloseablesManager implements Closeable { try { close(closeable); } catch (IllegalStateException e) { - // this can occur if already closed ( for example closed called from another thread ) + // this can occur if already closed ( for example closed is called from another thread ) } catch (IOException e) { LOG.error("", e); } From c2027c3e3df1e4cb11ff2523c32ac527e32e57ba Mon Sep 17 00:00:00 2001 From: Rob Austin Date: Wed, 24 Sep 2014 09:19:54 +0100 Subject: [PATCH 10/10] fixed 1.6 compatibility issue --- src/main/java/net/openhft/chronicle/map/CloseablesManager.java | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/main/java/net/openhft/chronicle/map/CloseablesManager.java b/src/main/java/net/openhft/chronicle/map/CloseablesManager.java index b520e7344..a115c234e 100644 --- a/src/main/java/net/openhft/chronicle/map/CloseablesManager.java +++ b/src/main/java/net/openhft/chronicle/map/CloseablesManager.java @@ -29,7 +29,7 @@ final class CloseablesManager implements Closeable { private static final Logger LOG = LoggerFactory.getLogger(CloseablesManager.class.getName()); private boolean isClosed = false; - private final List closeables = new ArrayList<>(); + private final List closeables = new ArrayList(); private void checkState() { if (isClosed)