mirror of
https://github.com/OpenHFT/Chronicle-Map.git
synced 2021-09-13 01:26:12 +00:00
Full proof read and edit
This commit is contained in:
+119
-117
@@ -74,15 +74,15 @@ this JVM process, rather than creating a new Chronicle Map data store. That mean
|
||||
|
||||
In this tutorial, the term `ChronicleMap` instance (or simply `ChronicleMap`) is used to refer to an on-heap object providing access to a Chronicle Map data store (or Chronicle Map key-value store, or Chronicle Map store, or simply Chronicle Map, with space between two words in contrast to `ChronicleMap`), which could be purely in-memory, or persisted to disk.
|
||||
|
||||
Currently the Java implementation doesn't allow creation of multiple accessor ChronicleMap` objects for a single *in-memory Chronicle Map store*; there is always a one-to-one correspondence.
|
||||
Currently the Java implementation doesn't allow creation of multiple accessor ChronicleMap` objects for a single *in-memory Chronicle Map store*; there is always a one-to-one relationship.
|
||||
|
||||
*Persisted Chronicle Map store*, however, does allow creation of multiple accessor `ChronicleMap` instances; either within a single JVM process (although it is not recommended), or from concurrent JVM processes.
|
||||
A *persisted Chronicle Map store*, however, does allow creation of multiple accessor `ChronicleMap` instances; either within a single JVM process (although it is not recommended), or from concurrent JVM processes.
|
||||
|
||||
When no processes access the file, it could be freely moved to another location in the system, or
|
||||
When no processes access the file, it could be moved to another location in the system, or
|
||||
even to another server. It could even run on different operating systems. When opened from another location you will observe the same data.
|
||||
|
||||
If you don't need the Chronicle Map instance to survive the server restart, i. e. you don't need
|
||||
persistence to disk, only multi-process access, choose the file to be mounted on http://en.wikipedia.org/wiki/Tmpfs[tmpfs]. For example, on `Linux` it is as easy as placing you file in `/dev/shm` directory.
|
||||
If you don't need the Chronicle Map instance to survive the server restart (that is, you don't need
|
||||
persistence to disk; only multi-process access), mount the file on http://en.wikipedia.org/wiki/Tmpfs[tmpfs]. For example, on `Linux` it is as easy as placing you file in `/dev/shm` directory.
|
||||
|
||||
|
||||
== Configure entries
|
||||
@@ -94,28 +94,34 @@ You should not put additional margin over the actual target number of entries. T
|
||||
See `ChronicleMapBuilder#entries()` in http://www.javadoc.io/doc/net.openhft/chronicle-map/[Javadocs] for more information.
|
||||
|
||||
|
||||
Once `ChronicleMap` instance is created, its configurations are sealed and cannot be changed
|
||||
though the `ChronicleMapBuilder` instance.
|
||||
Once a `ChronicleMap` instance is created, its configurations are sealed and cannot be changed
|
||||
using the `ChronicleMapBuilder` instance.
|
||||
|
||||
|
||||
== Single `ChronicleMap` instance per JVM
|
||||
If you want to access a Chronicle Map data store
|
||||
concurrently within a Java process, you should not create a separate `ChronicleMap` instance per thread. Within the JVM environment, `ChronicleMap` instance is a `ConcurrentMap`, and could be accessed concurrently the same way as, for example, `ConcurrentHashMap`.
|
||||
concurrently within a Java process, you should not create a separate `ChronicleMap` instance for each thread. Within the JVM environment, a `ChronicleMap` instance is a `ConcurrentMap`, and could be accessed concurrently the same way as, for example, a `ConcurrentHashMap`.
|
||||
|
||||
== Recovery
|
||||
|
||||
If a process, accessing a persisted Chronicle Map, terminated abnormally: crashed, `SIGKILL`ed, or
|
||||
terminated because the host operating system crashed, or the machine lost power, the Chronicle Map
|
||||
might remain in an inaccessible or corrupted state. When the Chronicle Map is opened next time from
|
||||
another process, it should be done via `.recoverPersistedTo()` method in `ChronicleMapBuilder`.
|
||||
Unlike `createPersistedTo()`, this method scans all memory of Chronicle Map store for
|
||||
inconsistencies, if some found, it cleans them up.
|
||||
If a process, accessing a persisted Chronicle Map, terminated abnormally, for example:
|
||||
|
||||
- crashed
|
||||
- `SIGKILL`ed
|
||||
- terminated because the host operating system crashed
|
||||
- terminated because the host machine lost power
|
||||
|
||||
then the Chronicle Map may remain in an inaccessible or corrupted state.
|
||||
|
||||
When the Chronicle Map is opened next time from another process, it should be done using `.recoverPersistedTo()` method in `ChronicleMapBuilder`.
|
||||
|
||||
Unlike `createPersistedTo()`, this method scans all the memory of the Chronicle Map store for
|
||||
inconsistencies and, if any are found, it cleans them up.
|
||||
|
||||
`.recoverPersistedTo()` needs to access the Chronicle Map exclusively. If a concurrent process is
|
||||
accessing the Chronicle Map while another process is attempting to perform recovery, result of
|
||||
operations on the accessing process side, and results of recovery are unspecified. The data could be
|
||||
corrupted further. You must ensure no other process is accessing the Chronicle Map store when
|
||||
calling for `.recoverPersistedTo()` on this store.
|
||||
accessing the Chronicle Map while another process is attempting to perform recovery, the results of
|
||||
operations on the accessing process side, and results of recovery are unspecified; the data could be corrupted further. You must ensure thst no other process is accessing the Chronicle Map store when
|
||||
calling `.recoverPersistedTo()`.
|
||||
|
||||
Example:
|
||||
|
||||
@@ -129,14 +135,17 @@ ChronicleMap<Integer, PostalCodeRange> cityPostalCodes = ChronicleMap
|
||||
```
|
||||
|
||||
The second parameter in `recoverPersistedTo()` method is called
|
||||
`sameBuilderConfigAndLibraryVersion`, it could be `true` only if `ChronicleMapBuilder` is configured
|
||||
`sameBuilderConfigAndLibraryVersion`. It has two possible values:
|
||||
|
||||
- `true` - if `ChronicleMapBuilder` is configured
|
||||
in exactly the same way, as when the Chronicle Map (persisted to the given file) was created, and
|
||||
using the same version of the Chronicle Map library, or `false`, if initial configurations are not
|
||||
known, or current version of Chronicle Map library could be different from the version, used to
|
||||
create this Chronicle Map initially.
|
||||
using the same version of the Chronicle Map library
|
||||
- `false` - if the initial configuration is not
|
||||
known, or the current version of Chronicle Map library could be different from the version originally used to
|
||||
create this Chronicle Map.
|
||||
|
||||
If `sameBuilderConfigAndLibraryVersion` is `true`, `recoverPersistedTo()` "knows" all the right
|
||||
configurations and what should be written to the header. It checks if the recovered Chronicle Map's
|
||||
configurations, and what should be written to the header. It checks if the recovered Chronicle Map's
|
||||
header memory (containing serialized configurations) is corrupted or not. If the header is
|
||||
corrupted, it is overridden, and the recovery process continues.
|
||||
|
||||
@@ -146,16 +155,14 @@ corrupted, `ChronicleHashRecoveryFailedException` is thrown.
|
||||
|
||||
However, the subject header memory is never updated on ordinary operations with Chronicle Map, so it
|
||||
couldn't be corrupted if an accessing process crashed, or the operating system crashed, or even the
|
||||
machine lost power. Only hardware memory or disk corruption or a bug in the file system could lead
|
||||
machine lost power. Only hardware, memory, or disk corruption, or a bug in the file system, could lead
|
||||
to Chronicle Map header memory corruption.
|
||||
|
||||
`.recoverPersistedTo()` is harmless if the previous process accessing the Chronicle Map terminated
|
||||
normally, however this is a computationally expensive procedure that should generally be avoided.
|
||||
normally; however this is a computationally expensive procedure that should generally be avoided.
|
||||
|
||||
Chronicle Map creation and recovery could be conveniently merged using a single call:
|
||||
`.createOrRecoverPersistedTo(persistenceFile, sameLibraryVersion)` in `ChronicleMapBuilder`, which
|
||||
acts like `createPersistedTo(persistenceFile)`, if the persistence file doesn't yet exist, and like
|
||||
`recoverPersistedTo(persistenceFile, sameLibraryVersion)`, if the file already exists, e. g.:
|
||||
Chronicle Map creation and recovery could be conveniently merged using a single call, `.createOrRecoverPersistedTo(persistenceFile, sameLibraryVersion)` in `ChronicleMapBuilder`. This acts like `createPersistedTo(persistenceFile)` if the persistence file doesn't yet exist, and like
|
||||
`recoverPersistedTo(persistenceFile, sameLibraryVersion)`, if the file already exists. For example:
|
||||
|
||||
```java
|
||||
ChronicleMap<Integer, PostalCodeRange> cityPostalCodes = ChronicleMap
|
||||
@@ -165,46 +172,42 @@ ChronicleMap<Integer, PostalCodeRange> cityPostalCodes = ChronicleMap
|
||||
.createOrRecoverPersistedTo(cityPostalCodesFile, false);
|
||||
```
|
||||
|
||||
If the Chronicle Map is configured to store entry checksums along with entries, recovery procedure
|
||||
checks for each entry that the checksums is correct, otherwise it assumes the entry is corrupted and
|
||||
deletes it from the Chronicle Map. If checksums are to stored, recovery procedure cannot guarantee
|
||||
correctness of entry data. See [Entry checksums](#entry-checksums) section for more information.
|
||||
If the Chronicle Map is configured to store entry checksums along with entries, then the recovery procedure checks that the checksum is correct for each entry.
|
||||
|
||||
Otherwise, it assumes the entry is corrupted and deletes it from the Chronicle Map. If checksums are to be stored, the recovery procedure cannot guarantee
|
||||
correctness of the entry data. See [Entry checksums](#entry-checksums) section for more information.
|
||||
|
||||
== Key and Value Types
|
||||
|
||||
Either key or value type of `ChronicleMap<K, V>` could be:
|
||||
The key, or value type, of `ChronicleMap<K, V>` could be:
|
||||
|
||||
- Types with best possible out of the box support:
|
||||
- Any [value interface](https://github.com/OpenHFT/Chronicle-Values)
|
||||
- Any class implementing [`Byteable`](
|
||||
http://openhft.github.io/Chronicle-Bytes/apidocs/net/openhft/chronicle/bytes/Byteable.html)
|
||||
interface from [Chronicle Bytes](https://github.com/OpenHFT/Chronicle-Bytes)
|
||||
- Any class implementing [`BytesMarshallable`](
|
||||
http://openhft.github.io/Chronicle-Bytes/apidocs/net/openhft/chronicle/bytes/BytesMarshallable.html)
|
||||
- Types with best possible out-of-the-box support:
|
||||
** Any https://github.com/OpenHFT/Chronicle-Values[value interface]
|
||||
** Any class implementing http://openhft.github.io/Chronicle-Bytes/apidocs/net/openhft/chronicle/bytes/Byteable.html[`Byteable`] interface from https://github.com/OpenHFT/Chronicle-Bytes[Chronicle Bytes].
|
||||
** Any class implementing http://openhft.github.io/Chronicle-Bytes/apidocs/net/openhft/chronicle/bytes/BytesMarshallable.html[`BytesMarshallable`].
|
||||
interface from Chronicle Bytes. The implementation class should have a public no-arg constructor.
|
||||
- `byte[]` and `ByteBuffer`
|
||||
- `CharSequence`, `String` and `StringBuilder`. Note that these char sequence types are
|
||||
** `byte[]` and `ByteBuffer`
|
||||
** `CharSequence`, `String` and `StringBuilder`. Note that these char sequence types are
|
||||
serialized using UTF-8 encoding by default. If you need a different encoding, refer to the
|
||||
example in the [custom `CharSequence` encoding](#custom-charsequence-encoding) section.
|
||||
- `Integer`, `Long` and `Double`
|
||||
example in the <<CM_Tutorial.adoc#custom-charsequence-encoding,custom `CharSequence` encoding>> section.
|
||||
** `Integer`, `Long` and `Double`
|
||||
|
||||
- Types supported out of the box, but not particularly efficiently. You might want to implement
|
||||
more efficient [custom serializers](#custom-serializers) for them:
|
||||
- Any class implementing `java.io.Externalizable`. The implementation class should have a public
|
||||
no-arg constructor.
|
||||
- Any type implementing `java.io.Serializable`, including boxed primitive types (except listed
|
||||
above) and array types
|
||||
- Types supported out-of-the-box, but that are not particularly efficiently. You may want to implement
|
||||
more efficient <<CM_Tutorial.adoc#custom-serializers,custom serializers>> for them:
|
||||
** Any class implementing `java.io.Externalizable`. The implementation class should have a public `no-arg` constructor.
|
||||
** Any type implementing `java.io.Serializable`, including boxed primitive types (except those listed
|
||||
above) and array types.
|
||||
|
||||
- Any other type, if [custom serializers](#custom-serializers) are provided.
|
||||
- Any other type, if <<CM_Tutorial.adoc#custom-serializers,custom serializers>> are provided.
|
||||
|
||||
https://github.com/OpenHFT/Chronicle-Values)[value interfaces] are preferred as they don't generate garbage, and have close to zero serialization/deserialization costs. They are preferable even to boxed primitives. For example, try to use `net.openhft.chronicle.core.values.IntValue` instead of `Integer`.
|
||||
https://github.com/OpenHFT/Chronicle-Values)[Value interfaces] are preferred as they do not generate garbage, and have close to zero serialization/deserialization costs. They are preferable even to boxed primitives. For example, try to use `net.openhft.chronicle.core.values.IntValue` instead of `Integer`.
|
||||
|
||||
Generally, you must hint the `ChronicleMapBuilder` with the average sizes of the keys and values, which are going to be inserted into the `ChronicleMap`. This is needed to allocate the proper volume of the shared memory. Do this using `averageKey()` (preferred) or `averageKeySize()`, and
|
||||
Generally, you must provide hints for the `ChronicleMapBuilder` with the average sizes of the keys and the values, which are going to be inserted into the `ChronicleMap`. This is required in order to allocate the proper amount of shared memory. Do this using `averageKey()` (preferred) or `averageKeySize()`, and
|
||||
`averageValue()` or `averageValueSize()` respectively.
|
||||
|
||||
In the example above, `averageKey("Amsterdam")` is called, because it is assumed that "Amsterdam" (9 bytes in UTF-8 encoding) is the average length for city names; some names are shorter (Tokyo, 5 bytes), some names are longer (San Francisco, 13 bytes).
|
||||
In the example above, `averageKey("Amsterdam")` is called, because it is assumed that "Amsterdam" (9 bytes in UTF-8 encoding) is the average length for city names. Some names are shorter (Tokyo, 5 bytes), some names are longer (San Francisco, 13 bytes).
|
||||
|
||||
Another example could be if values in your `ChronicleMap` are adjacency lists of some social graph, where nodes are represented as `long` ids, and adjacency lists are `long[]` arrays. The average number of friends is 150. You could configure the `ChronicleMap` as follows:
|
||||
Another example could be if values in your `ChronicleMap` are adjacency lists of some social graph, where nodes are represented as `long` identifiers, and adjacency lists are `long[]` arrays. For example, if the average number of friends is 150, you could configure the `ChronicleMap` as follows:
|
||||
|
||||
```java
|
||||
Map<Long, long[]> socialGraph = ChronicleMap
|
||||
@@ -215,9 +218,9 @@ Map<Long, long[]> socialGraph = ChronicleMap
|
||||
.create();
|
||||
```
|
||||
|
||||
You could omit specifying key or value average sizes, if their types are boxed Java primitives or value interfaces. They are constantly-sized and Chronicle Map knows about that.
|
||||
You could omit specifying key, or value, average sizes, if their types are boxed Java primitives or value interfaces. They are constantly-sized and Chronicle Map knows about that.
|
||||
|
||||
If the key or value type is constantly sized, or keys or values only of a certain size appear in your Chronicle Map domain, you should prefer to configure `constantKeySizeBySample()` or
|
||||
If the key or value type is constantly sized, or keys or values only of a certain size appear in your Chronicle Map domain, then preferably you should configure `constantKeySizeBySample()` or
|
||||
`constantValueSizeBySample()`, instead of `averageKey()` or `averageValue()`. For example:
|
||||
|
||||
```java
|
||||
@@ -232,9 +235,9 @@ ChronicleSet<UUID> uuids =
|
||||
|
||||
== Custom serializers
|
||||
|
||||
Chronicle Map allows you to configure custom marshallers for key or value types which are not supported out of the box. You can also serialize supported types like `String` in some custom way (encoded other than UTF-8), or serialize supported types more efficiently than by default.
|
||||
Chronicle Map allows you to configure custom marshallers for key or value types which are not supported out-of-the-box. You can also serialize supported types like `String` in some custom way (encoded other than UTF-8), or serialize supported types more efficiently than by default.
|
||||
|
||||
There are three pairs of serialization interfaces, only one of them should be chosen in a single implementation, and supplied to the `ChronicleMapBuilder` for the key or value type. These are:
|
||||
There are three pairs of serialization interfaces. Only one of them should be chosen in a single implementation, and supplied to the `ChronicleMapBuilder` for the key or value type. These are:
|
||||
|
||||
- link:CM_Tutorial_Bytes.adoc[BytesWriter and BytesReader]
|
||||
- link:CM_Tutorial_Sized.adoc[SizedWriter and SizedReader]
|
||||
@@ -292,14 +295,14 @@ There are three pairs of serialization interfaces, only one of them should be ch
|
||||
|
||||
=== Single-key queries
|
||||
|
||||
First of all, `ChronicleMap` supports all operations from [`Map`](
|
||||
https://docs.oracle.com/javase/8/docs/api/java/util/Map.html): `get()`, `put()`, etc, including
|
||||
methods added in Java 8, like `compute()` and `merge()`, and [`ConcurrentMap`](
|
||||
https://docs.oracle.com/javase/8/docs/api/java/util/concurrent/ConcurrentMap.html) interfaces:
|
||||
`putIfAbsent()`, `replace()`. All operations, including those which include "two steps", e. g.
|
||||
`compute()`, are correctly synchronized in terms of `ConcurrentMap` interface.
|
||||
`ChronicleMap` supports all operations from:
|
||||
|
||||
This means, you could use `ChronicleMap` instance just like a `HashMap` or `ConcurrentHashMap`:
|
||||
- https://docs.oracle.com/javase/8/docs/api/java/util/Map.html[`Map`] interfaces; `get()`, `put()`, etc, including
|
||||
methods added in Java 8, like `compute()` and `merge()`, and
|
||||
- https://docs.oracle.com/javase/8/docs/api/java/util/concurrent/ConcurrentMap.html[`ConcurrentMap`] interfaces;
|
||||
`putIfAbsent()`, `replace()`.
|
||||
|
||||
All operations, including those which include "two-steps", for example, `compute()`, are correctly synchronized in terms of the `ConcurrentMap` interface. This means that you could use a `ChronicleMap` instance just like a `HashMap` or `ConcurrentHashMap`.
|
||||
|
||||
```java
|
||||
PostalCodeRange amsterdamCodes = Values.newHeapInstance(PostalCodeRange.class);
|
||||
@@ -313,7 +316,7 @@ PostalCodeRange amsterdamCodes = cityPostalCodes.get("Amsterdam");
|
||||
```
|
||||
|
||||
However, this approach often generates garbage, because the values should be deserialized from
|
||||
off-heap memory to on-heap, the new value object are allocated. There are several possibilities to
|
||||
off-heap memory to on-heap memory when the new value objects are allocated. There are several possibilities to
|
||||
reuse objects efficiently:
|
||||
|
||||
==== Value interfaces instead of boxed primitives
|
||||
@@ -349,10 +352,8 @@ for (long id : orderIds) {
|
||||
|
||||
==== `chronicleMap.getUsing()`
|
||||
|
||||
Use `ChronicleMap#getUsing(K key, V using)` to reuse the value object. It works if:
|
||||
Use `ChronicleMap#getUsing(K key, V using)` to reuse the value object. It works if the value type is `CharSequence`. Pass `StringBuilder` as the `using` argument. For example:
|
||||
|
||||
- The value type is `CharSequence`, pass `StringBuilder` as the `using` argument.
|
||||
For example:
|
||||
```java
|
||||
ChronicleMap<LongValue, CharSequence> names = ...
|
||||
StringBuilder name = new StringBuilder();
|
||||
@@ -363,15 +364,15 @@ Use `ChronicleMap#getUsing(K key, V using)` to reuse the value object. It works
|
||||
}
|
||||
```
|
||||
|
||||
In this case, calling `names.getUsing(key, name)` is equivalent to
|
||||
In this case, calling `names.getUsing(key, name)` is equivalent to:
|
||||
|
||||
```java
|
||||
name.setLength(0);
|
||||
name.append(names.get(key));
|
||||
```
|
||||
|
||||
with the difference that it doesn't generate garbage.
|
||||
- The value type is value interface, pass heap instance to read the data into it without new object
|
||||
allocation:
|
||||
The difference is that it doesn't generate garbage. The value type is the value interface. Pass the heap instance to read the data into it without new object allocation:
|
||||
|
||||
```java
|
||||
ThreadLocal<PostalCodeRange> cachedPostalCodeRange =
|
||||
ThreadLocal.withInitial(() -> Values.newHeapInstance(PostalCodeRange.class));
|
||||
@@ -382,15 +383,18 @@ Use `ChronicleMap#getUsing(K key, V using)` to reuse the value object. It works
|
||||
cityPostalCodes.getUsing(city, range);
|
||||
// process the range...
|
||||
```
|
||||
- If the value type implements `BytesMarshallable`, or `Externalizable`, `ChronicleMap` attempts to
|
||||
|
||||
- If the value type implements `BytesMarshallable`, or `Externalizable`, then `ChronicleMap` attempts to
|
||||
reuse the given `using` object by deserializing the value into the given object.
|
||||
|
||||
- If custom marshaller is configured in the `ChronicleMapBuilder` via `.valueMarshaller()`,
|
||||
`ChronicleMap` attempts to reuse the given object by calling `readUsing()` method from the
|
||||
then `ChronicleMap` attempts to reuse the given object by calling the `readUsing()` method from the
|
||||
marshaller interface.
|
||||
|
||||
If `ChronicleMap` fails to reuse the object in `getUsing()`, it makes no harm, it falls back to
|
||||
object creation, like in `get()` method. In particular, even `null` is allowed to be passed as
|
||||
`using` object. It allows "lazy" using object initialization pattern:
|
||||
If `ChronicleMap` fails to reuse the object in `getUsing()`, it does no harm. It falls back to
|
||||
object creation, as in the `get()` method. In particular, even `null` is allowed to be passed as
|
||||
`using` object. It allows a "lazy" using object initialization pattern:
|
||||
|
||||
```java
|
||||
// a field
|
||||
PostalCodeRange cachedRange = null;
|
||||
@@ -401,15 +405,14 @@ PostalCodeRange cachedRange = null;
|
||||
cachedRange = cityPostalCodes.getUsing(city, cachedRange);
|
||||
// process the range...
|
||||
```
|
||||
In this example, `cachedRange` is `null` initially, on the first `getUsing()` call the heap value
|
||||
is allocated, and saved in a `cachedRange` field for later reuse.
|
||||
In this example, `cachedRange` is `null` initially. On the first `getUsing()` call, the heap value
|
||||
is allocated and saved in a `cachedRange` field for later reuse.
|
||||
|
||||
<i>If the value type is a value interface, <b>don't</b> use flyweight implementation as `getUsing()`
|
||||
argument.</i> This is dangerous, because on reusing flyweight points to the `ChronicleMap` memory
|
||||
directly, but the access is not synchronized. At least you could read inconsistent value state,
|
||||
at most - corrupt the `ChronicleMap` memory.
|
||||
NOTE: If the value type is a value interface, **do not** use flyweight implementation as the `getUsing()` argument. This is dangerous, because on reusing flyweight points to the `ChronicleMap` memory
|
||||
directly, but the access is not synchronized. At best you could read inconsistent value state;
|
||||
at worst you could corrupt the `ChronicleMap` memory.
|
||||
|
||||
For accessing the `ChronicleMap` value memory directly use the following technique:
|
||||
For accessing the `ChronicleMap` value memory directly use the following techniques.
|
||||
|
||||
==== Working with an entry within a context
|
||||
|
||||
@@ -432,7 +435,8 @@ try (ExternalMapQueryContext<CharSequence, PostalCodeRange, ?> c =
|
||||
|
||||
=== Multi-key queries
|
||||
|
||||
In this example, consistent graph edge addition and removals are implemented via multi-key queries:
|
||||
In this example, consistent graph edge addition and removals are implemented using multi-key queries:
|
||||
|
||||
```java
|
||||
public static boolean addEdge(
|
||||
ChronicleMap<Integer, Set<Integer>> graph, int source, int target) {
|
||||
@@ -562,6 +566,7 @@ private static boolean innerRemoveEdge(
|
||||
```
|
||||
|
||||
Usage:
|
||||
|
||||
```java
|
||||
HashSet<Integer> averageValue = new HashSet<>();
|
||||
for (int i = 0; i < AVERAGE_CONNECTIVITY; i++) {
|
||||
@@ -579,21 +584,22 @@ removeEdge(graph, 1, 2);
|
||||
```
|
||||
|
||||
== Close `ChronicleMap`
|
||||
Unlike ConcurrentHashMap, ChronicleMap stores its data off heap, often in a memory mapped file.
|
||||
Its recommended that you call close() once you have finished working with a ChronicleMap.
|
||||
Unlike `ConcurrentHashMap`, `ChronicleMap` stores its data off-heap; often in a memory mapped file.
|
||||
It is recommended that you call `close()` when you have finished working with a `ChronicleMap`.
|
||||
|
||||
``` java
|
||||
map.close()
|
||||
```
|
||||
|
||||
This is especially important when working with ChronicleMap replication, as failure to call close may prevent
|
||||
you from restarting a replicated map on the same port. In the event that your application crashes it may not
|
||||
be possible to call close(). Your operating system will usually close dangling ports automatically,
|
||||
so although it is recommended that you close() when you have finished with the map,
|
||||
its not something that you must do, it's just something that we recommend you should do.
|
||||
This is especially important when working with Chronicle Map replication, as failure to call close may prevent
|
||||
you from restarting a replicated map on the same port.
|
||||
|
||||
WARNING: *WARNING*. If you call `close()` too early before you have finished working with the map, this can cause
|
||||
your JVM to crash. Close MUST BE the last thing that you do with the map.
|
||||
In the event that your application crashes, it may not
|
||||
be possible to call `close()`. Your operating system will usually close dangling ports automatically. So, although it is recommended that you `close()` when you have finished with the map,
|
||||
it is not something that you must do; it's just something that we recommend you should do.
|
||||
|
||||
WARNING: If you call `close()` too early before you have finished working with the map, this can cause
|
||||
your JVM to crash. Close **MUST** be the last thing that you do with the map.
|
||||
|
||||
== Behaviour Customization
|
||||
|
||||
@@ -604,31 +610,27 @@ See <<CM_Tutorial_Behaviour.adoc#,CM_Tutorial_Behaviour.adoc>> for more details.
|
||||
|
||||
== Entry checksums
|
||||
|
||||
Chronicle Map 3 is able to store entry checksums along with entries. With entry checksums it is
|
||||
possible to identify partially written entries (in case of operating system or power failure while)
|
||||
and corrupted entries (in case of hardware memory or disk corruption) and clean them up during the <<CM_Tutorial.adoc#recovery,recovery>> procedure.
|
||||
Chronicle Map is able to store entry checksums along with entries. With entry checksums it is
|
||||
possible to identify partially written entries (in the case of operating system, or power failure),
|
||||
and corrupted entries (in the case of hardware, memory, or disk corruption) and clean them up during the <<CM_Tutorial.adoc#recovery,recovery>> procedure.
|
||||
|
||||
Entry checksums are 32-bit numbers, computed by a hash function with good avalanche effect.
|
||||
Theoretically there is still about a one-billionth chance that after entry corruption it passes the
|
||||
sum check.
|
||||
Entry checksums are `32-bit` numbers, computed by a hash function with good avalanche effect.
|
||||
Theoretically, there is still about a one-in-a-billion chance that after entry corruption, it passes the sum check.
|
||||
|
||||
By default, entry checksums are ON if the Chronicle Map is persisted to disk (i. e. created via
|
||||
`createPersistedTo()` method), and OFF if the Chronicle Map is purely in-memory. Storing checksums
|
||||
for a purely in-memory Chronicle Map hardly makes any practical sense, but you might want to disable
|
||||
storing checksums for a persisted Chronicle Map by calling `.checksumEntries(false)` on the
|
||||
`ChronicleMapBuilder` used to create a map. It makes sense if you don't need extra safety checksums
|
||||
provide.
|
||||
By default, entry checksums are:
|
||||
|
||||
- **`ON`** if the Chronicle Map is persisted to disk (i. e. created via
|
||||
`createPersistedTo()` method)
|
||||
- **`OFF`** if the Chronicle Map is purely in-memory.
|
||||
|
||||
Storing checksums for a purely in-memory Chronicle Map hardly makes any practical sense, but you might want to disable storing checksums for a persisted Chronicle Map by calling `.checksumEntries(false)` on the `ChronicleMapBuilder` used to create a map. It makes sense if you don't need extra safety that checksums provide.
|
||||
|
||||
Entry checksums are computed automatically when an entry is inserted into a Chronicle Map, and
|
||||
re-computed automatically on operations which update the whole value, e. g. `map.put()`,
|
||||
`map.replace()`, `map.compute()`, `mapEntry.doReplaceValue()` (See `MapEntry` interface in
|
||||
[Javadocs](http://www.javadoc.io/doc/net.openhft/chronicle-map/). But if you update values directly,
|
||||
bypassing Chronicle Map logic, keeping entry checksum up-to-date is also your responsibility.
|
||||
re-computed automatically on operations which update the whole value. For example, `map.put()`,
|
||||
`map.replace()`, `map.compute()`, `mapEntry.doReplaceValue()`. See the `MapEntry` interface in
|
||||
http://www.javadoc.io/doc/net.openhft/chronicle-map/[Javadocs]. If you update values directly, bypassing Chronicle Map logic, keeping the entry checksum up-to-date is also your responsibility.
|
||||
|
||||
It is strongly recommended to update off-heap memory of values directly only within a
|
||||
[context](#working-with-an-entry-within-a-context-section) and update or write lock held. Within a
|
||||
context, you are provided with an entry object of `MapEntry` type. To re-compute entry checksum
|
||||
manually, cast that object to `ChecksumEntry` type and call `.updateChecksum()` method on it:
|
||||
It is strongly recommended to update off-heap memory of values directly only within a context, and update or write lock held. Within a context, you are provided with an entry object of `MapEntry` type. To re-compute entry checksum manually, cast that object to `ChecksumEntry` type and call the `.updateChecksum()` method:
|
||||
|
||||
``` java
|
||||
try (ChronicleMap<Integer, LongValue> map = ChronicleMap
|
||||
|
||||
@@ -12,45 +12,44 @@ toc::[]
|
||||
|
||||
You could customize `ChronicleMap` behaviour on several levels:
|
||||
|
||||
- `ChronicleMapBuilder.entryOperations()` define the "inner" listening level, all operations with
|
||||
- `ChronicleMapBuilder.entryOperations()` define the "inner" listening level. All operations with
|
||||
entries, either during ordinary map method calls, remote calls, replication or modifications during
|
||||
iteration over the map, operate via this configured SPI.
|
||||
iteration over the map, operate using this configured SPI.
|
||||
|
||||
- `ChronicleMapBuilder.mapMethods()` is the higher-level of listening for local calls of Map
|
||||
methods. Methods in `MapMethods` interface correspond to `Map` interface methods with the same
|
||||
methods. Methods in the `MapMethods` interface correspond to `Map` interface methods with the same
|
||||
names, and define their implementations for `ChronicleMap`.
|
||||
|
||||
- `ChronicleMapBuilder.remoteOperations()` is for listening and customizing behaviour of remote
|
||||
calls, and replication events.
|
||||
|
||||
All executions around `ChronicleMap` go through the three tiers (or the two bottom):
|
||||
All executions around `ChronicleMap` go through all three tiers, or just the last two):
|
||||
|
||||
1. Query tier: `MapQueryContext` interface
|
||||
2. Entry tier: `MapEntry` and `MapAbsentEntry` interfaces
|
||||
3. Data tier: `Data` interface
|
||||
|
||||
`MapMethods` and `MapRemoteOperations` methods accept *query context*, i. e. these SPI is above
|
||||
the Query tier. `MapEntryOperations` methods accept `MapEntry` or `MapAbsentEntry`, i. e. this SPI
|
||||
is between Query and Entry tiers.
|
||||
`MapMethods` and `MapRemoteOperations` methods accept *query context*; SPI is above
|
||||
the Query tier.
|
||||
|
||||
Combined, interception SPI interfaces and `ChronicleMap.queryContext()` API are powerful enough to
|
||||
`MapEntryOperations` methods accept `MapEntry` or `MapAbsentEntry`; SPI is between Query and Entry tiers.
|
||||
|
||||
- Log all operations of some kind on `ChronicleMap` (e. g. all remove, insert or update operations)
|
||||
- Log some specific operations on `ChronicleMap` (e. g. log only acquireUsing() calls, which has
|
||||
created a new entry)
|
||||
- Forbid performing operations of some kind on the `ChronicleMap` instance
|
||||
- Backup all changes to `ChronicleMap` to some alternative storage, e. g. SQL database
|
||||
When combined, interception SPI interfaces and the `ChronicleMap.queryContext()` API are powerful enough to:
|
||||
|
||||
- Log all operations of some kind on `ChronicleMap` (for example, all remove, insert, or update operations).
|
||||
- Log some specific operations on `ChronicleMap` (for example, log only `acquireUsing()` calls, which has created a new entry).
|
||||
- Inhibit operations of some kind on the `ChronicleMap` instance.
|
||||
- Backup all changes to `ChronicleMap` to some alternative storage; for example, an SQL database.
|
||||
- Perform multi-Chronicle Map operations correctly in concurrent environment, by acquiring locks on
|
||||
all ChronicleMaps before updating them.
|
||||
- Perform multi-key operations on a single `ChronicleMap` correctly in concurrent environment, by
|
||||
acquiring locks on all keys before updating the entries
|
||||
- Define own replication/reconciliation logic for distributed Chronicle Maps
|
||||
- Dump statistics of the Chronicle Map instance -- each segment's load, size in bytes of each
|
||||
all ChronicleMaps, before updating them.
|
||||
- Perform multi-key operations on a single `ChronicleMap` correctly in a concurrent environment, by acquiring locks on all keys before updating the entries.
|
||||
- Define your own replication/reconciliation logic for distributed Chronicle Maps.
|
||||
- Dump statistics of the Chronicle Map instance; each segment's load, size in bytes of each
|
||||
entry, etc.
|
||||
|
||||
== Example - Simple logging
|
||||
|
||||
Just log all modification operations on `ChronicleMap`
|
||||
To log all modification operations on `ChronicleMap`:
|
||||
|
||||
```java
|
||||
class SimpleLoggingMapEntryOperations<K, V> implements MapEntryOperations<K, V, Void> {
|
||||
@@ -101,7 +100,7 @@ ChronicleMap<IntValue, IntValue> map = ChronicleMap
|
||||
|
||||
== Example - BiMap
|
||||
|
||||
Possible bidirectional map (i. e. a map that preserves the uniqueness of its values as well
|
||||
Possible bi-directional map (that is, a map that preserves the uniqueness of its values, as well
|
||||
as that of its keys) implementation over Chronicle Maps.
|
||||
|
||||
```java
|
||||
@@ -293,6 +292,7 @@ class BiMapEntryOperations<K, V> implements MapEntryOperations<K, V, DualLockSuc
|
||||
```
|
||||
|
||||
Usage:
|
||||
|
||||
```java
|
||||
BiMapEntryOperations<Integer, CharSequence> biMapOps1 = new BiMapEntryOperations<>();
|
||||
ChronicleMap<Integer, CharSequence> map1 = ChronicleMapBuilder
|
||||
|
||||
+21
-28
@@ -8,14 +8,13 @@ Neil Clifford
|
||||
|
||||
toc::[]
|
||||
|
||||
This pair of interfaces is configured via `ChronicleMapBuilder.keyMarshallers()` or
|
||||
`valueMarshallers()` for the key or value type of the map, respectively.
|
||||
This pair of interfaces is configured using `ChronicleMapBuilder.keyMarshallers()` or
|
||||
`valueMarshallers()`, for the key, or value type of the map, respectively.
|
||||
|
||||
This pair of interfaces is the most suitable, if the size of the serialized form is not known
|
||||
in advance, i. e. the easiest way to compute the size of the serialized form of some object of the
|
||||
type, is performing serialization itself and looking at the number of written bytes.
|
||||
These interfaces are the most suitable, where the size of the serialized form is not known in advance. The easiest way to compute the size of the serialized form of some object of the
|
||||
type, is performing serialization itself, and then looking at the number of written bytes.
|
||||
|
||||
This pair of interfaces is the least efficient and the simplest to implement, so it should also be
|
||||
This interface-pair is the least efficient, and the simplest to implement, so it should also be
|
||||
used when efficiency is not the top priority, or when gains of using other pairs of interfaces
|
||||
(which are more complicated to implement) are marginal.
|
||||
|
||||
@@ -26,8 +25,7 @@ Basically you should implement two serialization methods:
|
||||
- `T read(Bytes in, @Nullable T using)` from `BytesReader` interface, which reads the serialized
|
||||
object into the given `using` instance (if the serialized type is reusable, the `using` object is
|
||||
not `null`, and suitable for reusing for this particular serialized object), or a newly created
|
||||
instance. The returned object contains the serialized data; it may be identical or not identical to
|
||||
the passed `using` instance.
|
||||
instance. The returned object contains the serialized data; it may be identical, or not identical, to the passed `using` instance.
|
||||
|
||||
For example, here is the implementation of `BytesWriter` and `BytesReader` for `CharSequence[]`
|
||||
value type (array of CharSequences):
|
||||
@@ -106,27 +104,24 @@ try (ChronicleMap<String, CharSequence[]> map = ChronicleMap
|
||||
}
|
||||
```
|
||||
|
||||
The total size of serialization form for some `CharSequence[]` array is 4 bytes for storing the
|
||||
The total size of serialization-form for a `CharSequence[]` array is 4 bytes for storing the
|
||||
array length, plus the sum of sizes of all CharSequences, in UTF-8 encoding. Computing this size
|
||||
without actual encoding has comparable computational cost with performing actual encoding, that
|
||||
makes `CharSequence[]` type to meet the second criteria (see above) which makes `BytesWriter` and
|
||||
without actual encoding has comparable computational cost with performing actual encoding. That
|
||||
makes `CharSequence[]` type to meet the second criteria (see above). This makes `BytesWriter` and
|
||||
`BytesReader` the most suitable pair of serialization interfaces to implement for the type.
|
||||
|
||||
Note how `read()` implementation attempts to reuse not only the array object, but also the elements,
|
||||
Note how `read()` implementation attempts to reuse, not only the array object, but also the elements,
|
||||
minimizing the amount of produced garbage. This is a recommended practice.
|
||||
|
||||
Some additional notes:
|
||||
|
||||
- If the reader or writer interface implementation is not configurable and doesn't have
|
||||
per-instance cache or state fields, i. e. it doesn't have instance fields at all, there is
|
||||
a convention to make such implementation classes `final`, give them a `private` constructor and
|
||||
expose a single `INSTANCE` constant - a sole instance of this implementation in the JVM.
|
||||
- *Don't* make marshaller class `enum`, because there are some issues with `enum` serialization/
|
||||
deserialization.
|
||||
- For such no-state implementations, don't forget to implement `ReadResolvable`
|
||||
interface and return `INSTANCE`, otherwise you have no guarantee that `INSTANCE` constant is the
|
||||
only alive instance of this implementation in the JVM.
|
||||
- If *both* writer and reader interface implementations have no fields, it might be a good idea
|
||||
- If the reader, or writer, interface implementation is not configurable, and doesn't have
|
||||
per-instance cache or state fields (it doesn't have instance fields at all), there is
|
||||
a convention to make such implementation classes `final`, give them a `private` constructor, and then
|
||||
expose a single `INSTANCE` constant; a sole instance of this implementation in the JVM.
|
||||
- *Do not* make `marshaller` class `enum`, because there are some issues with `enum` serialization/deserialization.
|
||||
- For such no-state implementations, do not forget to implement the `ReadResolvable` interface and return `INSTANCE`. Otherwise you have no guarantee that `INSTANCE` constant is the only alive instance of this implementation in the JVM.
|
||||
- If *both* the writer and reader interface implementations have no fields, it might be a good idea
|
||||
to merge them into a single type, in order to keep writing and reading logic together.
|
||||
|
||||
=== Custom `CharSequence` encoding
|
||||
@@ -396,15 +391,13 @@ try (ChronicleMap<String, CharSequence> englishToChinese = ChronicleMap
|
||||
}
|
||||
```
|
||||
|
||||
Some notes on this case of custom serialization:
|
||||
Some notes on this form of custom serialization:
|
||||
|
||||
- Both `CharSequenceCustomEncodingBytesWriter` and `CharSequenceCustomEncodingBytesReader` have
|
||||
configurations (charset and input buffer size), hence they are implemented as normal classes rather
|
||||
configurations (charset and input buffer size). They are implemented as normal classes rather
|
||||
than classes with `private` constructors and a single `INSTANCE`.
|
||||
- Both writer and reader classes have some "cache" fields, their contents are mutated during
|
||||
writing and reading. That is why they have to implement `StatefulCopyable` interface. See
|
||||
[Understanding `StatefulCopyable`](#understanding-statefulcopyable) section for more infromation on
|
||||
this.
|
||||
- Both writer and reader classes have some "cache" fields; their contents are mutated during
|
||||
writing and reading. That is why they have to implement the `StatefulCopyable` interface.
|
||||
|
||||
|
||||
'''
|
||||
|
||||
@@ -8,25 +8,27 @@ Neil Clifford
|
||||
|
||||
toc::[]
|
||||
|
||||
This pair of interfaces is configured via `ChronicleMapBuilder.keyReaderAndDataAccess()` or
|
||||
`valueReaderAndDataAccess()` for the key, or value type, of the map, respectively.
|
||||
This pair of interfaces is configured using `ChronicleMapBuilder.keyReaderAndDataAccess()` or
|
||||
`valueReaderAndDataAccess()` for the key, or value type, of the map respectively.
|
||||
|
||||
The reader part, `SizedReader`, is the same as in <<CM_Tutorial_Sized.adoc#`SizedWriter` and `SizedReader`,`SizedWriter` and `SizedReader`>> pair, so `DataAccess` is an "advanced" interface to
|
||||
replace `SizedWriter`.
|
||||
The reader part, `SizedReader`, is the same as in <<CM_Tutorial_Sized.adoc#,`SizedWriter` and `SizedReader`>> pair. `DataAccess` is an "advanced" interface to replace `SizedWriter`.
|
||||
|
||||
The main method in `DataAccess` is `Data<T> getData(@NotNull T instance)`, it returns a `Data`
|
||||
accessor which is used to write "serialized" form of the instance to off-heap memory. `Data.size()`
|
||||
on the returned `Data` object is used for the same purpose as `SizedWriter.size()` method in
|
||||
<<CM_Tutorial_Sized.adoc#`SizedWriter` and `SizedReader`,`SizedWriter` and `SizedReader`>> pair interfaces. `Data.writeTo()`
|
||||
The main method in `DataAccess` is `Data<T> getData(@NotNull T instance)`. It returns a `Data`
|
||||
accessor which is used to write a "serialized" form of the instance to off-heap memory. `Data.size()`
|
||||
on the returned `Data` object is used for the same purpose as the `SizedWriter.size()` method in
|
||||
the <<CM_Tutorial_Sized.adoc,`SizedWriter` and `SizedReader`>> pair interfaces. `Data.writeTo()`
|
||||
is used instead of `SizedWriter.write()`.
|
||||
|
||||
`DataAccess` assumes that the `Data` object, returned from the `getData()` method is cached in some way. That is why it also has `uninit()` method to clear references to the serialized object after query operation to a Chronicle Map is complete to prevent memory leaks. This, in turn, implies that `DataAccess` implementation is stateful. Therefore, `DataAccess` is made a subinterface of
|
||||
`StatefulCopyable` to force all `DataAccess` implementations to implement `StatefulCopyable` as well. See [Understanding `StatefulCopyable`](#understanding-statefulcopyable) for more infromation
|
||||
on this. If your `DataAccess` implementation is not actually stateful, it is free to return `this` from the `StatefulCopyable.copy()` method.
|
||||
`DataAccess` assumes that the `Data` object, returned from the `getData()` method is cached in some way. That is why it also has an `uninit()` method to clear references to the serialized object after a query operation to a Chronicle Map is complete to prevent memory leaks. This, in turn, implies that the `DataAccess` implementation is stateful. Therefore, `DataAccess` is made a sub-interface of
|
||||
`StatefulCopyable` to force all `DataAccess` implementations to implement `StatefulCopyable` as well.
|
||||
|
||||
The `DataAccess` interface is primarily intended for "serializing" objects that are already sequences of bytes, and in fact doesn't require serialization, like `byte[]`, `ByteBuffer`, arrays of Java
|
||||
primitives. For such types of objects, `DataAccess` allows bypassing of the intermediate buffering, copying
|
||||
data directly from objects to Chronicle Map's off-heap memory.
|
||||
See <<CM_Tutorial_Understanding.adoc#, Understanding `StatefulCopyable`>> for more infromation
|
||||
on this.
|
||||
|
||||
If your `DataAccess` implementation is not actually stateful, it is free to return `this` from the `StatefulCopyable.copy()` method.
|
||||
|
||||
The `DataAccess` interface is primarily intended for "serializing" objects that are already sequences of bytes, and in fact do not require serialization; for example, `byte[]`, `ByteBuffer`, arrays of Java
|
||||
primitives. For such types of objects, `DataAccess` allows bypassing of the intermediate buffering, copying data directly from objects to Chronicle Map's off-heap memory.
|
||||
|
||||
For example, look at the `DataAccess` implementation for `byte[]`:
|
||||
|
||||
@@ -106,12 +108,12 @@ public final class ByteArrayDataAccess extends AbstractData<byte[]> implements D
|
||||
}
|
||||
```
|
||||
|
||||
Note that the `getData()` method returns `this`, and the `DataAccess` implementation implements `Data` interface as well. This is recommended practice, because it reduces the number of objects involved (hence pointer chasing), and keeps `DataAccess`, and `Data` logic together.
|
||||
The `getData()` method returns `this`, and the `DataAccess` implementation implements the `Data` interface as well. This is recommended practice, because it reduces the number of objects involved (hence pointer chasing), and keeps `DataAccess`, and `Data` logic together.
|
||||
|
||||
`Data` interface puts constraints on `equals()`, `hashCode()`, and `toString()` implementations. This is why `ByteArrayDataAccess` subclasses `AbstractData`, and inherits proper implementations from it.
|
||||
It is OK for serializer strategy implementation to have `equals()`, `hashCode()`, and `toString()` from a very different domain, because those methods are never called on serializers inside Chronicle Map.
|
||||
The `Data` interface puts constraints on `equals()`, `hashCode()`, and `toString()` implementations. This is why `ByteArrayDataAccess` sub-class'es `AbstractData`, and inherits proper implementations from it.
|
||||
A serializer strategy implementation can have `equals()`, `hashCode()`, and `toString()` from a very different domain, because those methods are never called on serializers inside Chronicle Map.
|
||||
|
||||
The easiest way to implement `equals()`, `hashCode()`, and `toString()` is to extend the `AbstractData` class. If it is not possible (the `Data` implementation already extends some other class), do this
|
||||
The easiest way to implement `equals()`, `hashCode()`, and `toString()` is to extend the `AbstractData` class. If it is not possible (perhaps the `Data` implementation already extends some other class), do this
|
||||
by delegating to `dataEquals()`, `dataHashCode()`, and `dataToString()` default methods, provided in the `Data` interface.
|
||||
|
||||
Corresponding `SizedReader` for `byte[]`:
|
||||
@@ -155,7 +157,7 @@ public final class ByteArraySizedReader
|
||||
}
|
||||
```
|
||||
|
||||
NOTE: No special coding is required to use this pair of serializers with `byte[]` keys of values in Chronicle Map. It is used as the default, if you configure `byte[]` key, or value type.
|
||||
NOTE: If you configure `byte[]` key, or value type, then this pair of serializers is used as the default.
|
||||
|
||||
|
||||
'''
|
||||
|
||||
+17
-20
@@ -8,41 +8,38 @@ Neil Clifford
|
||||
|
||||
toc::[]
|
||||
|
||||
This pair of interfaces is configured via `ChronicleMapBuilder.keyMarshallers()` or
|
||||
`valueMarshallers()` for the key or value type of the map, respectively (overloaded methods, those
|
||||
This pair of interfaces is configured using `ChronicleMapBuilder.keyMarshallers()` or
|
||||
`valueMarshallers()` for the key, or value type, of the map, respectively. Overloaded methods; those
|
||||
for [`BytesWriter` and `BytesReader`](#byteswriter-and-bytesreader) interfaces have the same names).
|
||||
|
||||
The main two methods to implement in `SizedWriter` interface:
|
||||
The main two methods to implement in `SizedWriter` interface are:
|
||||
|
||||
- `long size(@NotNull T toWrite);` returns the number of bytes, which is written by the subsequent
|
||||
`write()` method given the same `toWrite` instance of the serialized type.
|
||||
- `void write(Bytes out, long size, @NotNull T toWrite);` writes the given `toWrite` instance to
|
||||
the given `out` bytes sink. Additionally `size` is provided, which is the value computed for the
|
||||
same `toWrite` instance by calling `size()` method. Hence, when `SizedWriter` is used internally,
|
||||
`size()` method is always called before `write()`, caching logic in `SizedWriter` implementation
|
||||
may rely on this. This method should advance `writePosition()` of the given out `Bytes` exactly by
|
||||
the given `out` bytes sink. Additionally, `size` is provided, which is the value computed for the
|
||||
same `toWrite` instance by calling `size()` method. When `SizedWriter` is used internally,
|
||||
the `size()` method is always called before `write()`; caching logic in `SizedWriter` implementation
|
||||
may rely on this. This method should advance the `writePosition()` of the given out `Bytes` exactly by
|
||||
the given `size` number of bytes.
|
||||
|
||||
`SizedReader`'s `T read(Bytes in, long size, @Nullable T using);` method is similar to the
|
||||
corresponding `read()` method in `BytesReader` interface, except the size of the serialized object
|
||||
is provided.
|
||||
corresponding `read()` method in `BytesReader` interface, except that the size of the serialized object is provided.
|
||||
|
||||
This pair of interfaces is suitable, if
|
||||
This pair of interfaces is suitable, if:
|
||||
|
||||
- The serialized form of the type effectively includes the size of the rest of the serialized form
|
||||
in the beginning,
|
||||
- But the type is not a plain sequence of bytes like `byte[]` or `ByteBuffer` (for those types
|
||||
[`DataAccess` and `SizedReader`](#dataaccess-and-sizedreader) pair of interfaces is a much better
|
||||
choice).
|
||||
in the beginning.
|
||||
|
||||
Examples of such types include lists and arrays of constant-sized elements.
|
||||
- The type is not a plain sequence of bytes; for example, `byte[]` or `ByteBuffer`. For `byte[]` or `ByteBuffer`, the <<CM_Tutorial_DataAccess#, `DataAccess` and `SizedReader`>> pair of interfaces is a much better
|
||||
choice.
|
||||
|
||||
Compared to `BytesWriter` and `BytesReader`, this pair of interfaces allows to share the information
|
||||
about the serialized form between serialization logic and the Chronicle Map, saving a few bytes by
|
||||
storing the serialization size only once rather than twice. This pair of interfaces is not much more
|
||||
difficult to implement, but the gains are also not big.
|
||||
Examples of such types include lists, and arrays of constant-sized elements.
|
||||
|
||||
Example: serializing lists of simple `Point` structures:
|
||||
Compared to `BytesWriter` and `BytesReader`, this pair of interfaces allows the sharing of information
|
||||
about the serialized form, between serialization logic, and the Chronicle Map; this saves a few bytes by storing the serialization size only once, rather than twice. This pair of interfaces is not much more difficult to implement, but the gains are also not large.
|
||||
|
||||
Example: Serializing lists of simple `Point` structures:
|
||||
|
||||
```java
|
||||
public final class Point {
|
||||
|
||||
@@ -25,24 +25,23 @@ To avoid generating a lot of garbage, those intermediate objects should be cache
|
||||
- many methods in `Data`.
|
||||
+
|
||||
+
|
||||
It is usually essential to save some computation results between calls to those methods from inside the Chronicle Map implementation, while working some key or value during some query to Chronicle Map. This is because, otherwise expensive, computations are performed several times, and is very
|
||||
inefficient.
|
||||
It is usually essential to save some computation results between calls to those methods from inside the Chronicle Map implementation, while working some key or value during some query to Chronicle Map. This is because, otherwise expensive computations are performed several times, and is very inefficient.
|
||||
|
||||
These problems require to access some mutable, context-dependant fields, from within serializer
|
||||
interface implementations. Only a *single instance* of serializer implementation is configured for a Chronicle Map (either by `keyMarshallers()` or `valueMarshallers()` or
|
||||
`keyReaderAndDataAccess()` or `valueReaderAndDataAccess()` method in `ChronicleMapBuilder`). On the
|
||||
other hand, `ChronicleMap` is a `ConcurrentMap`, that is, serializer implementations could be accessed from multiple threads concurrently. Moreover, a single `ChronicleMapBuilder` (with a single pair of serializer instances configured for keys and values) could be used to construct many independent ``ChronicleMap``s, which could also be accessed concurrently.
|
||||
interface implementations. Only a *single instance* of the serializer implementation is configured for a Chronicle Map (either by `keyMarshallers()`, `valueMarshallers()`, `keyReaderAndDataAccess()`, or `valueReaderAndDataAccess()` methods in `ChronicleMapBuilder`). On the
|
||||
other hand, `ChronicleMap` is a `ConcurrentMap`. That is, serializer implementations could be accessed from multiple threads concurrently. Moreover, a single `ChronicleMapBuilder` (with a single pair of serializer instances configured for keys and values) could be used to construct many independent ``ChronicleMap``s, which could also be accessed concurrently.
|
||||
|
||||
An inefficient and fragile solution would be a single instance of serialization interface, static `ThreadLocal` fields in the serializer implementation class. It is inefficient, because ``ThreadLocal``s are accessed each time a serializer interface method is called; that is much slower than accessing vanilla object fields.
|
||||
An inefficient and fragile solution would be a single instance of serialization interface, static `ThreadLocal` fields in the serializer implementation class.
|
||||
|
||||
It is fragile, because if you need `ThreadLocal` fields for preserving some state between calls to
|
||||
- It is inefficient, because ``ThreadLocal``s are accessed each time a serializer interface method is called. That is much slower than accessing vanilla object fields.
|
||||
|
||||
- It is fragile, because if you need `ThreadLocal` fields for preserving some state between calls to
|
||||
multiple methods in serializer interfaces over a single query to a Chronicle Map (the 3rd point in
|
||||
the problems list above), then you should consider that a single serializer instance could be used for both keys and values of the same Chronicle Map. Calls to some methods of serializer interface for
|
||||
serializing the key and the value over a Chronicle Map query are interspersed in unspecified order.
|
||||
the problems list above), then you should consider that a single serializer instance could be used for both keys and values of the same Chronicle Map. Calls to some methods of serializer interface for serializing the key and the value over a Chronicle Map query are interspersed in unspecified order.
|
||||
|
||||
In addition, you should consider that a single serializer instance could be used to access multiple independent Chronicle Map instances in the same thread. Calls to serializers within maps could be interspersed via <<CM_Tutorial.adoc#Working with an entry within a context,Working with an entry within a context>>.
|
||||
|
||||
So, `ThreadLocal` fields should be isolated; not just per accessing thread, but also per serialized object domain, and per accessing Chronicle Map instance.
|
||||
So, `ThreadLocal` fields should be isolated; not just per-accessing thread, but also per-serialized object domain, and per-accessing Chronicle Map instance.
|
||||
|
||||
== Recommended solution
|
||||
Use `StatefulCopyable`.
|
||||
@@ -50,15 +49,21 @@ Use `StatefulCopyable`.
|
||||
A serializer implementation has ordinary instance
|
||||
fields for caching anything, and preserving the state between calls to different methods. It implements the `StatefulCopyable` interface with a single method, `copy()`. This is like `Object.clone()` but copies only the configuration fields of the serializer instance; not the cache, or state fields.
|
||||
|
||||
A call to the `copy()` method, at any point of a serializer instance lifetime, should return an instance of the same serializer implementation in the state; exactly equal to the state of the current instance at the moment after construction and initial configuration, with an empty cache and state.
|
||||
A call to the `copy()` method, at any point of a serializer instance lifetime, should return an instance of the same serializer implementation, in a state exactly equal to the state of the current instance, at the moment after construction and initial configuration, with an empty cache and state.
|
||||
|
||||
As a consequence, the `copy()` method is transitive. `serializer.copy().copy().copy()` should return an object in exactly the same state as after a single `copy()` call.
|
||||
|
||||
Chronicle Map implementation recognizes that configured `BytesWriter`, `BytesReader`, `SizedWriter`, `SizedReader`, or `DataAccess` instances implement `StatefulCopyable`, and populates it internally using `copy()` for each thread, Chronicle Map instances and serialized object domain (keys or values).
|
||||
|
||||
See examples of implementing this interface in <<document-b.adoc#custom `CharSequence` encoding,custom `CharSequence` encoding>>.
|
||||
See examples of implementing this interface in <<CM_Tutorial.adoc#custom-charsequence-encoding,custom `CharSequence` encoding>>.
|
||||
|
||||
It is permissable to return `this` from the `copy()` method, if the
|
||||
serializer implementation doesn't have a state. Typically this is the case when the serializer is
|
||||
serializer implementation doesn't have a state.
|
||||
|
||||
Typically this is the case when the serializer is
|
||||
configured with sub-serializers, which might be `StatefulCopyable`, or not. For example, the https://github.com/OpenHFT/Chronicle-Map/blob/master/src/main/java/net/openhft/chronicle/hash/serialization/ListMarshaller.java[`ListMarshaller`]
|
||||
class.
|
||||
|
||||
|
||||
'''
|
||||
<<CM_Tutorial.adoc,Back to Tutorial>>
|
||||
|
||||
Reference in New Issue
Block a user