diff --git a/src/main/java/net/openhft/chronicle/ChronicleHashBuilder.java b/src/main/java/net/openhft/chronicle/ChronicleHashBuilder.java index 505d9614b..e4a21799a 100644 --- a/src/main/java/net/openhft/chronicle/ChronicleHashBuilder.java +++ b/src/main/java/net/openhft/chronicle/ChronicleHashBuilder.java @@ -1,12 +1,16 @@ package net.openhft.chronicle; -import net.openhft.chronicle.map.*; +import net.openhft.chronicle.map.ChannelProvider; +import net.openhft.chronicle.map.ChronicleMap; +import net.openhft.chronicle.map.ChronicleMapBuilder; +import net.openhft.chronicle.map.ReplicationConfig; import net.openhft.chronicle.set.ChronicleSet; import net.openhft.chronicle.set.ChronicleSetBuilder; -import org.jetbrains.annotations.NotNull; - -import net.openhft.lang.io.serialization.*; +import net.openhft.lang.io.serialization.BytesMarshaller; +import net.openhft.lang.io.serialization.BytesMarshallerFactory; +import net.openhft.lang.io.serialization.ObjectSerializer; import net.openhft.lang.model.Byteable; +import org.jetbrains.annotations.NotNull; import java.io.File; import java.io.IOException; @@ -18,14 +22,12 @@ import java.util.concurrent.ConcurrentHashMap; import java.util.concurrent.TimeUnit; /** - * This interface defines the meaning of configurations, common to {@link ChronicleMapBuilder} - * and {@link ChronicleSetBuilder}, i. e. Chronicle hash container configurations. + * This interface defines the meaning of configurations, common to {@link ChronicleMapBuilder} and {@link + * ChronicleSetBuilder}, i. e. Chronicle hash container configurations. * * @param the type of keys in hash containers, created by this builder - * @param the container type, created by this builder, i. e. {@link ChronicleMap} - * or {@link ChronicleSet} - * @param the concrete builder type, i. e. {@link ChronicleMapBuilder} - * or {@link ChronicleSetBuilder} + * @param the container type, created by this builder, i. e. {@link ChronicleMap} or {@link ChronicleSet} + * @param the concrete builder type, i. e. {@link ChronicleMapBuilder} or {@link ChronicleSetBuilder} */ public interface ChronicleHashBuilder> extends Cloneable { @@ -41,14 +43,13 @@ public interface ChronicleHashBuilderIf key size varies moderately, specify the size higher than average, but lower than - * the maximum possible, to minimize average memory overuse. If key size varies in a wide range, - * it's better to use {@linkplain #entrySize(int) entry size} in "chunk" mode and configure it - * directly. + *

If key size varies moderately, specify the size higher than average, but lower than the maximum + * possible, to minimize average memory overuse. If key size varies in a wide range, it's better to use + * {@linkplain #entrySize(int) entry size} in "chunk" mode and configure it directly. * * @param keySize number of bytes, taken by serialized form of keys * @return this builder back @@ -58,15 +59,15 @@ public interface ChronicleHashBuilderIf keys are of boxed primitive type or {@link Byteable} subclass, i. e. if key size - * is known statically, it is automatically accounted and this method shouldn't be called. + *

If keys are of boxed primitive type or {@link Byteable} subclass, i. e. if key size is known + * statically, it is automatically accounted and this method shouldn't be called. * - *

If key size varies, method {@link #keySize(int)} or {@link #entrySize(int)} should be - * called instead of this one. + *

If key size varies, method {@link #keySize(int)} or {@link #entrySize(int)} should be called instead + * of this one. * * @param sampleKey the sample key * @return this builder back @@ -75,55 +76,43 @@ public interface ChronicleHashBuilder{@link ChronicleMap} and {@link ChronicleSet} store their data off-heap, so it is required - * to serialize key (and values, in {@code ChronicleMap} case) (unless they are direct - * {@link Byteable} instances). Serialized key bytes (+ serialized value bytes, - * in {@code ChronicleMap} case) + some metadata bytes comprise "entry space", which - * {@code ChronicleMap} or {@code ChronicleSet} should allocate. So entry size - * is a minimum allocation portion in the hash containers, created by this builder. - * E. g. if entry size is 100, the created container could only allocate 100, 200, 300... bytes - * for an entry. If say 150 bytes of entry space are required by the entry, 200 bytes - * will be allocated, 150 used and 50 wasted. To minimize memory overuse and improve speed, you - * should pay decent attention to this configuration. + *

{@link ChronicleMap} and {@link ChronicleSet} store their data off-heap, so it is required to + * serialize key (and values, in {@code ChronicleMap} case) (unless they are direct {@link Byteable} + * instances). Serialized key bytes (+ serialized value bytes, in {@code ChronicleMap} case) + some + * metadata bytes comprise "entry space", which {@code ChronicleMap} or {@code ChronicleSet} should + * allocate. So entry size is a minimum allocation portion in the hash containers, created by this + * builder. E. g. if entry size is 100, the created container could only allocate 100, 200, 300... bytes + * for an entry. If say 150 bytes of entry space are required by the entry, 200 bytes will be allocated, + * 150 used and 50 wasted. To minimize memory overuse and improve speed, you should pay decent attention + * to this configuration. * - *

There are three major patterns of this configuration usage: - *

    - *
  1. Key (and value, in {@code ChronicleMap} case) sizes are constant. Configure them - * via {@link #constantKeySizeBySample(Object)} and - * {@link ChronicleMapBuilder#constantValueSizeBySample(Object)} methods, and you will - * experience no memory waste at all.
  2. - *
  3. Key (and/or value size, in {@code ChronicleMap} case) varies moderately. Specify them - * using corresponding methods, or specify entry size directly by calling this method, - * by sizes somewhere between average and maximum possible. The idea is to have most - * (90% or more) entries to fit a single "entry size" with moderate memory waste - * (10-20% on average), rest 10% or less of entries should take 2 "entry sizes", thus with - * ~50% memory overuse.
  4. - *
  5. Key (and/or value size, in {@code ChronicleMap} case) varies in a wide range. - * Then it's best to use entry size configuration in chunk mode. Specify entry size - * so that most entries should take from 5 to several dozens of "chunks". With this - * approach, average memory waste should be very low. + *

    There are three major patterns of this configuration usage:

    1. Key (and value, in {@code + * ChronicleMap} case) sizes are constant. Configure them via {@link #constantKeySizeBySample(Object)} and + * {@link ChronicleMapBuilder#constantValueSizeBySample(Object)} methods, and you will experience no + * memory waste at all.
    2. Key (and/or value size, in {@code ChronicleMap} case) varies moderately. + * Specify them using corresponding methods, or specify entry size directly by calling this method, by + * sizes somewhere between average and maximum possible. The idea is to have most (90% or more) entries to + * fit a single "entry size" with moderate memory waste (10-20% on average), rest 10% or less of entries + * should take 2 "entry sizes", thus with ~50% memory overuse.
    3. Key (and/or value size, in {@code + * ChronicleMap} case) varies in a wide range. Then it's best to use entry size configuration in chunk + * mode. Specify entry size so that most entries should take from 5 to several dozens of "chunks". + * With this approach, average memory waste should be very low. * - *

      However, remember that - *

        - *
      • Operations with entries that span several "entry sizes" are a bit slower, than - * entries which take a single "entry size". That is why "chunk" approach - * is not recommended, when key (and/or value size) varies moderately.
      • - *
      • The maximum number of chunks could be taken by an entry is 64. - * {@link IllegalArgumentException} is thrown on attempt to insert too large entry, - * compared to the configured or computed entry size.
      • - *
      • The number of "entries" {@linkplain #entries(long) in the whole map} and - * {@linkplain #actualEntriesPerSegment(int) per segment} is actually the number - * "chunks".
      • - *
      + *

      However, remember that

      • Operations with entries that span several "entry sizes" are a bit + * slower, than entries which take a single "entry size". That is why "chunk" approach is not recommended, + * when key (and/or value size) varies moderately.
      • The maximum number of chunks could be taken by + * an entry is 64. {@link IllegalArgumentException} is thrown on attempt to insert too large entry, + * compared to the configured or computed entry size.
      • The number of "entries" {@linkplain + * #entries(long) in the whole map} and {@linkplain #actualEntriesPerSegment(int) per segment} is actually + * the number "chunks".
      * - *

      Example: if values in your {@code ChronicleMap} are adjacency lists of some social - * graph, where nodes are represented as {@code long} ids, and adjacency lists - * are serialized in efficient manner, for example as {@code long[]} arrays. Typical number - * of connections is 100-300, maximum is 3000. In this case entry size of - * 50 * (8 bytes for each id) = 400 bytes would be a good choice:

      {@code
      +     * 

      Example: if values in your {@code ChronicleMap} are adjacency lists of some social graph, where + * nodes are represented as {@code long} ids, and adjacency lists are serialized in efficient manner, for + * example as {@code long[]} arrays. Typical number of connections is 100-300, maximum is 3000. In this + * case entry size of + * 50 * (8 bytes for each id) = 400 bytes would be a good choice:

      {@code
            * Map socialGraph = ChronicleMapBuilder
            *     .of(Long.class, long[].class)
            *     // given that graph should have of 1 billion nodes, and 150 average adjacency list size
      @@ -131,11 +120,8 @@ public interface ChronicleHashBuilder
      -     *     It is minimum possible (because 3000 friends / 50 friends = 60 is close to 64 "max
      -     *     chunks by single entry" limit, and ensures moderate average memory overuse (not more
      -     *     than 20%).
      -     *     
    4. - *
    + * It is minimum possible (because 3000 friends / 50 friends = 60 is close to 64 "max chunks by single + * entry" limit, and ensures moderate average memory overuse (not more than 20%).
* * @param entrySize the "chunk size" in bytes * @return this builder back @@ -144,35 +130,28 @@ public interface ChronicleHashBuilder - *
  • If key size (and value size, in {@code ChronicleMap} case) is constant, this number - * is equal to the maximum number of entries (because each entry takes exactly one - * "entry size" memory unit).
  • - *
  • If key (and/or value, in {@code ChronicleMap} case) size varies moderately, you - * should pass to this method the maximum number of entries + 5-25%, depending on your data - * properties and configured {@linkplain #keySize(int) key}/{@linkplain - * ChronicleMapBuilder#valueSize(int) value}/{@linkplain #entrySize(int) entry} sizes.
  • - *
  • If your data size varies in a wide range, pass the maximum number of entries - * multiplied by average data size and divided by the configured "entry size" (i. e. chunk - * size). See an example in the documentation to {@link #entrySize(int)} method.
  • - * + *
    1. If key size (and value size, in {@code ChronicleMap} case) is constant, this number is equal + * to the maximum number of entries (because each entry takes exactly one "entry size" memory unit).
    2. + *
    3. If key (and/or value, in {@code ChronicleMap} case) size varies moderately, you should pass to this + * method the maximum number of entries + 5-25%, depending on your data properties and configured + * {@linkplain #keySize(int) key}/{@linkplain ChronicleMapBuilder#valueSize(int) value}/{@linkplain + * #entrySize(int) entry} sizes.
    4. If your data size varies in a wide range, pass the maximum + * number of entries multiplied by average data size and divided by the configured "entry size" (i. e. + * chunk size). See an example in the documentation to {@link #entrySize(int)} method.
    * - *

    You shouldn't put additional margin over the number, computed according the rules above. - * This bad practice was popularized by {@link HashMap#HashMap(int)} - * and {@link HashSet#HashSet(int)} constructors, which accept "capacity", that should - * be multiplied by "load factor" to obtain actual maximum expected number of entries. - * {@code ChronicleMap} and {@code ChronicleSet} don't have a notion of load factor. + *

    You shouldn't put additional margin over the number, computed according the rules above. This bad + * practice was popularized by {@link HashMap#HashMap(int)} and {@link HashSet#HashSet(int)} constructors, + * which accept "capacity", that should be multiplied by "load factor" to obtain actual maximum expected + * number of entries. {@code ChronicleMap} and {@code ChronicleSet} don't have a notion of load factor. * *

    Default value is 2^20 (~ 1 million). * - * @param entries maximum size of the created maps, in memory allocation units, - * so-called "entry size" + * @param entries maximum size of the created maps, in memory allocation units, so-called "entry size" * @return this builder back * @see #entrySize(int) */ @@ -192,6 +171,8 @@ public interface ChronicleHashBuilder keyMarshaller); /** - * Specifies that key objects, queried with the hash containers, created by this builder, - * are inherently immutable. Keys in {@link ChronicleMap} or {@link ChronicleSet} - * are not required to be immutable, as in ordinary {@link Map} or {@link Set} implementations, - * because they are serialized off-heap. However, {@code ChronicleMap} and {@code ChronicleSet} - * implementations can benefit from the knowledge that keys are not mutated between queries. + * Specifies that key objects, queried with the hash containers, created by this builder, are inherently + * immutable. Keys in {@link ChronicleMap} or {@link ChronicleSet} are not required to be immutable, as in + * ordinary {@link Map} or {@link Set} implementations, because they are serialized off-heap. However, + * {@code ChronicleMap} and {@code ChronicleSet} implementations can benefit from the knowledge that keys + * are not mutated between queries. * - *

    By default, {@code ChronicleHashBuilder}s detects immutability automatically - * only for very few standard JDK types (for example, for {@link String}), it is not recommended - * to rely on {@code ChronicleHashBuilder} to be smart enough about this. + *

    By default, {@code ChronicleHashBuilder}s detects immutability automatically only for very few + * standard JDK types (for example, for {@link String}), it is not recommended to rely on {@code + * ChronicleHashBuilder} to be smart enough about this. * * @return this builder back */ diff --git a/src/main/java/net/openhft/chronicle/set/ChronicleSetBuilder.java b/src/main/java/net/openhft/chronicle/set/ChronicleSetBuilder.java index 1e5eb3310..1b2571609 100644 --- a/src/main/java/net/openhft/chronicle/set/ChronicleSetBuilder.java +++ b/src/main/java/net/openhft/chronicle/set/ChronicleSetBuilder.java @@ -169,6 +169,12 @@ public class ChronicleSetBuilder return this; } + @Override + public ChronicleSetBuilder channel(ChannelProvider.ChronicleChannel chronicleChannel) { + chronicleMapBuilder.channel(chronicleChannel); + return this; + } + @Override public String toString() {