package net.openhft.chronicle; import net.openhft.chronicle.map.ChannelProvider; import net.openhft.chronicle.map.ChronicleMap; import net.openhft.chronicle.map.ChronicleMapBuilder; import net.openhft.chronicle.map.ReplicationConfig; import net.openhft.chronicle.set.ChronicleSet; import net.openhft.chronicle.set.ChronicleSetBuilder; import net.openhft.lang.io.serialization.BytesMarshaller; import net.openhft.lang.io.serialization.BytesMarshallerFactory; import net.openhft.lang.io.serialization.ObjectSerializer; import net.openhft.lang.model.Byteable; import org.jetbrains.annotations.NotNull; import java.io.File; import java.io.IOException; import java.util.HashMap; import java.util.HashSet; import java.util.Map; import java.util.Set; import java.util.concurrent.ConcurrentHashMap; import java.util.concurrent.TimeUnit; /** * This interface defines the meaning of configurations, common to {@link ChronicleMapBuilder} and {@link * ChronicleSetBuilder}, i. e. Chronicle hash container configurations. * * @param the type of keys in hash containers, created by this builder * @param the container type, created by this builder, i. e. {@link ChronicleMap} or {@link ChronicleSet} * @param the concrete builder type, i. e. {@link ChronicleMapBuilder} or {@link ChronicleSetBuilder} */ public interface ChronicleHashBuilder> extends Cloneable { B clone(); /** * Set minimum number of segments. See concurrencyLevel in {@link ConcurrentHashMap}. * * @param minSegments the minimum number of segments in containers, constructed by this builder * @return this builder object back */ B minSegments(int minSegments); /** * Configures the optimal number of bytes, taken by serialized form of keys, put into hash containers, * created by this builder. If key size is always the same, call {@link #constantKeySizeBySample(Object)} * method instead of this one. * *

If key size varies moderately, specify the size higher than average, but lower than the maximum * possible, to minimize average memory overuse. If key size varies in a wide range, it's better to use * {@linkplain #entrySize(int) entry size} in "chunk" mode and configure it directly. * * @param keySize number of bytes, taken by serialized form of keys * @return this builder back * @see #constantKeySizeBySample(Object) * @see #entrySize(int) */ B keySize(int keySize); /** * Configures the constant number of bytes, taken by serialized form of keys, put into hash containers, * created by this builder. This is done by providing the {@code sampleKey}, all keys should take the same * number of bytes in serialized form, as this sample object. * *

If keys are of boxed primitive type or {@link Byteable} subclass, i. e. if key size is known * statically, it is automatically accounted and this method shouldn't be called. * *

If key size varies, method {@link #keySize(int)} or {@link #entrySize(int)} should be called instead * of this one. * * @param sampleKey the sample key * @return this builder back * @see #keySize(int) */ B constantKeySizeBySample(K sampleKey); /** * Configures the size in bytes of allocation unit of hash container instances, created by this builder. * *

{@link ChronicleMap} and {@link ChronicleSet} store their data off-heap, so it is required to * serialize key (and values, in {@code ChronicleMap} case) (unless they are direct {@link Byteable} * instances). Serialized key bytes (+ serialized value bytes, in {@code ChronicleMap} case) + some * metadata bytes comprise "entry space", which {@code ChronicleMap} or {@code ChronicleSet} should * allocate. So entry size is a minimum allocation portion in the hash containers, created by this * builder. E. g. if entry size is 100, the created container could only allocate 100, 200, 300... bytes * for an entry. If say 150 bytes of entry space are required by the entry, 200 bytes will be allocated, * 150 used and 50 wasted. To minimize memory overuse and improve speed, you should pay decent attention * to this configuration. * *

There are three major patterns of this configuration usage:

  1. Key (and value, in {@code * ChronicleMap} case) sizes are constant. Configure them via {@link #constantKeySizeBySample(Object)} and * {@link ChronicleMapBuilder#constantValueSizeBySample(Object)} methods, and you will experience no * memory waste at all.
  2. Key (and/or value size, in {@code ChronicleMap} case) varies moderately. * Specify them using corresponding methods, or specify entry size directly by calling this method, by * sizes somewhere between average and maximum possible. The idea is to have most (90% or more) entries to * fit a single "entry size" with moderate memory waste (10-20% on average), rest 10% or less of entries * should take 2 "entry sizes", thus with ~50% memory overuse.
  3. Key (and/or value size, in {@code * ChronicleMap} case) varies in a wide range. Then it's best to use entry size configuration in chunk * mode. Specify entry size so that most entries should take from 5 to several dozens of "chunks". * With this approach, average memory waste should be very low. * *

    However, remember that

    • Operations with entries that span several "entry sizes" are a bit * slower, than entries which take a single "entry size". That is why "chunk" approach is not recommended, * when key (and/or value size) varies moderately.
    • The maximum number of chunks could be taken by * an entry is 64. {@link IllegalArgumentException} is thrown on attempt to insert too large entry, * compared to the configured or computed entry size.
    • The number of "entries" {@linkplain * #entries(long) in the whole map} and {@linkplain #actualEntriesPerSegment(int) per segment} is actually * the number "chunks".
    * *

    Example: if values in your {@code ChronicleMap} are adjacency lists of some social graph, where * nodes are represented as {@code long} ids, and adjacency lists are serialized in efficient manner, for * example as {@code long[]} arrays. Typical number of connections is 100-300, maximum is 3000. In this * case entry size of * 50 * (8 bytes for each id) = 400 bytes would be a good choice:

    {@code
         * Map socialGraph = ChronicleMapBuilder
         *     .of(Long.class, long[].class)
         *     // given that graph should have of 1 billion nodes, and 150 average adjacency list size
         *     // => values takes 3 chuncks on average
         *     .entries(1_000_000_000L * (150 / 50))
         *     .entrySize(50 * 8)
         *     .create();}
    * It is minimum possible (because 3000 friends / 50 friends = 60 is close to 64 "max chunks by single * entry" limit, and ensures moderate average memory overuse (not more than 20%).
* * @param entrySize the "chunk size" in bytes * @return this builder back * @see #entries(long) */ B entrySize(int entrySize); /** * Configures the maximum number of {@linkplain #entrySize(int) "entry size chunks"}, which could be taken * by the maximum number of entries, inserted into the hash containers, created by this builder. If you * try to insert more data, {@link IllegalStateException} might be thrown, because currently {@link * ChronicleMap} and {@link ChronicleSet} doesn't support resizing. * *
  1. If key size (and value size, in {@code ChronicleMap} case) is constant, this number is equal * to the maximum number of entries (because each entry takes exactly one "entry size" memory unit).
  2. *
  3. If key (and/or value, in {@code ChronicleMap} case) size varies moderately, you should pass to this * method the maximum number of entries + 5-25%, depending on your data properties and configured * {@linkplain #keySize(int) key}/{@linkplain ChronicleMapBuilder#valueSize(int) value}/{@linkplain * #entrySize(int) entry} sizes.
  4. If your data size varies in a wide range, pass the maximum * number of entries multiplied by average data size and divided by the configured "entry size" (i. e. * chunk size). See an example in the documentation to {@link #entrySize(int)} method.
* *

You shouldn't put additional margin over the number, computed according the rules above. This bad * practice was popularized by {@link HashMap#HashMap(int)} and {@link HashSet#HashSet(int)} constructors, * which accept "capacity", that should be multiplied by "load factor" to obtain actual maximum expected * number of entries. {@code ChronicleMap} and {@code ChronicleSet} don't have a notion of load factor. * *

Default value is 2^20 (~ 1 million). * * @param entries maximum size of the created maps, in memory allocation units, so-called "entry size" * @return this builder back * @see #entrySize(int) */ B entries(long entries); B actualEntriesPerSegment(int actualEntriesPerSegment); B actualSegments(int actualSegments); B lockTimeOut(long lockTimeOut, TimeUnit unit); B errorListener(ChronicleHashErrorListener errorListener); B largeSegments(boolean largeSegments); B metaDataBytes(int metaDataBytes); B replicators(byte identifier, ReplicationConfig... replicators); B channel(ChannelProvider.ChronicleChannel chronicleChannel); B timeProvider(TimeProvider timeProvider); B bytesMarshallerFactory(BytesMarshallerFactory bytesMarshallerFactory); B objectSerializer(ObjectSerializer objectSerializer); B keyMarshaller(@NotNull BytesMarshaller keyMarshaller); /** * Specifies that key objects, queried with the hash containers, created by this builder, are inherently * immutable. Keys in {@link ChronicleMap} or {@link ChronicleSet} are not required to be immutable, as in * ordinary {@link Map} or {@link Set} implementations, because they are serialized off-heap. However, * {@code ChronicleMap} and {@code ChronicleSet} implementations can benefit from the knowledge that keys * are not mutated between queries. * *

By default, {@code ChronicleHashBuilder}s detects immutability automatically only for very few * standard JDK types (for example, for {@link String}), it is not recommended to rely on {@code * ChronicleHashBuilder} to be smart enough about this. * * @return this builder back */ B immutableKeys(); C create(File file) throws IOException; C create() throws IOException; }