diff --git a/spec/0_purposes.md b/spec/0-purposes.md similarity index 100% rename from spec/0_purposes.md rename to spec/0-purposes.md diff --git a/spec/1_design_goals.md b/spec/1-design-goals.md similarity index 95% rename from spec/1_design_goals.md rename to spec/1-design-goals.md index 422917cd8..0a026d12a 100644 --- a/spec/1_design_goals.md +++ b/spec/1-design-goals.md @@ -3,7 +3,7 @@ - Chronicle Map is an in-memory, highly concurrent key-value store. Generally keys and values are arbitrary sequences of bytes. - Chronicle Map optionally persists to a *single file* via the memory-mapping facility, present in - Windows and POSIX-compatible operation systems (`mmap`). The file shouldn't necessarily materialize + Windows and POSIX-compatible operating systems (`mmap`). The file shouldn't necessarily materialize on disk. It's OK for Chronicle Map if the file resides a memory-mounted FS. The whole Chronicle Map state is contained in the file contents i. e. it is possible to move, copy or send the file to another machine, access it and observe exactly the same Chronicle Map state. Chronicle Map doesn't @@ -46,9 +46,9 @@ The two above points are true for CPUs with x86 and x86_64 architectures. #### Memory-mapping - - The memory-mapping implementation shouldn't corrupt the mapped file, even if operation system + - The memory-mapping implementation shouldn't corrupt the mapped file, even if operating system execution was interrupted in any way (black out, virtual machine crash, etc.) It means Chronicle - Map expects some memory might be "stale", i. e. the bits written shortly before operation system + Map expects some memory might be "stale", i. e. the bits written shortly before operating system failure might not be persisted, but it doesn't expect to read the bit values that have never been written to the memory. - Writes to disk should be at least 4 or 8 bytes atomic, i. e. either aligned 4- or 8-byte blocks @@ -80,11 +80,11 @@ If the above assumptions are met, Chronicle Map aims to satisfy the following gu Map could be observed. Reading corrupted or half written values by some keys, or observing some keys which were never stored during the iteration is disallowed. -If Chronicle Map is persisted to a file and the operation system fails to flush all dirty memory to +If Chronicle Map is persisted to a file and the operating system fails to flush all dirty memory to disk due to a power-off or any other failure, when the file is mapped to memory and accessed again, it might be needed to perform a special recovery procedure on the store first, which identifies and -purges corrupted entries from the Chronicle Map. Therefore, some entries updated shortly before the -failure could be lost. +purges corrupted entries from the Chronicle Map. Therefore, *some entries updated shortly before the +failure could be lost.* ## Goals diff --git a/spec/2-design-overview.md b/spec/2-design-overview.md new file mode 100644 index 000000000..5ac16076c --- /dev/null +++ b/spec/2-design-overview.md @@ -0,0 +1,158 @@ +# Chronicle Map Design Overview + +## Logic + +**Chronicle Map is split into N *completely independent*, ordered segments**. The number of segments +is chosen during Chronicle Map creation and is never changed. Each segment has an associated 3-level +lock (read, *update* and write levels). Typical steps of performing a query to a Chronicle Map: + + 1. Compute the hash code of the queried key. Hash code size is 64 bits. + 2. Choose the segment that should hold the key, by the hash code + 3. Acquire the segment lock on the needed level on + 4. Search for the entry with the queried key in the segment + 5. Perform the query operation on the entry (read the value, update the value, etc.) + 6. Release the segment lock + +For multi-key queries, + + 1. All segments in which the involved keys should be held are identified + 2. Acquire the locks of all involved segments. Within each involved Chronicle Map, segment locks + are acquired in the order of their segments. + + > Acquiring segment locks in the predictable order is needed to avoid dead-locks, e. g. when the + > first thread acquires lock of the segment #1 and then tries to acquire the lock of the + > segment #2, and the second thread does the opposite: locks the segment #2, then tries to lock the + > segment #1. + + 3. Perform the query operations + 4. Release the locks of all involved segment in reverse order + +Bulk operations (iterations, background tasks) visit each segment once in any order. For each +segment, they acquire the segment lock, then visit each entry stored in the segment, then release +the segment lock. + +### Lock levels + +**Read level** is shared, several threads and processes could hold the segment lock on read level at +the same time. This level is used for read queries on a single or multiple keys, like simple value +reading by the key. This level couldn't be upgraded to update or write level. + +> Read level couldn't upgrade to more exclusive levels, because this is dead-lock prone. Two threads +> could acquire the same lock on read level, then both try upgrade to write level - and blocked by +> each other. (Upgrading to write level through update level wouldn't eliminate this issue too.) + +**Update level** is partially exclusive: a thread could hold the segment lock on update level while +some other threads or processes are holding the same lock on read level, but not update or write +level. This level is used for "typically read" queries, that occasionally should be able to write +hence upgrade to write level without releasing the lock, for example reading value by the key, if +the entry is present or storing a new value, if the entry is absent. Also, bulk operations should +acquire segment locks at least on update level, even if they never perform writes. Lock on update +level could be upgraded to write level. + +**Write level** is fully exclusive: when a thread holds the segment lock on write level, no other +thread or process could hold the same lock on any level. Write level is used for write queries. + +## Lock implementation + +Lock is represented by two 32-bit words in Chronicle Map memory. *Count word* holds the counts of +threads holding the lock at the moment on read, update and write levels. *Wait word* holds the +number of pending threads, trying to acquire the lock on write level, at the moment. These two words +are updated only via compare-and-swap operations. In addition to the checks essential for keeping +exclusiveness invariants, described in the previous section, when a thread tries to acquire the lock +on read level, it should check there are no threads pending for write access, to prevent starvation +of the latter. + +As Chronicle Map is designed for inter-process communication and it doesn't use operating system +level locks, it doesn't have queues of threads, waiting for lock acquisition and conditional +wake-ups. Threads should try to acquire a lock in a spin loop, probably yielding and/or sleeping +between the attempts, after certain numbers of unsuccessful attempts. + +> Chronicle Map implementations specialized for single-process access may implement thread some kind +> of thread queues and wake-ups on their runtime level; certain lock acquisition strategy is out of +> scope of the Chronicle Map data structure specification. + +## The segment data structure + +Under normal conditions, a segment is represented by a single *tier*. + +### Segment tier + +Segment tier basically consists of two parts: + + 1. **Hash lookup**, a flat, power-of-two sized open addressing hash table with collision resolution + via linear probing, in which the role of keys is played by parts of the Chronicle Map's key hash + codes (different part of the hash code, from what is used for choosing the segment), the values in + this hash table are allocation identifiers within the segment tier's *entry space*. + + > In the reference Chronicle Map implementation hash lookup table size is chosen in order to keep + > table load factor between 0.33 and 0.66. + + 2. **Entry space** is the area to allocate blocks of memory to store the actual Chronicle Map's + entries (key and value bytes sequences), along with some metadata. + + > The specific memory allocation algorithm may vary, depending on characteristics of the keys + > and values stored in the specific Chronicle Map, time-space tradeoff chosen, etc. One important + > property that the allocator should likely obey, is *not* requiring to touch all it's memory up + > front, for example zeroing out or splitting the memory in blocks, going though all of them and + > creating an intrusive linked list. Touching entry space memory on demand takes advantage from + > the Linux memory mapping feature - lazy page allocation. This, in turn, allows to overcommit + > memory for segment tier's entry spaces without sacrificing actual memory usage much (because + > hash lookup slots, that should be overcommited too, are relatively small). This, in turn, + > mitigates variance of segments' filling rates and makes slow *tier chaining* (see below) very + > unlikely. + + > The reference Chronicle Map implementation uses the following allocation algorithm: it is + > initialized with the *chunk size* depending of the configured properties of keys and values, + > which are going to be stored in the Chronicle Map, virtually splits the entry space into M + > chunks, and a bit set, where each bit corresponds to a single chunk of the same index. Allocation + > procedure computes the number of chunks that cover the requested allocation size and finds the + > same number of continuous zero bits in the bit set. Allocation identifier is the index of the + > first chunk of the allocated block. + +Hash lookup slots' size is either 4 or 8 bytes. Allocation identifiers use the minimum required bits +to identify themselves within the entry space, the rest bits of hash lookup slots are used to store +bits of the Chronicle Map's key hash codes. + +### Tier chaining + +If segment tier is filled up, i. e. on some entry insertion request entry space fails to allocate +a memory block sufficient to place the new entry, *a whole new segment tier is allocated and chained +after the previous tier.* All tiers, either first in their segments or chained, are identical. + +> A segment with one or several chained tiers might be much slower for key search, than a "normal" +> segment with the single tier. Tier chaining is an *exceptional* mechanism for dealing with segment +> overflow. Chronicle Map shows it's efficiency properties when a very little fraction or none of +> it's segments has chained tiers. Chronicle Map should be configured accordingly. + +### Procedure of searching for a key within a segment + +
    +
  1. Extract the part of the queried key's hash code, that should serve as a key in hash lookup + tables.
    2. + +
  2. [For a segment tier, the first tier initially] + +

    Search for the slot with the extracted hash lookup key in the tier's hash lookup, using + standard search algorithm for hash tables with open addressing and collision resolution via linear + probing. + +

      +
    1. If such slot is found, go to the entry location within the tier's entry space using the value + in the found hash lookup slot. Compare the queried key with the key, stored in the entry + location, byte-by-byte.
      2. + +
        +
      1. If keys are identical, the entry for the queried within the segment is found, exit the + procedure.
        2. + +
      2. If keys are not identical (it means parts of their hash codes collide), continue the search + in the current tier's hash lookup table (step #2), without resetting the slot index.
        3. +
      + +
    2. If slot is not found, and there are no more chained tiers, the entry for the queried key + within the segment is not found, exit the procedure. + +

      If there is a chained tier, go to it and start searching again from step #2.

      3. +
    +
    3. +
diff --git a/spec/2_design_overview.md b/spec/2_design_overview.md deleted file mode 100644 index d5f43f5a2..000000000 --- a/spec/2_design_overview.md +++ /dev/null @@ -1,70 +0,0 @@ -# Chronicle Map Design Overview - -## Logic - -Chronicle Map is split into N *completely independent*, ordered segments. The number of segments is -chosen during Chronicle Map creation and is never changed. Each segment has an associated 3-level -lock (read, *update* and write levels). Typical steps of performing a query to a Chronicle Map: - - 1. Compute the hash code of the queried key - 2. Choose the segment that should hold the key, by the hash code - 3. Acquire the segment lock on the needed level on - 4. Search for the entry with the queried key in the segment - 5. Perform the query operation on the entry (read the value, update the value, etc.) - 6. Release the segment lock - -For multi-key queries, - - 1. All segments in which the involved keys should be held are identified - 2. Acquire the locks of all involved segments. Within each involved Chronicle Map segment locks - are acquired in the order of their segments. - - > Acquiring segment locks in the predictable order is needed to avoid dead-locks, e. g. when first - > thread acquires lock of the segment #1 and then tries to acquire the lock of the segment #2, and - > the second thread does the opposite: locks the segment #2, then tries to lock the segment #1. - - 3. Perform the query operations - 4. Release the locks of all involved segment in reverse order - -Bulk operations (iterations, background tasks) visit each segment once in any order. For each -segment, they acquire the segment lock, then visit each entry stored in the segment, then release -the segment lock. - -### Lock levels - -**Read level** is shared, several threads and processes could hold the segment lock on read level at -the same time. This level is used for read queries on a single or multiple keys, like simple value -reading by the key. This level couldn't be upgraded to update or write level. - -> Read level couldn't upgrade to more exclusive levels, because this is dead-lock prone. Two threads -> could acquire the same lock on read level, then both try upgrade to write level - and blocked by -> each other. (Upgrading to write level through update level wouldn't eliminate this issue too.) - -**Update level** is partially exclusive: a thread could hold the segment lock on update level while -some other threads or processes are holding the same lock on read level, but not update or write -level. This level is used for "typically read" queries, that occasionally should be able to write -hence upgrade to write level without releasing the lock, for example reading value by the key, if -the entry is present or storing a new value, if the entry is absent. Also, bulk operations should -acquire segment locks at least on update level, even if they never perform writes. Lock on update -level could be upgraded to write level. - -**Write level** is fully exclusive: when a thread holds the segment lock on write level, no other -thread or process could hold the same lock on any level. Write level is used for write queries. - -## Lock implementation - -Lock is represented by two 32-bit words in Chronicle Map memory. *Count word* holds the the counts -of threads holding the lock at the moment on read, update and write levels. *Wait word* holds the -number of pending threads, trying to acquire the lock on write level, at the moment. These two words -are updated only via compare-and-swap operations. In addition to the checks essential for keeping -exclusiveness invariants, described in the previous section, when a thread tries to acquire the lock -on read level, it should check there are no threads pending for write access, to prevent starvation -of the latter. - -## The segment data structure - -Under normal conditions, a segment is represented by a single *tier*. - -Segment tier basically consists of two parts: *hash lookup*, a flat, power-of-two sized open -addressing hash table - diff --git a/spec/3-memory-layout.md b/spec/3-memory-layout.md new file mode 100644 index 000000000..ed7050198 --- /dev/null +++ b/spec/3-memory-layout.md @@ -0,0 +1,54 @@ +# Chronicle Map Memory Layout + +Newly created Chronicle Map (or just mapped from an already existing persistence file) takes one big +continuous block of memory. It's structure, from lower addresses to higher: + + 1. Self-bootstrapping header + 2. Global mutable state + 3. Alignment to the next page boundary by addresses (page size of the current memory mapping), + if a new Chronicle Map is created, if the existing one (e. g. persisted) is loaded, the alignment + is specified in the global mutable state (however, specifically this field of global mutable state + is actually immutable). + + > The purpose of this alignment is to minimize the number of pages spanned by the following segment + > headers area. Segment headers area is frequently accessed and updated, so we want to minimize the + > number of TLB cache entries it almost constantly reside and the number of pages to flush to disk, + > if the Chronicle Map is persisted. + + > Need to store the alignment, because other accessing processes and/or implementations of + > Chronicle Map may use different page size for memory mapping. + + 4. Segment headers area + 5. Main segments area + 6. Zero to several *extra tier bulks*. If the Chronicle Map is persisted to a file, each extra tier + bulk is pre-aligned to the granularity of offsets in files supported by memory mapping facility in + the operating system running at the moment of extra tier bulk allocation, then these alignments are + stored for each extra tier bulk in the global mutable state. + + > The reason to make alignment before each extra tier bulk is different from that for alignment + > before segment headers area. Extra tier bulks are allocated in runtime, there are zero bulks in + > a newly created Chronicle Map. If the Chronicle Map is persisted, it should extend the persisted + > file and map the newly allocated file space into memory. In processes which will load this + > Chronicle Map later the already existing Chronicle Map space and the new extra tier bulk will + > be served by a single memory mapping, hence continuous range of addresses. But in the processes + > *currently working* with this Chronicle Map (including the process triggered extra tier bulk + > allocation itself), i. e. already using a *shorter* memory-mapping, this new extra tier bulk + > should be loaded as a *separate* memory mapping. This separate memory mapping with offset in + > the persistence file, should have the offset aligned accordingly. + + > Granularity of offsets in files, supported by memory mapping facility is equal to *native* page + > size in Linux, i. e. most likely 4KB, and "file allocation granularity" in Windows, i. e. most + > likely 64KB. + +## Self-Bootstrapping Header + +It's structure and initialization order is described in [Self Bootstrapping Data 1.0]( +https://github.com/OpenHFT/RFC/blob/master/Self-Bootstrapping-Data/Self-Bootstraping-Data-0.1.md) +specification. It could be in Binary Wire or Text Wire formats (see details in the Self +Bootstrapping Data specification). Once created, this header is never changed. It contains all +configurations, immutable for Chronicle Map during the instance "lifetime": number of segments, +various sizes, offsets, etc. See the specification of the fields on [Map Header +Fields](3_1-map-header-fields.md) page. + +## Global Mutable State + diff --git a/spec/3_1-map-header-fields.md b/spec/3_1-map-header-fields.md new file mode 100644 index 000000000..e69de29bb diff --git a/spec/README.md b/spec/README.md index 0712f4f7a..fdcd8a16b 100644 --- a/spec/README.md +++ b/spec/README.md @@ -4,6 +4,7 @@ This directory contains the Chronicle Map specification. This is currently work-in-progress. - - [Rational behind Chronicle Map Specification](0_purposes.md) - - [Chronicle Map Design Goals, Assumptions and Guarantees](1_design_goals.md) + - [Rational behind this Specification](0_purposes.md) + - [Design Goals, Assumptions and Guarantees](1_design_goals.md) - [Design overview](2_design_overview.md) + - [Memory layout](3-memory-layout.md) diff --git a/src/main/java/net/openhft/chronicle/hash/ChronicleHash.java b/src/main/java/net/openhft/chronicle/hash/ChronicleHash.java index d45a569fb..48674a45f 100644 --- a/src/main/java/net/openhft/chronicle/hash/ChronicleHash.java +++ b/src/main/java/net/openhft/chronicle/hash/ChronicleHash.java @@ -130,7 +130,7 @@ public interface ChronicleHash, SC extends HashSegment * Releases the off-heap memory, used by this hash container and resources, used by replication, * if any. However, if hash container (hence off-heap memory, used by it) is mapped to the file * and there are other instances mapping the same data on the server across JVMs, the memory - * won't be actually freed on operation system level. I. e. this method call doesn't affect + * won't be actually freed on operating system level. I. e. this method call doesn't affect * other {@link ChronicleMap} or {@link ChronicleSet} instances mapping the same data. * *

If you won't call this method, memory would be held at least until next garbage