mirror of
https://github.com/OpenHFT/Chronicle-Map.git
synced 2021-09-13 01:26:12 +00:00
Complete Design Overview spec page and start Memory Layout; operation system -> operating system
This commit is contained in:
leventov
@@ -3,7 +3,7 @@
|
||||
- Chronicle Map is an in-memory, highly concurrent key-value store. Generally keys and values are
|
||||
arbitrary sequences of bytes.
|
||||
- Chronicle Map optionally persists to a *single file* via the memory-mapping facility, present in
|
||||
Windows and POSIX-compatible operation systems (`mmap`). The file shouldn't necessarily materialize
|
||||
Windows and POSIX-compatible operating systems (`mmap`). The file shouldn't necessarily materialize
|
||||
on disk. It's OK for Chronicle Map if the file resides a memory-mounted FS. The whole Chronicle Map
|
||||
state is contained in the file contents i. e. it is possible to move, copy or send the file to
|
||||
another machine, access it and observe exactly the same Chronicle Map state. Chronicle Map doesn't
|
||||
@@ -46,9 +46,9 @@ The two above points are true for CPUs with x86 and x86_64 architectures.
|
||||
|
||||
#### Memory-mapping
|
||||
|
||||
- The memory-mapping implementation shouldn't corrupt the mapped file, even if operation system
|
||||
- The memory-mapping implementation shouldn't corrupt the mapped file, even if operating system
|
||||
execution was interrupted in any way (black out, virtual machine crash, etc.) It means Chronicle
|
||||
Map expects some memory might be "stale", i. e. the bits written shortly before operation system
|
||||
Map expects some memory might be "stale", i. e. the bits written shortly before operating system
|
||||
failure might not be persisted, but it doesn't expect to read the bit values that have never been
|
||||
written to the memory.
|
||||
- Writes to disk should be at least 4 or 8 bytes atomic, i. e. either aligned 4- or 8-byte blocks
|
||||
@@ -80,11 +80,11 @@ If the above assumptions are met, Chronicle Map aims to satisfy the following gu
|
||||
Map could be observed. Reading corrupted or half written values by some keys, or observing some
|
||||
keys which were never stored during the iteration is disallowed.
|
||||
|
||||
If Chronicle Map is persisted to a file and the operation system fails to flush all dirty memory to
|
||||
If Chronicle Map is persisted to a file and the operating system fails to flush all dirty memory to
|
||||
disk due to a power-off or any other failure, when the file is mapped to memory and accessed again,
|
||||
it might be needed to perform a special recovery procedure on the store first, which identifies and
|
||||
purges corrupted entries from the Chronicle Map. Therefore, some entries updated shortly before the
|
||||
failure could be lost.
|
||||
purges corrupted entries from the Chronicle Map. Therefore, *some entries updated shortly before the
|
||||
failure could be lost.*
|
||||
|
||||
## Goals
|
||||
|
||||
@@ -0,0 +1,158 @@
|
||||
# Chronicle Map Design Overview
|
||||
|
||||
## Logic
|
||||
|
||||
**Chronicle Map is split into N *completely independent*, ordered segments**. The number of segments
|
||||
is chosen during Chronicle Map creation and is never changed. Each segment has an associated 3-level
|
||||
lock (read, *update* and write levels). Typical steps of performing a query to a Chronicle Map:
|
||||
|
||||
1. Compute the hash code of the queried key. Hash code size is 64 bits.
|
||||
2. Choose the segment that should hold the key, by the hash code
|
||||
3. Acquire the segment lock on the needed level on
|
||||
4. Search for the entry with the queried key in the segment
|
||||
5. Perform the query operation on the entry (read the value, update the value, etc.)
|
||||
6. Release the segment lock
|
||||
|
||||
For multi-key queries,
|
||||
|
||||
1. All segments in which the involved keys should be held are identified
|
||||
2. Acquire the locks of all involved segments. Within each involved Chronicle Map, segment locks
|
||||
are acquired in the order of their segments.
|
||||
|
||||
> Acquiring segment locks in the predictable order is needed to avoid dead-locks, e. g. when the
|
||||
> first thread acquires lock of the segment #1 and then tries to acquire the lock of the
|
||||
> segment #2, and the second thread does the opposite: locks the segment #2, then tries to lock the
|
||||
> segment #1.
|
||||
|
||||
3. Perform the query operations
|
||||
4. Release the locks of all involved segment in reverse order
|
||||
|
||||
Bulk operations (iterations, background tasks) visit each segment once in any order. For each
|
||||
segment, they acquire the segment lock, then visit each entry stored in the segment, then release
|
||||
the segment lock.
|
||||
|
||||
### Lock levels
|
||||
|
||||
**Read level** is shared, several threads and processes could hold the segment lock on read level at
|
||||
the same time. This level is used for read queries on a single or multiple keys, like simple value
|
||||
reading by the key. This level couldn't be upgraded to update or write level.
|
||||
|
||||
> Read level couldn't upgrade to more exclusive levels, because this is dead-lock prone. Two threads
|
||||
> could acquire the same lock on read level, then both try upgrade to write level - and blocked by
|
||||
> each other. (Upgrading to write level through update level wouldn't eliminate this issue too.)
|
||||
|
||||
**Update level** is partially exclusive: a thread could hold the segment lock on update level while
|
||||
some other threads or processes are holding the same lock on read level, but not update or write
|
||||
level. This level is used for "typically read" queries, that occasionally should be able to write
|
||||
hence upgrade to write level without releasing the lock, for example reading value by the key, if
|
||||
the entry is present or storing a new value, if the entry is absent. Also, bulk operations should
|
||||
acquire segment locks at least on update level, even if they never perform writes. Lock on update
|
||||
level could be upgraded to write level.
|
||||
|
||||
**Write level** is fully exclusive: when a thread holds the segment lock on write level, no other
|
||||
thread or process could hold the same lock on any level. Write level is used for write queries.
|
||||
|
||||
## Lock implementation
|
||||
|
||||
Lock is represented by two 32-bit words in Chronicle Map memory. *Count word* holds the counts of
|
||||
threads holding the lock at the moment on read, update and write levels. *Wait word* holds the
|
||||
number of pending threads, trying to acquire the lock on write level, at the moment. These two words
|
||||
are updated only via compare-and-swap operations. In addition to the checks essential for keeping
|
||||
exclusiveness invariants, described in the previous section, when a thread tries to acquire the lock
|
||||
on read level, it should check there are no threads pending for write access, to prevent starvation
|
||||
of the latter.
|
||||
|
||||
As Chronicle Map is designed for inter-process communication and it doesn't use operating system
|
||||
level locks, it doesn't have queues of threads, waiting for lock acquisition and conditional
|
||||
wake-ups. Threads should try to acquire a lock in a spin loop, probably yielding and/or sleeping
|
||||
between the attempts, after certain numbers of unsuccessful attempts.
|
||||
|
||||
> Chronicle Map implementations specialized for single-process access may implement thread some kind
|
||||
> of thread queues and wake-ups on their runtime level; certain lock acquisition strategy is out of
|
||||
> scope of the Chronicle Map data structure specification.
|
||||
|
||||
## The segment data structure
|
||||
|
||||
Under normal conditions, a segment is represented by a single *tier*.
|
||||
|
||||
### Segment tier
|
||||
|
||||
Segment tier basically consists of two parts:
|
||||
|
||||
1. **Hash lookup**, a flat, power-of-two sized open addressing hash table with collision resolution
|
||||
via linear probing, in which the role of keys is played by parts of the Chronicle Map's key hash
|
||||
codes (different part of the hash code, from what is used for choosing the segment), the values in
|
||||
this hash table are allocation identifiers within the segment tier's *entry space*.
|
||||
|
||||
> In the reference Chronicle Map implementation hash lookup table size is chosen in order to keep
|
||||
> table load factor between 0.33 and 0.66.
|
||||
|
||||
2. **Entry space** is the area to allocate blocks of memory to store the actual Chronicle Map's
|
||||
entries (key and value bytes sequences), along with some metadata.
|
||||
|
||||
> The specific memory allocation algorithm may vary, depending on characteristics of the keys
|
||||
> and values stored in the specific Chronicle Map, time-space tradeoff chosen, etc. One important
|
||||
> property that the allocator should likely obey, is *not* requiring to touch all it's memory up
|
||||
> front, for example zeroing out or splitting the memory in blocks, going though all of them and
|
||||
> creating an intrusive linked list. Touching entry space memory on demand takes advantage from
|
||||
> the Linux memory mapping feature - lazy page allocation. This, in turn, allows to overcommit
|
||||
> memory for segment tier's entry spaces without sacrificing actual memory usage much (because
|
||||
> hash lookup slots, that should be overcommited too, are relatively small). This, in turn,
|
||||
> mitigates variance of segments' filling rates and makes slow *tier chaining* (see below) very
|
||||
> unlikely.
|
||||
|
||||
> The reference Chronicle Map implementation uses the following allocation algorithm: it is
|
||||
> initialized with the *chunk size* depending of the configured properties of keys and values,
|
||||
> which are going to be stored in the Chronicle Map, virtually splits the entry space into M
|
||||
> chunks, and a bit set, where each bit corresponds to a single chunk of the same index. Allocation
|
||||
> procedure computes the number of chunks that cover the requested allocation size and finds the
|
||||
> same number of continuous zero bits in the bit set. Allocation identifier is the index of the
|
||||
> first chunk of the allocated block.
|
||||
|
||||
Hash lookup slots' size is either 4 or 8 bytes. Allocation identifiers use the minimum required bits
|
||||
to identify themselves within the entry space, the rest bits of hash lookup slots are used to store
|
||||
bits of the Chronicle Map's key hash codes.
|
||||
|
||||
### Tier chaining
|
||||
|
||||
If segment tier is filled up, i. e. on some entry insertion request entry space fails to allocate
|
||||
a memory block sufficient to place the new entry, *a whole new segment tier is allocated and chained
|
||||
after the previous tier.* All tiers, either first in their segments or chained, are identical.
|
||||
|
||||
> A segment with one or several chained tiers might be much slower for key search, than a "normal"
|
||||
> segment with the single tier. Tier chaining is an *exceptional* mechanism for dealing with segment
|
||||
> overflow. Chronicle Map shows it's efficiency properties when a very little fraction or none of
|
||||
> it's segments has chained tiers. Chronicle Map should be configured accordingly.
|
||||
|
||||
### Procedure of searching for a key within a segment
|
||||
|
||||
<ol>
|
||||
<li>Extract the part of the queried key's hash code, that should serve as a key in hash lookup
|
||||
tables.</li>
|
||||
|
||||
<li>[For a segment tier, the first tier initially]
|
||||
|
||||
<p>Search for the slot with the extracted hash lookup key in the tier's hash lookup, using
|
||||
standard search algorithm for hash tables with open addressing and collision resolution via linear
|
||||
probing.
|
||||
|
||||
<ol>
|
||||
<li>If such slot is found, go to the entry location within the tier's entry space using the value
|
||||
in the found hash lookup slot. Compare the queried key with the key, stored in the entry
|
||||
location, byte-by-byte.</li>
|
||||
|
||||
<ol>
|
||||
<li>If keys are identical, the entry for the queried within the segment is found, exit the
|
||||
procedure.</li>
|
||||
|
||||
<li>If keys are not identical (it means parts of their hash codes collide), continue the search
|
||||
in the current tier's hash lookup table (step #2), without resetting the slot index.</li>
|
||||
</ol>
|
||||
|
||||
<li>If slot is not found, and there are no more chained tiers, the entry for the queried key
|
||||
within the segment is not found, exit the procedure.
|
||||
|
||||
<p>If there is a chained tier, go to it and start searching again from step #2.</li>
|
||||
</ol>
|
||||
</li>
|
||||
</ol>
|
||||
@@ -1,70 +0,0 @@
|
||||
# Chronicle Map Design Overview
|
||||
|
||||
## Logic
|
||||
|
||||
Chronicle Map is split into N *completely independent*, ordered segments. The number of segments is
|
||||
chosen during Chronicle Map creation and is never changed. Each segment has an associated 3-level
|
||||
lock (read, *update* and write levels). Typical steps of performing a query to a Chronicle Map:
|
||||
|
||||
1. Compute the hash code of the queried key
|
||||
2. Choose the segment that should hold the key, by the hash code
|
||||
3. Acquire the segment lock on the needed level on
|
||||
4. Search for the entry with the queried key in the segment
|
||||
5. Perform the query operation on the entry (read the value, update the value, etc.)
|
||||
6. Release the segment lock
|
||||
|
||||
For multi-key queries,
|
||||
|
||||
1. All segments in which the involved keys should be held are identified
|
||||
2. Acquire the locks of all involved segments. Within each involved Chronicle Map segment locks
|
||||
are acquired in the order of their segments.
|
||||
|
||||
> Acquiring segment locks in the predictable order is needed to avoid dead-locks, e. g. when first
|
||||
> thread acquires lock of the segment #1 and then tries to acquire the lock of the segment #2, and
|
||||
> the second thread does the opposite: locks the segment #2, then tries to lock the segment #1.
|
||||
|
||||
3. Perform the query operations
|
||||
4. Release the locks of all involved segment in reverse order
|
||||
|
||||
Bulk operations (iterations, background tasks) visit each segment once in any order. For each
|
||||
segment, they acquire the segment lock, then visit each entry stored in the segment, then release
|
||||
the segment lock.
|
||||
|
||||
### Lock levels
|
||||
|
||||
**Read level** is shared, several threads and processes could hold the segment lock on read level at
|
||||
the same time. This level is used for read queries on a single or multiple keys, like simple value
|
||||
reading by the key. This level couldn't be upgraded to update or write level.
|
||||
|
||||
> Read level couldn't upgrade to more exclusive levels, because this is dead-lock prone. Two threads
|
||||
> could acquire the same lock on read level, then both try upgrade to write level - and blocked by
|
||||
> each other. (Upgrading to write level through update level wouldn't eliminate this issue too.)
|
||||
|
||||
**Update level** is partially exclusive: a thread could hold the segment lock on update level while
|
||||
some other threads or processes are holding the same lock on read level, but not update or write
|
||||
level. This level is used for "typically read" queries, that occasionally should be able to write
|
||||
hence upgrade to write level without releasing the lock, for example reading value by the key, if
|
||||
the entry is present or storing a new value, if the entry is absent. Also, bulk operations should
|
||||
acquire segment locks at least on update level, even if they never perform writes. Lock on update
|
||||
level could be upgraded to write level.
|
||||
|
||||
**Write level** is fully exclusive: when a thread holds the segment lock on write level, no other
|
||||
thread or process could hold the same lock on any level. Write level is used for write queries.
|
||||
|
||||
## Lock implementation
|
||||
|
||||
Lock is represented by two 32-bit words in Chronicle Map memory. *Count word* holds the the counts
|
||||
of threads holding the lock at the moment on read, update and write levels. *Wait word* holds the
|
||||
number of pending threads, trying to acquire the lock on write level, at the moment. These two words
|
||||
are updated only via compare-and-swap operations. In addition to the checks essential for keeping
|
||||
exclusiveness invariants, described in the previous section, when a thread tries to acquire the lock
|
||||
on read level, it should check there are no threads pending for write access, to prevent starvation
|
||||
of the latter.
|
||||
|
||||
## The segment data structure
|
||||
|
||||
Under normal conditions, a segment is represented by a single *tier*.
|
||||
|
||||
Segment tier basically consists of two parts: *hash lookup*, a flat, power-of-two sized open
|
||||
addressing hash table
|
||||
|
||||
@@ -0,0 +1,54 @@
|
||||
# Chronicle Map Memory Layout
|
||||
|
||||
Newly created Chronicle Map (or just mapped from an already existing persistence file) takes one big
|
||||
continuous block of memory. It's structure, from lower addresses to higher:
|
||||
|
||||
1. Self-bootstrapping header
|
||||
2. Global mutable state
|
||||
3. Alignment to the next page boundary by addresses (page size of the current memory mapping),
|
||||
if a new Chronicle Map is created, if the existing one (e. g. persisted) is loaded, the alignment
|
||||
is specified in the global mutable state (however, specifically this field of global mutable state
|
||||
is actually immutable).
|
||||
|
||||
> The purpose of this alignment is to minimize the number of pages spanned by the following segment
|
||||
> headers area. Segment headers area is frequently accessed and updated, so we want to minimize the
|
||||
> number of TLB cache entries it almost constantly reside and the number of pages to flush to disk,
|
||||
> if the Chronicle Map is persisted.
|
||||
|
||||
> Need to store the alignment, because other accessing processes and/or implementations of
|
||||
> Chronicle Map may use different page size for memory mapping.
|
||||
|
||||
4. Segment headers area
|
||||
5. Main segments area
|
||||
6. Zero to several *extra tier bulks*. If the Chronicle Map is persisted to a file, each extra tier
|
||||
bulk is pre-aligned to the granularity of offsets in files supported by memory mapping facility in
|
||||
the operating system running at the moment of extra tier bulk allocation, then these alignments are
|
||||
stored for each extra tier bulk in the global mutable state.
|
||||
|
||||
> The reason to make alignment before each extra tier bulk is different from that for alignment
|
||||
> before segment headers area. Extra tier bulks are allocated in runtime, there are zero bulks in
|
||||
> a newly created Chronicle Map. If the Chronicle Map is persisted, it should extend the persisted
|
||||
> file and map the newly allocated file space into memory. In processes which will load this
|
||||
> Chronicle Map later the already existing Chronicle Map space and the new extra tier bulk will
|
||||
> be served by a single memory mapping, hence continuous range of addresses. But in the processes
|
||||
> *currently working* with this Chronicle Map (including the process triggered extra tier bulk
|
||||
> allocation itself), i. e. already using a *shorter* memory-mapping, this new extra tier bulk
|
||||
> should be loaded as a *separate* memory mapping. This separate memory mapping with offset in
|
||||
> the persistence file, should have the offset aligned accordingly.
|
||||
|
||||
> Granularity of offsets in files, supported by memory mapping facility is equal to *native* page
|
||||
> size in Linux, i. e. most likely 4KB, and "file allocation granularity" in Windows, i. e. most
|
||||
> likely 64KB.
|
||||
|
||||
## Self-Bootstrapping Header
|
||||
|
||||
It's structure and initialization order is described in [Self Bootstrapping Data 1.0](
|
||||
https://github.com/OpenHFT/RFC/blob/master/Self-Bootstrapping-Data/Self-Bootstraping-Data-0.1.md)
|
||||
specification. It could be in Binary Wire or Text Wire formats (see details in the Self
|
||||
Bootstrapping Data specification). Once created, this header is never changed. It contains all
|
||||
configurations, immutable for Chronicle Map during the instance "lifetime": number of segments,
|
||||
various sizes, offsets, etc. See the specification of the fields on [Map Header
|
||||
Fields](3_1-map-header-fields.md) page.
|
||||
|
||||
## Global Mutable State
|
||||
|
||||
+3
-2
@@ -4,6 +4,7 @@ This directory contains the Chronicle Map specification.
|
||||
|
||||
This is currently work-in-progress.
|
||||
|
||||
- [Rational behind Chronicle Map Specification](0_purposes.md)
|
||||
- [Chronicle Map Design Goals, Assumptions and Guarantees](1_design_goals.md)
|
||||
- [Rational behind this Specification](0_purposes.md)
|
||||
- [Design Goals, Assumptions and Guarantees](1_design_goals.md)
|
||||
- [Design overview](2_design_overview.md)
|
||||
- [Memory layout](3-memory-layout.md)
|
||||
|
||||
@@ -130,7 +130,7 @@ public interface ChronicleHash<K, E extends HashEntry<K>, SC extends HashSegment
|
||||
* Releases the off-heap memory, used by this hash container and resources, used by replication,
|
||||
* if any. However, if hash container (hence off-heap memory, used by it) is mapped to the file
|
||||
* and there are other instances mapping the same data on the server across JVMs, the memory
|
||||
* won't be actually freed on operation system level. I. e. this method call doesn't affect
|
||||
* won't be actually freed on operating system level. I. e. this method call doesn't affect
|
||||
* other {@link ChronicleMap} or {@link ChronicleSet} instances mapping the same data.
|
||||
*
|
||||
* <p>If you won't call this method, memory would be held at least until next garbage
|
||||
|
||||
Reference in New Issue
Block a user