From 1382e08961296263e2ffc8ea608d548710a8b37d Mon Sep 17 00:00:00 2001 From: Per Minborg Date: Fri, 21 Aug 2020 10:17:44 +0200 Subject: [PATCH] Fix format and language issues in the specification --- ...s\\buddy\\dev\\shm\\OPERAND_CHRONICLE_MAP" | Bin 0 -> 12288 bytes ...y\\dev\\shm\\OPERAND_ChronicleStampedLock" | Bin 0 -> 8192 bytes ...\OPERAND_ChronicleStampedLock=ReaderCount" | Bin 0 -> 8192 bytes ...\OPERAND_ChronicleStampedLock=WriterCount" | Bin 0 -> 8192 bytes spec/1-design-goals.md | 8 ++-- spec/2-design-overview.md | 1 - spec/3-memory-layout.md | 35 ++++++++++-------- spec/3_2-lock-structure.md | 10 +++-- spec/5-initialization.md | 4 +- 9 files changed, 33 insertions(+), 25 deletions(-) create mode 100644 "C:\\Users\\buddy\\dev\\shm\\OPERAND_CHRONICLE_MAP" create mode 100644 "C:\\Users\\buddy\\dev\\shm\\OPERAND_ChronicleStampedLock" create mode 100644 "C:\\Users\\buddy\\dev\\shm\\OPERAND_ChronicleStampedLock=ReaderCount" create mode 100644 "C:\\Users\\buddy\\dev\\shm\\OPERAND_ChronicleStampedLock=WriterCount" diff --git "a/C:\\Users\\buddy\\dev\\shm\\OPERAND_CHRONICLE_MAP" "b/C:\\Users\\buddy\\dev\\shm\\OPERAND_CHRONICLE_MAP" new file mode 100644 index 0000000000000000000000000000000000000000..191599e8eadada1067b0fb3af15a9c9187afd009 GIT binary patch literal 12288 zcmeH{PjAyO7{<#5#3UpH7bGs;#0iZkZMSaA1+`_>DmFTi(z}&9jaAo>XDj4-E+;cjtwg=6%651}zuxJMZokwj9#J#^4?8m`lP=YMKn`xk@Z!J*EYl6jY5 zDLu5W<}pFON+_S?5ue_YE1bv(Ge#1fQmrDwGIAtIC}#8)D?QAivzR%)5^;yB^)6Q1 zA|LRX;PF_=^fpS4~#EgNOuEna<9+hD0pDjAu4NS|pPJo0)H*VQX85KX z=`(pv5+iM{1-Qp6UCUtJYDf+8JVj=rFYHBRUF&BJQ>=8F(R3-qG>u1?r9@w*-tJ^G z`jO1D*x6>xXTm-K4_USAZM!(e0aaS^a+jR0US>7jL&uisdU2m~qA3{?jpNql3u+_3YwR86BOJ*$2?)ow+jauqY^kR$exJDDvv12&v0-gZz~;F=^P0;?(ASHMhOy z3)TOT@t-gev#|)I(-C|;8uYn_9%@DgitS^T+QCJ-f=h*Ym5m@2l!6{=Hk*2=H5WU1 zOM9_Y+tTPR`wK$Jc6Md|ox-RxCPVGDh2`EKEn2Ltx$>&JZKlGat+Z9Wb(#Fyc9U2L zEG~%6EzaDZLd<05oQzJmaZL}Mng{buBvQ3kht_Mf!F8Cxs;7Nc80&J}y~SQ_yYG3e z_TzB7Qm6s~KmZ5;0U!VbfWQM0c%pM1r z@=XH2Upzybr*HxUfB+Bx0zd!=00AHX1b_e#00KY&2mk>f00e*l5C8%|00;m9AOHk_ J01&7o@DDJ!dcObw literal 0 HcmV?d00001 diff --git "a/C:\\Users\\buddy\\dev\\shm\\OPERAND_ChronicleStampedLock" "b/C:\\Users\\buddy\\dev\\shm\\OPERAND_ChronicleStampedLock" new file mode 100644 index 0000000000000000000000000000000000000000..3132cb74ad5bb21475b07be68231acc214f2328b GIT binary patch literal 8192 zcmeH~!EO^V5QYl}kP1>cAaQjoj<8CTG_<)Tp;0SRQn6{u)!I0T6|cRry`?GYTkr%( zJOVc^T;NT32@Wv!?q)Y?T8TCV4(_Fr$Met3H|t-@$E~+dzWr#0(fyT`bxyP=BErW* z?FC~gI1L!_LL7O0%qe5Il^q`72)$oLXn-|tQ%3qkDk^v#ZB)E+$*Vl~UpEf@y>4&g z8QhwXX^UZ{>S$e0BZ7P_DIcW~pPrEeEY%n@MkHKW+{GBHv8RZnn9(z=p_oU*hNib1yxXS?& z0c2Xn#7u$%9b@K`QAoIk^G$PTsQ3}1T7%@(>@2Ol;{B&>A$`bWBvmbT5%YnWWM4HZ~P`^~l+nNPar5-iJ(<>_JdC%@45#8J=n`8~wOiLXhCUMlqEGDo^ z81BWz=qoZ!VmlXOJ`?r}NV}bvZQ%$9R71cC) z$gL)*7sR{6p(47D>?-Et1^H@qJL^0d zWxHbW!&Y}swZuIroPAM7q!}~2{C(b>YV!@&YWeIYXnRX)_e7dKYe7L8+^Hj5l!;N< z-pEcXrj--3>N!tX{DzG845bJMB2XS|ao|aELYRI3%A47duW=ZWK}Q6Wes`b4N`^pi zR>%x}1B+=1t~BP0JO-hl5QLFoHtRcfz-u~>h2Bu6>!S4_B(LXW_FZKjmZ0x}4QFnA z3*^!H9EaTJuC}NJ_HKL6^Q}XrpU*ANkWG*+m`&|MQuiBTrV?9bbi|D%r;hwiNTnP3 zp?wzc(3(tO-3AjE#u&ZQ>iO!*rGJkg^7zuI3pfIffFs}tI0BAl!Ay8n)V aBj5-)0*-(q;0QPZj({WJ2si?Nm%wijnXDWD literal 0 HcmV?d00001 diff --git "a/C:\\Users\\buddy\\dev\\shm\\OPERAND_ChronicleStampedLock=ReaderCount" "b/C:\\Users\\buddy\\dev\\shm\\OPERAND_ChronicleStampedLock=ReaderCount" new file mode 100644 index 0000000000000000000000000000000000000000..d85ddf55e93e053d9b2788dc4aca0c2ac73b0455 GIT binary patch literal 8192 zcmeH~!EO^V5QYN>kSY!wcmTKR5ms%|q-}0VXw-_9R&40q+D;NHj=i$I&=eJTH%=UR z2p$2cN5@I{#O(P+MdrCbs|NQgW87U8MzJB?5_JiI$s@2v7m_WuL#zPZCV-X=S?&y-&F#(1R9 zKv`!q?B8e=w&uZ1X}~P;EJdfD58XYaqTBm*TWm1RS~`|;lEmkfCxCZ}!|nYZeFd{W z#zHRL3&`A7WqUNH5i@AHmCusF#bJu>ZPIi|tHnn`079}L>Pc2RpQw8>V^tB9B6o4a%8=_vaxSAe&AaA_1B zVB`EnSrgrayURZk-I=j(aJRKL?_h(@TiSadv+8+*#xl&wE|JSo-DTvb4QutCz3R$N zSo|G|q=LXD>)=?5k$WO@0Xxh+R!2_?ZKznZI|zwNK^z4=yM5xkEV)q}%#OCM%PWNt zwcLhsF!J4$A|2p*&A03wh?48de8pb6eWff|$Ieqx?>0(y-Aj7Qo#hw8XAYoy=xs8j zT&L@ivyrf?F?)CKrP8KT9PZjzgAcyTB{h9qrqV8`U~BDqt+w*Nr?uMWXDe6rf}VgU z;0bsFo`5Id39OR90~C9_O2PkBYx5tQ`_n!BK1qLn^kgCoDQSIE-&yHj|Irih1Uvyx Wz!UHUJONL@6YvB)0Z-sB6Zi=V&3E(w literal 0 HcmV?d00001 diff --git "a/C:\\Users\\buddy\\dev\\shm\\OPERAND_ChronicleStampedLock=WriterCount" "b/C:\\Users\\buddy\\dev\\shm\\OPERAND_ChronicleStampedLock=WriterCount" new file mode 100644 index 0000000000000000000000000000000000000000..48886ec1afcfda365a5ce262ea4944a63f098658 GIT binary patch literal 8192 zcmeH~!EO^V5QYN>kSY!wcmTKJ2&*<}(l)mwG)hHFD>js~wVfnZ9D8MZp(zUR3Oo=e zPCO7&FG!5NyV(tdN=<{fxTn-J^Upt@osn|?`un$!XFurmgGyynfC*#_Vl*&8G*VKq zh(j=;ad1urCFt7b1mrG=gZXlgXh#NPx(UO2AG+ zBa=`b!f*n@;Q5X{v{b^FGh`k(3`%LI%XM{)k z43u>?#s0NMVQU`Dl={pP&r)>ix##X772P?g+G2xg*3z+*<0L+(JOR8*9B%*5=u4RX zH5PK|UO?t{QMN;48Zm>G8~H35TpXt8-XwL0bg}qI2tY^{M78b)UW}h6W=XuO0FIe9 ztQPxB|;qLN}M0;xN8{BK`&pX(l^OpAR%dC2yps@^dvPme2RIt4<+o-Jl?{TH_`N`Ti are written at addresses Ai, then 32- or 64-bit value Y is written at address B, there is a way to ensure some memory order. In particular, if a - concurrent thread reads value Y from address B, after that it will read Xii values from Ai addresses (but not the old values, stored at those addresses). > In terms of the Java Memory model, this assumption means "writes to Ai addresses diff --git a/spec/2-design-overview.md b/spec/2-design-overview.md index 097263695..ed7accc3f 100644 --- a/spec/2-design-overview.md +++ b/spec/2-design-overview.md @@ -161,7 +161,6 @@ after the previous tier.* All tiers, either first in their segments or chained,
  1. If the keys are identical, the entry for the queried key within the segment is found, exit the procedure.
    2. -
  2. If the keys are not identical (because of a full collision of parts of their hash codes for storing in hash lookup tables), continue the search in the current tier's hash lookup table (step #2), without resetting the hash lookup slot index.
    3. diff --git a/spec/3-memory-layout.md b/spec/3-memory-layout.md index cbaa7404b..2312eccaf 100644 --- a/spec/3-memory-layout.md +++ b/spec/3-memory-layout.md @@ -13,7 +13,7 @@ one big continuous block of memory. Its structure, from lower addresses to highe persisted to disk. 3. [The global mutable state](#global-mutable-state). - 4. + 4. [Segment header alignment](#segment-headers-alignment") The alignment to the next page boundary by addresses (page size of the current memory mapping), if a new Chronicle Map is created, if an existing one (i. e. persisted) is loaded, the alignment is read from the global mutable state (specifically this field of the global mutable state is @@ -21,7 +21,7 @@ one big continuous block of memory. Its structure, from lower addresses to highe > The purpose of this alignment is to minimize the number of pages spanned by the following > *segment headers area*. The segment headers area is frequently accessed and updated, so the pages - > it spans almost always reside in TLB cache and always need to be flushed to the disk. + > it spans almost always reside in the TLB cache and always need to be flushed to the disk. > The alignment (more precisely, the offset to the next area, the segment headers area) is stored > in the global mutable state, rather then computed each time a mapped Chronicle Map store is @@ -30,18 +30,18 @@ one big continuous block of memory. Its structure, from lower addresses to highe 5. [The segment headers area](#segment-headers-area). 6. [The main segments area](#main-segments-area). - 7. Zero or several [extra tier bulks](#extra-tier-bulks). + 7. Zero or more [extra tier bulks](#extra-tier-bulks). The process of initialization of the Chronicle Map's memory is described on [Initialization Order]( 5-initialization.md) page. ## Self-bootstrapping header -The concept is described in [Self Bootstrapping Data]( +The concept is described in the [Self Bootstrapping Data]( https://github.com/OpenHFT/RFC/blob/master/Self-Bootstrapping-Data/Self-Bootstraping-Data-0.1.md) specification. -The structure of this area is described in [Size Prefixed Blob]( +The structure of this area is described in the [Size Prefixed Blob]( https://github.com/OpenHFT/RFC/blob/master/Size-Prefixed-Blob/Size-Prefixed-Blob-0.1.md) specification. The first 8 bytes contains the hash value of bytes sequence from 9th byte to the end of this size-prefixed blob, computed by [xxHash](https://github.com/Cyan4973/xxHash/) algorithm @@ -63,12 +63,13 @@ The global mutable state is 33 bytes long. > global mutable state and segment headers (see below). 2. Bytes 8..10 - the number of allocated [extra tier bulks](#extra-tier-bulks). An unsigned 24-bit - value, stored in the little-endian order. + value, stored in little-endian order. See also the [extra tier bulk allocation](5-initialization.md#entra-tier-bulk-allocation) operation. + 3. Bytes 11..15 - the index of the first *free* segment tier. An unsigned 40-bit value, stored - in the little-endian order. Extra segment tiers are allocated in bulks, so there is usually a chain + in little-endian order. Extra segment tiers are allocated in bulks, so there is usually a chain of allocated, but unused yet segment tiers. This field points to the head of this chain, or has value `0`, if there are no free segment tiers. @@ -79,15 +80,18 @@ The global mutable state is 33 bytes long. [`actualSegments`](3_1-header-fields.md#actualsegments) − 1) has tier index `actualSegments`, the first tier of the first extra tier bulk has tier index `actualSegments` + 1, etc. Tier indexes are 1-counted, because value 0 has some special meaning. + 4. Bytes 16..20 - the number of used extra segment tiers. An unsigned 40-bit value, stored in - the little-endian order. + little-endian order. + 5. Bytes 21..24 - the offset of the segment headers area from the beginning of the memory of this - Chronicle Map store. An unsigned 32-bit value, stored in the little-endian ordered. This field + Chronicle Map store. An unsigned 32-bit value, stored in little-endian ordered. This field determines the size of [the 4th area of the general Chronicle Map structure]( #segment-headers-alignment). + 6. Bytes 25..32 - the Chronicle Map data store size, the offset to the end of the [main segments area](#main-segments area) or the last [extra tier bulk](#extra-tier-bulks). A non-negative 64-bit - value, stored in the little-endian order. + value, stored in little-endian order. > The reference Java implementation: [`VanillaGlobalMutableState` > ](../src/main/java/net/openhft/chronicle/hash/VanillaGlobalMutableState.java). @@ -97,7 +101,7 @@ The global mutable state is 33 bytes long. The offset to this area is stored in the 5th field of the [global mutable state ](#global-mutable-state). -The size (in bytes) of a segment headers area is [`actualSegments`]( +The size (in bytes) of a segment header's area is [`actualSegments`]( 3_1-header-fields.md#actualsegments) * [`segmentHeaderSize`](3_1-header-fields.md#segmentheadersize ). Each segment header starts at offsets from the start of the segment headers area, that are multiples of `segmentHeaderSize`. Each segment header is 32 bytes long. `segmentHeaderSize` − @@ -126,6 +130,7 @@ multiples of `segmentHeaderSize`. Each segment header is 32 bytes long. `segment It is a 64-bit value, stored in the little-endian order. If the value of this field is 0, this means there is no chained segment tier in this segment yet after the first tier, in other words, the first tier is the only one in the chain for the current segment. + 5. Bytes 24..31 - reserved for use by extensions. > The reference Java implementation: [`BigSegmentHeader` @@ -133,13 +138,13 @@ multiples of `segmentHeaderSize`. Each segment header is 32 bytes long. `segment ## Main segments area -This area contains first tiers of the Chronicle Map's segments. +This area contains the first tiers of the Chronicle Map's segments. A main segments area starts immediately after the end of a segment headers area without extra offsets and alignments. The size of a main segments area is [`actualSegments`](3_1-header-fields.md#actualsegments) * -[`tierSize`](3_1-header-fields.md#tiersize). Segment tiers in a main segments area doesn't have +[`tierSize`](3_1-header-fields.md#tiersize). Segment tiers in a main segments area don't have extra gaps in memory between each other. ### Segment tier structure @@ -147,7 +152,7 @@ extra gaps in memory between each other. See also [segment tiers design overview](2-design-overview.md#segment-tier) for more explanations about this structure. -Segment tier is [`tierSize`](3_1-header-fields.md#tiersize) bytes long. +A segment tier is [`tierSize`](3_1-header-fields.md#tiersize) bytes long. The segment tier structure: @@ -163,7 +168,7 @@ A hash lookup area starts at the same address as a segment tier, containing it. A hash lookup consists of [`tierHashLookupCapacity`](3_1-header-fields.md#tierhashlookupcapacity) slots each of [`tierHashLookupSlotSize`](3_1-header-fields.md#tierhashlookupslotsize) bytes. In each slot a 32-bit or 64-bit (if the `tierHashLookupSlotSize` is 4 or 8, respectively) value is stored in -little-endian order. The slot value of 0 designates an empty slot. +little-endian order. A slot value of 0 designates an empty slot. ##### Hash lookup key diff --git a/spec/3_2-lock-structure.md b/spec/3_2-lock-structure.md index 9d1730e93..83ed46d79 100644 --- a/spec/3_2-lock-structure.md +++ b/spec/3_2-lock-structure.md @@ -2,11 +2,13 @@ The lock structure is 8 bytes (64 bits) long. - 1. *Count word*: + *Count word*: 1. Bits 0..29 - read lock count (little-endian) 2. Bit 30 - update lock flag 3. Bit 31 - write lock flag - 2. *Wait word*: bits 32..63 - wait count (little-endian) + +*Wait word*: + 1. bits 32..63 - wait count (little-endian) ## Try acquire read lock @@ -107,6 +109,7 @@ procedure and call one depending on the context. + ## Release write lock, or write to update lock downgrade, or write to read lock downgrade Perform a CAS operation on the count word of the lock state, comparing 0x80000000 (i. e. a count @@ -114,7 +117,7 @@ word with the write lock flag set) and swapping with 0 (in case of releasing wri 0x40000000 (in case of write to update lock downgrade), or 1 (in case of write to read lock downgrade). The result of the CAS operation is the result of the procedure. -## Try upgrade update to write lock +## Try upgrade to write lock Perform a CAS operation on the count word of the lock state, comparing 0x40000000 (i. e. a count word with the update lock flag set, the write lock flag not set, the read lock count of zero) and @@ -140,6 +143,7 @@ procedure. + ## Time-limited write lock acquisition or update to write upgrade 1. Perform the corresponding *try acquire* procedure ([write lock](#try-acquire-write-lock) or diff --git a/spec/5-initialization.md b/spec/5-initialization.md index 2e613181e..8c46c66e1 100644 --- a/spec/5-initialization.md +++ b/spec/5-initialization.md @@ -17,7 +17,7 @@ existing Map, persisted to some file). [wait until the Chronicle Map is ready](#wait-until-chronicle-map-is-ready). Otherwise, acquire an exclusive file lock, and check the file size again. If the file size is non-zero now, release the file lock and [wait until the Chronicle Map is ready](#wait-until-chronicle-map-is-ready). If - the file is still empty, [write self bootstrapping header](#write-self-bootstrapping-header) to the + the file is still empty, [write the self bootstrapping header](#write-self-bootstrapping-header) to the beginning of the file, then release the file lock. > The purpose of acquiring file lock is to ensure that only one concurrent process (if any) @@ -49,7 +49,7 @@ existing Map, persisted to some file). out memory from the beginning of the tier to the start of the [entry space]( 3-memory-layout.md#entry-space), i. e. zero out this tier's hash lookup, segment tier counters area and free list. - 6. White the segment headers offset into the 5th field of the global mutable state. + 6. Write the segment headers offset into the 5th field of the global mutable state. 7. Write the offset to the end of the main segments area into the 6th field of the global mutable state. 8. If the Chronicle Map is persisted, ensure all data written to the file is flushed to the disk.