aboutsummaryrefslogtreecommitdiff
path: root/core/sync/atomic.odin
blob: 7e514a6b4832bf655a530789a41eafc7b581cff8 (plain)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
package sync

import "base:intrinsics"

/*
This procedure may lower CPU consumption or yield to a hyperthreaded twin
processor. It's exact function is architecture specific, but the intent is to
say that you're not doing much on a CPU.
*/
cpu_relax :: intrinsics.cpu_relax

/*
Describes memory ordering for an atomic operation.

Modern CPU's contain multiple cores and caches specific to those cores. When a
core performs a write to memory, the value is written to cache first. The issue
is that a core doesn't typically see what's inside the caches of other cores.
In order to make operations consistent CPU's implement mechanisms that
synchronize memory operations across cores by asking other cores or by
pushing data about writes to other cores.

Due to how these algorithms are implemented, the stores and loads performed by
one core may seem to happen in a different order to another core. It also may
happen that a core reorders stores and loads (independent of how compiler put
them into the machine code). This can cause issues when trying to synchronize
multiple memory locations between two cores. Which is why CPU's allow for
stronger memory ordering guarantees if certain instructions or instruction
variants are used.

In Odin there are 5 different memory ordering guaranties that can be provided
to an atomic operation:

- `Relaxed`: The memory access (load or store) is unordered with respect to
  other memory accesses. This can be used to implement an atomic counter.
  Multiple threads access a single variable, but it doesn't matter when
  exactly it gets incremented, because it will become eventually consistent.
- `Consume`: No loads or stores dependent on a memory location can be
  reordered before a load with consume memory order. If other threads released
  the same memory, it becomes visible.
- `Acquire`: No loads or stores on a memory location can be reordered before a
  load of that memory location with acquire memory ordering. If other threads
  release the same memory, it becomes visible.
- `Release`: No loads or stores on a memory location can be reordered after a
  store of that memory location with release memory ordering. All threads that
  acquire the same memory location will see all writes done by the current
  thread.
- `Acq_Rel`: Acquire-release memory ordering: combines acquire and release
  memory orderings in the same operation.
- `Seq_Cst`: Sequential consistency. The strongest memory ordering. A load will
  always be an acquire operation, a store will always be a release operation,
  and in addition to that all threads observe the same order of writes.

Non-explicit atomics will always be sequentially consistent.

	Atomic_Memory_Order :: enum {
		Relaxed = 0, // Unordered
		Consume = 1, // Monotonic
		Acquire = 2,
		Release = 3,
		Acq_Rel = 4,
		Seq_Cst = 5,
	}

**Note(i386, x64)**: x86 has a very strong memory model by default. It
guarantees that all writes are ordered, stores and loads aren't reordered. In
a sense, all operations are at least acquire and release operations. If `lock`
prefix is used, all operations are sequentially consistent. If you use explicit
atomics, make sure you have the correct atomic memory order, because bugs likely
will not show up in x86, but may show up on e.g. arm. More on x86 memory
ordering can be found
[[here; https://www.cs.cmu.edu/~410-f10/doc/Intel_Reordering_318147.pdf]]
*/
Atomic_Memory_Order :: intrinsics.Atomic_Memory_Order

/*
Establish memory ordering.

This procedure establishes memory ordering, without an associated atomic
operation.
*/
atomic_thread_fence :: intrinsics.atomic_thread_fence

/*
Establish memory ordering between a current thread and a signal handler.

This procedure establishes memory ordering between a thread and a signal
handler, that run on the same thread, without an associated atomic operation.
This procedure is equivalent to `atomic_thread_fence`, except it doesn't
issue any CPU instructions for memory ordering.
*/
atomic_signal_fence :: intrinsics.atomic_signal_fence

/*
Atomically store a value into memory.

This procedure stores a value to a memory location in such a way that no other
thread is able to see partial reads. This operation is sequentially-consistent.
*/
atomic_store :: intrinsics.atomic_store

/*
Atomically store a value into memory with explicit memory ordering.

This procedure stores a value to a memory location in such a way that no other
thread is able to see partial reads. The memory ordering of this operation is
as specified by the `order` parameter.
*/
atomic_store_explicit :: intrinsics.atomic_store_explicit

/*
Atomically load a value from memory.

This procedure loads a value from a memory location in such a way that the
received value is not a partial read. The memory ordering of this operation is
sequentially-consistent.
*/
atomic_load :: intrinsics.atomic_load

/*
Atomically load a value from memory with explicit memory ordering.

This procedure loads a value from a memory location in such a way that the
received value is not a partial read. The memory ordering of this operation
is as specified by the `order` parameter.
*/
atomic_load_explicit :: intrinsics.atomic_load_explicit

/*
Atomically add a value to the value stored in memory.

This procedure loads a value from memory, adds the specified value to it, and
stores it back as an atomic operation. This operation is an atomic equivalent
of the following:

	dst^ += val

The memory ordering of this operation is sequentially-consistent.
*/
atomic_add :: intrinsics.atomic_add

/*
Atomically add a value to the value stored in memory.

This procedure loads a value from memory, adds the specified value to it, and
stores it back as an atomic operation. This operation is an atomic equivalent
of the following:

	dst^ += val

The memory ordering of this operation is as specified by the `order` parameter.
*/
atomic_add_explicit :: intrinsics.atomic_add_explicit

/*
Atomically subtract a value from the value stored in memory.

This procedure loads a value from memory, subtracts the specified value from it,
and stores the result back as an atomic operation. This operation is an atomic
equivalent of the following:

	dst^ -= val

The memory ordering of this operation is sequentially-consistent.
*/
atomic_sub :: intrinsics.atomic_sub

/*
Atomically subtract a value from the value stored in memory.

This procedure loads a value from memory, subtracts the specified value from it,
and stores the result back as an atomic operation. This operation is an atomic
equivalent of the following:

	dst^ -= val

The memory ordering of this operation is as specified by the `order` parameter.
*/
atomic_sub_explicit :: intrinsics.atomic_sub_explicit

/*
Atomically replace the memory location with the result of AND operation with
the specified value.

This procedure loads a value from memory, calculates the result of AND operation
between the loaded value and the specified value, and stores it back into the
same memory location as an atomic operation. This operation is an atomic
equivalent of the following:

	dst^ &= val

The memory ordering of this operation is sequentially-consistent.
*/
atomic_and :: intrinsics.atomic_and

/*
Atomically replace the memory location with the result of AND operation with
the specified value.

This procedure loads a value from memory, calculates the result of AND operation
between the loaded value and the specified value, and stores it back into the
same memory location as an atomic operation. This operation is an atomic
equivalent of the following:

	dst^ &= val

The memory ordering of this operation is as specified by the `order` parameter.
*/
atomic_and_explicit :: intrinsics.atomic_and_explicit

/*
Atomically replace the memory location with the result of NAND operation with
the specified value.

This procedure loads a value from memory, calculates the result of NAND operation
between the loaded value and the specified value, and stores it back into the
same memory location as an atomic operation. This operation is an atomic
equivalent of the following:

	dst^ = ~(dst^ & val)

The memory ordering of this operation is sequentially-consistent.
*/
atomic_nand :: intrinsics.atomic_nand

/*
Atomically replace the memory location with the result of NAND operation with
the specified value.

This procedure loads a value from memory, calculates the result of NAND operation
between the loaded value and the specified value, and stores it back into the
same memory location as an atomic operation. This operation is an atomic
equivalent of the following:

	dst^ = ~(dst^ & val)

The memory ordering of this operation is as specified by the `order` parameter.
*/
atomic_nand_explicit :: intrinsics.atomic_nand_explicit

/*
Atomically replace the memory location with the result of OR operation with
the specified value.

This procedure loads a value from memory, calculates the result of OR operation
between the loaded value and the specified value, and stores it back into the
same memory location as an atomic operation. This operation is an atomic
equivalent of the following:

	dst^ |= val

The memory ordering of this operation is sequentially-consistent.
*/
atomic_or :: intrinsics.atomic_or

/*
Atomically replace the memory location with the result of OR operation with
the specified value.

This procedure loads a value from memory, calculates the result of OR operation
between the loaded value and the specified value, and stores it back into the
same memory location as an atomic operation. This operation is an atomic
equivalent of the following:

	dst^ |= val

The memory ordering of this operation is as specified by the `order` parameter.
*/
atomic_or_explicit :: intrinsics.atomic_or_explicit

/*
Atomically replace the memory location with the result of XOR operation with
the specified value.

This procedure loads a value from memory, calculates the result of XOR operation
between the loaded value and the specified value, and stores it back into the
same memory location as an atomic operation. This operation is an atomic
equivalent of the following:

	dst^ ~= val

The memory ordering of this operation is sequentially-consistent.
*/
atomic_xor :: intrinsics.atomic_xor

/*
Atomically replace the memory location with the result of XOR operation with
the specified value.

This procedure loads a value from memory, calculates the result of XOR operation
between the loaded value and the specified value, and stores it back into the
same memory location as an atomic operation. This operation is an atomic
equivalent of the following:

	dst^ ~= val

The memory ordering of this operation is as specified by the `order` parameter.
*/
atomic_xor_explicit :: intrinsics.atomic_xor_explicit

/*
Atomically exchange the value in a memory location, with the specified value.

This procedure loads a value from the specified memory location, and stores the
specified value into that memory location. Then the loaded value is returned,
all done in a single atomic operation. This operation is an atomic equivalent
of the following:

	tmp := dst^
	dst^ = val
	return tmp

The memory ordering of this operation is sequentially-consistent.
*/
atomic_exchange :: intrinsics.atomic_exchange

/*
Atomically exchange the value in a memory location, with the specified value.

This procedure loads a value from the specified memory location, and stores the
specified value into that memory location. Then the loaded value is returned,
all done in a single atomic operation. This operation is an atomic equivalent
of the following:

	tmp := dst^
	dst^ = val
	return tmp

The memory ordering of this operation is as specified by the `order` parameter.
*/
atomic_exchange_explicit :: intrinsics.atomic_exchange_explicit

/*
Atomically compare and exchange the value with a memory location.

This procedure checks if the value pointed to by the `dst` parameter is equal
to `old`, and if they are, it stores the value `new` into the memory location,
all done in a single atomic operation. This procedure returns the old value
stored in a memory location and a boolean value signifying whether `old` was
equal to `new`.

This procedure is an atomic equivalent of the following operation:

	old_dst := dst^
	if old_dst == old {
		dst^ = new
		return old_dst, true
	} else {
		return old_dst, false
	}

The strong version of compare exchange always returns true, when the returned
old value stored in location pointed to by `dst` and the `old` parameter are
equal.

Atomic compare exchange has two memory orderings: One is for the
read-modify-write operation, if the comparison succeeds, and the other is for
the load operation, if the comparison fails. The memory ordering for both of
of these operations is sequentially-consistent.
*/
atomic_compare_exchange_strong :: intrinsics.atomic_compare_exchange_strong

/*
Atomically compare and exchange the value with a memory location.

This procedure checks if the value pointed to by the `dst` parameter is equal
to `old`, and if they are, it stores the value `new` into the memory location,
all done in a single atomic operation. This procedure returns the old value
stored in a memory location and a boolean value signifying whether `old` was
equal to `new`.

This procedure is an atomic equivalent of the following operation:

	old_dst := dst^
	if old_dst == old {
		dst^ = new
		return old_dst, true
	} else {
		return old_dst, false
	}

The strong version of compare exchange always returns true, when the returned
old value stored in location pointed to by `dst` and the `old` parameter are
equal.

Atomic compare exchange has two memory orderings: One is for the
read-modify-write operation, if the comparison succeeds, and the other is for
the load operation, if the comparison fails. The memory ordering for these
operations is as specified by `success` and `failure` parameters respectively.
*/
atomic_compare_exchange_strong_explicit :: intrinsics.atomic_compare_exchange_strong_explicit

/*
Atomically compare and exchange the value with a memory location.

This procedure checks if the value pointed to by the `dst` parameter is equal
to `old`, and if they are, it stores the value `new` into the memory location,
all done in a single atomic operation. This procedure returns the old value
stored in a memory location and a boolean value signifying whether `old` was
equal to `new`.

This procedure is an atomic equivalent of the following operation:

	old_dst := dst^
	if old_dst == old {
		// may return false here
		dst^ = new
		return old_dst, true
	} else {
		return old_dst, false
	}

The weak version of compare exchange may return false, even if `dst^ == old`.
On some platforms running weak compare exchange in a loop is faster than a
strong version.

Atomic compare exchange has two memory orderings: One is for the
read-modify-write operation, if the comparison succeeds, and the other is for
the load operation, if the comparison fails. The memory ordering for both
of these operations is sequentially-consistent.
*/
atomic_compare_exchange_weak :: intrinsics.atomic_compare_exchange_weak

/*
Atomically compare and exchange the value with a memory location.

This procedure checks if the value pointed to by the `dst` parameter is equal
to `old`, and if they are, it stores the value `new` into the memory location,
all done in a single atomic operation. This procedure returns the old value
stored in a memory location and a boolean value signifying whether `old` was
equal to `new`.

This procedure is an atomic equivalent of the following operation:

	old_dst := dst^
	if old_dst == old {
		// may return false here
		dst^ = new
		return old_dst, true
	} else {
		return old_dst, false
	}

The weak version of compare exchange may return false, even if `dst^ == old`.
On some platforms running weak compare exchange in a loop is faster than a
strong version.

Atomic compare exchange has two memory orderings: One is for the
read-modify-write operation, if the comparison succeeds, and the other is for
the load operation, if the comparison fails. The memory ordering for these
operations is as specified by the `success` and `failure` parameters
respectively.
*/
atomic_compare_exchange_weak_explicit :: intrinsics.atomic_compare_exchange_weak_explicit