-
Notifications
You must be signed in to change notification settings - Fork 0
Expand file tree
/
Copy pathatom.xml
More file actions
614 lines (326 loc) · 325 KB
/
atom.xml
File metadata and controls
614 lines (326 loc) · 325 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
<?xml version="1.0" encoding="utf-8"?>
<feed xmlns="http://www.w3.org/2005/Atom">
<title>XBoom Dove</title>
<link href="/atom.xml" rel="self"/>
<link href="https://www.yuankang.top/"/>
<updated>2024-06-14T13:18:48.919Z</updated>
<id>https://www.yuankang.top/</id>
<author>
<name>XBoom Dove</name>
</author>
<generator uri="https://hexo.io/">Hexo</generator>
<entry>
<title>Protocol-05-RFC</title>
<link href="https://www.yuankang.top/2024/06/14/Protocol/Protocol-05-RFC/"/>
<id>https://www.yuankang.top/2024/06/14/Protocol/Protocol-05-RFC/</id>
<published>2024-06-14T02:25:38.000Z</published>
<updated>2024-06-14T13:18:48.919Z</updated>
<content type="html"><![CDATA[<p>最近开发<code>IPv6</code>需要查看<code>rfc</code>文档，简单记录一下如何查找相关标准文档</p><p>通过该<a href="https://www.rfc-editor.org/search/rfc_search_detail.php" target="_blank" rel="noopener">页面</a>，搜索领居发现协议 Neighbor Discovery 会等到如下的结果</p><img src="https://images-1307117474.cos.ap-guangzhou.myqcloud.com/blogs/Blog_3/202406121447240.png" alt="image-20240612144705604" style="zoom:60%;" /><p>这里主要关注的是 <code>Title</code> 以及 <code>Status</code>，通过 Additional Criteria 也能进一步详细的搜索，页面很清晰，不做过多介绍，它的状态一共包括一下几种</p><table><thead><tr><th>状态</th><th>解释</th></tr></thead><tbody><tr><td><strong>Internet-Draft</strong></td><td>是IETF标准制定过程的起点，是一种工作文档，有效期最长为六个月，并且可能被更新、替换或作废</td></tr><tr><td><strong>Proposed Standard</strong></td><td>技术规范已经完成所有必要的技术审查，并且正在被互联网社区广泛测试和验证</td></tr><tr><td><strong>Draft Standard</strong></td><td>技术规范在Proposed Standard阶段经过一段时间的社区实践和审查后，如果继续被证明是稳定和成熟的，并且满足所有必要的要求，它可能被提升为Draft Standard</td></tr><tr><td><strong>(Internet) Standard</strong></td><td>这是IETF的最高级别标准，技术规范被认为是稳定、普遍接受的，并且适用于互联网操作</td></tr><tr><td><strong>BCP (Best Current Practice)</strong></td><td>BCP文档提供建议，但不一定描述协议规范。它们通常涉及推荐的做法或配置建议</td></tr><tr><td><strong>Historic</strong></td><td>这些文档包含了不再推荐使用的技术或实践，但由于历史价值而保留</td></tr><tr><td><strong>Experimental</strong></td><td>用于那些正在开发中，尚未准备好进行更广泛测试或部署的技术规范</td></tr><tr><td><strong>Informational</strong></td><td>这类文档提供信息和讨论，但并不包含技术规范或标准</td></tr><tr><td><strong>Not Recommended</strong></td><td>这些文档包含的建议或规范不再被推荐使用，但可能仍然有参考价值</td></tr><tr><td><strong>Obsolete</strong></td><td>这些文档已经被新的标准或实践所取代，不再具有实际应用价值</td></tr></tbody></table><p>所以一般查看 <code>Standards</code> 的文档</p><img src="https://images-1307117474.cos.ap-guangzhou.myqcloud.com/blogs/Blog_3/202406121458049.png" alt="image-20240612145845983" style="zoom:80%;" /><p>然后就可以找到对于 IPv6 邻居协议的最新的 Draft Standard 版本</p><p><img src="https://images-1307117474.cos.ap-guangzhou.myqcloud.com/blogs/Blog_3/202406121501818.png" alt="image-20240612150113766" /></p><p>在 <strong>More Info</strong> 这一栏有补充信息</p><ul><li><strong>Errata</strong>: 指的是对已发布RFC文档的更正。这些更正通常涉及文档中的错误或遗漏，不改变技术规范的实质内容</li><li><strong>Obsoletes</strong>: 当一个RFC文档被新的RFC替代时，新的RFC会列出它所取代的旧RFC编号。例如，如果RFC 4861被标记为"Obsoletes RFC 2461"，这意味着RFC 2461已经被RFC 4861所替代，不再推荐使用</li><li><strong>Updated by</strong>: 当一个RFC文档需要更新或修正，但这些更新不足以构成一个全新的版本时，会使用这个术语。更新的RFC会列出它所更新的原始RFC编号。例如，如果RFC 5942被标记为"Updated by RFC 6980"，这意味着RFC 6980提供了对RFC 5942的更新或补充信息</li></ul><p>注意：</p><ol><li><p>更新的文档是当前文档的拓展修正，在后续的开发过程中可能有相关问题，所以也需要关注！</p></li><li><p>搜索展示的时候可能不全，别忘了查看是否已经显示了全部</p><p><img src="https://images-1307117474.cos.ap-guangzhou.myqcloud.com/blogs/Blog_3/202406130951608.png" alt="image-20240613095159529" /></p></li></ol><h3 id="参考文档"><a class="markdownIt-Anchor" href="#参考文档"></a> 参考文档</h3><ol><li><a href="https://www.rfc-editor.org/search/rfc_search_detail.php" target="_blank" rel="noopener">https://www.rfc-editor.org/search/rfc_search_detail.php</a></li></ol>]]></content>
<summary type="html">
<p>最近开发<code>IPv6</code>需要查看<code>rfc</code>文档，简单记录一下如何查找相关标准文档</p>
<p>通过该<a href="https://www.rfc-editor.org/search/rfc_search_detail.php"
</summary>
<category term="Protocol" scheme="https://www.yuankang.top/categories/Protocol/"/>
<category term="RFC" scheme="https://www.yuankang.top/tags/RFC/"/>
<category term="Protocol" scheme="https://www.yuankang.top/tags/Protocol/"/>
</entry>
<entry>
<title>Kernel-14-kfifo</title>
<link href="https://www.yuankang.top/2024/05/23/Kernel/Kernel-14-kfifo/"/>
<id>https://www.yuankang.top/2024/05/23/Kernel/Kernel-14-kfifo/</id>
<published>2024-05-23T13:41:15.918Z</published>
<updated>2024-05-25T09:27:28.059Z</updated>
<content type="html"><![CDATA[<p>做内核驱动开发的时候，从 Linux收包的时候用到了 <code>kfifo</code> 这种结构接收报文进行报文分析，这里学习一下</p><p><code>kfifo</code> 其实是一个无锁环形队列，主要用来解决进出速度不匹配的情况。当生产速度太快或多生产者的情况，消费者可能处理不过来可能导致服务崩溃</p><h3 id="基础结构"><a class="markdownIt-Anchor" href="#基础结构"></a> 基础结构</h3><p>源码位置：<code>linux/include/linux/kfifo.h</code></p><p>首先是 kfifo 数据结构</p><figure class="highlight c"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br></pre></td><td class="code"><pre><span class="line"><span class="class"><span class="keyword">struct</span> __<span class="title">kfifo</span> {</span></span><br><span class="line"><span class="keyword">unsigned</span> <span class="keyword">int</span>in;<span class="comment">//进队索引</span></span><br><span class="line"><span class="keyword">unsigned</span> <span class="keyword">int</span>out;<span class="comment">//出队索引</span></span><br><span class="line"><span class="keyword">unsigned</span> <span class="keyword">int</span>mask;<span class="comment">//大小掩码</span></span><br><span class="line"><span class="keyword">unsigned</span> <span class="keyword">int</span>esize;<span class="comment">//单个元素大小</span></span><br><span class="line"><span class="keyword">void</span>*data;<span class="comment">//队列缓存指针</span></span><br><span class="line">};</span><br></pre></td></tr></table></figure><p>上面就是一个普通的环形队列结构，但是重点却是下面这个数据结构</p><figure class="highlight c"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br></pre></td><td class="code"><pre><span class="line"><span class="meta">#<span class="meta-keyword">define</span> __STRUCT_KFIFO_COMMON(datatype, recsize, ptrtype) \</span></span><br><span class="line"><span class="keyword">union</span> { \</span><br><span class="line"><span class="class"><span class="keyword">struct</span> __<span class="title">kfifo</span><span class="title">kfifo</span>;</span> \</span><br><span class="line">datatype*type; \</span><br><span class="line"><span class="keyword">const</span> datatype*const_type; \</span><br><span class="line"><span class="keyword">char</span>(*rectype)[recsize]; \</span><br><span class="line">ptrtype*ptr; \</span><br><span class="line">ptrtype <span class="keyword">const</span>*ptr_const; \</span><br><span class="line">}</span><br></pre></td></tr></table></figure><p>定义了一个 <code>union</code> 允许不同的数据类型共享相同的内存位置。在编译的时候会分配一个最大的结构体内存空间也就是 <code>_kfifo</code>，其他字段的使用会覆盖已有的内容</p><p><code>rectype</code> 表示一个指向字符数组的指针类型</p><p>提供了一种灵活的方式来定义环形缓冲区，允许开发者根据需要选择不同的数据类型和指针类型，根据它自带例子 <code>record-example.c</code> 来看看它两种使用方式</p><h3 id="动态分配"><a class="markdownIt-Anchor" href="#动态分配"></a> 动态分配</h3><p>动态分配的流程步骤</p><p>第一步，定义一个 fifo 指针对象</p><figure class="highlight c"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment">//1. 定义一个 fifo 对象</span></span><br><span class="line"><span class="class"><span class="keyword">struct</span> <span class="title">kfifo_rec_ptr_1</span> <span class="title">test</span>;</span></span><br><span class="line"></span><br><span class="line"><span class="class"><span class="keyword">struct</span> <span class="title">kfifo_rec_ptr_1</span> __<span class="title">STRUCT_KFIFO_PTR</span>(<span class="title">unsigned</span> <span class="title">char</span>, 1, <span class="title">void</span>);</span></span><br><span class="line"><span class="class"><span class="keyword">struct</span> <span class="title">kfifo_rec_ptr_2</span> __<span class="title">STRUCT_KFIFO_PTR</span>(<span class="title">unsigned</span> <span class="title">char</span>, 2, <span class="title">void</span>);</span></span><br><span class="line"><span class="meta">#<span class="meta-keyword">define</span> __STRUCT_KFIFO_PTR(type, recsize, ptrtype) \</span></span><br><span class="line">{ \</span><br><span class="line">__STRUCT_KFIFO_COMMON(type, recsize, ptrtype); \</span><br><span class="line">typebuf[<span class="number">0</span>]; \</span><br><span class="line">}</span><br></pre></td></tr></table></figure><p>将 <code>kfifo_rec_ptr_1</code> 完全展开的到的是</p><figure class="highlight c"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br></pre></td><td class="code"><pre><span class="line"><span class="class"><span class="keyword">struct</span> <span class="title">kfifo_rec_ptr_1</span></span></span><br><span class="line"><span class="class">{</span></span><br><span class="line"> <span class="keyword">union</span></span><br><span class="line"> {</span><br><span class="line"> <span class="class"><span class="keyword">struct</span> __<span class="title">kfifo</span> <span class="title">kfifo</span>;</span></span><br><span class="line"> <span class="keyword">unsigned</span> <span class="keyword">char</span> *type;</span><br><span class="line"> <span class="keyword">const</span> <span class="keyword">unsigned</span> <span class="keyword">char</span> *const_type;</span><br><span class="line"> <span class="keyword">char</span> (*rectype)[<span class="number">1</span>];</span><br><span class="line"> <span class="keyword">void</span> *ptr;</span><br><span class="line"> <span class="keyword">void</span> <span class="keyword">const</span> *ptr_const;</span><br><span class="line"> };</span><br><span class="line"> <span class="keyword">unsigned</span> <span class="keyword">char</span> buf[<span class="number">0</span>];</span><br><span class="line">};</span><br></pre></td></tr></table></figure><p>第二步，为缓冲队列指针分配指定大小内存</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line">//2. 初始化 fifo 内存</span><br><span class="line">#define FIFO_SIZE128</span><br><span class="line"></span><br><span class="line">kfifo_alloc(&test, FIFO_SIZE, GFP_KERNEL);//GFP_KERNEL 为内核分配内存的标志，用来指定分配内存时的行为和特征</span><br></pre></td></tr></table></figure><p>这里用到了第二个比较特别的宏定义，将其展开得到:</p><figure class="highlight c"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br></pre></td><td class="code"><pre><span class="line"><span class="meta">#<span class="meta-keyword">define</span> kfifo_alloc(fifo, size, gfp_mask) \</span></span><br><span class="line"> __kfifo_int_must_check_helper( \</span><br><span class="line"> ({ \</span><br><span class="line"> typeof((fifo) + <span class="number">1</span>) __tmp = (fifo); \</span><br><span class="line"> struct __kfifo *__kfifo = &__tmp->kfifo; \</span><br><span class="line"> __is_kfifo_ptr(__tmp) ? __kfifo_alloc(__kfifo, <span class="built_in">size</span>, <span class="keyword">sizeof</span>(*__tmp->type), gfp_mask) : -EINVAL; \</span><br><span class="line"> }))</span><br></pre></td></tr></table></figure><ul><li><code>typeof((fifo) + 1) __tmp = (fifo);</code>：使用 <code>typeof</code> 运算符获取 <code>fifo</code> 指针的类型，并创建一个名为 <code>__tmp</code> 的新变量，初始化为 <code>fifo</code> 的值。<code>(fifo) + 1</code> 用于获取 <code>fifo</code> 的类型，而不改变 <code>fifo</code> 的值</li><li><code>struct __kfifo *__kfifo = &__tmp->kfifo;</code>：一个指向 <code>__kfifo</code> 结构的指针，将其初始化为 <code>__tmp</code> 中 <code>kfifo</code> 成员的地址</li><li><code>__is_kfifo_ptr(__tmp) ?</code>：条件运算符，函数用于检查 <code>__tmp</code> 是否是环形缓冲区指针的有效实例，如果是环形缓冲区有效实例，则分配一个环形缓冲区 <code>__kfifo_alloc</code></li></ul><p><strong>为什么使用 <code>(fifo) + 1</code> 而不是直接使用 <code>fifo</code> 呢？</strong></p><p><code>typeof((a) + 1</code>) 用来保证传递进来的变量 a 为指针，若部位指针则引发编译错误，而不是等运行时才爆发错误</p><figure class="highlight c"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">typedef</span> <span class="class"><span class="keyword">struct</span> <span class="title">demo_t</span></span></span><br><span class="line"><span class="class">{</span></span><br><span class="line"> <span class="keyword">int</span> a;</span><br><span class="line"> <span class="keyword">int</span> b;</span><br><span class="line">} <span class="keyword">demo_t</span>;</span><br><span class="line"></span><br><span class="line"><span class="keyword">demo_t</span> demo;</span><br><span class="line"></span><br><span class="line">typeof((&demo) + <span class="number">1</span>) b; <span class="comment">// 等效 demo_t* b</span></span><br><span class="line">typeof((&demo)) b; <span class="comment">// 等效 demo_t* b</span></span><br><span class="line">typeof((demo) + <span class="number">1</span>) b; <span class="comment">// 报错，结构体变量不能 + 1</span></span><br><span class="line">typeof((demo)) b; <span class="comment">// 不报错，等效 demo_t b</span></span><br></pre></td></tr></table></figure><p>由于 <code>+ 1</code> 不会改变 <code>fifo</code> 的实际值，它只是用于类型推断，所以它是安全的</p><p>回到分配上，核心逻辑如下:</p><figure class="highlight c"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">int</span> __kfifo_alloc(struct __kfifo *fifo, <span class="keyword">unsigned</span> <span class="keyword">int</span> <span class="built_in">size</span>,</span><br><span class="line"><span class="keyword">size_t</span> esize, <span class="keyword">gfp_t</span> gfp_mask)</span><br><span class="line">{</span><br><span class="line"><span class="comment">/*</span></span><br><span class="line"><span class="comment"> * round up to the next power of 2, since our 'let the indices</span></span><br><span class="line"><span class="comment"> * wrap' technique works only in this case.</span></span><br><span class="line"><span class="comment"> */</span></span><br><span class="line"><span class="built_in">size</span> = roundup_pow_of_two(<span class="built_in">size</span>);</span><br><span class="line"></span><br><span class="line">fifo->in = <span class="number">0</span>;</span><br><span class="line">fifo->out = <span class="number">0</span>;</span><br><span class="line">fifo->esize = esize;</span><br><span class="line"></span><br><span class="line"><span class="keyword">if</span> (<span class="built_in">size</span> < <span class="number">2</span>) {</span><br><span class="line">fifo->data = <span class="literal">NULL</span>;</span><br><span class="line">fifo->mask = <span class="number">0</span>;</span><br><span class="line"><span class="keyword">return</span> -EINVAL;</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line">fifo->data = kmalloc_array(esize, <span class="built_in">size</span>, gfp_mask);</span><br><span class="line"></span><br><span class="line"><span class="keyword">if</span> (!fifo->data) {</span><br><span class="line">fifo->mask = <span class="number">0</span>;</span><br><span class="line"><span class="keyword">return</span> -ENOMEM;</span><br><span class="line">}</span><br><span class="line">fifo->mask = <span class="built_in">size</span> - <span class="number">1</span>;</span><br><span class="line"></span><br><span class="line"><span class="keyword">return</span> <span class="number">0</span>;</span><br><span class="line">}</span><br></pre></td></tr></table></figure><h3 id="静态分配"><a class="markdownIt-Anchor" href="#静态分配"></a> 静态分配</h3><p>首先声明一个缓冲队列对象</p><figure class="highlight c"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br></pre></td><td class="code"><pre><span class="line"><span class="meta">#<span class="meta-keyword">define</span> STRUCT_KFIFO_REC_1(size) \</span></span><br><span class="line"><span class="class"><span class="keyword">struct</span> __<span class="title">STRUCT_KFIFO</span>(<span class="title">unsigned</span> <span class="title">char</span>, <span class="title">size</span>, 1, <span class="title">void</span>)</span></span><br><span class="line"><span class="class"></span></span><br><span class="line"><span class="class">#<span class="title">define</span> __<span class="title">STRUCT_KFIFO</span>(<span class="title">type</span>, <span class="title">size</span>, <span class="title">recsize</span>, <span class="title">ptrtype</span>) \</span></span><br><span class="line"><span class="class">{</span> \</span><br><span class="line">__STRUCT_KFIFO_COMMON(type, recsize, ptrtype); \</span><br><span class="line">typebuf[((<span class="built_in">size</span> < <span class="number">2</span>) || (<span class="built_in">size</span> & (<span class="built_in">size</span> - <span class="number">1</span>))) ? <span class="number">-1</span> : <span class="built_in">size</span>]; \</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">typedef</span> <span class="title">STRUCT_KFIFO_REC_1</span><span class="params">(FIFO_SIZE)</span> mytest</span>;</span><br><span class="line"></span><br><span class="line"><span class="keyword">static</span> mytest test;</span><br></pre></td></tr></table></figure><p>静态分配与动态分配不一样的地方声明了一个 <code>unsigned char</code> 类型的 buf数组，数组大小为 <code>((size < 2) || (size & (size - 1))) ? -1 : size</code></p><p>这里有两点需要注意：</p><ol><li><code>FIFO_SIZE</code> 默认是128，如果指定为的1，那么将编译失败</li><li>为什么要使用 <code>size & (size - 1)</code> 来判断大小是否是 2 的 n 次幂<ul><li>直接使用 <code>in/out</code> 进行&运算，避免进行取余计算</li><li>既然是 2 的 n 次幂，那么使用的时候需要注意，比如申请 大小为 100，实际其实是 128 字节，每次出对入队都是 10 字节，<strong>那么可能出现最后一次只保存 10 字节的情况！！！！</strong></li></ul></li></ol><p><strong>所以在进出队列的时候需要对比进入队列大小！！！！</strong></p><figure class="highlight c"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">if</span>(kfifo_in(&test, &hello, <span class="keyword">sizeof</span>(hello)) != <span class="keyword">sizeof</span>(hello))</span><br><span class="line">{</span><br><span class="line"> <span class="comment">//...</span></span><br><span class="line">}</span><br><span class="line"></span><br><span class="line"><span class="keyword">if</span>(kfifo_out(&test, buf, <span class="keyword">sizeof</span>(buf) != <span class="keyword">sizeof</span>(buf)){</span><br><span class="line"> <span class="comment">//...</span></span><br><span class="line">}</span><br></pre></td></tr></table></figure><p>静态分配会简单简单一些</p><figure class="highlight c"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br></pre></td><td class="code"><pre><span class="line">INIT_KFIFO(test);</span><br><span class="line"></span><br><span class="line"><span class="meta">#<span class="meta-keyword">define</span> INIT_KFIFO(fifo) \</span></span><br><span class="line">(<span class="keyword">void</span>)({ \</span><br><span class="line">typeof(&(fifo)) __tmp = &(fifo); \</span><br><span class="line">struct __kfifo *__kfifo = &__tmp->kfifo; \</span><br><span class="line">__kfifo->in = <span class="number">0</span>; \</span><br><span class="line">__kfifo->out = <span class="number">0</span>; \</span><br><span class="line">__kfifo->mask = __is_kfifo_ptr(__tmp) ? <span class="number">0</span> : ARRAY_SIZE(__tmp->buf) - <span class="number">1</span>;\</span><br><span class="line">__kfifo->esize = <span class="keyword">sizeof</span>(*__tmp->buf); \</span><br><span class="line">__kfifo->data = __is_kfifo_ptr(__tmp) ? <span class="literal">NULL</span> : __tmp->buf; \</span><br><span class="line">})</span><br></pre></td></tr></table></figure><h3 id="自旋锁"><a class="markdownIt-Anchor" href="#自旋锁"></a> 自旋锁</h3><p>在 API 接口中还提供了使用自旋锁保护的 kfifo 保护的接口</p><figure class="highlight c"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">kfifo_in_spinlocked</span><br><span class="line">kfifo_out_spinlocked</span><br></pre></td></tr></table></figure><p>为什么要使用自旋锁保护的接口，他这里有一段解释</p><figure class="highlight c"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment">/*</span></span><br><span class="line"><span class="comment"> * Note about locking: There is no locking required until only one reader</span></span><br><span class="line"><span class="comment"> * and one writer is using the fifo and no kfifo_reset() will be called.</span></span><br><span class="line"><span class="comment"> * kfifo_reset_out() can be safely used, until it will be only called</span></span><br><span class="line"><span class="comment"> * in the reader thread.</span></span><br><span class="line"><span class="comment"> * For multiple writer and one reader there is only a need to lock the writer.</span></span><br><span class="line"><span class="comment"> * And vice versa for only one writer and multiple reader there is only a need</span></span><br><span class="line"><span class="comment"> * to lock the reader.</span></span><br><span class="line"><span class="comment"> */</span></span><br></pre></td></tr></table></figure><p>根据读写者的数量来判断是否需要使用自旋接口，它的实现如下</p><figure class="highlight c"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br></pre></td><td class="code"><pre><span class="line"><span class="meta">#<span class="meta-keyword">define</span>kfifo_in_spinlocked(fifo, buf, n, lock) \</span></span><br><span class="line">({ \</span><br><span class="line"><span class="keyword">unsigned</span> <span class="keyword">long</span> __flags; \</span><br><span class="line"><span class="keyword">unsigned</span> <span class="keyword">int</span> __ret; \</span><br><span class="line">spin_lock_irqsave(lock, __flags); \</span><br><span class="line">__ret = kfifo_in(fifo, buf, n); \</span><br><span class="line">spin_unlock_irqrestore(lock, __flags); \</span><br><span class="line">__ret; \</span><br><span class="line">})</span><br></pre></td></tr></table></figure><p><code>spin_lock_irqsave</code> 自旋锁且禁用本地中断，自旋锁能保证在多 CPU 系统中，一次只有一个 CPU 可以执行临界区代码。而禁用中断可以防止临界区代码执行期间发生上下文切换</p><p>其他 API</p><figure class="highlight c"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br></pre></td><td class="code"><pre><span class="line">kfifo_is_empty(fifo)<span class="comment">//是否为空</span></span><br><span class="line">kfifo_len(fifo)<span class="comment">//当前元素数量</span></span><br><span class="line">kfifo_reset(fifo)<span class="comment">//重置索引位置</span></span><br><span class="line">kfifo_size(fifo)<span class="comment">//队列大小</span></span><br><span class="line">kfifo_initialized(fifo)<span class="comment">//检查是否初始化，返回 mask 的值</span></span><br><span class="line">kfifo_in(fifo, buf, n)<span class="comment">//进队</span></span><br><span class="line">kfifo_out(fifo, buf, n)<span class="comment">//出对</span></span><br></pre></td></tr></table></figure><h3 id="总结"><a class="markdownIt-Anchor" href="#总结"></a> 总结</h3><ol><li>声明的大小为 大于 size的最小的 2 的幂等数，所以在进出队列的时候需要判断返回值，防止截断</li><li>根据读写者数量来判断是否使用 <code>spinlock</code> 接口</li></ol><h3 id="参考文档"><a class="markdownIt-Anchor" href="#参考文档"></a> 参考文档</h3><ol><li><a href="https://zhuanlan.zhihu.com/p/500070147" target="_blank" rel="noopener">https://zhuanlan.zhihu.com/p/500070147</a></li><li><a href="https://blog.csdn.net/fangye945a/article/details/86617551" target="_blank" rel="noopener">https://blog.csdn.net/fangye945a/article/details/86617551</a></li><li><a href="https://blog.csdn.net/qq_30027083/article/details/120697813" target="_blank" rel="noopener">https://blog.csdn.net/qq_30027083/article/details/120697813</a></li></ol>]]></content>
<summary type="html">
<p>做内核驱动开发的时候，从 Linux收包的时候用到了 <code>kfifo</code> 这种结构接收报文进行报文分析，这里学习一下</p>
<p><code>kfifo</code> 其实是一个无锁环形队列，主要用来解决进出速度不匹配的情况。当生产速度太快或多生产者的情
</summary>
<category term="Kernel" scheme="https://www.yuankang.top/categories/Kernel/"/>
<category term="Linux" scheme="https://www.yuankang.top/tags/Linux/"/>
<category term="kfifo" scheme="https://www.yuankang.top/tags/kfifo/"/>
<category term="Kernel" scheme="https://www.yuankang.top/tags/Kernel/"/>
</entry>
<entry>
<title>TCPIP 网络编程-12-IPC-FIFO</title>
<link href="https://www.yuankang.top/2024/02/07/Unix%20NetWork/TCPIP%20%E7%BD%91%E7%BB%9C%E7%BC%96%E7%A8%8B-12-IPC-FIFO/"/>
<id>https://www.yuankang.top/2024/02/07/Unix%20NetWork/TCPIP%20%E7%BD%91%E7%BB%9C%E7%BC%96%E7%A8%8B-12-IPC-FIFO/</id>
<published>2024-02-07T15:00:00.000Z</published>
<updated>2024-05-18T07:05:22.879Z</updated>
<content type="html"><![CDATA[<p>FIFO也叫命名管道，与管道类似，最大的差别在于 FIFO 在文件系统中拥有一个名称，打开方式与普通文件一样。这样就能够将 FIFO 用于非相关进程之间的通信(如客户端和服务器)</p><p><strong>FIFO特点</strong></p><ol><li><p>与管道一样，也是字节流</p></li><li><p>与管道一样，FIFO 也有一个写入端和读取端，并且从管道中读取数据的顺序与写入的顺序是一样的</p></li><li><p>与管道一样，当<strong>所有引用 FIFO 的描述符都被关闭</strong>之后，所有未被读取的数据会被丢弃</p></li><li><p><strong>FIFO是双向通信的</strong>，允许两个进程在同一个FIFO文件中进行读写操作</p></li><li><p>一旦 FIFO 被创建，任何进程都能够打开它，只要它能够通过常规的文件权限检测</p></li><li><p>使用 <code>open(path, O_RDONLY)</code> 标记，打开 FIFO 以便读取数据，将会阻塞直到另一个进程打开 FIFO 以写入数据为止。相应地，打开一个 FIFO 以写入数据将会阻塞直到另一个进程打开 FIFO 以读取数据为止，虽然 <code>O_RDWR</code> 可以绕开FIFO阻塞行为，但是不要用</p><ul><li>虽然使用 <code>open(path, O_RDWR)</code>就会立即返回，但无法使用返回的文件描述 符在 FIFO 上读取和写入数据。SUSv3 明确指出以 <code>O_RDWR</code> 标记打开一个 FIFO 的结果是未知的，<strong>因此出于可移植性的原因，开发人员不应该使用这项技 术</strong>。可改为<code>O_NONBLOCK</code> 标记替代</li><li>使用 <code>O_RDWR</code> 调用 <code>open()</code> 之后，调用进程在从返回的文件描述符中读取数据时<strong>永远都不会看到文件结束</strong>，因为永远都至少存在一个文件描述符被打开着以等待数据被写入 FIFO，即进程从中读取数据的那个描述符。</li></ul></li></ol><p>使用命令 <code>$ mkfifo [-m mode] pathname</code> 构建FIFO，当使用 <code>ls –l</code> 列出文件时，FIFO 文件在第一列的类型为 p，<code>ls –F</code> 会在 FIFO 路径名后面附加上一个管道符（|）。</p><figure class="highlight shell"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">drwxrwxrwx 30 777 root 4096 Feb 29 08:02 directory</span><br><span class="line">-rwxrwxrwx 1 yuankang yuankang 4716 Sep 21 01:01 file</span><br><span class="line">prw-r--r-- 1 root root 0 Mar 7 04:44 hello#管道</span><br></pre></td></tr></table></figure><p>使用程序构建FIFO</p><figure class="highlight c"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br></pre></td><td class="code"><pre><span class="line"><span class="meta">#<span class="meta-keyword">include</span> <span class="meta-string"><sys/stat.h></span></span></span><br><span class="line"></span><br><span class="line"><span class="comment">/**</span></span><br><span class="line"><span class="comment"> * @brief 创建一个名为 pathname 的全新的 FIFO</span></span><br><span class="line"><span class="comment"> * @param pathname 管道路径</span></span><br><span class="line"><span class="comment"> * @param mode 参数指定了新 FIFO 的权限</span></span><br><span class="line"><span class="comment"> * @return 0 success; -1 failed</span></span><br><span class="line"><span class="comment"> * */</span></span><br><span class="line"><span class="function"><span class="keyword">int</span> <span class="title">mkfifo</span><span class="params">(<span class="keyword">const</span> <span class="keyword">char</span> *pathname, <span class="keyword">mode_t</span> mode)</span></span>;</span><br></pre></td></tr></table></figure><h3 id="双重管道线"><a class="markdownIt-Anchor" href="#双重管道线"></a> 双重管道线</h3><p>tee 命令，它将其从标准输入中读取到的数据 复制两份并输出：一份写入到标准输出，另一份写入到通过命令行参数指定的文件中。</p><figure class="highlight shell"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line"><span class="meta">$</span><span class="bash"> mkfifo hello</span></span><br><span class="line"><span class="meta">$</span><span class="bash"> wc -l < hello &</span></span><br><span class="line"><span class="meta">$</span><span class="bash"> ls -l | teel hello | sort -k5n</span></span><br></pre></td></tr></table></figure><p><img src="https://images-1307117474.cos.ap-guangzhou.myqcloud.com/blogs/Blog_3/202403071257117.png" alt="image-20240307125729988" /></p><p><code>ls -l</code>：这部分是一个ls命令，带有-l选项，表示以长格式（详细信息）显示当前目录中的文件和文件夹列表。</p><p><code>sort -k5n</code>：这部分是对ls -l命令的输出进行排序。sort命令用于对文本文件中的行进行排序，<code>-k5n</code>表示按照第5列的数值(文件大小)进行排序，n代表按照数字顺序排序</p><h3 id="cs服务"><a class="markdownIt-Anchor" href="#cs服务"></a> CS服务</h3><p>为了实现多客户端与服务端的通信</p><ol><li>虽然 FIFO 是双向的，但是当接收服务端响应的时候，一个 FIFO 会在多个客户端之间发生竞争。所以每一个进程创建一个 FIFO 用来接收消息。那样一个服务器也只需要一个 FIFO</li><li>服务端收到多个客户端的消息之后，如何给客户端响应。就需要在消息中带上客户端的管道名称</li><li>由于 FIFO 也是字节流，所以需要指定消息的长度，这里使用 proto 进行消息传输</li></ol><p>于是就有了示例代码 <code>network-ip/apps/ipcs/02</code></p><p>服务端运行结果：</p><figure class="highlight shell"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line"><span class="meta">[root]#</span><span class="bash">./fifo_svr </span></span><br><span class="line">[server/server.c:45 main info] recv client request ./client_fifo_file_0 22 </span><br><span class="line">[server/server.c:61 main info] write finished </span><br><span class="line">[server/server.c:65 main info] child exit </span><br><span class="line">[server/server.c:73 main info] parent exit</span><br></pre></td></tr></table></figure><p>客户端运行结果：</p><figure class="highlight shell"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line"><span class="meta">[root]#</span><span class="bash">./fifo_cli </span></span><br><span class="line">[client/client.c:74 main info] send fifo :./client_fifo_file_0 22 </span><br><span class="line">[client/client.c:50 main info] child recv hello world </span><br><span class="line">[client/client.c:56 main info] child exit </span><br><span class="line">[client/client.c:82 main info] parent exit</span><br></pre></td></tr></table></figure><h3 id="参考文档"><a class="markdownIt-Anchor" href="#参考文档"></a> 参考文档</h3><ol><li>《UNIX 网络编程卷 2》</li><li>《UNIX 系统编程 44 章》</li></ol>]]></content>
<summary type="html">
<p>FIFO也叫命名管道，与管道类似，最大的差别在于 FIFO 在文件系统中拥有一个名称，打开方式与普通文件一样。这样就能够将 FIFO 用于非相关进程之间的通信(如客户端和服务器)</p>
<p><strong>FIFO特点</strong></p>
<ol>
<li>
<p
</summary>
<category term="Unix NetWork" scheme="https://www.yuankang.top/categories/Unix-NetWork/"/>
<category term="Unix" scheme="https://www.yuankang.top/tags/Unix/"/>
<category term="Unix NetWork" scheme="https://www.yuankang.top/tags/Unix-NetWork/"/>
</entry>
<entry>
<title>TCPIP 网络编程-11-IPC-管道</title>
<link href="https://www.yuankang.top/2024/01/31/Unix%20NetWork/TCPIP%20%E7%BD%91%E7%BB%9C%E7%BC%96%E7%A8%8B-11-IPC-%E7%AE%A1%E9%81%93/"/>
<id>https://www.yuankang.top/2024/01/31/Unix%20NetWork/TCPIP%20%E7%BD%91%E7%BB%9C%E7%BC%96%E7%A8%8B-11-IPC-%E7%AE%A1%E9%81%93/</id>
<published>2024-01-31T15:00:00.000Z</published>
<updated>2024-05-18T07:06:50.056Z</updated>
<content type="html"><![CDATA[<p>管道是linux系统中最常用的进程间通信方式，在命令行中使用如下命令：</p><figure class="highlight shell"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">ls | wc -l</span><br></pre></td></tr></table></figure><p>就是使用了管道在 ls进程 和 wc进程间传递数据</p><img src="https://images-1307117474.cos.ap-guangzhou.myqcloud.com/blogs/Blog_3/image-20240306223602517.png" alt="image-20240306223602517" style="zoom:50%;" /><p>实际上，这两个进程并不知道管道的存在，它们只是从标准文件描述符中读取数据和写入数据</p><p>管道的特点</p><ol><li>一个管道是一个字节流，及消息不存在边界可以读取任意大小的数据块，从管道中读取出来的字节顺序与写入时一致，无法使用 <code>lseek()</code> 随机访问数据</li><li>从一个当前为空的管道中读取数据将会被阻塞直到管道有数据为止。如果管道的写入端被关闭，那么从管道中读取数据的进程在读完管道中剩余的所有数据之后将会看到文件结束(<strong>即 read()返回 0</strong>）</li><li>管道是单向的</li><li>写入不超过 <code>PIPE_BUF</code> 字节的操作是原子的，除非写入操作被一个信号处理器中断了(返回已经成功传输到管道中的字节数)。如果多个进程写入同一个管道，如果他们在同一时刻写入的数据量不超过 <code>PIPE_BUF</code> 字节，那么就可以确保写入的数据不会发生相互混合。否则内核可能会将数据分割成几个较小的片段来传输(<code>write()</code>调用会阻塞直到所有数据就被写入到管道中)</li><li>管道的容量是有限的</li></ol><img src="https://images-1307117474.cos.ap-guangzhou.myqcloud.com/blogs/Blog_3/image-20240302203900843.png" alt="image-20240302203900843" style="zoom:50%;" /><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br></pre></td><td class="code"><pre><span class="line"></span><br><span class="line"></span><br><span class="line">#include <limits.h></span><br><span class="line">#include <stdio.h></span><br><span class="line"></span><br><span class="line">int main() {</span><br><span class="line"> printf("%ld\n", (long)PIPE_BUF);</span><br><span class="line"> return 0;</span><br><span class="line">}</span><br></pre></td></tr></table></figure><h3 id="创建和使用管道"><a class="markdownIt-Anchor" href="#创建和使用管道"></a> 创建和使用管道</h3><figure class="highlight c"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br></pre></td><td class="code"><pre><span class="line"><span class="meta">#<span class="meta-keyword">include</span> <span class="meta-string">"unistd.h"</span></span></span><br><span class="line"></span><br><span class="line"><span class="comment">/**</span></span><br><span class="line"><span class="comment"> * @brief 构建管道</span></span><br><span class="line"><span class="comment"> * @param</span></span><br><span class="line"><span class="comment"> * filedes[0] 通过管道接收数据时使用的文件描述符，管道出口</span></span><br><span class="line"><span class="comment"> * filedes[1] 通过管道传输数据时使用的文件描述符，管道入口</span></span><br><span class="line"><span class="comment"> * @return 0 成功; -1 失败</span></span><br><span class="line"><span class="comment">*/</span></span><br><span class="line"><span class="function"><span class="keyword">int</span> <span class="title">pipe</span><span class="params">(<span class="keyword">int</span> filedes[<span class="number">2</span>])</span></span>;</span><br></pre></td></tr></table></figure><p>**问题 1：**有了管道之后，如何与两个进程通信？</p><p>想到了之前的 <code>fork</code>，会复制文件描述符号，那样一个进程使用一边就能进行通信了。</p><img src="https://images-1307117474.cos.ap-guangzhou.myqcloud.com/blogs/Blog_3/Snipaste_2024-03-02_15-49-50.png" alt="Snipaste_2024-03-02_15-49-50" style="zoom:50%;" /><p>示例地址：<a href="https://github.com/XBoom/network-ip.git" target="_blank" rel="noopener">https://github.com/XBoom/network-ip.git</a> 中的 <code>apps/socket/15/</code></p><figure class="highlight c"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">[root]#./server </span><br><span class="line">[<span class="number">15</span>/server.c:<span class="number">28</span> main info] parent recv Who are you?</span><br></pre></td></tr></table></figure><p>**问题 2：**既然父子进程都有管道的出入口，可否使用一个管道进行双向通信？</p><img src="https://images-1307117474.cos.ap-guangzhou.myqcloud.com/blogs/Blog_3/image-20240302155045671.png" alt="image-20240302155045671" style="zoom:50%;" /><p>示例地址：<a href="https://github.com/XBoom/network-ip.git" target="_blank" rel="noopener">https://github.com/XBoom/network-ip.git</a> 中的 <code>apps/socket/15/</code></p><figure class="highlight c"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">[root]#./client </span><br><span class="line">[<span class="number">15</span>/client.c:<span class="number">24</span> main info] child recv Who are you?</span><br></pre></td></tr></table></figure><p>答案是不行的，同一个管道，先执行的 <code>read</code> 会将 <code>write</code> 全部读出来，所以就需要创建两个管道</p><img src="https://images-1307117474.cos.ap-guangzhou.myqcloud.com/blogs/Blog_3/image-20240302155144907.png" alt="image-20240302155144907" style="zoom:50%;" /><p>最后，构建一个回声示例，服务端将收到与回复的消息都存入到文件中</p><p>示例地址：<a href="https://github.com/XBoom/network-ip.git" target="_blank" rel="noopener">https://github.com/XBoom/network-ip.git</a> 中的 <code>apps/socket/16/</code></p><figure class="highlight shell"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br></pre></td><td class="code"><pre><span class="line"><span class="meta">[root]#</span><span class="bash">./server 5555</span></span><br><span class="line">[16/server.c:105 main info] write hello world success </span><br><span class="line">[16/server.c:77 main info] fwrite hello world 30 </span><br><span class="line">[16/server.c:10 child_proc info] Child 1054 exited </span><br><span class="line"></span><br><span class="line"><span class="meta">[root]#</span><span class="bash">./client 127.0.0.1 5555</span></span><br><span class="line">[16/client.c:37 main info] recv hello world</span><br></pre></td></tr></table></figure><p>这里有一个注意事项，就是 <strong>必须要关闭管道的读取端和写入端的未使用 文件描述符</strong></p><ol><li>读取数据的进程需要关闭其持有的管道的写入描述符，这样<ul><li>当其他进程完成输出并关闭其写入描述符之后，读者就能够看到文件结束(在读完管道中的数据之后)。否则，其他进程关闭了写入描述符之后，读者看不到文件结束，<code>read()</code>将会阻塞以等待数据，因为内核知道至少还存在一个管道的写入描述符打开着，即读取进程自己打开了这个描述符</li></ul></li><li>写入进程关闭其持有的管道的读取描述符。是因为<ul><li>当一个进程试图向一个管道中写入数据但没有任何进程拥有该管道的打开着的读取描述符时，内核会向写入进程发送 一个 <code>SIGPIPE</code> 信号。在默认情况下，这个信号会杀死一个进程。但进程可以捕获或忽略该信号，这样就会导致管道上的 <code>write()</code> 操作因 <code>EPIPE</code> 错误（已损坏的管道）而失败。收到 SIGPIPE 信号或得到 EPIPE 错误对于标示出管道的状态是有用的，这就是为何需要关闭管道的未使用读取描述符的原因</li><li>如果写入进程没有关闭管道的读取端，那么即使在其他进程已经关闭了管道的读取端之 后写入进程仍然能够向管道写入数据，最后写入进程会将数据充满整个管道，后续的写入请求会被永远阻塞。</li><li>只有当所有进程中所有引用一个管道的文件描 述符被关闭之后才会销毁该管道以及释放该管道占用的资源以供其他进程复用。此时，管道中所有未读取的数据都会丢失。</li></ul></li></ol><h3 id="进程同步"><a class="markdownIt-Anchor" href="#进程同步"></a> 进程同步</h3><p>使用管道来同步父进程和子进程的动作以防止出现竞争条件，创建多个子进程，每个子进程都完成某个动作，父进程等待直到所有子进程完成了自己的动作为止</p><p>示例地址：<a href="https://github.com/XBoom/network-ip.git" target="_blank" rel="noopener">https://github.com/XBoom/network-ip.git</a> 中的 <code>apps/ipcs/01/01.c</code></p><figure class="highlight shell"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br></pre></td><td class="code"><pre><span class="line"><span class="meta">[root]#</span><span class="bash">./demo </span></span><br><span class="line">[01/01.c:26 main info] child 408 sleep 0 to close pipe </span><br><span class="line">[01/01.c:26 main info] child 407 sleep 1 to close pipe </span><br><span class="line">[01/01.c:26 main info] child 409 sleep 1 to close pipe </span><br><span class="line">[01/01.c:26 main info] child 406 sleep 3 to close pipe </span><br><span class="line">[01/01.c:26 main info] child 410 sleep 3 to close pipe </span><br><span class="line">[01/01.c:36 main info] program exit 0</span><br></pre></td></tr></table></figure><p>可以看到，<code>read</code> 阻塞直到所有的写端都关闭为止，而多个信号无法排队的事实使得信号不适用于这种情形。反过来其实也行，类型一个进程广播其他进程(《UNIX 系统编程手册》中说不能像信号一样广播给其他进程，其实也是可以的)</p><p>示例地址：<a href="https://github.com/XBoom/network-ip.git" target="_blank" rel="noopener">https://github.com/XBoom/network-ip.git</a> 中的 <code>apps/ipcs/01/02.c</code></p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br></pre></td><td class="code"><pre><span class="line">[root]#./demo </span><br><span class="line">[01/02.c:29 main info] program exit </span><br><span class="line">[01/02.c:23 main info] child 488 read end </span><br><span class="line">[01/02.c:23 main info] child 490 read end </span><br><span class="line">[01/02.c:23 main info] child 489 read end </span><br><span class="line">[01/02.c:23 main info] child 491 read end </span><br><span class="line">[root]#[01/02.c:23 main info] child 492 read end</span><br></pre></td></tr></table></figure><p>不能通过管道发送消息实现广播，因为管道的数据只会被一个进程读取到</p><h3 id="过滤器"><a class="markdownIt-Anchor" href="#过滤器"></a> 过滤器</h3><p><strong>当管道被创建之后，为管道的两端分配的文件描述符是可用描述符中数值最小的两个</strong>。通常情况下，进程已经使用了描述符 0、1 和 2，因此会为管道分配一些数值更大的描述符。</p><p>使用管道连接两个过滤器（即从 <code>stdin</code> 读取和写入到 <code>stdout</code> 的程序）使得一个程序的标准输出被定向到管道中，而另一个程序的标准输入则从管道中读取</p><img src="https://images-1307117474.cos.ap-guangzhou.myqcloud.com/blogs/Blog_3/image-20240306223602517.png" alt="image-20240306223602517" style="zoom:50%;" /><p>示例地址：<a href="https://github.com/XBoom/network-ip.git" target="_blank" rel="noopener">https://github.com/XBoom/network-ip.git</a> 中的 <code>apps/ipcs/01/03.c</code></p><p>将 <code>fds[1]</code> 重定向到标准输出 <code>(STDOUT_FILENO)</code>，将 <code>fds[0]</code> 重定向到标准输入 <code>(STDIN_FILENO)</code>，<code>ls</code> 命令会将当前目录的内容写入到标准输出，由于标准输出已经被重定向到管道，所以内容会被写入到管道中，<code>wc</code> 命令会统计从标准输入读取到的文件数量，并显示在屏幕上</p><blockquote><p>为什么最后的结果会显示在屏幕上</p><p>因为 第二个子进程的 <code>STDOUT_FILENO</code> 并没有指向管道，<strong>记住描述符是会复制的</strong>，第二个程序的输出就显示在屏幕上了</p></blockquote><figure class="highlight shell"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line">// 使用 dup 函数复制文件描述符 0</span><br><span class="line">int fd1 = dup(0);</span><br><span class="line"></span><br><span class="line">// 使用 dup2 函数复制文件描述符 0 到文件描述符 3</span><br><span class="line">int fd2 = dup2(0, 3);</span><br></pre></td></tr></table></figure><p><code>execlp</code> 是一个 Unix 系统函数，用于执行一个新的程序，并替换当前进程</p><figure class="highlight c"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line"><span class="function"><span class="keyword">int</span> <span class="title">execlp</span><span class="params">(<span class="keyword">const</span> <span class="keyword">char</span> *file, <span class="keyword">const</span> <span class="keyword">char</span> *arg0, ..., <span class="keyword">const</span> <span class="keyword">char</span> *argN, (<span class="keyword">char</span> *)<span class="literal">NULL</span>)</span></span>;</span><br></pre></td></tr></table></figure><p>标准输入输出与程序输出的结果一致</p><figure class="highlight shell"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line"><span class="meta">[root]#</span><span class="bash">./demo </span></span><br><span class="line">5</span><br><span class="line"><span class="meta">[root]#</span><span class="bash">ls | wc -l</span></span><br><span class="line">5</span><br></pre></td></tr></table></figure><h3 id="管道与-shell"><a class="markdownIt-Anchor" href="#管道与-shell"></a> 管道与 Shell</h3><p>管道还能用于执行 shell 命令并读取其输出或向其发送一些输入</p><figure class="highlight c"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line"><span class="meta">#<span class="meta-keyword">include</span> <span class="meta-string"><stdio.h></span></span></span><br><span class="line"></span><br><span class="line"><span class="function">FILE *<span class="title">popen</span><span class="params">(<span class="keyword">const</span> <span class="keyword">char</span> *command, <span class="keyword">const</span> <span class="keyword">char</span> *mode)</span></span>;</span><br><span class="line"><span class="function"><span class="keyword">int</span> <span class="title">pclose</span><span class="params">(FILE *stream)</span></span>;</span><br></pre></td></tr></table></figure><ul><li><code>popen()</code> 函数创建了一个管道，然后创建了一个子进程来执行 shell，而 shell 又创建了一个子进程来执行 <code>command</code> 字符串。<code>mode</code> 参数是一个字符串，它确定调用进程是从管道中读取数据（mode == r）还是将数据写入到管道中（mode == w）。</li><li><code>pclose()</code>函数关闭管道并等待子进程中的 shell 终止，而 <code>fclose()</code> 不会等待</li></ul><p>由于管道是单向的，因此无法在执行的 <code>command</code> 中进行双向通信。mode 的取值确定了所执行的命令的标准输出是连接到管 道的写入端还是将其标准输入连接到管道的读取端</p><img src="https://images-1307117474.cos.ap-guangzhou.myqcloud.com/blogs/Blog_3/image-20240307001827277.png" alt="image-20240307001827277" style="zoom:50%;" /><p>示例地址：<a href="https://github.com/XBoom/network-ip.git" target="_blank" rel="noopener">https://github.com/XBoom/network-ip.git</a> 中的 <code>apps/ipcs/01/04.c</code>，程序重复读取一个文件名通配符模式，然后使用 <code>popen()</code>或者这个模式传入 <code>ls</code> 命令之后的结果</p><figure class="highlight c"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br></pre></td><td class="code"><pre><span class="line">[root]#./demo </span><br><span class="line">pattern: d</span><br><span class="line">[<span class="number">01</span>/<span class="number">04.</span>c:<span class="number">35</span> main info] popenCmd:ls | grep d </span><br><span class="line">[<span class="number">01</span>/<span class="number">04.</span>c:<span class="number">44</span> main info] file[<span class="number">0</span>] name: README.md </span><br><span class="line">[<span class="number">01</span>/<span class="number">04.</span>c:<span class="number">44</span> main info] file[<span class="number">1</span>] name: demo </span><br><span class="line">[<span class="number">01</span>/<span class="number">04.</span>c:<span class="number">44</span> main info] file[<span class="number">2</span>] name: demo.o </span><br><span class="line">pattern: de</span><br><span class="line">[<span class="number">01</span>/<span class="number">04.</span>c:<span class="number">35</span> main info] popenCmd:ls | grep de </span><br><span class="line">[<span class="number">01</span>/<span class="number">04.</span>c:<span class="number">44</span> main info] file[<span class="number">0</span>] name: demo </span><br><span class="line">[<span class="number">01</span>/<span class="number">04.</span>c:<span class="number">44</span> main info] file[<span class="number">1</span>] name: demo.o</span><br></pre></td></tr></table></figure><p>如果使用 <code>popen(pathname, "w")</code> 来写入数据，为了让子进程在管道的另一端能立即读取到数据，需要调用 <code>fflush()</code> 或者 使用 <code>setbuf(fp, NULL)</code> 来禁用 <code>stdio</code> 缓冲</p><h3 id="参考文档"><a class="markdownIt-Anchor" href="#参考文档"></a> 参考文档</h3><ol><li>《TCPIP 网络编程》</li><li>《Unix 系统编程 44 章》</li></ol>]]></content>
<summary type="html">
<p>管道是linux系统中最常用的进程间通信方式，在命令行中使用如下命令：</p>
<figure class="highlight shell"><table><tr><td class="gutter"><pre><span class="line">1</span><br
</summary>
<category term="Unix NetWork" scheme="https://www.yuankang.top/categories/Unix-NetWork/"/>
<category term="Unix" scheme="https://www.yuankang.top/tags/Unix/"/>
<category term="Unix NetWork" scheme="https://www.yuankang.top/tags/Unix-NetWork/"/>
</entry>
<entry>
<title>TCPIP 网络编程-09-多进程服务器</title>
<link href="https://www.yuankang.top/2024/01/25/Unix%20NetWork/TCPIP%20%E7%BD%91%E7%BB%9C%E7%BC%96%E7%A8%8B-09-%E5%A4%9A%E8%BF%9B%E7%A8%8B%E6%9C%8D%E5%8A%A1%E5%99%A8/"/>
<id>https://www.yuankang.top/2024/01/25/Unix%20NetWork/TCPIP%20%E7%BD%91%E7%BB%9C%E7%BC%96%E7%A8%8B-09-%E5%A4%9A%E8%BF%9B%E7%A8%8B%E6%9C%8D%E5%8A%A1%E5%99%A8/</id>
<published>2024-01-25T15:00:00.000Z</published>
<updated>2024-03-04T15:16:04.675Z</updated>
<content type="html"><![CDATA[<p>到目前为止都是循环客户端连接，处理完一个客户端再处理另外一个。现实是一个服务器可以同时处理多个客户端请求。这里通过多进程实现网络服务器</p><p><strong>第一步，使用 <code>fork</code> 分配多个进程处理客户端请求</strong></p><p>所有进程都会从操作系统分配到 ID，<strong>1 要分配给操作系统启动后的首个进程，因此用户进程无法得到进程 ID 1</strong></p><figure class="highlight c"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line"><span class="meta">#<span class="meta-keyword">include</span> <span class="meta-string"><unistd.h></span></span></span><br><span class="line"></span><br><span class="line"><span class="comment">//成功时返回子进程 ID，失败返回 -1</span></span><br><span class="line"><span class="function"><span class="keyword">pid_t</span> <span class="title">fork</span><span class="params">(<span class="keyword">void</span>)</span></span>;</span><br></pre></td></tr></table></figure><blockquote><p>这样记返回值：因为父进程无法获取子进程 id，所以需要在 fork 直接将 pid_t 返回给父进程，通过 <code>getppid</code> 可以获取父进程 id</p></blockquote><p>示例地址：<a href="https://github.com/XBoom/network-ip.git" target="_blank" rel="noopener">https://github.com/XBoom/network-ip.git</a> 中的 <code>apps/socket/12/</code></p><figure class="highlight shell"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br></pre></td><td class="code"><pre><span class="line"><span class="meta">[root]#</span><span class="bash">./client </span></span><br><span class="line">[12/client.c:17 main info] parent before 0 0 0 0 </span><br><span class="line">[12/client.c:22 main info] parent after 2 2 2 2 </span><br><span class="line">[12/client.c:36 main info] main after 2 2 2 </span><br><span class="line">[12/client.c:28 main info] son before 0 0 0 0 </span><br><span class="line">[12/client.c:33 main info] son after 3 3 3 3 </span><br><span class="line">[12/client.c:36 main info] main after 3 3 3</span><br></pre></td></tr></table></figure><p>从上面的运行结果来看</p><ul><li>虽然子进程复制了父进程的内存结构，但是两者拥有的是独立的内存结构，不会相互影响</li><li><code>fork</code> 后面的代码指令也会复制一份，独立运行</li></ul><p><strong>第二步，有了子进程，那么如何管理子进程的死亡</strong></p><p>父子进程的退出情况一共存在以下三种情况</p><ul><li>父进程比子进程先退出</li><li>父进程比子进程后退出</li><li>子进程一直不退出</li></ul><p>孤儿进程(Orphan Process)和僵尸进程(Zombie Process)是与进程状态相关的两个概念，通常在操作系统中用来描述进程的状态变化。</p><ol><li><strong>孤儿进程（Orphan Process）:</strong><ul><li>当一个父进程创建了子进程，但父进程在子进程之前终止，而子进程继续运行时，子进程就成为孤儿进程。</li><li>孤儿进程会被操作系统的init进程（在Unix/Linux系统中通常是进程号1的init）接管。init进程会成为孤儿进程的新父进程，并负责收养和管理这个孤儿进程。</li><li>孤儿进程不会影响系统的正常运行，因为它已经有了新的父进程（init进程）来照顾它。</li></ul></li><li><strong>僵尸进程（Zombie Process）:</strong><ul><li>当一个子进程终止时，它并不会立即从系统中移除，而是留下一个僵尸进程。</li><li>僵尸进程仍然占用系统资源，但它不再执行任何代码。僵尸进程的存在是为了让父进程获取子进程的退出状态。</li><li>父进程可以通过调用<code>wait()</code>或<code>waitpid()</code>等系统调用来获取子进程的退出状态，一旦获取完成，僵尸进程就会被完全移除。</li><li>如果父进程没有及时处理子进程的退出状态，可能会导致系统中积累大量的僵尸进程，从而占用系统资源。</li></ul></li></ol><p>防止僵尸进程的方法通常是在父进程中使用合适的系统调用来等待子进程的终止，并获取其退出状态。</p><blockquote><p>孤儿进程会被 init 进程（PID为1的进程）领养，init 进程会负责回收孤儿进程的资源，因此孤儿进程不会变成僵尸进程</p></blockquote><p>也就是说，如果父进程比子进程后退出，且父进程并没有管子进程，那么就会僵尸进程，为了防止僵尸进程，可用如下函数销毁僵尸进程</p><figure class="highlight c"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line"><span class="meta">#<span class="meta-keyword">include</span> <span class="meta-string"><sys/wait.h></span></span></span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">pid_t</span> <span class="title">wait</span><span class="params">(<span class="keyword">int</span> * statloc)</span></span>;</span><br></pre></td></tr></table></figure><p>如果调用此函数的时候已经有子进程终止，那么子进程 终止时传递的返回值(exit函数返回值、main函数的返回值) 将保存到该函数的参数所指内存空间</p><p>函数参数 <code>statloc</code> 所指向的单元中还含有其他信息，需要宏进行分离</p><ul><li><code>WIFEXITED</code> 子进程正常终止时返回 true（错误系统idaoyong，信号，异常退出，资源耗尽等表示异常退出）</li><li><code>WEXITSTATUS</code> 返回子进程的返回值</li></ul><figure class="highlight c"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">if</span> (WIFEXITED(status)) {</span><br><span class="line"> <span class="comment">// 子进程正常终止</span></span><br><span class="line"> <span class="built_in">printf</span>(<span class="string">"子进程退出状态: %d\n"</span>, WEXITSTATUS(status));</span><br><span class="line">}</span><br></pre></td></tr></table></figure><p>调用 <code>wait</code> 函数时，如果没有已终止的子进程，那么程序将阻塞直到有子进程终止，就有了第二种</p><figure class="highlight c"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br></pre></td><td class="code"><pre><span class="line"><span class="meta">#<span class="meta-keyword">include</span> <span class="meta-string"><sys/wait.h></span></span></span><br><span class="line"><span class="comment">/*</span></span><br><span class="line"><span class="comment"> * pid 等待终止的目标子进程的ID,若传递-1，则与wait函数相同，可以等待任意子进程终止</span></span><br><span class="line"><span class="comment"> * statloc 与wait的含义相同</span></span><br><span class="line"><span class="comment"> * option WNOHANG,即使没有终止的子进程也不会进入阻塞状态，而是返回0并退出函数</span></span><br><span class="line"><span class="comment">*/</span></span><br><span class="line"><span class="function"><span class="keyword">pid_t</span> <span class="title">waitpid</span><span class="params">(<span class="keyword">pid_t</span> pid, <span class="keyword">int</span> *staloc, <span class="keyword">int</span> options)</span></span>;</span><br></pre></td></tr></table></figure><p>示例地址：<a href="https://github.com/XBoom/network-ip.git" target="_blank" rel="noopener">https://github.com/XBoom/network-ip.git</a> 中的 <code>apps/socket/13/</code> ，运行结果</p><figure class="highlight c"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br></pre></td><td class="code"><pre><span class="line">[root]#./client </span><br><span class="line">[<span class="number">13</span>/client.c:<span class="number">20</span> main info] wait child </span><br><span class="line">[<span class="number">13</span>/client.c:<span class="number">20</span> main info] wait child </span><br><span class="line">[<span class="number">13</span>/client.c:<span class="number">20</span> main info] wait child </span><br><span class="line">[<span class="number">13</span>/client.c:<span class="number">20</span> main info] wait child </span><br><span class="line">[<span class="number">13</span>/client.c:<span class="number">12</span> main info] child <span class="built_in">end</span> </span><br><span class="line">[<span class="number">13</span>/client.c:<span class="number">20</span> main info] wait child </span><br><span class="line">[<span class="number">13</span>/client.c:<span class="number">23</span> main info] child has exited <span class="number">7</span> </span><br><span class="line">[<span class="number">13</span>/client.c:<span class="number">25</span> main info] child pid <span class="number">119</span> status <span class="number">7</span></span><br></pre></td></tr></table></figure><p>可以看到 <code>waitpid</code> 并没有阻塞，而是看一下当前有没有子进程退出，如果没有则返回 0 并退出。</p><p>但是父进程不能啥也不干，就循环判断子进程是否退出了，这个时候就用到了信号</p><figure class="highlight c"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment">/**</span></span><br><span class="line"><span class="comment">* 返回类型：void (*)(int)，表示返回一个指向参数为整型、返回类型为 void 的函数的指针</span></span><br><span class="line"><span class="comment">* 函数名：signal，是函数的名称。</span></span><br><span class="line"><span class="comment">* 参数列表：(int signo, void (*func)(int))，包括两个参数：</span></span><br><span class="line"><span class="comment"> signo：整型，表示信号的编号。</span></span><br><span class="line"><span class="comment"> (*func)(int)：一个指向参数为整型、返回类型为 void 的函数的指针，用于指定处理该信号时要调用的函数</span></span><br><span class="line"><span class="comment">*/</span></span><br><span class="line"><span class="keyword">void</span> (*signal(<span class="keyword">int</span> signo, <span class="keyword">void</span> (*func)(<span class="keyword">int</span>)))(<span class="keyword">int</span>);</span><br><span class="line"></span><br><span class="line"></span><br><span class="line"><span class="comment">/**</span></span><br><span class="line"><span class="comment"> * signo 信号信息</span></span><br><span class="line"><span class="comment"> * act 对应与第一个参数的信号处理函数</span></span><br><span class="line"><span class="comment"> * oldact 通过次参数获取之前注册的信号处理函数指针，若不需要则传递 0</span></span><br><span class="line"><span class="comment">*/</span></span><br><span class="line"><span class="function"><span class="keyword">int</span> <span class="title">sigaction</span><span class="params">(<span class="keyword">int</span> signo, <span class="keyword">const</span> struct sigaction *act, struct sigaction * oldact)</span></span>;</span><br></pre></td></tr></table></figure><p><code>signal</code> 很少使用，仅仅是为了向前兼容。所以最好使用 <code>sigaction</code> 处理信号, 常用的处理信号有</p><ul><li><code>SIGALRM</code>: 已经通过调用alarm函数注册的时间，通过 alarm 函数注册的时间表示：在接收到 SIGALRM 信号之前的秒数。当调用 <code>alarm(seconds)</code> 后，系统会在指定的秒数后发送 SIGALRM 信号给调用进程</li><li><code>SIGINT</code>: 输入 CTRL + C</li><li><code>SIGCHLD</code>: 子进程终止</li></ul><p><strong>第四步，基于前面的内容实现一个并发回声服务器</strong></p><p>示例地址：<a href="https://github.com/XBoom/network-ip.git" target="_blank" rel="noopener">https://github.com/XBoom/network-ip.git</a> 中的 <code>apps/socket/14/</code> ，运行结果</p><figure class="highlight shell"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br></pre></td><td class="code"><pre><span class="line"><span class="meta">[root]#</span><span class="bash">./server 5555</span></span><br><span class="line">[14/server.c:83 main info] client disconnected </span><br><span class="line">[14/server.c:19 read_childproc info] remove proc id: 308 </span><br><span class="line">[14/server.c:83 main info] client disconnected </span><br><span class="line">[14/server.c:19 read_childproc info] remove proc id: 310 </span><br><span class="line"></span><br><span class="line"><span class="meta">#</span><span class="bash">client1</span></span><br><span class="line"><span class="meta">[root]#</span><span class="bash">./client 127.0.0.1 5555</span></span><br><span class="line">[14/client.c:30 main info] recv hello world </span><br><span class="line"></span><br><span class="line"><span class="meta">#</span><span class="bash">client2</span></span><br><span class="line"><span class="meta">[root]#</span><span class="bash">./client 127.0.0.1 5555</span></span><br><span class="line">[14/client.c:30 main info] recv hello world</span><br></pre></td></tr></table></figure><p>中间使用 <code>fork</code> 出来的子进程处理客户端发送的请求，通过信号处理子进程退出</p><figure class="highlight c"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line"><span class="class"><span class="keyword">struct</span> <span class="title">sigaction</span> <span class="title">act</span>;</span></span><br><span class="line">act.sa_handler = read_childproc;</span><br><span class="line">sigemptyset(&act.sa_mask);</span><br><span class="line">act.sa_flags = <span class="number">0</span>;</span><br><span class="line">sigaction(SIGCHLD, &act, <span class="number">0</span>); <span class="comment">//SIGCHLD 子进程终止</span></span><br></pre></td></tr></table></figure><p>而信号处理函数仅干了一件事，就是处理任意退出的子进程</p><figure class="highlight c"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment">//子进程处理</span></span><br><span class="line"><span class="function"><span class="keyword">void</span> <span class="title">read_childproc</span><span class="params">(<span class="keyword">int</span> sig)</span></span></span><br><span class="line"><span class="function"></span>{</span><br><span class="line"> <span class="keyword">pid_t</span> pid;</span><br><span class="line"> <span class="keyword">int</span> status;</span><br><span class="line"> <span class="comment">/**</span></span><br><span class="line"><span class="comment"> * @brief 等待特定或者指定的子进程</span></span><br><span class="line"><span class="comment"> * @param</span></span><br><span class="line"><span class="comment"> * -1 表示等待任何子进程</span></span><br><span class="line"><span class="comment"> * status 存储终止进程的状态信息</span></span><br><span class="line"><span class="comment"> * WNOHANG 使waitpid 函数变味非阻塞。如果没有终止的子进程，会立即返回 0，而</span></span><br><span class="line"><span class="comment"> * 不会组阻塞程序的执行</span></span><br><span class="line"><span class="comment"> */</span></span><br><span class="line"> pid=waitpid(<span class="number">-1</span>, &status, WNOHANG); </span><br><span class="line"> LOG_INFO(<span class="string">"remove proc id: %d"</span>, pid);</span><br><span class="line">}</span><br></pre></td></tr></table></figure><p>另外每次循环子进程都会关闭一次 <code>server_socket</code> 与 <code>client_socket</code>，父进程关闭 <code>client_socket</code></p><figure class="highlight c"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">if</span>(child_pid == <span class="number">0</span>) <span class="comment">//子进程</span></span><br><span class="line">{</span><br><span class="line"> <span class="built_in">close</span>(server_socket); <span class="comment">//子进程会复制父进程内存，所以这里需要关闭</span></span><br><span class="line"> <span class="comment">//...</span></span><br><span class="line"></span><br><span class="line"> <span class="built_in">close</span>(client_socket);</span><br><span class="line">}</span><br><span class="line"><span class="keyword">else</span></span><br><span class="line">{</span><br><span class="line"> <span class="built_in">close</span>(client_socket); <span class="comment">//父进程关闭子进程</span></span><br><span class="line">}</span><br><span class="line"></span><br><span class="line"><span class="comment">//...</span></span><br><span class="line"><span class="built_in">close</span>(server_socket);</span><br></pre></td></tr></table></figure><h3 id="参考文档"><a class="markdownIt-Anchor" href="#参考文档"></a> 参考文档</h3><ol><li>《TCPIP 网络编程》</li></ol>]]></content>
<summary type="html">
<p>到目前为止都是循环客户端连接，处理完一个客户端再处理另外一个。现实是一个服务器可以同时处理多个客户端请求。这里通过多进程实现网络服务器</p>
<p><strong>第一步，使用 <code>fork</code> 分配多个进程处理客户端请求</strong></p>
<p
</summary>
<category term="Unix NetWork" scheme="https://www.yuankang.top/categories/Unix-NetWork/"/>
<category term="Unix" scheme="https://www.yuankang.top/tags/Unix/"/>
<category term="Unix NetWork" scheme="https://www.yuankang.top/tags/Unix-NetWork/"/>
</entry>
<entry>
<title>TCPIP 网络编程-08-套接字选项</title>
<link href="https://www.yuankang.top/2024/01/19/Unix%20NetWork/TCPIP%20%E7%BD%91%E7%BB%9C%E7%BC%96%E7%A8%8B-08-%E5%A5%97%E6%8E%A5%E5%AD%97%E9%80%89%E9%A1%B9/"/>
<id>https://www.yuankang.top/2024/01/19/Unix%20NetWork/TCPIP%20%E7%BD%91%E7%BB%9C%E7%BC%96%E7%A8%8B-08-%E5%A5%97%E6%8E%A5%E5%AD%97%E9%80%89%E9%A1%B9/</id>
<published>2024-01-19T15:00:00.000Z</published>
<updated>2024-01-28T11:01:31.674Z</updated>
<content type="html"><![CDATA[<p>套接字选项支持对套接字进行设置</p><figure class="highlight c"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br></pre></td><td class="code"><pre><span class="line"><span class="meta">#<span class="meta-keyword">include</span> <span class="meta-string"><sys/socket.h></span></span></span><br><span class="line"></span><br><span class="line"><span class="comment">/**</span></span><br><span class="line"><span class="comment"> * @brief 查看套接字选项</span></span><br><span class="line"><span class="comment"> * sock 用于查看选项套接字文件描述符</span></span><br><span class="line"><span class="comment"> * level 可查看的可选项的协议层</span></span><br><span class="line"><span class="comment"> * optname 要查看的可选项名</span></span><br><span class="line"><span class="comment"> * optval 保存查看结果的缓存地址值</span></span><br><span class="line"><span class="comment"> * optlen 第四个参数 optval 的参缓存大小，返回四个参数的字节数</span></span><br><span class="line"><span class="comment"> * @return 成功返回 0，失败返回-1</span></span><br><span class="line"><span class="comment">*/</span></span><br><span class="line"><span class="function"><span class="keyword">int</span> <span class="title">getsockopt</span><span class="params">(<span class="keyword">int</span> sock, <span class="keyword">int</span> level, <span class="keyword">int</span> optname, <span class="keyword">void</span> *optval, <span class="keyword">socklen_t</span> *optlen)</span></span>;</span><br><span class="line"></span><br><span class="line"><span class="comment">/**</span></span><br><span class="line"><span class="comment"> * @brief 查看套接字选项</span></span><br><span class="line"><span class="comment"> * sock 用于更改选项套接字文件描述符</span></span><br><span class="line"><span class="comment"> * level 可更改的可选项的协议层</span></span><br><span class="line"><span class="comment"> * optname 要更改的可选项名</span></span><br><span class="line"><span class="comment"> * optval 保存要更改的选项信息的缓存地址值</span></span><br><span class="line"><span class="comment"> * optlen 第四个参数 optval 的字节数</span></span><br><span class="line"><span class="comment"> * @return 成功返回 0，失败返回-1</span></span><br><span class="line"><span class="comment">*/</span></span><br><span class="line"><span class="function"><span class="keyword">int</span> <span class="title">setsockopt</span><span class="params">(<span class="keyword">int</span> sock, <span class="keyword">int</span> level, <span class="keyword">int</span> optname, <span class="keyword">const</span> <span class="keyword">void</span> *optval, <span class="keyword">socklen_t</span> oplen)</span></span>;</span><br></pre></td></tr></table></figure><p>示例地址：<a href="https://github.com/XBoom/network-ip.git" target="_blank" rel="noopener">https://github.com/XBoom/network-ip.git</a> 中的 <code>apps/socket/11/</code> 来看看</p><p>应用如下：</p><ol><li><p>查看套接字类型，运行结果如下</p><figure class="highlight shell"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line"><span class="meta">[root]#</span><span class="bash">./client </span></span><br><span class="line">[11/client.c:23 main info] get tcp[1] sock opt :1 4 </span><br><span class="line">[11/client.c:27 main info] get udp[2] sock opt :2 4</span><br></pre></td></tr></table></figure></li></ol><p>但是！！！ 套接字类型只能在创建时设置，以后不能再改变</p><ol start="2"><li><p>查看缓冲区大小</p><figure class="highlight shell"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line">[11/client.c:33 main info] tcp send buff size:16384 </span><br><span class="line">[11/client.c:36 main info] tcp recv buff size:131072 </span><br><span class="line">[11/client.c:39 main info] udp send buff size:212992 </span><br><span class="line">[11/client.c:42 main info] udp recv buff size:212992</span><br></pre></td></tr></table></figure></li><li><p><code>time-wait</code> 重用</p><figure class="highlight c"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">int</span> option = <span class="number">1</span>;</span><br><span class="line">optlen = <span class="keyword">sizeof</span>(option);</span><br><span class="line">ret = setsockopt(tcp_sock, SOL_SOCKET, SO_REUSEADDR, </span><br><span class="line"> (<span class="keyword">void</span> *)&option, optlen);</span><br></pre></td></tr></table></figure><p><code>time_wait</code> 在四次挥手过程中会等待 <code>2msl</code> 时间(大多数系统是 120s)，是为了等待最后的 <code>ack</code> 未收到。如果服务端发起 Fin，那么最后会因为 <code>time_wait</code> 而等待端口释放，那么再异常重启状态下，就需要等待大概 4 分钟端口才能恢复(客户端因为端口是随机分配的，所以不太在意立即释放)。</p><figure class="highlight shell"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">net.ipv4.tcp_tw_reuse # 1 表示启用了端口重用，即允许在 TIME_WAIT 状态下的连接所使用的端口被新的连接使用。 0 表示禁止</span><br></pre></td></tr></table></figure></li><li><p>Nagle算法</p><p>Nagle算法是一种用于减少小包传输的网络优化算法。它的目标是减少网络上的小型数据包数量，提高网络效率。会等待当前缓冲区中的数据被确认之后才发送新的数据。这有助于避免发送大量的小数据包，从而提高网络的效率</p><img src="https://images-1307117474.cos.ap-guangzhou.myqcloud.com/blogs/Blog_3/image-20240114181938761.png" alt="image-20240114181938761" style="zoom:50%;" /><p>Nagle算法的原理如下：</p><ol><li>如果长度达到 MSS，则允许发送</li><li>如果数据包中包含<code>FIN</code>（断开连接标致），则允许发送</li><li>当设置了<code>TCP_NODELAY</code>选项，则允许发送，禁止Nagle算法</li><li>未设置<code>TCP_CORK</code>选项时，若所有发送的小数据包（包含长度小于MSS）均被确认，则设置该选项后，内核会尽力把小的数据包拼成一个大的数据包（一个MTU）在发送出去，若到指定时间，一般为200ms，仍未组成一个MTU，也要立刻发送。</li></ol><figure class="highlight c"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment">//禁用 Nagle</span></span><br><span class="line"><span class="keyword">int</span> flag = <span class="number">1</span>;</span><br><span class="line">setsockopt(socket_fd, IPPROTO_TCP, TCP_NODELAY, (<span class="keyword">char</span>*)&flag, <span class="keyword">sizeof</span>(<span class="keyword">int</span>));</span><br></pre></td></tr></table></figure><p>在某些情况下，Nagle算法可能会导致一定的延迟，特别是对于那些需要低延迟的应用程序。为了避免这种情况，可以通过设置<code>TCP_NODELAY</code>选项来禁用Nagle算法，强制立即发送数据(默认是开启的)</p></li></ol><p><strong>粘包</strong> 问题的缘由可能发生在发送端也可能发生在接收端：</p><ul><li>由 Nagle 算法造成的发送端的粘包：当提交一段数据给 TCP 发送时，TCP 并不立刻发送此段数据，而是等待一小段时间看看在等待期间是否还有要发送的数据，若有则会一次把这好几段数据发送出去。</li><li>接收端接受不及时造成的接收端粘包：TCP 会把接收到的数据存在自己的缓冲区中，然后通应用层取数据。当应用层由于某些原因不能及时地把 TCP 的数据取出来，就会造成 TCP 缓冲区中存放了几段数据</li></ul><p>解决粘包</p><ul><li>定长消息：协议提前约定好包的长度为多少，每当接收端接收到固定长度的字节就确定一个包；</li><li>消息分隔符：利用特殊符号标志着消息的开始或者结束，例如 HTTP 协议中的换行符；</li><li>长度前缀：先发送N个字节代表包的大小（注意大端和小端问题），后续解析也按长度读取解析。</li></ul><h3 id="参考文档"><a class="markdownIt-Anchor" href="#参考文档"></a> 参考文档</h3><ol><li>《TCPIP 网络编程》</li></ol>]]></content>
<summary type="html">
<p>套接字选项支持对套接字进行设置</p>
<figure class="highlight c"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</s
</summary>
<category term="Unix NetWork" scheme="https://www.yuankang.top/categories/Unix-NetWork/"/>
<category term="Unix" scheme="https://www.yuankang.top/tags/Unix/"/>
<category term="Unix NetWork" scheme="https://www.yuankang.top/tags/Unix-NetWork/"/>
</entry>
<entry>
<title>TCPIP 网络编程-07-域名地址转换</title>
<link href="https://www.yuankang.top/2024/01/14/Unix%20NetWork/TCPIP%20%E7%BD%91%E7%BB%9C%E7%BC%96%E7%A8%8B-07-%E5%9F%9F%E5%90%8D%E5%9C%B0%E5%9D%80%E8%BD%AC%E6%8D%A2/"/>
<id>https://www.yuankang.top/2024/01/14/Unix%20NetWork/TCPIP%20%E7%BD%91%E7%BB%9C%E7%BC%96%E7%A8%8B-07-%E5%9F%9F%E5%90%8D%E5%9C%B0%E5%9D%80%E8%BD%AC%E6%8D%A2/</id>
<published>2024-01-14T15:00:00.000Z</published>
<updated>2024-01-28T11:01:15.787Z</updated>
<content type="html"><![CDATA[<p>访问服务端的时候不能总是通过 IP 访问而是通过域名，于是就有了域名与地址转换</p><figure class="highlight c"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br></pre></td><td class="code"><pre><span class="line"><span class="meta">#<span class="meta-keyword">include</span> <span class="meta-string"><netdb.h></span></span></span><br><span class="line"></span><br><span class="line"><span class="comment">/**</span></span><br><span class="line"><span class="comment">* @brief 通过传递字符串格式的域名获取IP地址</span></span><br><span class="line"><span class="comment">* @return 成功时返回hostent结构体地址，失败时返回 NULL指针</span></span><br><span class="line"><span class="comment">*/</span></span><br><span class="line"><span class="function">struct hostent *<span class="title">gethostbyname</span><span class="params">(<span class="keyword">const</span> <span class="keyword">char</span>* hostname)</span></span>;</span><br><span class="line"></span><br><span class="line"><span class="class"><span class="keyword">struct</span> <span class="title">hostent</span></span></span><br><span class="line"><span class="class">{</span></span><br><span class="line"> <span class="keyword">char</span> * h_name; <span class="comment">//官网域名，代表着某一个主页(可能很多注明的域名并未注册官方域名)</span></span><br><span class="line"> <span class="keyword">char</span> ** h_aliases; <span class="comment">//通过多个域名访问同一个主页。同一个IP可以绑定多个域名，所以除了官方域名还可以指定其他域名</span></span><br><span class="line"> <span class="keyword">int</span> h_addrtype; <span class="comment">//地址类型 IPv4则是AF_INET</span></span><br><span class="line"> <span class="keyword">int</span> h_length; <span class="comment">//保存IP地址长度(单位B)。若是IPv4地址，则是4; IPv6 则是16</span></span><br><span class="line"> <span class="keyword">char</span> ** h_addr_list; <span class="comment">//以证书形式保存域名对应的IP地址，可能多个IP分配给同一个域名</span></span><br><span class="line">}</span><br><span class="line"></span><br><span class="line"><span class="comment">/**</span></span><br><span class="line"><span class="comment">* 利用IP地址获取域名</span></span><br><span class="line"><span class="comment">* addr 含有IP地址信息的in_addr结构体指针。为了同时传递IPv4地址之外的其他细心，该变量类型声明为char指针</span></span><br><span class="line"><span class="comment">* len 第一个参数的字节数，IPv4时为4，IPv6 时为16</span></span><br><span class="line"><span class="comment">* family 传递协议族信息，IPv4时为AF_INET，IPv6 时为AF_INET6</span></span><br><span class="line"><span class="comment">* 成功返回hostent结构体变量地址值，失败返回NULL指针</span></span><br><span class="line"><span class="comment">*/</span></span><br><span class="line"><span class="function">struct hostent *<span class="title">gethostbyaddr</span><span class="params">(<span class="keyword">const</span> <span class="keyword">char</span> *addr, <span class="keyword">socklen_t</span> len, <span class="keyword">int</span> family)</span></span>;</span><br></pre></td></tr></table></figure><p>有了上面的函数解析 示例地址：<a href="https://github.com/XBoom/network-ip.git" target="_blank" rel="noopener">https://github.com/XBoom/network-ip.git</a> 中的 <code>apps/socket/10/</code> 来看看</p><figure class="highlight c"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br></pre></td><td class="code"><pre><span class="line">[root]#./client www.naver.com</span><br><span class="line">[<span class="number">10</span>/client.c:<span class="number">12</span> main info] www.naver.com <span class="built_in">get</span> host name: </span><br><span class="line">[<span class="number">10</span>/client.c:<span class="number">13</span> main info] h_name:www.naver.com.nheos.com </span><br><span class="line">[<span class="number">10</span>/client.c:<span class="number">14</span> main info] h_aliases: </span><br><span class="line">[<span class="number">10</span>/client.c:<span class="number">17</span> main info] [<span class="number">0</span>]:www.naver.com </span><br><span class="line">[<span class="number">10</span>/client.c:<span class="number">19</span> main info] type:IPv4 </span><br><span class="line">[<span class="number">10</span>/client.c:<span class="number">20</span> main info] h_length:<span class="number">4</span> </span><br><span class="line">[<span class="number">10</span>/client.c:<span class="number">21</span> main info] h_addr_list: </span><br><span class="line">[<span class="number">10</span>/client.c:<span class="number">24</span> main info] [<span class="number">0</span>]:<span class="number">223.130</span><span class="number">.200</span><span class="number">.104</span> </span><br><span class="line">[<span class="number">10</span>/client.c:<span class="number">24</span> main info] [<span class="number">1</span>]:<span class="number">223.130</span><span class="number">.200</span><span class="number">.107</span> </span><br><span class="line">[root]#./server <span class="number">223.130</span><span class="number">.200</span><span class="number">.104</span></span><br><span class="line">[<span class="number">10</span>/server.c:<span class="number">14</span> main error] gethostbyaddr failed Unknown host </span><br><span class="line">[root]#./server <span class="number">223.130</span><span class="number">.200</span><span class="number">.107</span></span><br><span class="line">[<span class="number">10</span>/server.c:<span class="number">14</span> main error] gethostbyaddr failed Unknown host</span><br></pre></td></tr></table></figure><p>并不能总是通过 IP 地址获取到域名</p><p>通过抓去端口 53 的报文(DNS)可以得到</p><p><img src="https://images-1307117474.cos.ap-guangzhou.myqcloud.com/blogs/Blog_3/image-20240113080658112.png" alt="image-20240113080658112" /></p><p>通过 UDP 去发送 DNS 请求，一次返回多个 IP 地址，而无法获取域名则如下返回 <code>No such name</code></p><p><img src="https://images-1307117474.cos.ap-guangzhou.myqcloud.com/blogs/Blog_3/image-20240113080753643.png" alt="image-20240113080753643" /></p><p>需要注意的就是 <code>inet_ntoa(*(struct in_addr *)addr_info->h_addr_list[i])</code> 为什么是一个结构体，而要使用 <code>char*</code> 进行存储</p><img src="https://images-1307117474.cos.ap-guangzhou.myqcloud.com/blogs/Blog_3/image-20240113081503692.png" alt="image-20240113081503692" style="zoom:50%;" /><p>这是因为 这个结构体并不一定有 IPv4，也有可能是 IPv6；另外在定义这个标准之前还没有 <code>void *</code> 标准，所以使用了 <code>char*</code> 定义</p><p>不过目前比较常用的是 <code>getaddrinfo</code> 更加灵活</p><figure class="highlight c"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br></pre></td><td class="code"><pre><span class="line"><span class="meta">#<span class="meta-keyword">include</span> <span class="meta-string"><sys/types.h></span></span></span><br><span class="line"><span class="meta">#<span class="meta-keyword">include</span> <span class="meta-string"><sys/socket.h></span></span></span><br><span class="line"><span class="meta">#<span class="meta-keyword">include</span> <span class="meta-string"><netdb.h></span></span></span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">int</span> <span class="title">getaddrinfo</span><span class="params">(<span class="keyword">const</span> <span class="keyword">char</span> *node, <span class="keyword">const</span> <span class="keyword">char</span> *service,</span></span></span><br><span class="line"><span class="function"><span class="params"> <span class="keyword">const</span> struct addrinfo *hints, struct addrinfo **res)</span></span>;</span><br><span class="line"><span class="comment">// node 参数是要解析的主机名或 IP 地址，可以为 NULL。</span></span><br><span class="line"><span class="comment">// service 参数是服务名或端口号的字符串表示，可以为 NULL。</span></span><br><span class="line"><span class="comment">// hints 参数是一个指向 struct addrinfo 结构的指针，用于提供解析的提示信息。</span></span><br><span class="line"><span class="comment">// res 参数是一个指向 struct addrinfo 结构链表的指针，用于保存解析结果。</span></span><br><span class="line"></span><br><span class="line"><span class="class"><span class="keyword">struct</span> <span class="title">addrinfo</span> {</span></span><br><span class="line"><span class="keyword">int</span>ai_flags;<span class="comment">/* AI_PASSIVE, AI_CANONNAME, AI_NUMERICHOST */</span></span><br><span class="line"><span class="keyword">int</span>ai_family;<span class="comment">/* PF_xxx */</span></span><br><span class="line"><span class="keyword">int</span>ai_socktype;<span class="comment">/* SOCK_xxx */</span></span><br><span class="line"><span class="keyword">int</span>ai_protocol;<span class="comment">/* 0 or IPPROTO_xxx for IPv4 and IPv6 */</span></span><br><span class="line"><span class="keyword">socklen_t</span> ai_addrlen;<span class="comment">/* length of ai_addr */</span></span><br><span class="line"><span class="keyword">char</span>*ai_canonname;<span class="comment">/* canonical name for hostname */</span></span><br><span class="line"><span class="class"><span class="keyword">struct</span><span class="title">sockaddr</span> *<span class="title">ai_addr</span>;</span><span class="comment">/* binary address */</span></span><br><span class="line"><span class="class"><span class="keyword">struct</span><span class="title">addrinfo</span> *<span class="title">ai_next</span>;</span><span class="comment">/* next structure in linked list */</span></span><br><span class="line">};</span><br></pre></td></tr></table></figure><p>最终输出结果为</p><figure class="highlight c"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">getaddrinfo: </span><br><span class="line">[<span class="number">10</span>/client.c:<span class="number">55</span> main info] ip: <span class="number">182.61</span><span class="number">.200</span><span class="number">.7</span> </span><br><span class="line">[<span class="number">10</span>/client.c:<span class="number">55</span> main info] ip: <span class="number">182.61</span><span class="number">.200</span><span class="number">.6</span></span><br></pre></td></tr></table></figure><p>最后还有两个小知识点</p><ol><li><p><code>hstrerror(h_errno)</code> ：<code>h_errno</code> 是一个全局错误码，而 <code>hstrerror</code> 用来解析 host相关的错误</p></li><li><p><code>inet_ntop（Internet Network TO Presentation）</code>二进制字符串转换</p><figure class="highlight c"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br></pre></td><td class="code"><pre><span class="line"><span class="meta">#<span class="meta-keyword">include</span> <span class="meta-string"><arpa/inet.h></span></span></span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">const</span> <span class="keyword">char</span> *<span class="title">inet_ntop</span><span class="params">(<span class="keyword">int</span> af, <span class="keyword">const</span> <span class="keyword">void</span> *src, <span class="keyword">char</span> *dst, <span class="keyword">socklen_t</span> <span class="built_in">size</span>)</span></span>;</span><br><span class="line"><span class="comment">// af 是地址族，可以是 AF_INET（IPv4）或 AF_INET6（IPv6）。</span></span><br><span class="line"><span class="comment">// src 是指向存储二进制形式地址的指针。</span></span><br><span class="line"><span class="comment">// dst 是一个用于存储字符串形式地址的缓冲区。</span></span><br><span class="line"><span class="comment">// size 是缓冲区的大小</span></span><br></pre></td></tr></table></figure></li></ol><h3 id="参考文档"><a class="markdownIt-Anchor" href="#参考文档"></a> 参考文档</h3><ol><li>《TCPIP 网络编程》</li></ol>]]></content>
<summary type="html">
<p>访问服务端的时候不能总是通过 IP 访问而是通过域名，于是就有了域名与地址转换</p>
<figure class="highlight c"><table><tr><td class="gutter"><pre><span class="line">1</span><br
</summary>
<category term="Unix NetWork" scheme="https://www.yuankang.top/categories/Unix-NetWork/"/>
<category term="Unix" scheme="https://www.yuankang.top/tags/Unix/"/>
<category term="Unix NetWork" scheme="https://www.yuankang.top/tags/Unix-NetWork/"/>
</entry>
<entry>
<title>TCPIP 网络编程-06-套接字断开</title>
<link href="https://www.yuankang.top/2024/01/08/Unix%20NetWork/TCPIP%20%E7%BD%91%E7%BB%9C%E7%BC%96%E7%A8%8B-06-%E5%A5%97%E6%8E%A5%E5%AD%97%E6%96%AD%E5%BC%80/"/>
<id>https://www.yuankang.top/2024/01/08/Unix%20NetWork/TCPIP%20%E7%BD%91%E7%BB%9C%E7%BC%96%E7%A8%8B-06-%E5%A5%97%E6%8E%A5%E5%AD%97%E6%96%AD%E5%BC%80/</id>
<published>2024-01-08T15:00:00.000Z</published>
<updated>2024-01-28T11:01:08.763Z</updated>
<content type="html"><![CDATA[<p><code>close</code> 是关闭套接字，调用之后无法再通过 <code>write</code> 与 <code>read</code> 读写数据。但是通过查看 TCP 的四次挥手我们知道</p><img src="https://images-1307117474.cos.ap-guangzhou.myqcloud.com/blogs/Blog_3/Snipaste_2023-10-13_22-40-02.png" alt="Snipaste_2023-10-13_22-40-02" style="zoom:50%;" /><p>在 <code>close</code> 之后，对端可能还会发送一部分数据，这部分数据其实是丢失的。所以就引入了半关闭，只能发送数据但是不能接收 或者 只能接收数据但是不能发送。</p><p>这里提到了一个需求：客户端向服务器发起请求，然后服务端将文件传输给客户端之后，客户端接收完文件，然后告知服务端接收完毕了，客户端自动退出。</p><p>这里有一个疑问，客户端怎么知道接收完文件了？</p><ol><li>从 <code>read</code> 函数返回-1表示接收结束，但是<code>read</code> 返回-1 表示对端关闭了连接，那么需求中接收完文件还要发送响应就不行了</li><li>写一个标志位，告知这个标志位就表示文件结束。但是这样的话就必须让文件中没有与标志符一样的内容</li></ol><p>这个时候就用到了半关闭</p><figure class="highlight c"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment">/**</span></span><br><span class="line"><span class="comment"> * @brief 半关闭</span></span><br><span class="line"><span class="comment"> * @param</span></span><br><span class="line"><span class="comment"> * sock 要被关闭的套接字</span></span><br><span class="line"><span class="comment"> * howto 传递断开的方式</span></span><br><span class="line"><span class="comment"> * SHUT_RD: 断开输入流</span></span><br><span class="line"><span class="comment"> * SHUT_WR: 断开输出流</span></span><br><span class="line"><span class="comment"> * SHUT_RDWR: 同时断开输入输出流</span></span><br><span class="line"><span class="comment"> * @return 成功返回 0；失败返回 -1 </span></span><br><span class="line"><span class="comment">*/</span></span><br><span class="line"><span class="function"><span class="keyword">int</span> <span class="title">shutdown</span><span class="params">(<span class="keyword">int</span> sock, <span class="keyword">int</span> howto)</span></span>;</span><br></pre></td></tr></table></figure><p>知道了 <code>shutdown</code> 函数之后，看看是如何实现的 示例地址：<a href="https://github.com/XBoom/network-ip.git" target="_blank" rel="noopener">https://github.com/XBoom/network-ip.git</a> 中的 <code>apps/socket/09/</code></p><p>运行结果如下</p><figure class="highlight shell"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line"><span class="meta">[root]#</span><span class="bash">./server 5555</span></span><br><span class="line">[09/server.c:57 main info] server recv recv all file </span><br><span class="line"></span><br><span class="line"><span class="meta">[root]#</span><span class="bash">./client 127.0.0.1 5555</span></span><br></pre></td></tr></table></figure><p>另外通过抓包可以看出跟平时的四次挥手不一样的地方(5555 表示服务端)</p><p><img src="https://images-1307117474.cos.ap-guangzhou.myqcloud.com/blogs/Blog_3/image-20240111225428108.png" alt="image-20240111225428108" /></p><ol><li>首先通过三次握手(1~3)</li><li>服务端第一次发送数据给客户端，客户端确认(4~5)</li><li>服务端第二次发送数据给客户端，客户端确认(6、8)</li><li>在服务端第二次发送数据的时候，同时发送了<code>FIN</code> 结束(7)，也就是调用的 <code>shutdown(client_sock, SHUT_WR);</code></li><li>客户端接着向服务端发送数据，客户端发送响应(9-10)，发现客户端调用的 <code>shutdown(server_sock, SHUT_RD);</code> 并没有作用，因为已经收到了服务端的 <code>FIN</code> 报文</li><li>发送结束之后，客户端通过 <code>close</code> 发起 <code>FIN</code></li></ol><p>对于两个 <code>shutdown</code> 可以看成下图这样</p><p><img src="https://images-1307117474.cos.ap-guangzhou.myqcloud.com/blogs/Blog_3/image-20240111225821497.png" alt="image-20240111225821497" /></p><p><code>shotdown</code> 分别对应的是两个流，当一端关闭之后就不需要再次关闭了。而一条流的关闭也不会影响另外一条流</p><p>那么如果改变一下服务端的关闭流程，先关闭读，再关闭写。改成如下</p><figure class="highlight c"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment">//关闭读</span></span><br><span class="line"><span class="built_in">shutdown</span>(client_sock, SHUT_RD);</span><br><span class="line"></span><br><span class="line"><span class="built_in">memset</span>(buff, <span class="number">0</span> , <span class="keyword">sizeof</span>(buff));</span><br><span class="line"><span class="keyword">while</span>(<span class="built_in">read</span>(client_sock, buff, <span class="keyword">sizeof</span>(buff)))</span><br><span class="line">{</span><br><span class="line"> LOG_INFO(<span class="string">"server recv %s"</span>, buff);</span><br><span class="line">}</span><br><span class="line"><span class="comment">//关闭写</span></span><br><span class="line"><span class="built_in">shutdown</span>(client_sock, SHUT_WR);</span><br></pre></td></tr></table></figure><p>程序运行结果如下，好像服务端没有什么</p><figure class="highlight c"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line">[root]#./server <span class="number">5555</span></span><br><span class="line"> </span><br><span class="line">[root]#./client <span class="number">127.0</span><span class="number">.0</span><span class="number">.1</span> <span class="number">5555</span></span><br><span class="line">[root]#</span><br></pre></td></tr></table></figure><p>看看交互报文</p><p><img src="https://images-1307117474.cos.ap-guangzhou.myqcloud.com/blogs/Blog_3/image-20240111231443159.png" alt="image-20240111231443159" /></p><ol><li>服务端发送完数据之后，关闭了读 <code>shutdown(client_sock, SHUT_RD);</code></li><li>客户端接收完数据之后也关闭了读，导致客户端与服务端都阻塞等待</li></ol><h3 id="参考文档"><a class="markdownIt-Anchor" href="#参考文档"></a> 参考文档</h3><ol><li>《TCPIP 网络编程》</li></ol>]]></content>
<summary type="html">
<p><code>close</code> 是关闭套接字，调用之后无法再通过 <code>write</code> 与 <code>read</code> 读写数据。但是通过查看 TCP 的四次挥手我们知道</p>
<img src="https://images-1307117
</summary>
<category term="Unix NetWork" scheme="https://www.yuankang.top/categories/Unix-NetWork/"/>
<category term="Unix" scheme="https://www.yuankang.top/tags/Unix/"/>
<category term="Unix NetWork" scheme="https://www.yuankang.top/tags/Unix-NetWork/"/>
</entry>
<entry>
<title>TCPIP 网络编程-05-基于UDP的CS</title>
<link href="https://www.yuankang.top/2024/01/02/Unix%20NetWork/TCPIP%20%E7%BD%91%E7%BB%9C%E7%BC%96%E7%A8%8B-05-%E5%9F%BA%E4%BA%8EUDP%E7%9A%84CS/"/>
<id>https://www.yuankang.top/2024/01/02/Unix%20NetWork/TCPIP%20%E7%BD%91%E7%BB%9C%E7%BC%96%E7%A8%8B-05-%E5%9F%BA%E4%BA%8EUDP%E7%9A%84CS/</id>
<published>2024-01-02T15:00:00.000Z</published>
<updated>2024-01-28T11:01:00.544Z</updated>
<content type="html"><![CDATA[<p>在 《TCPIP 网络编程-02-套接字类型与协议设置》 中通过 示例地址：<a href="https://github.com/XBoom/network-ip.git" target="_blank" rel="noopener">https://github.com/XBoom/network-ip.git</a> 中的 <code>apps/socket/03/</code>学习了基于UDP的CS通信，可以发现</p><ol><li><code>Client</code> 是不需要发起<code>connect</code>的，直接使用 <code>sendto</code> 进行发送</li><li><code>Server</code> 不需要 <code>listen/accept</code>，而是直接通过<code>recvfrom</code>进行数据接收</li></ol><p>新增了两个函数 <code>sendto</code> 与 <code>recvfrom</code></p><figure class="highlight c"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br></pre></td><td class="code"><pre><span class="line"><span class="meta">#<span class="meta-keyword">include</span> <span class="meta-string"><sys/socket.h></span></span></span><br><span class="line"></span><br><span class="line"><span class="comment">/**</span></span><br><span class="line"><span class="comment"> * @brief 使用UDP套接字进行发送 </span></span><br><span class="line"><span class="comment"> * @param </span></span><br><span class="line"><span class="comment"> * sock 用于传输数据的UDP套接字文件描述符</span></span><br><span class="line"><span class="comment"> * buff 保存待传输数据的缓冲地址值</span></span><br><span class="line"><span class="comment"> * nbytes 待传输的数据长度，以B为单位</span></span><br><span class="line"><span class="comment"> * flags 可选参数、没有则为0</span></span><br><span class="line"><span class="comment"> * to 存有目的地址信息的sockaddr解耦提变量的地址值</span></span><br><span class="line"><span class="comment"> * addrlen 参数to的地址值结构体变量长度</span></span><br><span class="line"><span class="comment"> * @return 成功返回传输的字节数，失败时返回-1</span></span><br><span class="line"><span class="comment">*/</span></span><br><span class="line"><span class="function"><span class="keyword">ssize_t</span> <span class="title">sendto</span><span class="params">(<span class="keyword">int</span> sock, <span class="keyword">void</span> *buff, <span class="keyword">size_t</span> nbytes, <span class="keyword">int</span> flags,</span></span></span><br><span class="line"><span class="function"><span class="params"> struct sockaddr *to, <span class="keyword">socklen_t</span> addrlen)</span></span>;</span><br><span class="line"></span><br><span class="line"></span><br><span class="line"><span class="comment">/**</span></span><br><span class="line"><span class="comment"> * @brief 使用UDP套接字接收数据</span></span><br><span class="line"><span class="comment"> * @param</span></span><br><span class="line"><span class="comment"> * sock 用于接收数据的UDP套接字文件描述符</span></span><br><span class="line"><span class="comment"> * buff 保存接收数据的缓冲地址值</span></span><br><span class="line"><span class="comment"> * nbytes 可接收的最大字节数，故无法超过参数buff所指的缓冲大小 </span></span><br><span class="line"><span class="comment"> * flags 可选项参数，若没有则传入0</span></span><br><span class="line"><span class="comment"> * from 存有发送端地址信息的sockaddr结构体变量的地址值</span></span><br><span class="line"><span class="comment"> * addrlen 保存参数from的结构体变量长度的变量值</span></span><br><span class="line"><span class="comment"> * @return 成功返回传输的字节数，失败时返回-1</span></span><br><span class="line"><span class="comment"> */</span></span><br><span class="line"><span class="function"><span class="keyword">ssize_t</span> <span class="title">recvfrom</span><span class="params">(<span class="keyword">int</span> sock, <span class="keyword">void</span> *buff, <span class="keyword">size_t</span> nbytes, <span class="keyword">int</span> flags,</span></span></span><br><span class="line"><span class="function"><span class="params"> struct sockaddr *from, <span class="keyword">socklen_t</span> *addrlen)</span></span>;</span><br></pre></td></tr></table></figure><p>写一个基于UDP的回声CS，示例地址：<a href="https://github.com/XBoom/network-ip.git" target="_blank" rel="noopener">https://github.com/XBoom/network-ip.git</a> 中的 <code>apps/socket/07/</code>。运行结果如下</p><figure class="highlight shell"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br></pre></td><td class="code"><pre><span class="line"><span class="meta">[root]#</span><span class="bash">./client 127.0.0.1 5555</span></span><br><span class="line">12345</span><br><span class="line">[07/client.c:28 main info] client send 12345 to 127.0.0.1 </span><br><span class="line">[07/client.c:36 main info] client recv 12345 from 127.0.0.1 </span><br><span class="line"> </span><br><span class="line"><span class="meta">[root]#</span><span class="bash">./server 5555</span></span><br><span class="line">[07/server.c:33 main info] recv message 12345 from 127.0.0.1</span><br></pre></td></tr></table></figure><p>从上面可以看出，**CS都不需要连接，**那么客户端地址又是如何分配的，答案是在 <code>sendto</code>的时候，首次调用 <code>sendto</code> 的时候会自动给套接字分配 IP 与端口，然后一直沿用到套接字关闭。</p><p>所以</p><ol><li><p>客户端其实可以使用 <code>connect</code> 当然此处的作用就不是连接，而是分配地址(TCP 也是在这个时候分配的地址);</p></li><li><p>又因为套接字有了地址之后，就可以不用 <code>sendto</code> 而直接使用 <code>write</code> 进行发送;</p></li><li><p>而且<strong>CS均只需要一个套接字(即只需要一个套接字就能和多个主机通信)</strong></p></li></ol><img src="https://images-1307117474.cos.ap-guangzhou.myqcloud.com/blogs/Blog_3/202401052119873.png" alt="image-20240105211959767" style="zoom:50%;" /><p>所以最后将示例 7 修改为 示例8：<a href="https://github.com/XBoom/network-ip.git" target="_blank" rel="noopener">https://github.com/XBoom/network-ip.git</a> 中的 <code>apps/socket/08/</code>，效果一致。通过 <code>connect</code> 进行地址分配，通过 <code>read/write</code> 进行消息发送接收。运行结果如下：</p><figure class="highlight c"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br></pre></td><td class="code"><pre><span class="line">[root]#./server <span class="number">5555</span></span><br><span class="line">[<span class="number">08</span>/server.c:<span class="number">33</span> main info] recv message <span class="number">12345</span> from <span class="number">127.0</span><span class="number">.0</span><span class="number">.1</span> </span><br><span class="line"></span><br><span class="line">[root]#./client <span class="number">127.0</span><span class="number">.0</span><span class="number">.1</span> <span class="number">5555</span></span><br><span class="line"><span class="number">12345</span></span><br><span class="line">[<span class="number">08</span>/client.c:<span class="number">36</span> main info] recv server response:<span class="number">12345</span></span><br></pre></td></tr></table></figure><blockquote><p>TIPS:</p><p><code>sendto</code> 函数主要分为三个阶段</p><ul><li>向 UDP 套接字注册 IP 地址与端口</li><li>传输数据</li><li>删除 UDP 套接字中注册的 IP 地址与套接字</li></ul><p><code>sendto</code> 函数每次调用都执行上述三个阶段，将未注册 IP 地址与端口的称为未连接的 UDP 套接字。用于向不同的目标发送数据</p><p>相反，通过 <code>connect</code> 将 IP 地址与端口注册到 UDP 套接字，称为 已连接的 UDP 套接字。用于向同一个目标发送数据</p></blockquote><h3 id="参考文档"><a class="markdownIt-Anchor" href="#参考文档"></a> 参考文档</h3><ol><li>《TCPIP 网络编程》</li></ol>]]></content>
<summary type="html">
<p>在 《TCPIP 网络编程-02-套接字类型与协议设置》 中通过 示例地址：<a href="https://github.com/XBoom/network-ip.git" target="_blank" rel="noopener">https://github.com
</summary>
<category term="Unix NetWork" scheme="https://www.yuankang.top/categories/Unix-NetWork/"/>
<category term="Unix" scheme="https://www.yuankang.top/tags/Unix/"/>
<category term="Unix NetWork" scheme="https://www.yuankang.top/tags/Unix-NetWork/"/>
</entry>
<entry>
<title>TCPIP 网络编程-04-基于TCP的CS</title>
<link href="https://www.yuankang.top/2023/12/27/Unix%20NetWork/TCPIP%20%E7%BD%91%E7%BB%9C%E7%BC%96%E7%A8%8B-04-%E5%9F%BA%E4%BA%8ETCP%E7%9A%84CS/"/>
<id>https://www.yuankang.top/2023/12/27/Unix%20NetWork/TCPIP%20%E7%BD%91%E7%BB%9C%E7%BC%96%E7%A8%8B-04-%E5%9F%BA%E4%BA%8ETCP%E7%9A%84CS/</id>
<published>2023-12-27T15:00:00.000Z</published>
<updated>2024-01-28T11:00:44.653Z</updated>
<content type="html"><![CDATA[<p>有了地址的设置函数之后，接下来就是开始进行连接</p><p>服务端存在两个函数</p><figure class="highlight c"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br></pre></td><td class="code"><pre><span class="line"><span class="meta">#<span class="meta-keyword">include</span> <span class="meta-string"><sys/socket.h></span></span></span><br><span class="line"></span><br><span class="line"><span class="comment">/**</span></span><br><span class="line"><span class="comment"> * 通过调用 listen 函数进入等待连接请求状态。(只有调用 listen，客户端才能发起 connect)</span></span><br><span class="line"><span class="comment"> * sock 希望进入等待连接请求的状态的套接字文件描述</span></span><br><span class="line"><span class="comment"> * backlog 连接请求等待队列(Queue)的长度，若位 5，表示最多使5个连接请求进入队列</span></span><br><span class="line"><span class="comment">*/</span></span><br><span class="line"><span class="function"><span class="keyword">int</span> <span class="title">listen</span><span class="params">(<span class="keyword">int</span> sock, <span class="keyword">int</span> backlog)</span></span>;</span><br><span class="line"></span><br><span class="line"><span class="comment">/**</span></span><br><span class="line"><span class="comment"> * 受理客户端连接请求，返回一个套接字连接到发起请求的客户端</span></span><br><span class="line"><span class="comment"> * sock 服务端套接字的文件描述符</span></span><br><span class="line"><span class="comment"> * addr 保存发起连接请求的客户端地址信息的变量地址值</span></span><br><span class="line"><span class="comment"> * addrlen 第二个参数 addr 结构体的长度</span></span><br><span class="line"><span class="comment"> * 成功返回创建的套接字文件描述符，失败返回-1</span></span><br><span class="line"><span class="comment">*/</span></span><br><span class="line"><span class="function"><span class="keyword">int</span> <span class="title">accept</span><span class="params">(<span class="keyword">int</span> sock, struct sockaddr * addr, <span class="keyword">socklen_t</span> * addr_len)</span></span>;</span><br></pre></td></tr></table></figure><p><code>accept</code> 函数受理连接请求等待队列中待处理的客户端连接请求</p><p>客户端使用 connect发起连接请求</p><figure class="highlight c"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment">/**</span></span><br><span class="line"><span class="comment"> * 向服务端发起请求连接</span></span><br><span class="line"><span class="comment"> * sock 客户端套接字文件描述符</span></span><br><span class="line"><span class="comment"> * servaddr 保存目标服务端地址信息的变量地址值</span></span><br><span class="line"><span class="comment"> * addrlen 第二个参数结构体的长度</span></span><br><span class="line"><span class="comment">*/</span></span><br><span class="line"><span class="function"><span class="keyword">int</span> <span class="title">connect</span><span class="params">(<span class="keyword">int</span> sock, struct sockaddr * server_addr, <span class="keyword">socklen_t</span> addr_len)</span></span>;</span><br></pre></td></tr></table></figure><p>客户端调用 <code>connect</code> 之后，可能发生以下情况之一才返回(完成函数调用)</p><ol><li><p>服务端接收连接请求</p></li><li><p>发生断网导致一场情况而中断连接请求(阻塞的)</p></li></ol><p>客户端<code>connect</code>的时候并没有指定客户端的地址，那么是什么时候分配的ip 与端口？其实是自动分配的，所以客户端无需调用标记的 <code>bind</code> 函数进行分配</p><blockquote><p>TIPS: 接收连接并不意味着服务端调用 <code>accept</code> 函数，而是服务端把连接请求信息记录到等待队列。因此 <code>connect</code> 函数返回之后并不立即进行数据交换，那么问题来了，客户端既然不知道服务端是否已经 <code>accept</code> 了，那么如果客户端立即 <code>write</code> 发送数据，数据到服务端会是一个什么状态</p><p>答：如果服务器端尚未调用 <code>accept</code>，连接处于已建立但未接受的状态。在这种情况下，发送的数据将保留在等待队列中，等待服务器端调用 <code>accept</code> 来接受连接。一旦服务器端调用了 <code>accept</code>，则可以开始在客户端和服务器端之间进行数据交换</p></blockquote><p>所以函数的调用关系如下</p><img src="https://images-1307117474.cos.ap-guangzhou.myqcloud.com/blogs/Blog_3/Snipaste_2024-01-02_21-49-18.png" alt="Snipaste_2024-01-02_21-49-18" style="zoom:50%;" /><p>这章要求是写一个回写 CS，代码：<a href="https://github.com/XBoom/network-ip.git" target="_blank" rel="noopener">https://github.com/XBoom/network-ip.git</a> 中的 <code>apps/socket/06/</code>执行结果，服务端接收到消息之后恢复给客户端，有两种情况，每一次写入都原样写回；第二种是每次的写入收集起来一次写回</p><p>要求：</p><ol><li>服务器端在同一时刻只能与一个客户端相连，并提供回声服务</li><li>服务器端一次向5个客户端提供服务并退出</li><li>客户端接收用户输入的字符串并发送到服务器端</li><li>服务器端将接收的字符串数据传回客户端，即“回声服务”</li><li>服务器端与客户端之间的字符串回声一致执行到客户端输入Q</li></ol><figure class="highlight shell"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br></pre></td><td class="code"><pre><span class="line"><span class="meta">[root]#</span><span class="bash">./server 5555</span></span><br><span class="line">[06/server.c:44 main info] accept socket 4 </span><br><span class="line">[06/server.c:49 main info] recv message:client1 </span><br><span class="line">[06/server.c:55 main info] reponse len 7 message:client1 </span><br><span class="line">[06/server.c:44 main info] accept socket 4 </span><br><span class="line">[06/server.c:49 main info] recv message:client2 </span><br><span class="line">[06/server.c:55 main info] reponse len 7 message:client2 </span><br><span class="line">[06/server.c:44 main info] accept socket 4 </span><br><span class="line">[06/server.c:49 main info] recv message:client3 </span><br><span class="line">[06/server.c:55 main info] reponse len 7 message:client3 </span><br><span class="line">[06/server.c:44 main info] accept socket 4 </span><br><span class="line">[06/server.c:49 main info] recv message:client4 </span><br><span class="line">[06/server.c:55 main info] reponse len 7 message:client4 </span><br><span class="line">[06/server.c:44 main info] accept socket 4 </span><br><span class="line">[06/server.c:49 main info] recv message:client5 </span><br><span class="line">[06/server.c:55 main info] reponse len 7 message:client5 </span><br><span class="line">[06/server.c:60 main info] reponse message end</span><br></pre></td></tr></table></figure><p>读写操作 <code>read/write</code> 由于是无边界，数据存储在IO缓存区等待读写</p><img src="https://images-1307117474.cos.ap-guangzhou.myqcloud.com/blogs/Blog_3/202401052042916.png" alt="image-20240105204229710" style="zoom:50%;" /><p>IO缓存区有以下特点</p><ul><li>IO缓冲在每个TCP套接字中单独存在</li><li>IO缓冲在创建套接字时自动生成</li><li>关闭套接字会继续传递<strong>输出缓冲</strong>中遗留的数据</li><li>关闭套接字会丢失<strong>输入缓冲</strong>中的数据</li></ul><blockquote><p>TIPS: 输出不丢失是因为系统自动进行发送操作，而输入丢失是因为套接字关闭了输入缓冲中的数据也没有人能够读到了，也就丢掉</p></blockquote><h3 id="参考文档"><a class="markdownIt-Anchor" href="#参考文档"></a> 参考文档</h3><ol><li>《TCPIP 网络编程》</li></ol>]]></content>
<summary type="html">
<p>有了地址的设置函数之后，接下来就是开始进行连接</p>
<p>服务端存在两个函数</p>
<figure class="highlight c"><table><tr><td class="gutter"><pre><span class="line">1</span><b
</summary>
<category term="Unix NetWork" scheme="https://www.yuankang.top/categories/Unix-NetWork/"/>
<category term="Unix" scheme="https://www.yuankang.top/tags/Unix/"/>
<category term="Unix NetWork" scheme="https://www.yuankang.top/tags/Unix-NetWork/"/>
</entry>
<entry>
<title>TCPIP 网络编程-03-地址族与数据序列</title>
<link href="https://www.yuankang.top/2023/12/22/Unix%20NetWork/TCPIP%20%E7%BD%91%E7%BB%9C%E7%BC%96%E7%A8%8B-03-%E5%9C%B0%E5%9D%80%E6%97%8F%E4%B8%8E%E6%95%B0%E6%8D%AE%E5%BA%8F%E5%88%97/"/>
<id>https://www.yuankang.top/2023/12/22/Unix%20NetWork/TCPIP%20%E7%BD%91%E7%BB%9C%E7%BC%96%E7%A8%8B-03-%E5%9C%B0%E5%9D%80%E6%97%8F%E4%B8%8E%E6%95%B0%E6%8D%AE%E5%BA%8F%E5%88%97/</id>
<published>2023-12-22T15:00:00.000Z</published>
<updated>2024-01-28T11:00:36.698Z</updated>
<content type="html"><![CDATA[<p>上一章《套接字类型与协议设置》是构建套接字，这一章就是给地址分配 IP 地址和端口号</p><p>首先是结构体</p><figure class="highlight c"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br></pre></td><td class="code"><pre><span class="line"><span class="class"><span class="keyword">struct</span> <span class="title">sockaddr_in</span></span></span><br><span class="line"><span class="class">{</span></span><br><span class="line"> <span class="keyword">sa_family_t</span> sin_family; <span class="comment">//地址族(Address Family) __uint8_t</span></span><br><span class="line"> <span class="keyword">uint16_t</span> sin_port; <span class="comment">//16 位 TCP/UDP 端口号</span></span><br><span class="line"> <span class="class"><span class="keyword">struct</span> <span class="title">in_addr</span> <span class="title">sin_addr</span>;</span> <span class="comment">//32 位 IP 地址</span></span><br><span class="line"> <span class="keyword">char</span> sin_zero[<span class="number">8</span>]; <span class="comment">//不使用</span></span><br><span class="line">}</span><br><span class="line"></span><br><span class="line"><span class="class"><span class="keyword">struct</span> <span class="title">in_addr</span></span></span><br><span class="line"><span class="class">{</span></span><br><span class="line"> <span class="keyword">in_addr_t</span> s_addr; <span class="comment">//32 位 IPv4 地址 (__uint32_t) </span></span><br><span class="line">}</span><br></pre></td></tr></table></figure><p>其中 <code>sin_zero</code> 是为了 使 <code>struct sockaddr_in</code> 与 <code>struct sockaddr</code> 结构大小一致而插入的成员，必须填充为0</p><p><code>struct sockaddr</code> 的结构如下</p><figure class="highlight c"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line"><span class="class"><span class="keyword">struct</span> <span class="title">sockaddr</span></span></span><br><span class="line"><span class="class">{</span></span><br><span class="line"> <span class="keyword">sa_family_t</span> sin_family; <span class="comment">//协议族</span></span><br><span class="line"> <span class="keyword">char</span> sa_data[<span class="number">14</span>]; <span class="comment">//地址信息</span></span><br><span class="line">}</span><br></pre></td></tr></table></figure><p><code>sa_data</code> 保存的地址信息包含了地址与端口号，剩余部分填充为0</p><blockquote><p>tips: 服务端套接字虽然要绑定一个端口，但是同一个端口 TCP套接字和UDP套接字是可以共用的（虽然一般不共用）</p></blockquote><p>字节序转换有以下内容</p><ol><li>网络字节序规定统一使用大端</li><li>如何记住大小端：端(即 开端)，地址从低到高，低字节在低地址就是小端，高字节在低地址就是大端</li><li>即使系统本身是大端，最好也调用一下，反正也不会有任何变化。提升代码兼容性</li></ol><p>字节序转换函数</p><figure class="highlight c"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line"><span class="function"><span class="keyword">unsigned</span> short <span class="title">htons</span><span class="params">(<span class="keyword">unsigned</span> short)</span></span>;</span><br><span class="line"><span class="function"><span class="keyword">unsigned</span> short <span class="title">ntohs</span><span class="params">(<span class="keyword">unsigned</span> short)</span></span>;</span><br><span class="line"><span class="function"><span class="keyword">unsigned</span> <span class="keyword">long</span> <span class="title">ntohl</span><span class="params">(<span class="keyword">unsigned</span> <span class="keyword">long</span>)</span></span>;</span><br><span class="line"><span class="function"><span class="keyword">unsigned</span> <span class="keyword">long</span> <span class="title">ntohl</span><span class="params">(<span class="keyword">unsigned</span> <span class="keyword">long</span>)</span></span>;</span><br></pre></td></tr></table></figure><p>网络地址的转换</p><figure class="highlight c"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br></pre></td><td class="code"><pre><span class="line"><span class="meta">#<span class="meta-keyword">include</span> <span class="meta-string"><arpa/inet.h></span></span></span><br><span class="line"><span class="comment">//成功时返回32位大端序整数型值，失败返回INADDR_NONE(检查无效IP地址)</span></span><br><span class="line"><span class="function"><span class="keyword">in_addr_t</span> <span class="title">inet_addr</span><span class="params">(<span class="keyword">const</span> <span class="keyword">char</span> * <span class="built_in">string</span>)</span></span>;</span><br><span class="line"></span><br><span class="line"><span class="comment">//与上面功能一样，成功返回1，失败返回0</span></span><br><span class="line"><span class="function"><span class="keyword">int</span> <span class="title">inet_aton</span><span class="params">(<span class="keyword">const</span> <span class="keyword">char</span> * <span class="built_in">string</span>, struct in_addr *addr)</span></span>;</span><br><span class="line"><span class="comment">//将网络字节序地址转换为字符串形式(注意返回结果的释放)</span></span><br><span class="line"><span class="function"><span class="keyword">char</span> *<span class="title">inet_ntoa</span><span class="params">(struct in_addr *addr)</span></span>;</span><br></pre></td></tr></table></figure><p><code>ntoa</code> 可以理解为 <code>network to address</code>，<code>aton</code> 同理</p><p>重写代码地址：<a href="https://github.com/XBoom/network-ip.git" target="_blank" rel="noopener">https://github.com/XBoom/network-ip.git</a> 中的 <code>apps/socket/05/</code>执行结果</p><figure class="highlight shell"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br></pre></td><td class="code"><pre><span class="line">[05/server.c:16 main info] host ordered port: 0x1234 </span><br><span class="line">[05/server.c:17 main info] net ordered addr: 0x3412 </span><br><span class="line">[05/server.c:18 main info] host ordered post: 0x12345678 </span><br><span class="line">[05/server.c:19 main info] net ordered post: 0x12345678 </span><br><span class="line">[05/server.c:31 main info] inet addr 127.0.0.1 success 0x100007f </span><br><span class="line">[05/server.c:37 main error] inet addr 127.0.0.258 failed </span><br><span class="line">[05/server.c:53 main info] inet_aton 127.0.0.1 success 0x100007f </span><br><span class="line">[05/server.c:58 main error] inet_aton 127.0.0.258 failed </span><br><span class="line">[05/server.c:69 main info] inet ntoa 127.0.0.1 127.0.0.1</span><br></pre></td></tr></table></figure><h3 id="参考文档"><a class="markdownIt-Anchor" href="#参考文档"></a> 参考文档</h3><ol><li>《TCPIP 网络编程》</li></ol>]]></content>
<summary type="html">
<p>上一章《套接字类型与协议设置》是构建套接字，这一章就是给地址分配 IP 地址和端口号</p>
<p>首先是结构体</p>
<figure class="highlight c"><table><tr><td class="gutter"><pre><span class="
</summary>
<category term="Unix NetWork" scheme="https://www.yuankang.top/categories/Unix-NetWork/"/>
<category term="Unix" scheme="https://www.yuankang.top/tags/Unix/"/>
<category term="Unix NetWork" scheme="https://www.yuankang.top/tags/Unix-NetWork/"/>
</entry>
<entry>
<title>TCPIP 网络编程-02-套接字类型与协议设置</title>
<link href="https://www.yuankang.top/2023/12/13/Unix%20NetWork/TCPIP%20%E7%BD%91%E7%BB%9C%E7%BC%96%E7%A8%8B-02-%E5%A5%97%E6%8E%A5%E5%AD%97%E7%B1%BB%E5%9E%8B%E4%B8%8E%E5%8D%8F%E8%AE%AE%E8%AE%BE%E7%BD%AE/"/>
<id>https://www.yuankang.top/2023/12/13/Unix%20NetWork/TCPIP%20%E7%BD%91%E7%BB%9C%E7%BC%96%E7%A8%8B-02-%E5%A5%97%E6%8E%A5%E5%AD%97%E7%B1%BB%E5%9E%8B%E4%B8%8E%E5%8D%8F%E8%AE%AE%E8%AE%BE%E7%BD%AE/</id>
<published>2023-12-13T15:00:00.000Z</published>
<updated>2024-01-28T11:00:28.962Z</updated>
<content type="html"><![CDATA[<p>第一章《TCPIP 网络编程-01-入门》讲到了 socket 函数，这里将详细看看套接字逻辑</p><figure class="highlight c"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br></pre></td><td class="code"><pre><span class="line"><span class="meta">#<span class="meta-keyword">include</span> <span class="meta-string"><sys/socket.h></span></span></span><br><span class="line"><span class="comment">/**</span></span><br><span class="line"><span class="comment"> * @brief 创建套接字</span></span><br><span class="line"><span class="comment"> * domain 套接字中使用的协议族信息</span></span><br><span class="line"><span class="comment"> * type 套接字数据传输类型的信息</span></span><br><span class="line"><span class="comment"> * protocol 计算机见通信中使用的协议信息</span></span><br><span class="line"><span class="comment">*/</span></span><br><span class="line"><span class="function"><span class="keyword">int</span> <span class="title">socket</span><span class="params">(<span class="keyword">int</span> domain, <span class="keyword">int</span> type, <span class="keyword">int</span> protocol)</span></span>;</span><br></pre></td></tr></table></figure><p><strong>domain参数</strong>代表的协议族分类可分为以下几类</p><ul><li><strong>PF_INET</strong>: IPv4 互联网协议族</li><li><strong>PF_INET6</strong>: IPv6 互联网协议族</li><li><strong>PF_LOCAL</strong>: 本地通信的 UNIX 协议族</li><li><strong>PF_PACKET</strong>: 底层套接字的协议族</li><li><strong>PF_IPX</strong>: IPX Novell 协议族</li></ul><p><strong>type 参数</strong>代表的套接字数据的传输方式</p><ul><li><strong>SOCK_STREAM</strong>: 面向连接的套接字(TCP)。 面向连接的特点，传输过程中数据不会消失，按序传输数据，传输的数据不存在数据边界(面向连接的可靠字节流)<ul><li>可靠的(不丢失)</li><li>按序传输数据</li><li>传输的数据没有边界</li></ul></li><li><strong>SOCK_DGRAM</strong>: 面向消息的套接字(UDP)。有以下特点<ul><li>强调快速而非传输顺序</li><li>传输的数据可能丢失也可能损毁</li><li>传输数据有边界</li><li>限制每次传输的数据大小</li></ul></li></ul><p><strong>protocol 参数</strong>代表决定了最终的协议，为什么前两个参数无法决定？</p><p>因为同一协议族中可能存在多个数据传输方式相同的协议。所以一般可以传一个 0 表示使用默认协议</p><blockquote><p>Tips:</p><ol><li>为了接收数据，套接字内部有字节数组缓冲区。调用 read 函数从缓冲区读取部分数据，所以缓冲区并不总是满的。即使读的比较慢，缓冲区满了，套接字无法再接受数据，也不会发生数据丢失。因为套接字会根据接收端的状态传输数据(滑动窗口)，传输错误也会有重传服务</li></ol></blockquote><p>示例地址：<a href="https://github.com/XBoom/network-ip.git" target="_blank" rel="noopener">https://github.com/XBoom/network-ip.git</a> 中的 <code>apps/socket/03/</code></p><p>运行结果如下：</p><figure class="highlight shell"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line">[03/server.c:44 main info] server recv 48 len msssage:</span><br><span class="line">client message </span><br><span class="line">client message </span><br><span class="line">client message</span><br></pre></td></tr></table></figure><p>可以看到，即使<strong>一个一个字符的读取也是能读到所有的内容</strong>(注意字符串结束符会影响打印)</p><p>问题 1：既然数据是没有边界的，那么上层是如何获取对应的数据结构的？</p><ul><li>固定长度的消息：每次读取固定长度，既浪费空间也不灵活</li><li>消息头包含长度信息：在消息头包含一个字段，表示整个消息的长度。接收方先读消息头，然后根据长度信息读取对应长度的数据</li><li>使用消息格式协议：使用 JSON、 XML 等消息格式协议</li><li>使用消息起始标志与结束标志：通过检测这些标志来确定消息的边界</li></ul><p>问题 2：如何使用 <code>SOCK_DGRAM</code> 又如何编程？</p><p>示例地址：<a href="https://github.com/XBoom/network-ip.git" target="_blank" rel="noopener">https://github.com/XBoom/network-ip.git</a> 中的 <code>apps/socket/04/</code></p><p>区别在于 客户端并不需要执行连接请求，只需要在发送的时候指定发送地址。而服务端也不需要 <code>bind/listen</code></p><figure class="highlight c"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">for</span>(<span class="keyword">int</span> i = <span class="number">0</span>; i < <span class="number">3</span>; i++) </span><br><span class="line">{</span><br><span class="line"> <span class="keyword">char</span> message[MAX_BUFF_LEN] = <span class="string">"client message \n"</span>;</span><br><span class="line"></span><br><span class="line"> ret = sendto(server_sock, message, <span class="built_in">strlen</span>(message) + <span class="number">1</span>, </span><br><span class="line"> <span class="number">0</span>, (<span class="keyword">const</span> struct sockaddr*)&server_addr, server_addr_len);</span><br><span class="line"> CHECK_RET(ret == <span class="number">-1</span>, <span class="string">"sendto failed"</span>);</span><br><span class="line">}</span><br></pre></td></tr></table></figure><p>而服务端也不需要监听，直接读取数据</p><figure class="highlight c"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">while</span>((read_len = recvfrom(server_sock, &message[str_len], MAX_BUFF_LEN, <span class="number">0</span>,</span><br><span class="line"> (struct sockaddr *)&client_addr, &clietn_addr_len)) > <span class="number">0</span>)</span><br><span class="line">{</span><br><span class="line"> LOG_INFO(<span class="string">"recv %d %d"</span>, str_len, read_len);</span><br><span class="line"> str_len += read_len;</span><br><span class="line">}</span><br></pre></td></tr></table></figure><p>而且数据也不一个一个读取，这是读取 <code>MAX_BUFF_LEN</code> 的结果</p><figure class="highlight shell"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">[04/server.c:32 main info] recv 0 17 </span><br><span class="line">[04/server.c:32 main info] recv 17 17 </span><br><span class="line">[04/server.c:32 main info] recv 34 17</span><br></pre></td></tr></table></figure><p>而这是一次读取一个的情况</p><figure class="highlight shell"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">[04/server.c:32 main info] recv 0 1 </span><br><span class="line">[04/server.c:32 main info] recv 1 1 </span><br><span class="line">[04/server.c:32 main info] recv 2 1</span><br></pre></td></tr></table></figure><p>每次读取一个之后整个数据其实就丢了</p><p>最后来看看正常情况下是不是实现了 UDP 通信</p><img src="https://images-1307117474.cos.ap-guangzhou.myqcloud.com/blogs/Blog_3/Snipaste_2024-01-01_18-59-41.png" alt="Snipaste_2024-01-01_18-59-41" style="zoom:50%;" /><p>可以看到 UDP 并不需要三次握手也就不需要 connect 以及 listen 这类操作直接进行数据发送与接收，详细的过程后续在介绍 UDP 在详细查看</p><h3 id="参考文档"><a class="markdownIt-Anchor" href="#参考文档"></a> 参考文档</h3><ol><li>《TCPIP 网络编程》</li></ol>]]></content>
<summary type="html">
<p>第一章《TCPIP 网络编程-01-入门》讲到了 socket 函数，这里将详细看看套接字逻辑</p>
<figure class="highlight c"><table><tr><td class="gutter"><pre><span class="line">1</
</summary>
<category term="Unix NetWork" scheme="https://www.yuankang.top/categories/Unix-NetWork/"/>
<category term="Unix" scheme="https://www.yuankang.top/tags/Unix/"/>
<category term="Unix NetWork" scheme="https://www.yuankang.top/tags/Unix-NetWork/"/>
</entry>
<entry>
<title>TCPIP 网络编程-01-入门</title>
<link href="https://www.yuankang.top/2023/12/07/Unix%20NetWork/TCPIP%20%E7%BD%91%E7%BB%9C%E7%BC%96%E7%A8%8B-01-%E5%85%A5%E9%97%A8/"/>
<id>https://www.yuankang.top/2023/12/07/Unix%20NetWork/TCPIP%20%E7%BD%91%E7%BB%9C%E7%BC%96%E7%A8%8B-01-%E5%85%A5%E9%97%A8/</id>
<published>2023-12-07T15:00:00.000Z</published>
<updated>2024-01-28T11:00:20.591Z</updated>
<content type="html"><![CDATA[<p>带着问题看世界：</p><ol><li>套接字在网络编程中的作用是什么？为何称为套接字</li><li>比较说明 listen 函数和 accept 函数的作用</li><li>创建套接字后一般会为它分配地址，为什么？为了完成地址分配需要调用哪些函数</li></ol><p>入门例子就是写一个客户端与服务器通信，示例地址：<a href="https://github.com/XBoom/network-ip.git" target="_blank" rel="noopener">https://github.com/XBoom/network-ip.git</a> 中的 <code>apps/socket/01/</code></p><figure class="highlight c"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br></pre></td><td class="code"><pre><span class="line"><span class="meta">#<span class="meta-keyword">include</span> <span class="meta-string"><sys/socket.h></span></span></span><br><span class="line"></span><br><span class="line"><span class="comment">/**</span></span><br><span class="line"><span class="comment"> * 类似打电话的过程</span></span><br><span class="line"><span class="comment"> * 1. 拥有一个电话机</span></span><br><span class="line"><span class="comment"> * 2. 准备自己的电话号码</span></span><br><span class="line"><span class="comment"> * 3. 插上电话线准备监听</span></span><br><span class="line"><span class="comment"> * 4. 拿起电话进行对话</span></span><br><span class="line"><span class="comment"></span></span><br><span class="line"><span class="comment"> * 1. 有人打电话</span></span><br><span class="line"><span class="comment">*/</span></span><br><span class="line"></span><br><span class="line"><span class="comment">//构建套接字，成功返回文件描述符，失败返回-1</span></span><br><span class="line"><span class="function"><span class="keyword">int</span> <span class="title">socket</span><span class="params">(<span class="keyword">int</span> domain, <span class="keyword">int</span> type, <span class="keyword">int</span> protocol)</span></span>;</span><br><span class="line"></span><br><span class="line"><span class="comment">//给套接字分配地址(IP PORT)，成功返回0，失败返回-1</span></span><br><span class="line"><span class="function"><span class="keyword">int</span> <span class="title">bind</span><span class="params">(<span class="keyword">int</span> sockfd, struct sockaddr *myaddr, <span class="keyword">socklen_t</span> addrlen)</span></span>;</span><br><span class="line"></span><br><span class="line"><span class="comment">//开始监听(将套接字转为可接受请求状态)，成功返回 0，失败返回-1</span></span><br><span class="line"><span class="function"><span class="keyword">int</span> <span class="title">listen</span><span class="params">(<span class="keyword">int</span> sockfd, <span class="keyword">int</span> backlog)</span></span>;</span><br><span class="line"></span><br><span class="line"><span class="comment">//处理连接请求，成功返回文件描述符，失败时返回-1</span></span><br><span class="line"><span class="function"><span class="keyword">int</span> <span class="title">accept</span><span class="params">(<span class="keyword">int</span> sockfd, struct sockaddr *addr, <span class="keyword">socklen_t</span> *addrlen)</span></span>;</span><br><span class="line"></span><br><span class="line"><span class="comment">//发起套接字请求</span></span><br><span class="line"><span class="function"><span class="keyword">int</span> <span class="title">connect</span><span class="params">(<span class="keyword">int</span> sockfd, struct sockaddr *serv_addr, <span class="keyword">socklen_t</span> addrlen)</span></span>;</span><br><span class="line"></span><br><span class="line"></span><br><span class="line"><span class="meta">#<span class="meta-keyword">include</span> <span class="meta-string"><arpa/inet.h></span></span></span><br><span class="line"></span><br><span class="line"><span class="comment">//用于将点分十进制的IP地址字符串转换为网络字节序（big-endian）的32位IPv4地址</span></span><br><span class="line"><span class="function"><span class="keyword">in_addr_t</span> <span class="title">inet_addr</span><span class="params">(<span class="keyword">const</span> <span class="keyword">char</span> *cp)</span></span>;</span><br></pre></td></tr></table></figure><p><code>sockaddr</code> 与 <code>sockaddr_in</code> 的区别</p><ul><li><code>sockaddr</code> 用来代表套接字地址，IPv4 or IPv6</li><li><code>sockaddr_in</code> 用来特指 IPv4</li></ul><p>所以套接字接口使用的是 <code>struct sockaddr</code>，如果定义结构如果使用的是 <code>struct sockaddr_in</code> 则注意强制转换</p><p>为了对比，这里也把书中的文件例子重写了一遍，从client.txt读取文件然后写入到另外一个文件server.txt 中</p><p>示例地址：<a href="https://github.com/XBoom/network-ip.git" target="_blank" rel="noopener">https://github.com/XBoom/network-ip.git</a> 中的 apps/socket/02/</p><figure class="highlight c"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment">/**</span></span><br><span class="line"><span class="comment"> * 打开文件 成功时返回文件描述符，失败时返回-1</span></span><br><span class="line"><span class="comment"> * path 文件名的字符串地址</span></span><br><span class="line"><span class="comment"> * flag 文件打开模式信息</span></span><br><span class="line"><span class="comment"> * O_CREAT 必要时创建文件</span></span><br><span class="line"><span class="comment"> * O_TRUNC 删除全部现有文件</span></span><br><span class="line"><span class="comment"> * O_APPEND 维持现有数据，保存到其后面</span></span><br><span class="line"><span class="comment"> * O_RDONLY 只读打开</span></span><br><span class="line"><span class="comment"> * O_WRONLY 只写打开</span></span><br><span class="line"><span class="comment"> * O_RDWR 读写打开</span></span><br><span class="line"><span class="comment">*/</span></span><br><span class="line"><span class="function"><span class="keyword">int</span> <span class="title">open</span><span class="params">(<span class="keyword">const</span> <span class="keyword">char</span> *path, <span class="keyword">int</span> flag)</span></span>;</span><br><span class="line"></span><br><span class="line"><span class="comment">/**</span></span><br><span class="line"><span class="comment"> * 关闭文件</span></span><br><span class="line"><span class="comment"> * 成功时返回 0，失败时返回-1</span></span><br><span class="line"><span class="comment">*/</span></span><br><span class="line"><span class="function"><span class="keyword">int</span> <span class="title">close</span><span class="params">(<span class="keyword">int</span> fd)</span></span>;</span><br><span class="line"></span><br><span class="line"><span class="comment">/**</span></span><br><span class="line"><span class="comment"> * 数据写入文件</span></span><br><span class="line"><span class="comment"> * fd 文件描述符</span></span><br><span class="line"><span class="comment"> * buf 数据的缓冲地址值</span></span><br><span class="line"><span class="comment"> * nbytes 要传输的字节数</span></span><br><span class="line"><span class="comment">*/</span></span><br><span class="line"><span class="function"><span class="keyword">ssize_t</span> <span class="title">write</span><span class="params">(<span class="keyword">int</span> fd, <span class="keyword">const</span> <span class="keyword">void</span> *buf, <span class="keyword">size_t</span> nbytes)</span></span>;</span><br><span class="line"></span><br><span class="line"><span class="comment">/**</span></span><br><span class="line"><span class="comment"> * 读取文件中的数据</span></span><br><span class="line"><span class="comment"> * fd 文件描述符</span></span><br><span class="line"><span class="comment"> * buf 数据的缓冲地址值</span></span><br><span class="line"><span class="comment"> * nbytes 要传输的字节数</span></span><br><span class="line"><span class="comment">*/</span></span><br><span class="line"><span class="function"><span class="keyword">ssize_t</span> <span class="title">read</span><span class="params">(<span class="keyword">int</span> fd, <span class="keyword">void</span> *buf, <span class="keyword">size_t</span> nbytes)</span></span>;</span><br></pre></td></tr></table></figure><p>服务端运行结果</p><figure class="highlight shell"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line"><span class="meta">[root]#</span><span class="bash">./server 5555</span></span><br><span class="line">[02/server.c:9 write_file info] ./02/server.txt TCP/IP 网络编程 client.txt </span><br><span class="line">[02/server.c:18 write_file info] wirte success </span><br><span class="line">[02/server.c:66 main info] write file success</span><br></pre></td></tr></table></figure><p>客户端结果</p><figure class="highlight shell"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line"><span class="meta">[root]#</span><span class="bash">./client 127.0.0.1 5555</span></span><br><span class="line">[02/client.c:50 main info] client send msg success </span><br><span class="line">[02/client.c:56 main info] receive server reponse:write file success</span><br></pre></td></tr></table></figure><h3 id="参考文档"><a class="markdownIt-Anchor" href="#参考文档"></a> 参考文档</h3><ol><li>《TCPIP 网络编程》</li></ol>]]></content>
<summary type="html">
<p>带着问题看世界：</p>
<ol>
<li>套接字在网络编程中的作用是什么？为何称为套接字</li>
<li>比较说明 listen 函数和 accept 函数的作用</li>
<li>创建套接字后一般会为它分配地址，为什么？为了完成地址分配需要调用哪些函数</li>
</
</summary>
<category term="Unix NetWork" scheme="https://www.yuankang.top/categories/Unix-NetWork/"/>
<category term="Unix" scheme="https://www.yuankang.top/tags/Unix/"/>
<category term="Unix NetWork" scheme="https://www.yuankang.top/tags/Unix-NetWork/"/>
</entry>
<entry>
<title>TCPIP 网络编程-00-简介</title>
<link href="https://www.yuankang.top/2023/12/06/Unix%20NetWork/TCPIP%20%E7%BD%91%E7%BB%9C%E7%BC%96%E7%A8%8B-00%20%E7%AE%80%E4%BB%8B/"/>
<id>https://www.yuankang.top/2023/12/06/Unix%20NetWork/TCPIP%20%E7%BD%91%E7%BB%9C%E7%BC%96%E7%A8%8B-00%20%E7%AE%80%E4%BB%8B/</id>
<published>2023-12-06T15:00:00.000Z</published>
<updated>2024-01-28T11:00:12.545Z</updated>
<content type="html"><![CDATA[<p>学习《TCP&IP 网络编程》，将书中的代码结合自己的项目结构重新写一遍</p><p>项目地址在 <a href="https://github.com/XBoom/network-ip.git" target="_blank" rel="noopener">https://github.com/XBoom/network-ip.git</a> 目录 <code>apps/socket/</code> 对应章节如下</p><ul><li>01 第一章 理解网络编程和套接字</li><li>02 第一章 理解网络编程-从文件读写发送</li></ul><p>编译方法只需要在 <code>apps/socket/</code> 目录下执行 <code>make DIR=<directory></code> 即可编译对应目录下的文件</p><h3 id="参考文档"><a class="markdownIt-Anchor" href="#参考文档"></a> 参考文档</h3><ol><li>《TCPIP 网络编程》</li></ol>]]></content>
<summary type="html">
<p>学习《TCP&amp;IP 网络编程》，将书中的代码结合自己的项目结构重新写一遍</p>
<p>项目地址在 <a href="https://github.com/XBoom/network-ip.git" target="_blank" rel="noopener">ht
</summary>
<category term="Unix NetWork" scheme="https://www.yuankang.top/categories/Unix-NetWork/"/>
<category term="Unix" scheme="https://www.yuankang.top/tags/Unix/"/>
<category term="Unix NetWork" scheme="https://www.yuankang.top/tags/Unix-NetWork/"/>
</entry>
<entry>
<title>数据结构-02-Heap</title>
<link href="https://www.yuankang.top/2023/09/29/Data%20Structure/%E6%95%B0%E6%8D%AE%E7%BB%93%E6%9E%84-02-Heap/"/>
<id>https://www.yuankang.top/2023/09/29/Data%20Structure/%E6%95%B0%E6%8D%AE%E7%BB%93%E6%9E%84-02-Heap/</id>
<published>2023-09-28T16:16:00.000Z</published>
<updated>2023-09-28T16:22:33.072Z</updated>
<content type="html"><![CDATA[<p>在阅读 libevent 库中另外一个用来存储事件的第二个结构就是小根堆，这里重新学习一下它的实现</p><p>代码路径:<code>minheap-internal.h</code></p><figure class="highlight c"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">typedef</span> <span class="class"><span class="keyword">struct</span> <span class="title">min_heap</span></span></span><br><span class="line"><span class="class">{</span></span><br><span class="line"><span class="class"><span class="keyword">struct</span> <span class="title">event</span>** <span class="title">p</span>;</span></span><br><span class="line"><span class="keyword">size_t</span> n, a;<span class="comment">// n 表示堆的元素数目，a 表示当前分配的内存空间大小</span></span><br><span class="line">} <span class="keyword">min_heap_t</span>;</span><br></pre></td></tr></table></figure><p><code>struct event** p</code> 区别与使用数组进行存储，这样就可以实现动态扩容</p><h3 id="堆初始化"><a class="markdownIt-Anchor" href="#堆初始化"></a> 堆初始化</h3><p>一开始没有元素，而要做的就是初始化一个最小堆对象 <code>min_heap_t</code></p><figure class="highlight c"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment">//event.c:650 event_base_new_with_config</span></span><br><span class="line">min_heap_ctor_(&base->timeheap);</span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">void</span> <span class="title">min_heap_ctor_</span><span class="params">(<span class="keyword">min_heap_t</span>* s)</span> </span>{ s->p = <span class="number">0</span>; s->n = <span class="number">0</span>; s->a = <span class="number">0</span>; }</span><br></pre></td></tr></table></figure><p>初始化堆，这里有一个小技巧就是将 <code>s->p= 0</code> , <strong>就是不指向任何对象，相当于 NULL</strong>，<code>min_heap_t* a = 0;</code>，同时堆的分配内容大小 a 一开始并没指定**(懒加载形式)**</p><p>其他辅助函数包括</p><figure class="highlight c"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br></pre></td><td class="code"><pre><span class="line"><span class="function"><span class="keyword">int</span> <span class="title">min_heap_empty_</span><span class="params">(<span class="keyword">min_heap_t</span>* s)</span> </span>{ <span class="keyword">return</span> <span class="number">0</span> == s->n; }<span class="comment">//堆是否为空</span></span><br><span class="line"><span class="function"><span class="keyword">size_t</span> <span class="title">min_heap_size_</span><span class="params">(<span class="keyword">min_heap_t</span>* s)</span> </span>{ <span class="keyword">return</span> s->n; }<span class="comment">//堆的节点数量</span></span><br><span class="line"><span class="function">struct event* <span class="title">min_heap_top_</span><span class="params">(<span class="keyword">min_heap_t</span>* s)</span> </span>{ <span class="keyword">return</span> s->n ? *s->p : <span class="number">0</span>; }<span class="comment">//堆顶节点</span></span><br><span class="line"><span class="function"><span class="keyword">void</span> <span class="title">min_heap_dtor_</span><span class="params">(<span class="keyword">min_heap_t</span>* s)</span> </span>{ <span class="keyword">if</span> (s->p) mm_free(s->p); }<span class="comment">//释放堆</span></span><br><span class="line"><span class="comment">//节点为堆中最大索引(最后一个节点)</span></span><br><span class="line"><span class="function"><span class="keyword">void</span> <span class="title">min_heap_elem_init_</span><span class="params">(struct event* e)</span> </span>{ e->ev_timeout_pos.min_heap_idx = EV_SIZE_MAX; }</span><br></pre></td></tr></table></figure><h3 id="添加节点"><a class="markdownIt-Anchor" href="#添加节点"></a> 添加节点</h3><figure class="highlight c"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment">//event.c:2217 event_assign</span></span><br><span class="line"></span><br><span class="line">min_heap_elem_init_(ev);</span><br><span class="line"><span class="function"><span class="keyword">void</span> <span class="title">min_heap_elem_init_</span><span class="params">(struct event* e)</span> </span>{ e->ev_timeout_pos.min_heap_idx = EV_SIZE_MAX; }</span><br></pre></td></tr></table></figure><p>在添加节点的时候，首先指定节点在堆中的索引是 <code>EV_SIZE_MAX</code>，表示并没有加入到堆中，接着加入到堆中</p><figure class="highlight c"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br></pre></td><td class="code"><pre><span class="line"><span class="function"><span class="keyword">int</span> <span class="title">min_heap_push_</span><span class="params">(<span class="keyword">min_heap_t</span>* s, struct event* e)</span></span></span><br><span class="line"><span class="function"></span>{</span><br><span class="line"><span class="keyword">if</span> (min_heap_reserve_(s, s->n + <span class="number">1</span>))</span><br><span class="line"><span class="keyword">return</span> <span class="number">-1</span>;</span><br><span class="line">min_heap_shift_up_(s, s->n++, e);</span><br><span class="line"><span class="keyword">return</span> <span class="number">0</span>;</span><br><span class="line">}</span><br></pre></td></tr></table></figure><p>这里就发生了两步操作</p><ol><li>因为可能堆已经满了，所以需要将堆进行扩容 <code>min_heap_reserve_</code></li><li>堆化，需要调整成小顶堆</li></ol><h4 id="扩容"><a class="markdownIt-Anchor" href="#扩容"></a> 扩容</h4><p>首次分配的容量大小是8，后续依次加倍</p><blockquote><p>问题1：如果节点新增很多，会不会导致内存分配暴涨？</p><p>golang 的 map 有一个 2倍增长到了一定数量再1.25倍增长的过程</p></blockquote><figure class="highlight c"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br></pre></td><td class="code"><pre><span class="line"><span class="function"><span class="keyword">int</span> <span class="title">min_heap_reserve_</span><span class="params">(<span class="keyword">min_heap_t</span>* s, <span class="keyword">size_t</span> n)</span></span></span><br><span class="line"><span class="function"></span>{</span><br><span class="line"><span class="keyword">if</span> (s->a < n)<span class="comment">//判断容量是否不够</span></span><br><span class="line">{</span><br><span class="line"><span class="class"><span class="keyword">struct</span> <span class="title">event</span>** <span class="title">p</span>;</span></span><br><span class="line"><span class="keyword">size_t</span> a = s->a ? s->a * <span class="number">2</span> : <span class="number">8</span>;<span class="comment">// 小于8则使用8，否则加倍增长</span></span><br><span class="line"><span class="keyword">if</span> (a < n)<span class="comment">//如果仍然小于n, 则直接使用 n </span></span><br><span class="line">a = n;</span><br><span class="line"><span class="keyword">if</span> (!(p = (struct event**)mm_realloc(s->p, a * <span class="keyword">sizeof</span> *p)))</span><br><span class="line"><span class="keyword">return</span> <span class="number">-1</span>;</span><br><span class="line">s->p = p;</span><br><span class="line">s->a = a;</span><br><span class="line">}</span><br><span class="line"><span class="keyword">return</span> <span class="number">0</span>;</span><br><span class="line">}</span><br></pre></td></tr></table></figure><ol><li>当容量不够的时候，需要进行扩容操作，不需要考虑 golang map 那种因为装载量过大而影响查询效率的情况</li><li>容量首先是8，然后依次递增，当等待事件比较多，注意内存占用(一大块内存)</li><li>使用 <code>realloc</code> 操作，直接保留了原来的内容(<strong>realloc 的底层并不一定在原有的基础上，因为无法保证内存后面是否仍然有足够的空闲空间</strong>）</li></ol><h4 id="向上调整"><a class="markdownIt-Anchor" href="#向上调整"></a> 向上调整</h4><p>元素入堆的基本流程是：</p><ol><li><p>将元素添加到堆尾(堆最后一个元素)</p></li><li><p>将元素与父节点进行比较</p><p>a. 如果比父节点大则与父节点交换位置 然后重复 1～2 操作</p><p>b. 如果不比父节点大则停止入堆，完成元素添加</p></li></ol><p>接着看它的实际操作</p><figure class="highlight c"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment">//min_heap_shift_up_(s, s->n++, e);</span></span><br><span class="line"><span class="function"><span class="keyword">void</span> <span class="title">min_heap_shift_up_</span><span class="params">(<span class="keyword">min_heap_t</span>* s, <span class="keyword">size_t</span> hole_index, struct event* e)</span></span></span><br><span class="line"><span class="function"></span>{</span><br><span class="line"> <span class="comment">//找到节点的父节点</span></span><br><span class="line"> <span class="keyword">size_t</span> parent = (hole_index - <span class="number">1</span>) / <span class="number">2</span>;</span><br><span class="line"> <span class="keyword">while</span> (hole_index && min_heap_elem_greater(s->p[parent], e))</span><br><span class="line"> {</span><br><span class="line"> (s->p[hole_index] = s->p[parent])->ev_timeout_pos.min_heap_idx = hole_index;</span><br><span class="line"> hole_index = parent;</span><br><span class="line"> parent = (hole_index - <span class="number">1</span>) / <span class="number">2</span>;</span><br><span class="line"> }</span><br><span class="line"> (s->p[hole_index] = e)->ev_timeout_pos.min_heap_idx = hole_index;</span><br><span class="line">}</span><br></pre></td></tr></table></figure><ol><li><code>hole_index</code> 一开始为堆的最后一个元素的索引，即当前事件 e 的位置</li><li>找到最后一个节点的父节点 <code>(hole_index - 1) / 2;</code></li><li>如果当前位置 <code>hole_index</code> 还不是堆顶(<code>hole_index != 0</code> )且 父节点超时时间比自己大<ul><li>当前元素等于父节点，索引位置更新为 <code>hole_index</code></li><li>当前索引 <code>hole_index</code> 为父节点索引，<code>hole_index = parent;</code></li><li>找到父节点位置</li><li>重复操作 3</li></ul></li><li>更新当前节点为e，设置索引为 <code>hole_index</code></li></ol><p>比较逻辑则是使用超时时间进行对比</p><figure class="highlight c"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br></pre></td><td class="code"><pre><span class="line"><span class="meta">#<span class="meta-keyword">define</span> min_heap_elem_greater(a, b) \</span></span><br><span class="line">(evutil_timercmp(&(a)->ev_timeout, &(b)->ev_timeout, >))</span><br><span class="line"></span><br><span class="line"><span class="comment">//如果秒相同就是用毫秒</span></span><br><span class="line"><span class="meta">#<span class="meta-keyword">define</span>evutil_timercmp(tvp, uvp, cmp)\</span></span><br><span class="line">(((tvp)->tv_sec == (uvp)->tv_sec) ?\</span><br><span class="line"> ((tvp)->tv_usec cmp (uvp)->tv_usec) :\</span><br><span class="line"> ((tvp)->tv_sec cmp (uvp)->tv_sec))</span><br></pre></td></tr></table></figure><h3 id="弹出节点"><a class="markdownIt-Anchor" href="#弹出节点"></a> 弹出节点</h3><p>有增加就有删除，出堆常规操作是</p><ol><li>将堆顶与堆尾元素进行交换</li><li>将堆尾元素出堆</li><li>从对顶元素开始从上到下，与子节点比较交换</li></ol><p>看看它是如何做到的</p><figure class="highlight c"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br></pre></td><td class="code"><pre><span class="line"><span class="function">struct event* <span class="title">min_heap_pop_</span><span class="params">(<span class="keyword">min_heap_t</span>* s)</span></span></span><br><span class="line"><span class="function"></span>{</span><br><span class="line"><span class="keyword">if</span> (s->n)</span><br><span class="line">{</span><br><span class="line"><span class="class"><span class="keyword">struct</span> <span class="title">event</span>* <span class="title">e</span> = *<span class="title">s</span>-><span class="title">p</span>;</span></span><br><span class="line">min_heap_shift_down_(s, <span class="number">0</span>, s->p[--s->n]);</span><br><span class="line">e->ev_timeout_pos.min_heap_idx = EV_SIZE_MAX;</span><br><span class="line"><span class="keyword">return</span> e;</span><br><span class="line">}</span><br><span class="line"><span class="keyword">return</span> <span class="number">0</span>;</span><br><span class="line">}</span><br></pre></td></tr></table></figure><ol><li><code>--s->n</code> 因为要弹出，所以 数量减 1，<code>s->p[--s->n]</code> 指向堆尾</li><li>如果堆元素数量大于 0，记录堆顶事件。否则返回 0(与前面一样，0 指向指针表示 NULL)</li><li>向下调整，注意最后一个节点指向的是最后一个节点</li><li>更新出堆事件的索引(表示不在堆里面)</li></ol><h4 id="向下调整"><a class="markdownIt-Anchor" href="#向下调整"></a> 向下调整</h4><p>将 <code>hole_index</code> 与 <code>struct event* e 进行交换</code></p><figure class="highlight c"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br></pre></td><td class="code"><pre><span class="line"><span class="function"><span class="keyword">void</span> <span class="title">min_heap_shift_down_</span><span class="params">(<span class="keyword">min_heap_t</span>* s, <span class="keyword">size_t</span> hole_index, struct event* e)</span></span></span><br><span class="line"><span class="function"></span>{</span><br><span class="line"><span class="keyword">size_t</span> min_child = <span class="number">2</span> * (hole_index + <span class="number">1</span>);</span><br><span class="line"><span class="keyword">while</span> (min_child <= s->n)</span><br><span class="line">{</span><br><span class="line">min_child -= min_child == s->n || min_heap_elem_greater(s->p[min_child], s->p[min_child - <span class="number">1</span>]);</span><br><span class="line"><span class="keyword">if</span> (!(min_heap_elem_greater(e, s->p[min_child])))</span><br><span class="line"> <span class="keyword">break</span>;</span><br><span class="line">(s->p[hole_index] = s->p[min_child])->ev_timeout_pos.min_heap_idx = hole_index;</span><br><span class="line">hole_index = min_child;</span><br><span class="line">min_child = <span class="number">2</span> * (hole_index + <span class="number">1</span>);</span><br><span class="line">}</span><br><span class="line"> (s->p[hole_index] = e)->ev_timeout_pos.min_heap_idx = hole_index;</span><br><span class="line">}</span><br></pre></td></tr></table></figure><ol><li><p>找到节点的右子节点 <code>min_child</code></p></li><li><p>如果右节点存在(不应该超过数目上线)</p><ul><li><p>找到左右节点种较大的</p></li><li><p>如果当前超时时间比子节点小则停止</p><blockquote><p>为什么有 <code>min_child == s->n</code> 的比较操作，感觉没有必要！！因为 <code>while (min_child <= s->n)</code> 已经判断了</p></blockquote></li><li><p>当前节点为子节点，并更新索引</p></li><li><p>更新当前节点位置</p></li><li><p>找到当前节点的右子节点</p></li></ul></li><li><p>当前位置设置为 e，并更新事件的索引</p></li></ol><h3 id="删除元素"><a class="markdownIt-Anchor" href="#删除元素"></a> 删除元素</h3><p>删除元素常用于事件发生异常，需要从堆中直接删除而不是从堆顶，这个时候就会出现堆空洞的情况</p><figure class="highlight c"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br></pre></td><td class="code"><pre><span class="line"><span class="function"><span class="keyword">int</span> <span class="title">min_heap_erase_</span><span class="params">(<span class="keyword">min_heap_t</span>* s, struct event* e)</span></span></span><br><span class="line"><span class="function"></span>{</span><br><span class="line"><span class="keyword">if</span> (EV_SIZE_MAX != e->ev_timeout_pos.min_heap_idx)</span><br><span class="line">{</span><br><span class="line"><span class="class"><span class="keyword">struct</span> <span class="title">event</span> *<span class="title">last</span> = <span class="title">s</span>-><span class="title">p</span>[--<span class="title">s</span>-><span class="title">n</span>];</span></span><br><span class="line"><span class="keyword">size_t</span> parent = (e->ev_timeout_pos.min_heap_idx - <span class="number">1</span>) / <span class="number">2</span>;</span><br><span class="line"><span class="keyword">if</span> (e->ev_timeout_pos.min_heap_idx > <span class="number">0</span> && min_heap_elem_greater(s->p[parent], last))</span><br><span class="line">min_heap_shift_up_unconditional_(s, e->ev_timeout_pos.min_heap_idx, last);</span><br><span class="line"><span class="keyword">else</span></span><br><span class="line">min_heap_shift_down_(s, e->ev_timeout_pos.min_heap_idx, last);</span><br><span class="line">e->ev_timeout_pos.min_heap_idx = EV_SIZE_MAX;</span><br><span class="line"><span class="keyword">return</span> <span class="number">0</span>;</span><br><span class="line">}</span><br><span class="line"><span class="keyword">return</span> <span class="number">-1</span>;</span><br><span class="line">}</span><br></pre></td></tr></table></figure><ul><li>如果 <code>e->ev_timeout_pos.min_heap_idx != EV_SIZE_MAX</code> 表示已经不在堆中了，添加和弹出的时候都会更新这个索引</li><li>获取堆尾元素</li><li>获取事件的父节点</li><li>如果事件不在堆顶而且父结点大于最后一个节点<ul><li>向上调整</li></ul></li><li>如果事件在堆顶，类似 <code>min_heap_pop_</code><ul><li>向下调整</li></ul></li></ul><h4 id="向上调整2"><a class="markdownIt-Anchor" href="#向上调整2"></a> 向上调整2</h4><p>这里为了防止空洞，出现了一个不一样的操作 <code>min_heap_shift_up_unconditional_</code></p><figure class="highlight c"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br></pre></td><td class="code"><pre><span class="line"><span class="function"><span class="keyword">void</span> <span class="title">min_heap_shift_up_unconditional_</span><span class="params">(<span class="keyword">min_heap_t</span>* s, <span class="keyword">size_t</span> hole_index, struct event* e)</span></span></span><br><span class="line"><span class="function"></span>{</span><br><span class="line"> <span class="keyword">size_t</span> parent = (hole_index - <span class="number">1</span>) / <span class="number">2</span>;</span><br><span class="line"> <span class="keyword">do</span></span><br><span class="line"> {</span><br><span class="line">(s->p[hole_index] = s->p[parent])->ev_timeout_pos.min_heap_idx = hole_index;</span><br><span class="line">hole_index = parent;</span><br><span class="line">parent = (hole_index - <span class="number">1</span>) / <span class="number">2</span>;</span><br><span class="line"> } <span class="keyword">while</span> (hole_index && min_heap_elem_greater(s->p[parent], e));</span><br><span class="line"> (s->p[hole_index] = e)->ev_timeout_pos.min_heap_idx = hole_index;</span><br><span class="line">}</span><br></pre></td></tr></table></figure><ol><li>首先找到当前元素的父节点</li><li>将父节点更新到当前位置，并跟新索引</li><li>设置当前位置为父节点，找到父节点的父节点 <code>parent2</code></li><li>如果 <code>parent2</code> 不是堆顶，而且 <code>parent2</code> 大于 last，那么重复 2~3操作（感觉不会，因为是小根堆）</li><li>更新 last 到合适的位置</li></ol><p>也就是说通过 将 last 填补到要删除元素的位置，解决空洞问题。</p><p>问题1：<code>min_heap_shift_up_unconditional_</code> 与 <code>min_heap_shift_up_</code> 不同的是，后者先判断了 当前位置以及 当前位置事件与 e 的大小，为什么要做这个区别？</p><p>答：是因为前者的位置肯定不是堆顶(走乡下条)</p><p>问题2：<code>min_heap_erase_</code> 删除元素中为什么会出现向下调整与向上调整</p><p>答：首先需要明白向上调整与向下调整的作用分别是什么，</p><blockquote><p>向下调整：被用于在删除堆顶元素后，将最后一个元素移到堆顶并逐步将其“下沉”至合适的位置，以保持堆的有序性质。</p><p>向上调整：被用于在向堆中插入新元素后，将其逐步“上浮”至合适的位置，以保持堆的有序性质</p></blockquote><p>如果删除位置不是堆顶元素，且父节点大于 last 。那么根据小根堆的性质，last 应该从这个位置开始向上调整</p><p>如果删除位置是堆顶元素或者父节点小于last。那么根据小根堆的性质，last应该在删除位置的下发（或不变）</p><h3 id="参考链接"><a class="markdownIt-Anchor" href="#参考链接"></a> 参考链接</h3><ol><li><a href="https://www.hello-algo.com/chapter_heap/heap/" target="_blank" rel="noopener">https://www.hello-algo.com/chapter_heap/heap/</a></li><li><a href="https://www.hello-algo.com/chapter_sorting/heap_sort/" target="_blank" rel="noopener">https://www.hello-algo.com/chapter_sorting/heap_sort/</a></li></ol>]]></content>
<summary type="html">
<p>在阅读 libevent 库中另外一个用来存储事件的第二个结构就是小根堆，这里重新学习一下它的实现</p>
<p>代码路径:<code>minheap-internal.h</code></p>
<figure class="highlight c"><table><tr>
</summary>
<category term="Data Structure" scheme="https://www.yuankang.top/categories/Data-Structure/"/>
<category term="Heap" scheme="https://www.yuankang.top/tags/Heap/"/>
<category term="Data Structure" scheme="https://www.yuankang.top/tags/Data-Structure/"/>
<category term="Libevent" scheme="https://www.yuankang.top/tags/Libevent/"/>
</entry>
<entry>
<title>算法-01-Heap</title>
<link href="https://www.yuankang.top/2023/09/29/Algorithm/%E7%AE%97%E6%B3%95-01-Heap/"/>
<id>https://www.yuankang.top/2023/09/29/Algorithm/%E7%AE%97%E6%B3%95-01-Heap/</id>
<published>2023-09-28T16:15:00.000Z</published>
<updated>2023-09-28T16:21:25.689Z</updated>
<content type="html"><![CDATA[<p>堆(Heap)是具有以下性质的<strong>完全二叉树</strong></p><ul><li><p>每个结点的值都<strong>大于或等于其左右孩子</strong>结点的值，称为<strong>大顶堆</strong></p><p><img src="https://images-1307117474.cos.ap-guangzhou.myqcloud.com/blogs/Blog_3/Snipaste_2023-09-28_22-16-36.png" alt="Snipaste_2023-09-28_22-16-36" /></p></li><li><p>每个结点的值都<strong>小于或等于其左右孩子</strong>结点的值，称为<strong>小顶堆</strong></p><p><img src="https://images-1307117474.cos.ap-guangzhou.myqcloud.com/blogs/Blog_3/Snipaste_2023-09-28_22-17-26.png" alt="Snipaste_2023-09-28_22-17-26" /></p></li></ul><p><strong>完全二叉树</strong>：除了最后一层，其他层的节点个数都是满的，最后一层的节点都靠左排列</p><p>**问题 **：那左右子节点有谁大谁小？<strong>没有要求！！！</strong></p><p>大根堆，每个子结点都要小于父结点，不区分左右儿子谁大谁小，<strong>也不必保证某个 孙子结点 一定要小于另一个 儿子结点</strong><br />小根堆，每个子结点都要大于父结点，不区分左右儿子谁大谁小，**也不必保证某个 孙子结点 一定要大于另一个 儿子结点 **</p><p>也就是说，<strong>即使组成了 大小根堆，它们的排列还不是有序的</strong>，只是保证</p><p><strong>大顶堆：arr[i] >= arr[2i+1] && arr[i] >= arr[2i+2]</strong></p><p><strong>小顶堆：arr[i] <= arr[2i+1] && arr[i] <= arr[2i+2]</strong></p><h3 id="实现原理"><a class="markdownIt-Anchor" href="#实现原理"></a> 实现原理</h3><p>首先明白两个概念：<strong>向下调整</strong> 与 <strong>向上调整</strong></p><p><strong>向下调整</strong>：指的是为了满足堆的性质而向下不断比较调整父子节点的关系，一般用于堆删除一个元素</p><ul><li>从堆的根节点开始(通常是数组的第一个元素)，比较当前节点与其子节点的值。</li><li>如果当前节点的值小于其子节点中的一个或多个节点的值，那么将当前节点与最大的子节点进行交换，以满足大顶堆的性质。</li><li>重复上述步骤，直到节点没有子节点或者节点的值大于等于其子节点的值</li></ul><p>这里删除堆顶元素之后还向下调整的原因是，会将最后一个元素放置到堆顶</p><p><strong>向上调整</strong>：指的是为了满足堆的性质而向上不断比较调整父子节点的关系，一般用于堆插入一个元素</p><ul><li>在堆的最后一个位置(通常是数组的最后一个元素)插入新元素。</li><li>比较新插入的元素与其父节点的值。</li><li>如果新元素的值大于其父节点的值（对于大顶堆），则交换新元素与父节点，以满足堆性质。</li><li>重复上述步骤，直到新元素的值小于等于其父节点的值或者新元素已经成为根节点</li></ul><p>实际应用中还有一种情况，比如需要从堆中直接删除一个元素(元素已经失效，需要释放该元素重新调整<strong>小根堆</strong>的结构)，此时会出现一个<strong>空洞</strong></p><ul><li>将最后一个元素 last 放到被删除的位置</li><li>如果删除位置不是堆顶元素，且父节点大于 last 。那么根据小根堆的性质，last 应该从这个位置开始向上调整</li><li>如果删除位置是堆顶元素或者父节点小于last。那么根据小根堆的性质，last应该在删除位置的下发(或不变)</li></ul><p>所以上面其实就是堆的插入和删除过程了，原理就是只在头尾处理，然后进行向上或者向下调整，保证大根堆、小根堆的性质</p><p><strong>问题 1</strong>：为什么不在删除对顶元素之后，直接将第二大元素放到堆顶呢？</p><p>答：这样的后出现空洞的情况，就需要进一步处理</p><img src="https://static001.geekbang.org/resource/image/59/81/5916121b08da6fc0636edf1fc24b5a81.jpg?wh=1142*867" alt="img" style="zoom:50%;" /><p>所以，这里改变思路：将最后一个节点与堆顶交换，然后删除最后一个元素，最后利用同样的父子节点对比方法。对于不满足父子节点大小关系的，互换两个节点，并且重复进行这个过程，直到父子节点之间满足大小关系为止。这就是<strong>从上往下</strong>的堆化方法</p><img src="https://static001.geekbang.org/resource/image/11/60/110d6f442e718f86d2a1d16095513260.jpg?wh=1142*856" alt="img" style="zoom:50%;" /><p><strong>问题 2</strong>：知道了插入和删除，它首要条件是有一个堆，所以堆从哪里来</p><p>答：方案有两种</p><ul><li>从前往后，尽管数组中包含 n 个数据，假设起初堆中只包含一个数据，就是下标为 0 的数据。然后不断的插入将下标从 1 到 n - 1 的数据依次插入到堆中。每插入一个元素都向上调整到合适位置</li><li>从后往前，依次与子节点进行比较，因为叶子节点无法与子节点比较，所以就需要从最后一个非叶子节点比较。每次都向下调整到合适的位置</li></ul><p><strong>问题 3</strong>：最后一个非叶子节点如何表示</p><p>答：假设最后一个非叶子节点下表是 i (<strong>i 从 0 开始</strong>)</p><ul><li>如果它的左子节点是最后一个节点，那么最后一个节点下标就是 <code>2i + 1</code>。整个二叉树的长度就是 <code>len = 2i + 2</code></li><li>如果他的右子节点是最后一个节点，那么最后一个节点下标就是 <code>2i + 2</code>。整个二叉树的长度就是 <code>len = 2i + 3</code></li><li>所以最后一个非叶子节点的下标就是 <code>i = len / 2 - 1</code></li></ul><p><strong>问题 4</strong>：建堆的过程有两种，哪种更好</p><p>答：第二种方式可以节省一半的时间</p><ul><li>第一种方式需要对<code>n-1</code>个节点进行堆化，每个节点堆化的时间复杂度为 <code>O(logn)</code>，所以它精确的时间复杂度为<code>(n-1)*O(logn)</code></li><li>第二种方式从非叶子节点开始，需要对 <code>n/2</code>个节点进行堆化，精确的时间复杂度为<code>(n/2)*O(logn)</code></li></ul><p><strong>问题 5</strong>：既然不关心左右节点大小，那样怎么样做到整个有序呢？</p><p>答：就像删除堆顶元素一样。当堆顶元素移除之后，把下标为 n 的元素放到堆顶，然后向下调整，让剩下的 n−1 个元素重新构建成堆。堆化完成之后，再取堆顶的元素，放到下标是 n−1 的位置，一直重复这个过程，直到最后堆中只剩下标为 1 的一个元素，排序工作就完成了。</p><img src="https://static001.geekbang.org/resource/image/23/d1/23958f889ca48dbb8373f521708408d1.jpg?wh=1142*790" alt="img" style="zoom:50%;" /><p>一个包含 n 个节点的完全二叉树，树的高度不会超过 <code>log2n</code>，堆化的过程是顺着节点所在路径比较交换的，所以堆化的时间复杂度跟树的高度成正比，也就是 <code>O(logn)</code>。</p><p>插入数据和删除堆顶元素的主要逻辑就是堆化，所以，往堆中插入一个元素和删除堆顶元素的时间复杂度都是 <code>O(logn)</code></p><p>因为每个元素都需要进行删除的过程，所以堆排序整体的时间复杂度为 <code>O(nlogn)</code></p><h3 id="代码实现"><a class="markdownIt-Anchor" href="#代码实现"></a> 代码实现</h3><figure class="highlight c++"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br><span class="line">45</span><br><span class="line">46</span><br><span class="line">47</span><br><span class="line">48</span><br><span class="line">49</span><br><span class="line">50</span><br><span class="line">51</span><br><span class="line">52</span><br><span class="line">53</span><br><span class="line">54</span><br><span class="line">55</span><br><span class="line">56</span><br><span class="line">57</span><br><span class="line">58</span><br><span class="line">59</span><br><span class="line">60</span><br><span class="line">61</span><br><span class="line">62</span><br><span class="line">63</span><br><span class="line">64</span><br><span class="line">65</span><br><span class="line">66</span><br></pre></td><td class="code"><pre><span class="line"><span class="meta">#<span class="meta-keyword">include</span><span class="meta-string"><bits/stdc++.h></span></span></span><br><span class="line"><span class="keyword">using</span> <span class="keyword">namespace</span> <span class="built_in">std</span>;</span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">void</span> <span class="title">print</span><span class="params">(<span class="keyword">int</span> a[], <span class="keyword">int</span> n)</span></span></span><br><span class="line"><span class="function"></span>{</span><br><span class="line"> <span class="keyword">int</span> i;</span><br><span class="line"> <span class="keyword">for</span>(i = <span class="number">0</span>; i < n; i++)</span><br><span class="line"> {</span><br><span class="line"> <span class="built_in">cout</span> << a[i] << <span class="string">" "</span>;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="built_in">cout</span> << <span class="built_in">endl</span>;</span><br><span class="line">}</span><br><span class="line"> </span><br><span class="line"><span class="comment">//heapAdjust 作用从上往下构建，保证数组 a[] 中 第low个元素为堆顶的元素构成的结构为大顶堆</span></span><br><span class="line"><span class="function"><span class="keyword">void</span> <span class="title">heapAdjust</span><span class="params">(<span class="keyword">int</span> a[], <span class="keyword">int</span> low, <span class="keyword">int</span> high)</span><span class="comment">//调整的过程</span></span></span><br><span class="line"><span class="function"></span>{</span><br><span class="line"> <span class="keyword">int</span> pivotKey = a[low - <span class="number">1</span>];<span class="comment">//注意数组的索引从0开始</span></span><br><span class="line"> <span class="keyword">int</span> i; <span class="comment">//当前节点下标</span></span><br><span class="line"> <span class="keyword">for</span>(i = <span class="number">2</span> * low; i <= high; i *= <span class="number">2</span>) </span><br><span class="line"> {</span><br><span class="line"> <span class="keyword">if</span>(i < high && a[i - <span class="number">1</span>] < a[i])<span class="comment">//找到左右子节点中较大的那个</span></span><br><span class="line"> {</span><br><span class="line"> i++; <span class="comment">//i指向较大值</span></span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span>(pivotKey >= a[i - <span class="number">1</span>])<span class="comment">//如果大于等于子节点说明已经是大顶堆</span></span><br><span class="line"> {</span><br><span class="line"> <span class="keyword">break</span>;</span><br><span class="line"> }</span><br><span class="line"> </span><br><span class="line"> a[low - <span class="number">1</span>] = a[i - <span class="number">1</span>];<span class="comment">//否则将子节点更换到父节点的位置</span></span><br><span class="line"> low = i; <span class="comment">//然后继续判断子节点与孙子节点的大小,保证子节点与起构成的子节点也是大堆顶</span></span><br><span class="line"> }</span><br><span class="line"> a[low - <span class="number">1</span>] = pivotKey; <span class="comment">//将最开始的堆顶放到合适的位置</span></span><br><span class="line">}</span><br><span class="line"> </span><br><span class="line"><span class="function"><span class="keyword">void</span> <span class="title">heapSort</span><span class="params">(<span class="keyword">int</span> a[], <span class="keyword">int</span> n)</span></span></span><br><span class="line"><span class="function"></span>{</span><br><span class="line"> <span class="keyword">int</span> i, tmp;</span><br><span class="line"> <span class="comment">//堆化</span></span><br><span class="line"> <span class="keyword">for</span>(i = n/<span class="number">2</span>; i > <span class="number">0</span>; i--) <span class="comment">//从非叶子节点开始，开始调整(最后一个非叶子节点是 len/2 - 1)</span></span><br><span class="line"> {</span><br><span class="line"> heapAdjust(a, i, n);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">for</span>(i = n; i > <span class="number">1</span>; i--)<span class="comment">//将堆顶元素放置最后，开始调整</span></span><br><span class="line"> {</span><br><span class="line"> <span class="comment">//调整最后一个元素于第一个元素的位置</span></span><br><span class="line"> tmp = a[i <span class="number">-1</span>];</span><br><span class="line"> a[i - <span class="number">1</span>] = a[<span class="number">0</span>];</span><br><span class="line"> a[<span class="number">0</span>] = tmp;</span><br><span class="line"> <span class="comment">//向下调整</span></span><br><span class="line"> heapAdjust(a, <span class="number">1</span>, i - <span class="number">1</span>); </span><br><span class="line"> <span class="built_in">print</span>(a, n);</span><br><span class="line"> }</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">int</span> <span class="title">main</span><span class="params">()</span></span></span><br><span class="line"><span class="function"></span>{</span><br><span class="line"> <span class="keyword">int</span> a[] = {<span class="number">30</span>, <span class="number">20</span>, <span class="number">40</span>, <span class="number">10</span>, <span class="number">0</span>, <span class="number">60</span>, <span class="number">80</span>, <span class="number">70</span>};</span><br><span class="line"> <span class="keyword">int</span> n = <span class="keyword">sizeof</span>(a) / <span class="keyword">sizeof</span>(a[<span class="number">0</span>]); <span class="comment">//计算数组长度</span></span><br><span class="line"> heapSort(a, n);</span><br><span class="line"> <span class="built_in">print</span>(a, n);</span><br><span class="line"> <span class="keyword">return</span> <span class="number">0</span>;</span><br><span class="line">}</span><br></pre></td></tr></table></figure><h3 id="堆排序应用"><a class="markdownIt-Anchor" href="#堆排序应用"></a> 堆排序应用</h3><ol><li><p>Top K问题</p><p>从堆的定义来说，很容易想到堆顶元素是TOP 1，那么如何求经典的TOP K问题?</p><p>如果求最大的Top K 元素，是建立大顶堆，还是小顶堆</p><ol><li>如果用大顶堆，堆顶是最大的元素，新加入的元素必须和所有的堆中元素做比对，显然不够划算；</li><li>如果用小顶堆，那堆顶元素是最小的元素，新加入的元素只要和堆顶元素做对比，如果比堆顶元素大，就可以把堆顶的元素删除，将新加入的元素加入到堆中</li></ol><p>每次求TopK的问题的时候，只需要直接返回堆中所有元素即可，每次堆化的算法复杂度为O(logK),那么N个元素堆化的时间为O(nlogK),如果不用堆，每次新增元素都要重新排序，再取前面N个</p></li><li><p>定时器应用</p><p>独立线程，以超时时间建立了一个小顶堆，如果堆顶元素为2分钟，清理连接的线程的休眠时间设置为2分钟，2分钟后取堆顶元素，执行连接关闭操作。</p></li><li><p><strong>求中位数和99%的响应时间</strong></p><p>动态数组取中位数：</p><ul><li>如果n为偶数，前n/2数据存储在大顶堆中，n/2存储小顶堆中，则大顶堆的堆顶元素为要找的中位数</li><li>如果n为奇数，可以将前n/2+1个数据存储在大顶堆中，后n/2存储在小顶堆中</li></ul><p>如果添加新数据，则将数据与大顶堆中的堆顶元素比较</p><ul><li>如果小于等于大顶堆中的元素，就插入到大顶堆中</li><li>如果比大顶堆的堆顶大，那就插入到小顶堆中</li><li>如果插入数据后不满足要求两个堆的数量为n/2和n/2 或n/2 和n+1/2 的要求，需要调整两个堆的大小，从大顶堆中删除堆顶元素，或小顶堆中删除堆顶元素，移动到另外一个堆中即可。</li></ul><blockquote><p>99%的响应时间：如果一个接口的有100个访问请求，分别耗时1ms，2ms，3ms…100ms，那么按照访问时间从小到大排列，排在第99位的访问时间，就是99%的访问时间，我们维护两个堆，大顶堆的元素个数为n * 99%，小顶堆的元素个数为n*1%，那么大顶堆中的堆顶元素即是所求的99%的响应时间，和中位数的应用一样，只是中位数中的应用更特殊一点</p></blockquote></li><li><p>优先级队列</p><p>优先级队列和通常的栈和队列一样，只不过里面的每一个元素都有一个"优先级”，在处理的时候，首先处理优先级最高的。如果两个元素具有相同的优先级，则按照他们插入到队列中的先后顺序处理</p></li></ol><h3 id="参考链接"><a class="markdownIt-Anchor" href="#参考链接"></a> 参考链接</h3><ol><li><a href="https://www.cnblogs.com/chengxiao/p/6129630.html" target="_blank" rel="noopener">https://www.cnblogs.com/chengxiao/p/6129630.html</a></li><li><a href="https://mp.weixin.qq.com/s/OqpSdFfK12NhPpcsXTMvtA" target="_blank" rel="noopener">https://mp.weixin.qq.com/s/OqpSdFfK12NhPpcsXTMvtA</a></li><li><a href="https://blog.csdn.net/weixin_45728685/article/details/105115912" target="_blank" rel="noopener">https://blog.csdn.net/weixin_45728685/article/details/105115912</a></li></ol>]]></content>
<summary type="html">
<p>堆(Heap)是具有以下性质的<strong>完全二叉树</strong></p>
<ul>
<li>
<p>每个结点的值都<strong>大于或等于其左右孩子</strong>结点的值，称为<strong>大顶堆</strong></p>
<p><img src="htt
</summary>
<category term="Algorithms" scheme="https://www.yuankang.top/categories/Algorithms/"/>
<category term="Algorithms" scheme="https://www.yuankang.top/tags/Algorithms/"/>
<category term="Heap" scheme="https://www.yuankang.top/tags/Heap/"/>
</entry>
<entry>
<title>数据结构-01-List</title>
<link href="https://www.yuankang.top/2023/09/18/Data%20Structure/%E6%95%B0%E6%8D%AE%E7%BB%93%E6%9E%84-01-List/"/>
<id>https://www.yuankang.top/2023/09/18/Data%20Structure/%E6%95%B0%E6%8D%AE%E7%BB%93%E6%9E%84-01-List/</id>
<published>2023-09-18T14:58:29.000Z</published>
<updated>2023-09-28T16:22:28.149Z</updated>
<content type="html"><![CDATA[<p>在阅读 Libevent 库的时候看到 <code>TAILQ_ENTRY </code>的宏定义有点特别，这里记录一下，代码路径 <code>libevent/compat/sys/queue.h</code></p><p>首先它实现的类似泛型的功能</p><figure class="highlight c"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br></pre></td><td class="code"><pre><span class="line"><span class="meta">#<span class="meta-keyword">define</span> TAILQ_ENTRY(type)\</span></span><br><span class="line"><span class="class"><span class="keyword">struct</span> {</span>\</span><br><span class="line"><span class="class"><span class="keyword">struct</span> <span class="title">type</span> *<span class="title">tqe_next</span>;</span><span class="comment">/* next element */</span>\</span><br><span class="line"><span class="class"><span class="keyword">struct</span> <span class="title">type</span> **<span class="title">tqe_prev</span>;</span><span class="comment">/* address of previous next element */</span>\</span><br><span class="line">}</span><br><span class="line"><span class="meta">#<span class="meta-keyword">endif</span> <span class="comment">/* !TAILQ_ENTRY */</span></span></span><br></pre></td></tr></table></figure><blockquote><p>C11 中还能通过 _Generic关键字 实现泛型编程</p></blockquote><p>但是有一个奇怪的地方，<code>tqe_prev</code> 为什么是指针的指针，这里可能跟平时的好像不太一样?</p><p>平时定义的双向链表节点</p><figure class="highlight c"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment">// 定义一个结构体，表示链表中的节点</span></span><br><span class="line"><span class="class"><span class="keyword">struct</span> <span class="title">Node</span> {</span></span><br><span class="line"> <span class="keyword">int</span> data;</span><br><span class="line"> <span class="class"><span class="keyword">struct</span> <span class="title">Node</span>* <span class="title">next</span>;</span> <span class="comment">// 后一个节点的指针</span></span><br><span class="line"> <span class="class"><span class="keyword">struct</span> <span class="title">Node</span>* <span class="title">prev</span>;</span> <span class="comment">// 前一个节点的指针</span></span><br><span class="line">};</span><br></pre></td></tr></table></figure><p>由它构建的双向链表结构</p><p><img src="https://images-1307117474.cos.ap-guangzhou.myqcloud.com/blogs/Blog_3/%E5%8F%8C%E5%90%91%E9%93%BE%E8%A1%A8.png" alt="双向链表" /></p><p>删除节点的操作</p><figure class="highlight c"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment">// 在链表中删除指定值的节点</span></span><br><span class="line"><span class="function"><span class="keyword">void</span> <span class="title">deleteNode</span><span class="params">(struct Node** head, <span class="keyword">int</span> target)</span> </span>{</span><br><span class="line"> <span class="class"><span class="keyword">struct</span> <span class="title">Node</span>* <span class="title">current</span> = *<span class="title">head</span>;</span></span><br><span class="line"></span><br><span class="line"> <span class="comment">// 找到要删除的节点</span></span><br><span class="line"> <span class="keyword">while</span> (current != <span class="literal">NULL</span> && current->data != target) {</span><br><span class="line"> current = current->next;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="comment">// 如果节点存在，则删除它</span></span><br><span class="line"> <span class="keyword">if</span> (current != <span class="literal">NULL</span>) {</span><br><span class="line"> <span class="keyword">if</span> (current->prev != <span class="literal">NULL</span>) {<span class="comment">//头节点不用调整 prev 指向</span></span><br><span class="line"> current->prev->next = current->next;<span class="comment">//前一个节点的 next 指向下一个节点</span></span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">if</span> (current->next != <span class="literal">NULL</span>) {<span class="comment">//尾节点不用调整 next 指向</span></span><br><span class="line"> current->next->prev = current->prev;<span class="comment">//下一个节点的前一个节点指向当前的前一个节点</span></span><br><span class="line"> }</span><br><span class="line"> <span class="built_in">free</span>(current);</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure><p>而平时定义的单向链表</p><figure class="highlight c"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line"><span class="class"><span class="keyword">struct</span> <span class="title">node</span> {</span></span><br><span class="line"><span class="keyword">int</span> data;</span><br><span class="line"><span class="class"><span class="keyword">struct</span> <span class="title">Node</span>* <span class="title">next</span>;</span><span class="comment">//指向下一个节点</span></span><br><span class="line">};</span><br></pre></td></tr></table></figure><p>由它构建的单向链表</p><p><img src="https://images-1307117474.cos.ap-guangzhou.myqcloud.com/blogs/Blog_3/%E5%8D%95%E9%93%BE%E8%A1%A8.png" alt="单链表" /></p><p>而单链表的删除操作是这样的</p><figure class="highlight c"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment">// 在链表中删除指定值的节点</span></span><br><span class="line"><span class="function"><span class="keyword">void</span> <span class="title">deleteNode</span><span class="params">(struct Node** head, <span class="keyword">int</span> target)</span> </span>{</span><br><span class="line"> <span class="class"><span class="keyword">struct</span> <span class="title">Node</span>* <span class="title">current</span> = *<span class="title">head</span>;</span></span><br><span class="line"> <span class="class"><span class="keyword">struct</span> <span class="title">Node</span>* <span class="title">prev</span> = <span class="title">NULL</span>;</span></span><br><span class="line"></span><br><span class="line"> <span class="comment">// 找到要删除的节点</span></span><br><span class="line"> <span class="keyword">while</span> (current != <span class="literal">NULL</span> && current->data != target) {</span><br><span class="line"> prev = current;</span><br><span class="line"> current = current->next;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="comment">// 如果节点存在，则删除它</span></span><br><span class="line"> <span class="keyword">if</span> (current != <span class="literal">NULL</span>) {</span><br><span class="line"> <span class="keyword">if</span> (prev == <span class="literal">NULL</span>) {</span><br><span class="line"> <span class="comment">// 要删除的节点是头节点</span></span><br><span class="line"> *head = current->next;</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> prev->next = current->next;</span><br><span class="line"> }</span><br><span class="line"> <span class="built_in">free</span>(current);</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure><p>于是我又去看了 linux 内核中的链表结构，发现了它的使用</p><p>代码路径：<code>libevent/compat/sys/queue.h</code></p><figure class="highlight c"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br></pre></td><td class="code"><pre><span class="line"><span class="class"><span class="keyword">struct</span> <span class="title">list_head</span> {</span></span><br><span class="line"><span class="class"><span class="keyword">struct</span> <span class="title">list_head</span> *<span class="title">next</span>, *<span class="title">prev</span>;</span></span><br><span class="line">};</span><br><span class="line"></span><br><span class="line"><span class="class"><span class="keyword">struct</span> <span class="title">hlist_head</span> {</span></span><br><span class="line"><span class="class"><span class="keyword">struct</span> <span class="title">hlist_node</span> *<span class="title">first</span>;</span></span><br><span class="line">};</span><br><span class="line"></span><br><span class="line"><span class="class"><span class="keyword">struct</span> <span class="title">hlist_node</span> {</span></span><br><span class="line"><span class="class"><span class="keyword">struct</span> <span class="title">hlist_node</span> *<span class="title">next</span>, **<span class="title">pprev</span>;</span></span><br><span class="line">};</span><br></pre></td></tr></table></figure><p>它在定义双向链表的同时，还定义了一个<code>hlist_node</code>用于 hash 表的桶链表，头节点使用 <code>hlist_head</code>表示。它的删除操作</p><figure class="highlight c"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">static</span> <span class="keyword">inline</span> <span class="keyword">void</span> __hlist_del(struct hlist_node *n)</span><br><span class="line">{</span><br><span class="line"><span class="class"><span class="keyword">struct</span> <span class="title">hlist_node</span> *<span class="title">next</span> = <span class="title">n</span>-><span class="title">next</span>;</span></span><br><span class="line"><span class="class"><span class="keyword">struct</span> <span class="title">hlist_node</span> **<span class="title">pprev</span> = <span class="title">n</span>-><span class="title">pprev</span>;</span></span><br><span class="line"></span><br><span class="line">WRITE_ONCE(*pprev, next);</span><br><span class="line"><span class="keyword">if</span> (next)</span><br><span class="line">WRITE_ONCE(next->pprev, pprev);</span><br><span class="line">}</span><br></pre></td></tr></table></figure><p>其中 <code>WRITE_ONCE</code>的宏是 Linux 内核中用于确保写入操作是原子的宏，也就是说 <code>WRITE_ONCE(*pprev, next);</code> 相当于原子操作 <code>*pprev = next</code>, 那么这个双向链表的结构是</p><p><img src="https://images-1307117474.cos.ap-guangzhou.myqcloud.com/blogs/Blog_3/%E5%8F%8C%E5%90%91%E9%93%BE%E8%A1%A8%202.png" alt="双向链表 2" /></p><p>跟自己定义的区别就是将 <code>prev</code> 的指针指向的是前一个节点的 <code>next</code> 指针，为什么这样做其实从删除操作可以看出来，<strong>不需要额外判断是否是头节点与尾节点</strong></p><p>回到刚才的宏定义，有个双向链表节点，看它是如何完整的实现双向链表的</p><p>首先是初始化一个头部节点</p><figure class="highlight c"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br></pre></td><td class="code"><pre><span class="line"><span class="class"><span class="keyword">struct</span> <span class="title">event_list</span> <span class="title">events</span>;</span></span><br><span class="line">TAILQ_INIT(&events);</span><br><span class="line"><span class="meta">#<span class="meta-keyword">define</span>TAILQ_INIT(head) do {\</span></span><br><span class="line">(head)->tqh_first = <span class="literal">NULL</span>;\</span><br><span class="line">(head)->tqh_last = &(head)->tqh_first;\</span><br><span class="line">} <span class="keyword">while</span> (<span class="comment">/*CONSTCOND*/</span><span class="number">0</span>)</span><br></pre></td></tr></table></figure><p>也就是说 <code>head</code> 是一个包含 <code>tqh_first</code> 与 <code>tqh_last</code> 的节点</p><figure class="highlight c"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br></pre></td><td class="code"><pre><span class="line">TAILQ_HEAD (event_list, event);</span><br><span class="line"></span><br><span class="line"><span class="meta">#<span class="meta-keyword">define</span> TAILQ_HEAD(name, type)\</span></span><br><span class="line"><span class="class"><span class="keyword">struct</span> <span class="title">name</span> {</span>\</span><br><span class="line"><span class="class"><span class="keyword">struct</span> <span class="title">type</span> *<span class="title">tqh_first</span>;</span>\</span><br><span class="line"><span class="class"><span class="keyword">struct</span> <span class="title">type</span> **<span class="title">tqh_last</span>;</span>\</span><br><span class="line">}</span><br><span class="line"><span class="meta">#<span class="meta-keyword">endif</span></span></span><br><span class="line"></span><br><span class="line">TAILQ_ENTRY(event)</span><br><span class="line"><span class="meta">#<span class="meta-keyword">define</span> TAILQ_ENTRY(type)\</span></span><br><span class="line"><span class="class"><span class="keyword">struct</span> {</span>\</span><br><span class="line"><span class="class"><span class="keyword">struct</span> <span class="title">type</span> *<span class="title">tqe_next</span>;</span><span class="comment">/* next element */</span>\</span><br><span class="line"><span class="class"><span class="keyword">struct</span> <span class="title">type</span> **<span class="title">tqe_prev</span>;</span><span class="comment">/* address of previous next element */</span>\</span><br><span class="line">}</span><br></pre></td></tr></table></figure><p><code>TAILQ_HEAD</code> 定义了一个名字为 <code>event_list</code> 的双向链表，记录了第一个节点与最后一个节点</p><p>它的删除操作</p><figure class="highlight c"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br></pre></td><td class="code"><pre><span class="line"><span class="meta">#<span class="meta-keyword">define</span> TAILQ_REMOVE(head, elm, field) do {\</span></span><br><span class="line"><span class="keyword">if</span> (((elm)->field.tqe_next) != <span class="literal">NULL</span>)\</span><br><span class="line">(elm)->field.tqe_next->field.tqe_prev =\</span><br><span class="line"> (elm)->field.tqe_prev;\</span><br><span class="line"><span class="keyword">else</span>\</span><br><span class="line">(head)->tqh_last = (elm)->field.tqe_prev;\</span><br><span class="line">*(elm)->field.tqe_prev = (elm)->field.tqe_next;\</span><br><span class="line">} <span class="keyword">while</span> (<span class="number">0</span>)</span><br></pre></td></tr></table></figure><p>其中 <code>(elm)->field</code> 表示的是 <code>elm 指针</code>的成员字段 <code>field</code></p><p>而 <code>(elm)->field.tqe_next->field.tqe_prev = (elm)->field.tqe_prev;</code> 与上面的 WRITE_ONCE(next->pprev, pprev); 效果一样</p><p>那么 <code>*(elm)->field.tqe_prev = (elm)->field.tqe_next;</code> 理解为 <code>WRITE_ONCE(*pprev, next);</code>，所以这里的 <code>*(elm)</code> 而不是 <code>elm</code></p><p>这里不同的是，头节点记录的链表的头节点与尾节点，所以它的结构是</p><p><img src="https://images-1307117474.cos.ap-guangzhou.myqcloud.com/blogs/Blog_3/%E5%8F%8C%E5%90%91%E9%93%BE%E8%A1%A8%203.png" alt="双向链表 3" /></p><p>这样的好处是 <strong>可以同时在双向链表的头尾都进行增删操作，变成了一个双向队列</strong>，那为什么 linux 中的 hash 表没有弄成一个双向队列，因为新增的值直接加入到链表头，不需要头尾都都操作</p>]]></content>
<summary type="html">
<p>在阅读 Libevent 库的时候看到 <code>TAILQ_ENTRY </code>的宏定义有点特别，这里记录一下，代码路径 <code>libevent/compat/sys/queue.h</code></p>
<p>首先它实现的类似泛型的功能</p>
<figu
</summary>
<category term="Data Structure" scheme="https://www.yuankang.top/categories/Data-Structure/"/>
<category term="Data Structure" scheme="https://www.yuankang.top/tags/Data-Structure/"/>
<category term="List" scheme="https://www.yuankang.top/tags/List/"/>
<category term="Libevent" scheme="https://www.yuankang.top/tags/Libevent/"/>
</entry>
<entry>
<title>Protocol-04-DHCP</title>
<link href="https://www.yuankang.top/2023/09/07/Protocol/Protocol-04-DHCP/"/>
<id>https://www.yuankang.top/2023/09/07/Protocol/Protocol-04-DHCP/</id>
<published>2023-09-07T02:25:38.000Z</published>
<updated>2024-06-14T13:14:43.897Z</updated>
<content type="html"><![CDATA[<p>动态主机配置协议<code>DHCP</code>(Dynamic Host Configuration Protocol) 是一种网络管理协议，用于集中对用户 IP 地址进行动态管理和配置。<br /><strong>DHCP协议</strong>由 <code>RFC 2131</code> 定义，采用 Client /服务器通信模式，由 <strong>客户端(DHCP Client)</strong> 向 **服务器(DHCP Server)**提出配置申请，<strong>Server</strong>为网络上的每个设备动态分配 <strong>IP 地址</strong>、<strong>子网掩码</strong>、<strong>默认网关地址</strong>，<strong>域名服务器(DNS)地址</strong>等参数</p><p>相关协议(按时间从远到近排序) <a href="http://www.ietf.org/rfc/rfc1531.txt" target="_blank" rel="noopener">RFC1531</a> 、<a href="http://www.ietf.org/rfc/rfc1541.txt" target="_blank" rel="noopener">RFC1541</a>、<a href="http://www.ietf.org/rfc/rfc2131.txt" target="_blank" rel="noopener">RFC2131</a> 、<a href="http://www.ietf.org/rfc/rfc3396.txt" target="_blank" rel="noopener">RFC3396</a></p><h3 id="dhcp好处"><a class="markdownIt-Anchor" href="#dhcp好处"></a> DHCP好处</h3><p>网络中，每个连接Internet的设备都需要分配唯一的 IP。DHCP 使网络管理员能<strong>从中心结点监控和分配IP地址</strong>，从以下几个方面降低了配置和部署设备的时间</p><ul><li>减少 IP 地址冲突：每个连接的设备都必须有一个IP地址。但是，每个地址只能使用一次，重复的地址将导致无法连接一个或两个设备的冲突。</li><li>IP地址管理的自动化：如果没有 DHCP，网络管理员将需要手动分配和撤消地址。手动需要随时知道设备何时需要访问网络以及何时需要离开网络。</li><li>高效的变更管理：<strong>DHCP 的使用使更改地址，范围或端点变得非常简单</strong>。例如，组织可能希望将其IP寻址方案从一个范围更改为另一个范围。Server配置有新信息，该信息将传播到新端点。同样，如果升级并更换了网络设备，则不需要网络配置</li></ul><h3 id="dhcp-工作原理"><a class="markdownIt-Anchor" href="#dhcp-工作原理"></a> DHCP 工作原理</h3><p>DHCP 协议采用 <strong>UDP</strong> 作为传输协议，<strong>Client</strong> 发送请求消息到 <strong>Server</strong> 的 <strong>Port 67</strong>，<strong>Server</strong> 回应应答消息给 <strong>Client</strong> 的 <strong>Port 68</strong>，只有跟 Client 在同一个网段的 Server 才能收到 Client 广播的 <strong>DISCOVER报文</strong>。当 Client 与 Server 不在同一个网段时，必须部署DHCP中继来转发 Client 和Server之间的DHCP报文</p><h4 id="非中继"><a class="markdownIt-Anchor" href="#非中继"></a> 非中继</h4><p>在没有部署 <strong>DHCP中继</strong> 的场景下，首次接入网络 Client 与 Server 的报文交互过程，该过程称为<strong>DHCP报文四步交互</strong></p><p><img src="https://images-1307117474.cos.ap-guangzhou.myqcloud.com/blogs/Blog_3/download.png" alt="无中继场景时 Client 首次接入网络的报文交互示意图" /></p><p><strong>第一步：发现阶段</strong></p><p>首次接入网络的 Client 不知道 Server 的IP地址，为了学习到 Server 的IP地址，Client 以广播方式发送<strong>DISCOVER报文</strong>(目的IP地址为255.255.255.255)给同一网段内的所有设备(包括Server或中继)。<strong>DISCOVER报文</strong>中携带了 Client 的MAC地址等信息</p><p><strong>第二步：提供阶段</strong></p><p>与 Client 位于同一网段的 Server 都会接收到 <strong>DISCOVER报文</strong>，Server 选择跟接收 <strong>DISCOVER报文</strong>接口的IP地址处于同一网段的地址池，并且从中选择一个可用的IP地址，然后通过<strong>OFFER报文</strong>发送给Client，Server 在地址池中为 Client 分配IP地址的顺序如下：</p><ol><li>Server 上已配置的与 Client MAC地址静态绑定的 IP 地址。</li><li>Client 发送的 <strong>DISCOVER报文</strong>中 <code>Option50</code>(请求IP地址选项)指定的地址</li><li>地址池内查找 <code>Expired</code> 状态的IP地址，即曾经分配给 Client 的超过租期的IP地址</li><li>在地址池内随机查找一个 <code>Idle</code> 状态的IP地址。</li><li>如果未找到可供分配的 IP 地址，则地址池依次自动回收超过租期的 <code>Expired</code> 和处于冲突状态 <code>Conflict</code> 的IP地址。回收后如果找到可用的IP地址，则进行分配；否则，Client 等待应答超时后，重新发送<strong>DHCP DISCOVER报文</strong>来申请IP地址</li></ol><p>通常，<strong>Server</strong> 的地址池中会指定IP地址的租期，如果 <strong>Client</strong> 发送的 <strong>DISCOVER报文</strong> 中携带了期望租期，服务器会将 Client 请求的<strong>期望租期</strong>与其<strong>指定的租期</strong>进行比较，选择其中时间<strong>较短的租期</strong></p><p>为了防止分配出去的IP地址跟网络中其他 Client 的IP地址冲突，比如Client所在网段已经手工配置了一个地址池中的地址用于DNS服务器，Server 在发送<strong>DHCP OFFER报文</strong>前通过发送 Source IP = ServerIP地址、Target IP 为预分配出去IP地址的 <strong>ICMP ECHO REQUEST报文</strong> 对分配的IP地址进行地址冲突探测。</p><ul><li>如果在指定的时间内没有收到应答报文，表示网络中没有 Client 使用这个IP地址，可以分配给 Client ；</li><li>如果指定时间内收到应答报文，表示网络中已经存在使用此IP地址的 Client ，则把此地址列为冲突地址，然后等待重新接收到DHCP DISCOVER报文后重新选择（TODO 这个时间会不会很长，为什么不直接找一个IP地址分配）</li></ul><p>此阶段 Server 分配给 Client 的IP地址不一定是最终确定使用的IP地址，因为<strong>DHCP OFFER报文</strong>发送给 Client 等待<strong>16秒</strong> 后如果没有收到 Client 的响应，此地址就可以继续分配给其他 Client 。</p><p><strong>第三步：选择阶段</strong></p><p>在发现阶段 <code>DISCOVER报文</code> 是广播报文，也就是说有可能有多个 Server 向 Client 回应 <strong>DHCP OFFER报文</strong>，则 Client 一般只接收第一个收到的 DHCP OFFER报文，然后以<strong>广播方式</strong>发送 DHCP REQUEST报文，该报文中包含 Client 想选择的 Server标识符(即Option50，填充了接收的DHCP OFFER报文中<strong>yiaddr字段</strong>的IP地址)</p><p>Client 广播发送 DHCP REQUEST报文通知所有的Server，它将选择某个Server提供的IP地址，其他Server可以重新将曾经分配给 Client 的IP地址分配给其他 Client 。</p><blockquote><p>Server怎么判断这个报文是来请求自己分配出去的IP，还是 Client 用来告诉 Server 不用这个地址（两个DHCP 服务器分配的IP地址可能还都一样，不能通过IP判断是否是自己给的）？</p></blockquote><p><strong>第四步：确认阶段</strong></p><p>当Server收到 Client 发送的<strong>DHCP REQUEST报文</strong>后，Server回应 <strong>DHCP ACK报文</strong>，表示 <strong>DHCP REQUEST报文</strong> 中请求的IP地址(Option50 填充的)分配给 Client 使用。</p><ol><li><p>Client 收到 <strong>DHCP ACK报文</strong>，会广播发送免费<strong>ARP报文</strong>，探测本网段是否有其他终端使用服务器分配的IP地址</p><ul><li><p>如果在指定时间内没有收到回应，表示 Client 可以使用此地址。</p></li><li><p>如果收到了回应，说明有其他终端使用了此地址， Client 会向服务器发送 <strong>DHCP DECLINE报文</strong>，并重新向服务器请求IP地址，同时，服务器会将此地址列为<strong>冲突地址</strong>。当服务器没有空闲地址可分配时，再选择冲突地址进行分配，尽量减少分配出去的地址冲突</p><blockquote><p>DECLINE 是广播报文吗， 如果发送失败或者 Server无法分配其他地址呢？</p></blockquote></li></ul></li><li><p>当 Server 收到 Client 发送的 <strong>DHCP REQUEST报文</strong> 后，如果Client 由于某些原因（例如协商出错或者由于发送REQUEST过慢导致服务器已经把此地址分配给其他 Client )无法分配 <strong>DHCP REQUEST报文</strong>中 <code>Option50</code> 填充的IP地址，则发送<strong>DHCP NAK报文</strong>作为应答，通知 Client 无法分配此IP地址。** Client 需要重新发送DHCP DISCOVER报文来申请新的IP地址**</p></li></ol><h4 id="中继"><a class="markdownIt-Anchor" href="#中继"></a> 中继</h4><p>有DHCP中继的场景中，首次接入网络的 Client 和Server的工作原理与<strong>无中继场景时 Client 首次接入网络的工作原理</strong>相同。主要差异是DHCP中继在Server和 Client 之间转发DHCP报文，以保证Server和 Client 可以正常交互。在 Client 看来，DHCP中继就像Server；在Server看来，DHCP中继就像 Client</p><p>如下图所示，在部署DHCP中继的场景下，首次接入网络 Client 与Server的报文交互过程</p><p><img src="https://images-1307117474.cos.ap-guangzhou.myqcloud.com/blogs/Blog_3/download-20230907220000104.png" alt="有中继场景时 Client 首次接入网络的报文交互示意图" /></p><p><strong>第一步：发现阶段</strong></p><p>DHCP中继接收到 Client 广播发送的 DHCP DISCOVER报文后，进行如下处理：</p><ol><li><p>检查DHCP报文中的<code>hops</code>字段，<strong>如果大于16，则丢弃DHCP报</strong>文；否则，将<code>hops</code>字段加1(表明经过一次DHCP中继)，并继续下面的操作。</p></li><li><p>检查DHCP报文中的<code>giaddr</code>字段。如果是0，将<code>giaddr</code>字段设置为接收DHCP DISCOVER报文的接口IP地址。如果不是0，则不修改该字段，继续下面的操作。</p><blockquote><p>记录 <code>giaddr</code> 的目的就是在 DHCP 地址池中分配相同网段的 IP 地址</p></blockquote></li><li><p>将DHCP报文的目的IP地址改为<strong>Server或下一跳中继的IP地址</strong>，<strong>源地址改为中继连接 Client 的接口地址</strong>，通过路由转发将DHCP报文<strong>单播</strong>发送到Server或下一跳中继</p></li></ol><p>如果 Client 与Server之间存在多个DHCP中继，后面的中继接收到DHCP DISCOVER报文的处理流程同前面所述</p><p><strong>第二步：提供阶段</strong></p><p>Server接收到DHCP DISCOVER报文后，<strong>选择与报文中<code>giaddr</code>字段为同一网段的地址池</strong>，并为 Client 分配IP地址等参数，然后向<code>giaddr</code>字段标识的DHCP中继单播发送DHCP OFFER报文。</p><p>DHCP中继收到DHCP OFFER报文后，会进行如下处理：</p><ol><li><p>检查报文中的<code>giaddr</code>字段，<strong>如果不是接口的地址，则丢弃该报文</strong>；否则，继续下面的操作。</p><blockquote><p>DHCP 服务器怎么知道是不是中继过来的？通过 hops 吗？</p></blockquote></li><li><p>DHCP中继检查报文的<strong>广播标志位</strong>。如果广播标志位为1，则将DHCP OFFER报文广播发送给 Client ；否则将DHCP OFFER报文单播发送给 Client 。</p><blockquote><p>广播标志位是什么时候设置的？</p><p>为什么 DHCP OFFER 需要官博发送给 DHCP Client</p></blockquote></li></ol><p><strong>第三步：选择阶段</strong></p><p>中继接收到来自 Client 的DHCP REQUEST报文的处理过程同无中继场景下的选择阶段。</p><p><strong>第四步：确认阶段</strong></p><p>中继接收到来自服务器的DHCP ACK报文的处理过程同无中继场景下的确认阶段。</p><h4 id="续租"><a class="markdownIt-Anchor" href="#续租"></a> 续租</h4><p>DHCP提供了两种地址分配机制，可以根据网络需求为不同的主机选择不同的分配策略。</p><ul><li>动态分配机制：通过DHCP为主机分配一个有使用期限的IP地址</li><li>静态分配机制：网络管理员通过DHCP为指定的主机分配固定的IP地址。</li></ul><p>Client 向 Server 申请地址时可以携带期望租期，将分配租期与期望租期比较，**分配其中一个较短的租期给 Client **。租期到期或者 Client 下线释放地址后，服务器会收回该IP地址，收回的IP地址可以继续分配给其他 Client 使用。这种机制可以提高IP地址的利用率，避免 Client 下线后IP地址继续被占用。如果 Client 希望继续使用该地址，需要更新IP地址的租期(如延长IP地址租期)，更新租期过程如下</p><p><img src="https://images-1307117474.cos.ap-guangzhou.myqcloud.com/blogs/Blog_3/download-20230907220722271.png" alt=" Client 更新租期示意图" /></p><ol><li>当租期达到<strong>50%(T1)<strong>时，Client 会自动以</strong>单播</strong>的方式向 Server 发送 <strong>DHCP REQUEST报文</strong>，请求更新IP地址租期<ul><li>如果收到 Server 回应的 <strong>DHCP ACK报文</strong>，则租期更新成功(即租期从0开始计算)；</li><li>如果收到 <strong>DHCP NAK报文</strong>，则重新发送 <strong>DHCP DISCOVER报文</strong>请求新的IP地址。</li></ul></li><li>当租期达到<strong>87.5%(T2)<strong>时，如果仍未收到Server的应答，Client 会自动以</strong>广播</strong>的方式向 Server 发送 <strong>DHCP REQUEST报文</strong>，请求更新IP地址租期。后续处理逻辑同上</li><li>如果租期时间到时都没有收到服务器的回应， Client 停止使用此IP地址，重新发送DHCP DISCOVER报文请求新的IP地址。</li></ol><p>Client 在租期时间到之前，如果不想使用分配的IP地址(例如 Client 网络位置需要变更)，会触发 Client 向 Server 发送 <strong>DHCP RELEASE报文</strong>，通知 Server 释放IP地址的租期。Server 会保留这个 Client 的配置信息，将IP地址列为曾经分配过的IP地址中，以便后续重新分配给该 Client 或其他 Client 。 Client 可以通过发送 <strong>DHCP INFORM报文</strong>向服务器请求更新配置信息</p><p>部署DHCP中继时，更新租期的过程与上述过程相似。</p><p><img src="https://images-1307117474.cos.ap-guangzhou.myqcloud.com/blogs/Blog_3/download-20230907220805159.png" alt=" Client 通过DHCP中继更新租期示意图" /></p><p>**如果不是首次呢，比如客户端重启了 ？**如果客户端重启后，会自动发送 <strong>DHCP REQUEST广播</strong> 给 Server 以便请求继续租用原来使用的IP地址，</p><ul><li>如果服务器收到这个消息，确认客户端可以使用该地址，则回答一个<strong>DHCP ACK</strong>确认消息</li><li>如果此IP地址已经无法再分配，则返回一个 <strong>DHCP NCK</strong>否认信息，客户端收到这个信息以后，则必须重新发送 DHCP Discovery消息获取新的地址(启动首次登录的流程)。 <strong>若没有得到响应，则尝试与网关通信，如果通信正常，这个租期还没到期的话，则可以继续使用，但是不能和网关通信的话，则会获得169.254.0.1~169.254.255.254之间的IP地址，然后每隔5min尝试更新租约</strong></li></ul><h3 id="dhcp-报文"><a class="markdownIt-Anchor" href="#dhcp-报文"></a> DHCP 报文</h3><p>Server 与 Client 之间通过DHCP报文进行通信。DHCP报文是基于UDP协议传输的。Client 使用的端口号为68，Server 使用的端口号为67。目前DHCP定义了如下八种类型报文，<strong>每种报文的格式相同，只是某些字段的取值不同</strong></p><table><thead><tr><th>报文名称</th><th>说明</th></tr></thead><tbody><tr><td>DHCP DISCOVER</td><td>Client 首次登录网络时进行DHCP交互过程发送的第一个报文，用来寻找Server。</td></tr><tr><td>DHCP OFFER</td><td>Server用来响应<strong>DHCP DISCOVER报文</strong>，此报文携带了各种配置信息。</td></tr><tr><td>DHCP REQUEST</td><td>此报文用于以下三种用途。<br />1. Client 初始化后，发送广播的 <strong>DHCP REQUEST报文</strong>来回应服务器的<strong>DHCP OFFER报文</strong>。<br />2. Client 重启后，发送广播的 <strong>DHCP REQUEST报文</strong>来确认先前被分配的IP地址等配置信息。<br />3. 当 Client 已经和某个IP地址绑定后，发送 <strong>DHCP REQUEST单播或广播报文</strong>来更新IP地址的租约。</td></tr><tr><td>DHCP ACK</td><td>Server 对 Client 的<strong>DHCP REQUEST报文</strong>的确认响应报文，Client收到此报文后，才真正获得了IP地址和相关的配置信息。</td></tr><tr><td>DHCP NAK</td><td>服务器对Client 的<strong>DHCP REQUEST报文</strong>的拒绝响应报文，例如Server收到DHCP REQUEST报文后，没有找到相应的租约记录，则发送DHCP NAK 报文作为应答，告知 Client 无法分配合适IP地址。</td></tr><tr><td>DHCP DECLINE</td><td>当 Client 发现服务器分配给它的IP地址发生冲突时会通过发送此报文来通知服务器，并且会重新向服务器申请地址。</td></tr><tr><td>DHCP RELEASE</td><td>Client 可通过发送此报文主动释放服务器分配给它的IP地址，当服务器收到此报文后，可将这个IP地址分配给其它的 Client 。</td></tr><tr><td>DHCP INFORM</td><td>Client 获取IP地址后，如果需要向Server获取更为详细的配置信息(网关地址、DNS服务器地址)，则向Server发送DHCP INFORM请求报文。</td></tr></tbody></table><p>报文地址：<a href="https://wiki.wireshark.org/uploads/__moin_import__/attachments/Development/PcapNg/dhcp.pcapng" target="_blank" rel="noopener">https://wiki.wireshark.org/uploads/__moin_import__/attachments/Development/PcapNg/dhcp.pcapng</a></p><p>DHCP 四步交互如下</p><img src="https://images-1307117474.cos.ap-guangzhou.myqcloud.com/blogs/Blog_3/image-20230907230303654.png" alt="image-20230907230303654" style="zoom:80%;" /><p>服务端的端口是 67，客户端的端口是 68</p><p>这里有一个疑问就是 192.168.0.10 是从哪里来的，DHCP 服务器分配之后直接写进去了</p><h4 id="discover"><a class="markdownIt-Anchor" href="#discover"></a> Discover</h4><img src="https://images-1307117474.cos.ap-guangzhou.myqcloud.com/blogs/Blog_3/image-20230907230549875.png" alt="image-20230907230549875" style="zoom:50%;" /><p>DHCP 报文格式一样，只是最后的 Option 不同，前面包含的字段如下：</p><ul><li><strong>Message type</strong>：报文的操作类型。分为请求报文和响应报文，1为请求报文，2为响应报文。具体报文类型在option字段中标识</li><li><strong>Hardware type</strong>：DHCP客户端的硬件地址类型。1表示ethernet地址</li><li><strong>Hardeare address length</strong>：DHCP客户端的硬件地址长度。Ethernet地址为6</li><li><strong>Hops</strong>：DHCP报文经过的DHCP中继的数目。初始为0，报文每经过一个DHCP中继，该字段就会增加1</li><li><strong>Transaction ID</strong>: 客户端发起一次请求时选择的随机数，用来标识一次地址请求过程</li><li><strong>Seconds elapsed</strong>: Client 开始DHCP请求后所经过的时间。目前尚未使用，固定取0</li><li><strong>Bootp flags</strong>：DHCP服务器响应报文是采用单播还是广播方式发送。<strong>只使用第0比特位，0表示采用单播方式，1表示采用广播方式</strong>，其余比特保留不用</li><li><strong>Client IP address</strong>: DHCP客户端的IP地址</li><li><strong>Your(client) IP address</strong>: DHCP服务器分配给客户端的IP地址</li><li><strong>Next server IP address</strong>：DHCP客户端获取IP地址等信息的服务器IP地址</li><li><strong>Relay agent IP address</strong>: DHCP客户端发出请求报文后经过的第一个DHCP中继的IP地址</li><li><strong>Client MAC address</strong>: DHCP客户端的硬件地址</li><li><strong>Server host name</strong>: DHCP客户端获取IP地址等信息的服务器名称</li><li><strong>Boot file name</strong>: DHCP服务器为DHCP客户端指定的启动配置文件名称及路径信息</li><li><strong>Magic cookie</strong>: 标识和解释DHCP消息的元素，有助于确保不同设备之间的协议兼容性，并提供了消息类型和选项格式的关键信息</li><li><strong>Option</strong>：可选变长选项字段，包含报文的类型、有效租期、DNS服务器的IP地址和WINS服务器的IP地址等配置信息。</li></ul><p>因为是广播报文，所以目的地址是 <code>255.255.255.255</code>，其中</p><ul><li>Option(53): 表示 DHCP 消息类型，Discover(1)</li><li>Option(50): 用来记录请求的 IP</li><li>Option(55): 用来请求其他参数</li><li>Option(61): 客户端标识符</li></ul><h4 id="offer"><a class="markdownIt-Anchor" href="#offer"></a> Offer</h4><img src="https://images-1307117474.cos.ap-guangzhou.myqcloud.com/blogs/Blog_3/image-20230907230844722.png" alt="image-20230907230844722" style="zoom:50%;" /><p>这里也包含了很多 Option:</p><ul><li>Option(53): DHCP 消息类型，Offer(2)</li><li>Option(1): 子网掩码， <code>255.255.255.0</code></li><li>Option(58): 续租时间 T1，一般为租期的 50%</li><li>Option(59): 重新发起T2, 租约响应超时，重新发起广播时间，一般位租期的 87.5%</li><li>Option(51): 租约</li><li>Option(54): DHCP服务器标识符选项</li></ul><h4 id="request"><a class="markdownIt-Anchor" href="#request"></a> Request</h4><img src="https://images-1307117474.cos.ap-guangzhou.myqcloud.com/blogs/Blog_3/image-20230907231535947.png" alt="image-20230907231535947" style="zoom:50%;" /><p>这里也包含了很多 Option:</p><ul><li>Option(53): DHCP 消息类型，Request(3)</li><li>Option(61): 客户段标识</li><li>Option(50): 请求的 IP</li><li>Option(54): 服务端标识</li><li>Option(55): 请求的参数</li></ul><h4 id="ack"><a class="markdownIt-Anchor" href="#ack"></a> ACK</h4><img src="https://images-1307117474.cos.ap-guangzhou.myqcloud.com/blogs/Blog_3/image-20230907231557794.png" alt="image-20230907231557794" style="zoom:50%;" /><p>这里也包含了很多 Option:</p><ul><li>Option(53): DHCP 消息类型，Ack(5)</li><li>Option(1): 子网掩码， <code>255.255.255.0</code></li><li>Option(58): 续租时间 T1，一般为租期的 50%</li><li>Option(59): 重新发起T2, 租约响应超时，重新发起广播时间，一般位租期的 87.5%</li><li>Option(51): 租约</li><li>Option(54): DHCP服务器标识符选项</li></ul><h3 id="dhcp-snooping"><a class="markdownIt-Anchor" href="#dhcp-snooping"></a> DHCP Snooping</h3><p>DHCP Snooping 是DHCP的一种安全特性，用于保证 DHCP客户端从合法的 DHCP服务器获取IP地址，并记录DHCP客户端IP地址与MAC地址等参数的对应关系。<code>DHCP Snooping</code> 用来抵御网络中针对DHCP的各种攻击</p><blockquote><ol><li>DHCP攻击有哪些</li><li>它的原理是什么</li></ol></blockquote><h4 id="dhcp-攻击"><a class="markdownIt-Anchor" href="#dhcp-攻击"></a> DHCP 攻击</h4><h5 id="1-dhcp-server仿冒者攻击"><a class="markdownIt-Anchor" href="#1-dhcp-server仿冒者攻击"></a> 1. DHCP Server仿冒者攻击</h5><p>由于 Server 和 Client 之间没有认证机制，而 DHCP Discover报文 是以<strong>广播</strong>形式发送，如果在网络上随意添加一台 DHCP服务器，它就可以为 Client 分配IP地址以及其他网络参数。</p><p><img src="https://images-1307117474.cos.ap-guangzhou.myqcloud.com/blogs/Blog_3/download-20230908224500248.png" alt="DHCP Client发送DHCP Discover报文示意图" /></p><p>如果此时 DHCP Server仿冒者 回应给 Client 仿冒信息，如错误的网关地址、错误的DNS(Domain Name System)服务器、错误的IP等信息。Client 将无法获取正确的IP地址和相关信息，导致合法客户无法正常访问网络或信息安全受到严重威胁。</p><p><img src="https://images-1307117474.cos.ap-guangzhou.myqcloud.com/blogs/Blog_3/download.png" alt="img" /></p><p>解决办法：为了防止DHCP Server仿冒者攻击，可配置设备接口的 <code>信任(Trusted)/非信任(Untrusted)</code>工作模式</p><p>将与合法DHCP服务器直接或间接连接的接口设置为信任接口，其他接口设置为非信任接口。此后，从<code>非信任(Untrusted)</code>接口上收到的DHCP回应报文将被直接丢弃，这样可以有效防止DHCP Server仿冒者的攻击</p><p><img src="https://images-1307117474.cos.ap-guangzhou.myqcloud.com/blogs/Blog_3/download-20230908224631147.png" alt="img" /></p><h5 id="2-防止非dhcp用户攻击"><a class="markdownIt-Anchor" href="#2-防止非dhcp用户攻击"></a> 2. 防止非DHCP用户攻击</h5><p>在DHCP网络中，静态获取IP地址的用户（非DHCP用户）对网络可能存在多种攻击，譬如仿冒DHCP Server、构造虚假DHCP Request报文等。这将为合法DHCP用户正常使用网络带来了一定的安全隐患</p><p>解决办法：开启设备根据 <strong>DHCP Snooping绑定表</strong> 生成接口的静态MAC表项功能，设备将根据接口下所有的DHCP用户对应的DHCP Snooping绑定表项自动执行命令生成这些用户的静态MAC表项，并同时关闭接口学习动态MAC表项的能力。此时，只有源MAC与静态MAC表项匹配的报文才能够通过该接口，否则报文会被丢弃。因此对于该接口下的非DHCP用户，只有管理员手动配置了此类用户的静态MAC表项其报文才能通过，否则报文将被丢弃。</p><p><strong>动态MAC表项是设备自动学习并生成的，静态MAC表项则是根据命令配置而成的</strong>。MAC表项中包含用户的MAC、所属VLAN、连接的接口号等信息，设备可根据MAC表项对报文进行二层转发</p><h5 id="3-防止仿冒dhcp报文攻击"><a class="markdownIt-Anchor" href="#3-防止仿冒dhcp报文攻击"></a> 3. 防止仿冒DHCP报文攻击</h5><p>已获取到IP地址的合法用户通过向服务器发送 DHCP Request 或 DHCP Release 报文用以续租或释放IP地址。</p><ul><li>如果攻击者冒充合法用户不断向DHCP Server发送DHCP Request报文来续租IP地址，会导致这些到期的IP地址无法正常回收，以致一些合法用户不能获得IP地址；</li><li>而若攻击者仿冒合法用户的DHCP Release报文发往DHCP Server，将会导致用户异常下线</li></ul><p>解决办法：利用DHCP Snooping绑定表的功能。设备通过将DHCP Request续租报文和DHCP Release报文与绑定表进行匹配操作能够有效的判别报文是否合法(主要是检查报文中的VLAN、IP、MAC、接口信息是否匹配动态绑定表)，若匹配成功则转发该报文，匹配不成功则丢弃</p><h5 id="4-防止dhcp-server服务拒绝攻击"><a class="markdownIt-Anchor" href="#4-防止dhcp-server服务拒绝攻击"></a> 4. 防止DHCP Server服务拒绝攻击</h5><p>若设备接口if1 下存在大量攻击者恶意申请IP地址，会导致DHCP Server中IP地址快速耗尽而不能为其他合法用户提供IP地址分配服务。</p><p>另一方面，DHCP Server通常仅根据DHCP Request报文中的(Client Hardware Address)字段来确认客户端的MAC地址。如果某一攻击者通过不断改变<strong>CHADDR字段</strong>向DHCP Server申请IP地址，同样将会导致DHCP Server上的地址池被耗尽，从而无法为其他正常用户提供IP地址</p><p><img src="https://images-1307117474.cos.ap-guangzhou.myqcloud.com/blogs/Blog_3/download-20230908230901387.png" alt="img" /></p><p>解决办法：</p><ul><li><p>为了抑制大量 Client 恶意申请IP地址，在使能设备的DHCP Snooping功能后，可配置设备或接口允许接入的最大DHCP用户数，当接入的用户数达到该值时，则不再允许任何用户通过此设备或接口成功申请到IP地址。</p></li><li><p>通过改变DHCP Request报文中的CHADDR字段方式的攻击，可使能设备检测DHCP Request报文帧头MAC与DHCP数据区中<code>CHADDR</code>字段是否一致功能，此后设备将检查上送的DHCP Request报文中的帧头MAC地址是否与<strong>CHADDR值</strong>相等</p></li></ul><h5 id="5-ldra功能感知用户位置信息"><a class="markdownIt-Anchor" href="#5-ldra功能感知用户位置信息"></a> 5. LDRA功能感知用户位置信息</h5><p><strong>LDRA</strong>称为轻量级DHCPv6中继代理，该<strong>中继代理能够记录用户位置信息</strong>并将其发送到 DHCPv6 Server，从而使得DHCPv6 Server能够获取到用户详细的物理位置信息，以实现对用户客户端部署诸如地址分配、计费、接入控制等策略</p><p><img src="https://images-1307117474.cos.ap-guangzhou.myqcloud.com/blogs/Blog_3/download-20230908231042126.png" alt="LDRA应用组网图" /></p><p>解决办法：在使能Switch的DHCP Snooping功能之后，可使能其LDRA功能。Switch既能够获取用户详细的位置信息并将其发送到DHCPv6 Server。DHCPv6 Server即可根据用户的详细位置信息为其部署地址分配策略或其他安全策略</p><p>LDRA功能仅是记录了DHCPv6用户的详细位置信息并通过 RELAY-FORW报文 将该信息发送给DHCPv6 Server，对不同的用户部署诸如地址分配、计费、接入控制等策略，由DHCPv6 Server实现</p><h4 id="实现原理"><a class="markdownIt-Anchor" href="#实现原理"></a> 实现原理</h4><p>DHCP Snooping分为DHCPv4 Snooping和DHCPv6 Snooping，两者实现原理相似。开启 DHCP Snooping 的接口会根据 DHCP 请求与响应生成 DHCP Snoop 绑定表，后续设备再从盖接口接受用户发送的 DHCP 报文时，会进行匹配检查</p><p><img src="https://images-1307117474.cos.ap-guangzhou.myqcloud.com/blogs/Blog_3/download-20230908231452273.png" alt="img" /></p><p>如下图所示的DHCP场景中，连接在二层接入设备上的PC配置为自动获取IP地址。PC作为DHCP客户端通过广播形式发送DHCP请求报文，使能了DHCP Snooping功能的二层接入设备将其通过信任接口转发给DHCP服务器。最后DHCP服务器将含有IP地址信息的DHCP ACK报文通过单播的方式发送给PC。在这个过程中，二层接入设备收到DHCP ACK报文后，会从该报文中提取关键信息（包括PC的MAC地址以及获取到的IP地址、地址租期），并获取与PC连接的使能了DHCP Snooping功能的接口信息（包括接口编号及该接口所属的VLAN，根据这些信息生成DHCP Snooping绑定表。</p><p><img src="https://images-1307117474.cos.ap-guangzhou.myqcloud.com/blogs/Blog_3/download-20230908231606817.png" alt="img" /></p><p><strong>DHCP Snooping绑定表 根据DHCP租期进行老化或根据用户释放IP地址时发出的DHCP Release报文自动删除对应表项</strong></p><p>由于DHCP Snooping绑定表记录了DHCP客户端IP地址与MAC地址等参数的对应关系，故通过对报文与DHCP Snooping绑定表进行匹配检查，能够有效防范非法用户的攻击。</p><p>为了保证设备在生成DHCP Snooping绑定表时能够获取到用户MAC等参数，DHCP Snooping功能<strong>需应用于二层网络中的接入设备或第一个DHCP Relay上</strong>。</p><p>在DHCP中继使能DHCP Snooping场景中，<strong>DHCP Relay设备不需要设置信任接口</strong>。因为DHCP Relay收到DHCP请求报文后进行源目的IP、MAC转换处理，然后以单播形式发送给指定的合法DHCP服务器，所以DHCP Relay收到的DHCP ACK报文都是合法的，生成的DHCP Snooping绑定表也是正确的</p><p><strong>总结来说，是有一个 DHCP Snooping 绑定表用来记录以及校验 DHCP 报文</strong>，另外还有校验 二层 MAC 地址与 DHCP 的 <code>Client mac address</code> 的功能</p><h3 id="dhcpv6"><a class="markdownIt-Anchor" href="#dhcpv6"></a> DHCPv6</h3><p><code>546</code>号端口用于 <code>DHCPv6 Client</code>，而不用于DHCPv4，是为<code>DHCP failover</code>服务，这是需要特别开启的服务，<code>DHCP failover</code>是用来做<code>双机热备</code>的</p><h3 id="参考链接"><a class="markdownIt-Anchor" href="#参考链接"></a> 参考链接</h3><ol><li><a href="https://info.support.huawei.com/info-finder/encyclopedia/zh/DHCP.html" target="_blank" rel="noopener">https://info.support.huawei.com/info-finder/encyclopedia/zh/DHCP.html</a></li><li><a href="https://support.huawei.com/hedex/hdx.do?docid=EDOC1100087046&id=ZH-CN_CONCEPT_0176371515" target="_blank" rel="noopener">https://support.huawei.com/hedex/hdx.do?docid=EDOC1100087046&id=ZH-CN_CONCEPT_0176371515</a></li><li><a href="https://support.huawei.com/hedex/hdx.do?docid=EDOC1100087046&id=ZH-CN_CONCEPT_0176371535" target="_blank" rel="noopener">https://support.huawei.com/hedex/hdx.do?docid=EDOC1100087046&id=ZH-CN_CONCEPT_0176371535</a></li><li><a href="https://info.support.huawei.com/info-finder/encyclopedia/zh/ICMP.html" target="_blank" rel="noopener">https://info.support.huawei.com/info-finder/encyclopedia/zh/ICMP.html</a></li><li><a href="https://wiki.wireshark.org/DHCP.md" target="_blank" rel="noopener">https://wiki.wireshark.org/DHCP.md</a></li><li><a href="https://info.support.huawei.com/info-finder/encyclopedia/zh/DHCP+Snooping.html" target="_blank" rel="noopener">https://info.support.huawei.com/info-finder/encyclopedia/zh/DHCP+Snooping.html</a></li></ol>]]></content>
<summary type="html">
<p>动态主机配置协议<code>DHCP</code>(Dynamic Host Configuration Protocol) 是一种网络管理协议，用于集中对用户 IP 地址进行动态管理和配置。<br />
<strong>DHCP协议</strong>由 <code>RFC
</summary>
<category term="Protocol" scheme="https://www.yuankang.top/categories/Protocol/"/>
<category term="Protocol" scheme="https://www.yuankang.top/tags/Protocol/"/>
<category term="DHCP" scheme="https://www.yuankang.top/tags/DHCP/"/>
</entry>
<entry>
<title>Go-27-SyncPool</title>
<link href="https://www.yuankang.top/2023/07/15/Go/Go-27-SyncPool/"/>
<id>https://www.yuankang.top/2023/07/15/Go/Go-27-SyncPool/</id>
<published>2023-07-15T05:32:45.000Z</published>
<updated>2023-07-15T05:38:19.201Z</updated>
<content type="html"><![CDATA[<p>前面一篇讲解了<strong>Sync.Pool</strong>的底层数据结构 <a href="https://www.yuankang.top/2023/07/01/Go/Go%E5%85%A5%E9%97%A825-PoolDequeue/">poolDequeue</a>，接着看看<code>Sync.Pool</code>的具体实现原理。如果想看看 <code>Sync.Pool</code> 的使用 可以看看我的 <a href="https://www.yuankang.top/2023/07/13/Go/Go%E5%85%A5%E9%97%A826-Issue/">Go 入门 26-Issue</a>，使用的时候最好分配固定大小的对象否则注意清理</p><p>带着问题看世界</p><ol><li><code>Sync.Pool</code> 与 <code>Goroutine</code> 的关系</li><li><code>Sync.Pool</code> 是如何释放的(并没有主动释放接口)</li></ol><p>代码：<code>src/sync/pool.go</code>，1.20版本</p><p>首先来看内存池的说明，翻译过来就是就是：</p><ol><li>存放在<code>Pool</code> 中的元素任何时候都有可能在没有被其他引用的情况下释放掉</li><li><code>Pool</code> 是并发安全的</li><li>使用 <code>Pool</code> 之后不能再复制它。<em>假设缓存池对象 A 被对象 B 拷贝了，如果 A 被清空，B 的缓存对象指针指向的对象将会不可控</em></li></ol><p>先来看看整体的结构图</p><p><img src="https://images-1307117474.cos.ap-guangzhou.myqcloud.com/blogs/Blog_3/syncpools1.png" alt="syncpools1" /></p><h3 id="全局变量"><a class="markdownIt-Anchor" href="#全局变量"></a> 全局变量</h3><figure class="highlight go"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">var</span> poolRaceHash [<span class="number">128</span>]<span class="keyword">uint64</span></span><br><span class="line"></span><br><span class="line"><span class="keyword">var</span> (</span><br><span class="line">allPoolsMu Mutex<span class="comment">//互斥锁，用于更新 allPools，因为所有的 sync.pool 都存放在里面 </span></span><br><span class="line"></span><br><span class="line"><span class="comment">// allPools is the set of pools that have non-empty primary</span></span><br><span class="line"><span class="comment">// caches. Protected by either 1) allPoolsMu and pinning or 2)</span></span><br><span class="line"><span class="comment">// STW.</span></span><br><span class="line">allPools []*Pool</span><br><span class="line"></span><br><span class="line"><span class="comment">// oldPools is the set of pools that may have non-empty victim</span></span><br><span class="line"><span class="comment">// caches. Protected by STW.</span></span><br><span class="line">oldPools []*Pool</span><br><span class="line">)</span><br></pre></td></tr></table></figure><ul><li><code>allPoolsMu</code>：用于对<code>allPools</code>的更新进行保护</li><li><code>allPools</code>: <code>[]*Pool</code>切片，存储具有非空私有缓存的对象池。可以被多个 <code>goroutine</code> 访问</li><li><code>oldPools</code>：<code>[]*Pool</code>类型的切片，用于存储可能具有非空受害者缓存的对象池，由于只有在 <code>STW</code> 时候才会更新，不会被并发访问</li></ul><h3 id="基本结构"><a class="markdownIt-Anchor" href="#基本结构"></a> 基本结构</h3><figure class="highlight go"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">type</span> Pool <span class="keyword">struct</span> {</span><br><span class="line"> noCopy noCopy <span class="comment">// 提示不要复制</span></span><br><span class="line"> </span><br><span class="line"> local unsafe.Pointer <span class="comment">// 指向每个 P(处理器) 的本地池的指针。它的类型是 [P]poolLocal，其中P是处理器数量</span></span><br><span class="line"> localSize <span class="keyword">uintptr</span> <span class="comment">// 本地数组的大小</span></span><br><span class="line"></span><br><span class="line"> victim unsafe.Pointer <span class="comment">// GC上一个周期中的本地池的指针。在当前周期中，这个本地池变成了受害者。它的类型是 unsafe.Pointer</span></span><br><span class="line"> victimSize <span class="keyword">uintptr</span> <span class="comment">// 受害者缓存数组的大小，以字节为单位</span></span><br><span class="line"></span><br><span class="line"> <span class="comment">//可选地指定一个函数，当 Get 方法本来会返回 nil 时，可以用于生成一个值。</span></span><br><span class="line"> New <span class="function"><span class="keyword">func</span><span class="params">()</span> <span class="title">any</span></span></span><br><span class="line">}</span><br></pre></td></tr></table></figure><ul><li><p><code>noCopy</code> 用于提示不要进行对象复制</p></li><li><p><code>local</code> 与 <code>victim</code> 的关系在后面 <strong>内存池清理</strong> 中说明</p></li><li><p><code>New</code> 则是自定义的分配对象函数</p></li></ul><h4 id="nocopy"><a class="markdownIt-Anchor" href="#nocopy"></a> noCopy</h4><p><code>noCopy</code> 支持 使用 <code>go vet</code> 检查对象是否被复制，它是一个内置的空结构体类型，当然也可以自行实现类似功能</p><figure class="highlight go"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">type</span> noCopy <span class="keyword">struct</span>{}</span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">func</span> <span class="params">(n noCopy)</span> <span class="title">Lock</span><span class="params">()</span></span> {</span><br><span class="line"><span class="comment">//TODO implement me</span></span><br><span class="line"><span class="built_in">panic</span>(<span class="string">"implement me"</span>)</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">func</span> <span class="params">(n noCopy)</span> <span class="title">Unlock</span><span class="params">()</span></span> {</span><br><span class="line"><span class="comment">//TODO implement me</span></span><br><span class="line"><span class="built_in">panic</span>(<span class="string">"implement me"</span>)</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line"><span class="keyword">var</span> _ sync.Locker = (*noCopy)()</span><br><span class="line"></span><br><span class="line"><span class="keyword">type</span> Person <span class="keyword">struct</span> {</span><br><span class="line">noCopy</span><br><span class="line">age <span class="keyword">int</span></span><br><span class="line">height <span class="keyword">int</span></span><br><span class="line">}</span><br></pre></td></tr></table></figure><p>它第一次使用后不能被复制，其实代码是能够编译通过运行的，只是在 <code>go vet</code> 或者 <code>部分编辑器</code>会提示而已。可以看看没有成为标准的原因 <a href="https://golang.org/issues/8005#issuecomment-190753527" target="_blank" rel="noopener">https://golang.org/issues/8005#issuecomment-190753527</a></p><h4 id="poollocal"><a class="markdownIt-Anchor" href="#poollocal"></a> poolLocal</h4><p>每个处理器 <code>P</code> 都有一个 <code>poolLocal</code> 的本地池对象</p><figure class="highlight go"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment">// 每个 P 的本地对象池</span></span><br><span class="line"><span class="keyword">type</span> poolLocalInternal <span class="keyword">struct</span> {</span><br><span class="line">private any <span class="comment">// 仅能被当前 P 进行读写</span></span><br><span class="line">shared poolChain <span class="comment">// 共享，当前 P 能对其进行 pushHead/popHead; 任何 P 都能执行popTail</span></span><br><span class="line">}</span><br><span class="line"></span><br><span class="line"><span class="keyword">type</span> poolLocal <span class="keyword">struct</span> {</span><br><span class="line">poolLocalInternal</span><br><span class="line"></span><br><span class="line"><span class="comment">// Prevents false sharing on widespread platforms with</span></span><br><span class="line"><span class="comment">// 128 mod (cache line size) = 0 .</span></span><br><span class="line">pad [<span class="number">128</span> - unsafe.Sizeof(poolLocalInternal{})%<span class="number">128</span>]<span class="keyword">byte</span></span><br><span class="line">}</span><br></pre></td></tr></table></figure><ul><li><code>private</code> 是一个仅用于当前 P 进行读写的字段(即没有并发读写的问题)</li><li><code>shared</code> 可以在多个 P 之间进行共享读写，是一个 <code>poolChain</code> 链式队列结构， 当前 P 上可以进行 <code>pushHead</code> 和 <code>popHead</code> 操作(队头读写)， 在所有 P 上都可以进行 <code>popTail</code> (队尾出队)操作</li><li><code>pad</code> 用于 <strong>伪共享</strong> 保证 <code>poolLocal</code> 的大小是 128 字节的倍数</li></ul><h4 id="runtime_procpin"><a class="markdownIt-Anchor" href="#runtime_procpin"></a> runtime_procPin</h4><figure class="highlight go"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment">//go:nosplit</span></span><br><span class="line"><span class="function"><span class="keyword">func</span> <span class="title">procPin</span><span class="params">()</span> <span class="title">int</span></span> {</span><br><span class="line">gp := getg()<span class="comment">//获取当前goroutine的指针</span></span><br><span class="line">mp := gp.m<span class="comment">//当前goroutine 对应的 协程代表</span></span><br><span class="line"></span><br><span class="line"> mp.locks++<span class="comment">//这个变量用于跟踪当前线程(m)持有的锁的数量</span></span><br><span class="line"> <span class="keyword">return</span> <span class="keyword">int</span>(mp.p.ptr().id)<span class="comment">//代表当前线程(m)执行的处理器(p)的唯一标识符</span></span><br><span class="line">}</span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">func</span> <span class="title">procUnpin</span><span class="params">()</span></span> {</span><br><span class="line">gp := getg()</span><br><span class="line">gp.m.locks--</span><br><span class="line">}</span><br></pre></td></tr></table></figure><ul><li><code>locks</code>:通过增加锁的计数，表明当前线程被固定（pinned）在处理器上</li><li><code>mp.p</code>表示当前线程所绑定的处理器，<code>.ptr()</code>方法返回处理器的指针，<code>.id</code>表示处理器的唯一标识符</li></ul><h4 id="pinslow"><a class="markdownIt-Anchor" href="#pinslow"></a> pinSlow</h4><p>将当前的<code>goroutine</code>绑定到<code>Pool</code>中的一个<code>poolLocal</code>上，并返回该<code>poolLocal</code>及其索引</p><figure class="highlight go"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br></pre></td><td class="code"><pre><span class="line"><span class="function"><span class="keyword">func</span> <span class="params">(p *Pool)</span> <span class="title">pinSlow</span><span class="params">()</span> <span class="params">(*poolLocal, <span class="keyword">int</span>)</span></span> {</span><br><span class="line"> <span class="comment">//1. 解除当前goroutine与之前绑定的poolLocal的绑定关系，使当前goroutine可以进行重新绑定</span></span><br><span class="line"> <span class="comment">//使用 mutex 时候 P 必须可抢占</span></span><br><span class="line"> runtime_procUnpin()</span><br><span class="line"> </span><br><span class="line"> <span class="comment">//2. 获取池的全局锁 </span></span><br><span class="line"> allPoolsMu.Lock()</span><br><span class="line"> <span class="keyword">defer</span> allPoolsMu.Unlock()</span><br><span class="line"> </span><br><span class="line"> <span class="comment">//3. 将当前的goroutine绑定到一个poolLocal上，并返回其索引pid。</span></span><br><span class="line"><span class="comment">//再次固定 P 时 poolCleanup 不会被调用</span></span><br><span class="line"> pid := runtime_procPin()</span><br><span class="line"> </span><br><span class="line"> <span class="comment">//获取未越界返回poolLocal</span></span><br><span class="line"> s := p.localSize </span><br><span class="line"> l := p.local</span><br><span class="line"> <span class="keyword">if</span> <span class="keyword">uintptr</span>(pid) < s {</span><br><span class="line"> <span class="keyword">return</span> indexLocal(l, pid), pid</span><br><span class="line"> }</span><br><span class="line"> </span><br><span class="line"> <span class="comment">// 如果数组为空，将其添加到 allPools，垃圾回收器从这里获取所有 Pool 实例</span></span><br><span class="line"> <span class="keyword">if</span> p.local == <span class="literal">nil</span> {</span><br><span class="line"> allPools = <span class="built_in">append</span>(allPools, p)</span><br><span class="line"> }</span><br><span class="line"> </span><br><span class="line"><span class="comment">// 根据 P 数量创建 slice，如果 GOMAXPROCS 在 GC 间发生变化</span></span><br><span class="line"><span class="comment">// 我们重新分配此数组并丢弃旧的</span></span><br><span class="line">size := runtime.GOMAXPROCS(<span class="number">0</span>) <span class="comment">// 获取当前的GOMAXPROCS值，即当前系统的最大并发数</span></span><br><span class="line">local := <span class="built_in">make</span>([]poolLocal, size) <span class="comment">//创建本地池</span></span><br><span class="line"> </span><br><span class="line"> <span class="comment">// 将底层数组起始指针保存到 p.local，并设置 p.localSize</span></span><br><span class="line">atomic.StorePointer(&p.local, unsafe.Pointer(&local[<span class="number">0</span>]))</span><br><span class="line">runtime_StoreReluintptr(&p.localSize, <span class="keyword">uintptr</span>(size)) </span><br><span class="line"> </span><br><span class="line"> <span class="comment">// 返回所需的 pollLocal</span></span><br><span class="line"><span class="keyword">return</span> &local[pid], pid</span><br><span class="line">}</span><br></pre></td></tr></table></figure><p>这个逻辑的前提是当前 P 已经发生了动态调整，需要重新计算<code>localPool</code></p><ul><li>首先解除 <code>goroutine</code> 与 <code>Process</code> 的绑定，让 <code>goroutine</code> 可以重新绑定</li><li>获取 <code>allPools</code>的全局锁</li><li>将当前 <code>goroutine</code> 与 <code>Process</code> 重新绑定</li><li>如果<code>Process</code> 未发生变化，返回 <code>Process</code> 的 <code>localPool</code></li><li>如果没有变化则<ul><li>如果 <code>Pool.local</code>为空，则需要将 <code>Pool</code> 加入到 <code>allPools</code> 中，用于 GC 扫描回收</li><li>重新创建 <code>[p]poolLocal</code></li><li>重新将 <code>Pool.local</code> 指向 <code>[p]poolLocal</code></li></ul></li></ul><h4 id="pin"><a class="markdownIt-Anchor" href="#pin"></a> pin</h4><p>获取当前 <code>Process</code> 中的 <code>poolLocal</code>，将当前的<code>goroutine</code>绑定到一个特定的<code>Process</code>上，禁用抢占并返回<code>Process</code>的<code>poolLocal</code>本地池和<code>P</code>的标识</p><figure class="highlight go"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br></pre></td><td class="code"><pre><span class="line"><span class="function"><span class="keyword">func</span> <span class="params">(p *Pool)</span> <span class="title">pin</span><span class="params">()</span> <span class="params">(*poolLocal, <span class="keyword">int</span>)</span></span> {</span><br><span class="line"> <span class="comment">// 1. 将当前的goroutine绑定到一个P上，并返回P的标识pid。</span></span><br><span class="line">pid := runtime_procPin()</span><br><span class="line"> </span><br><span class="line">s := runtime_LoadAcquintptr(&p.localSize) <span class="comment">// load-acquire</span></span><br><span class="line">l := p.local <span class="comment">// load-consume</span></span><br><span class="line"> <span class="comment">//可能存在动态的 P（运行时调整 P 的个数）procresize/GOMAXPROCS，如果 P.id 没有越界，则直接返回</span></span><br><span class="line"><span class="keyword">if</span> <span class="keyword">uintptr</span>(pid) < s {</span><br><span class="line"><span class="keyword">return</span> indexLocal(l, pid), pid</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line"><span class="keyword">return</span> p.pinSlow()</span><br><span class="line">}</span><br></pre></td></tr></table></figure><p>尝试通过加载<code>local</code>和<code>localSize</code>字段的方式来判断是否可以直接返回一个可用的<code>poolLocal</code>，如果不满足条件，则调用<code>pinSlow</code>方法来重新分配并返回一个新的<code>poolLocal</code>。调用者在使用完<code>poolLocal</code>之后，必须调用<code>runtime_procUnpin()</code>来解除与<code>P</code>的绑定关系。</p><figure class="highlight go"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line"><span class="function"><span class="keyword">func</span> <span class="title">indexLocal</span><span class="params">(l unsafe.Pointer, i <span class="keyword">int</span>)</span> *<span class="title">poolLocal</span></span> {</span><br><span class="line">lp := unsafe.Pointer(<span class="keyword">uintptr</span>(l) + <span class="keyword">uintptr</span>(i)*unsafe.Sizeof(poolLocal{}))</span><br><span class="line"><span class="keyword">return</span> (*poolLocal)(lp)</span><br><span class="line">}</span><br></pre></td></tr></table></figure><h3 id="内存池清理"><a class="markdownIt-Anchor" href="#内存池清理"></a> 内存池清理</h3><p>在使用 init 仅执行了一个逻辑，就是注册内存池回收机制</p><figure class="highlight go"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line"><span class="function"><span class="keyword">func</span> <span class="title">init</span><span class="params">()</span></span> {</span><br><span class="line">runtime_registerPoolCleanup(poolCleanup)</span><br><span class="line">}</span><br></pre></td></tr></table></figure><p><code>poolCleanup</code> 用于实现内存池的清理</p><figure class="highlight go"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br></pre></td><td class="code"><pre><span class="line"><span class="function"><span class="keyword">func</span> <span class="title">poolCleanup</span><span class="params">()</span></span> {</span><br><span class="line"><span class="comment">// Because the world is stopped, no pool user can be in a</span></span><br><span class="line"><span class="comment">// pinned section (in effect, this has all Ps pinned).</span></span><br><span class="line"></span><br><span class="line"><span class="comment">// Drop victim caches from all pools.</span></span><br><span class="line"><span class="keyword">for</span> _, p := <span class="keyword">range</span> oldPools {</span><br><span class="line">p.victim = <span class="literal">nil</span></span><br><span class="line">p.victimSize = <span class="number">0</span></span><br><span class="line">}</span><br><span class="line"></span><br><span class="line"><span class="comment">// Move primary cache to victim cache.</span></span><br><span class="line"><span class="keyword">for</span> _, p := <span class="keyword">range</span> allPools {</span><br><span class="line">p.victim = p.local</span><br><span class="line">p.victimSize = p.localSize</span><br><span class="line">p.local = <span class="literal">nil</span></span><br><span class="line">p.localSize = <span class="number">0</span></span><br><span class="line">}</span><br><span class="line"></span><br><span class="line"><span class="comment">// The pools with non-empty primary caches now have non-empty</span></span><br><span class="line"><span class="comment">// victim caches and no pools have primary caches.</span></span><br><span class="line">oldPools, allPools = allPools, <span class="literal">nil</span></span><br><span class="line">}</span><br></pre></td></tr></table></figure><p>垃圾回收的策略就是</p><ul><li>将 <code>oldPools</code> 中也就是所有<code>localPool</code>的 <code>victim</code> 对象丢弃</li><li>将 <code>allPools</code> 的 <code>local</code> 复制给<code>victim</code>，并 <code>local</code> 重置</li><li>最后将 <code>allPools</code> 复制给 <code>oldPools</code>，<code>allPools</code> 置空</li></ul><h3 id="get"><a class="markdownIt-Anchor" href="#get"></a> Get</h3><p>整体流程如下</p><img src="https://images-1307117474.cos.ap-guangzhou.myqcloud.com/blogs/Blog_3/syncpool4.png" alt="syncpool4" style="zoom:50%;" /><ol><li>首先获取当前 <code>Process</code> 的 <code>poolLocal</code>，<strong>也就是说当 Goroutine 在哪个 Process 运行的时候就会从哪个 Process 的 localPool中获取对象</strong></li><li>优先从 <code>private</code> 中选择对象，并将 <code>private = nil</code></li><li>若取不到，则尝试从 <code>shared</code> 队列的队头进行读取</li><li>若取不到，则尝试从其他的 <code>Process</code> 中进行偷取 <code>getSlow</code><strong>（跨 Process 读写）</strong></li><li>若还是取不到，则使用自定义的 <code>New</code> 方法新建对象</li></ol><p>获取对象代码如下操作</p><figure class="highlight go"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br></pre></td><td class="code"><pre><span class="line"><span class="function"><span class="keyword">func</span> <span class="params">(p *Pool)</span> <span class="title">Get</span><span class="params">()</span> <span class="title">any</span></span> {</span><br><span class="line"> <span class="comment">//...race</span></span><br><span class="line"> <span class="comment">//1. 获取一个poolLocal</span></span><br><span class="line"> l, pid := p.pin()</span><br><span class="line"> <span class="comment">//2. 先从private获取对象</span></span><br><span class="line">x := l.private</span><br><span class="line">l.private = <span class="literal">nil</span></span><br><span class="line"><span class="keyword">if</span> x == <span class="literal">nil</span> {</span><br><span class="line"><span class="comment">// 尝试从 localPool 的 shared 队列队头读取</span></span><br><span class="line">x, _ = l.shared.popHead()</span><br><span class="line"><span class="keyword">if</span> x == <span class="literal">nil</span> {</span><br><span class="line"> <span class="comment">// 如果取不到，则获取新的缓存对象</span></span><br><span class="line">x = p.getSlow(pid)</span><br><span class="line">}</span><br><span class="line">}</span><br><span class="line">runtime_procUnpin()</span><br><span class="line"><span class="comment">//...race</span></span><br><span class="line"> <span class="comment">// 如果 getSlow 还是获取不到，则 New 一个</span></span><br><span class="line"><span class="keyword">if</span> x == <span class="literal">nil</span> && p.New != <span class="literal">nil</span> {</span><br><span class="line">x = p.New()</span><br><span class="line">}</span><br><span class="line"><span class="keyword">return</span> x</span><br><span class="line">}</span><br></pre></td></tr></table></figure><p>其中的 <code>getSlow</code> 就是从 其他 <code>Process</code> 或者 <code>victim</code> 中获取</p><figure class="highlight go"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br></pre></td><td class="code"><pre><span class="line"><span class="function"><span class="keyword">func</span> <span class="params">(p *Pool)</span> <span class="title">getSlow</span><span class="params">(pid <span class="keyword">int</span>)</span> <span class="title">any</span></span> {</span><br><span class="line"><span class="comment">//1. 加载本地池</span></span><br><span class="line">size := runtime_LoadAcquintptr(&p.localSize) <span class="comment">// load-acquire</span></span><br><span class="line">locals := p.local <span class="comment">// load-consume</span></span><br><span class="line"><span class="comment">// Try to steal one element from other procs.</span></span><br><span class="line"> <span class="comment">//2. 尝试从其他 process 偷取元素</span></span><br><span class="line"><span class="keyword">for</span> i := <span class="number">0</span>; i < <span class="keyword">int</span>(size); i++ {</span><br><span class="line">l := indexLocal(locals, (pid+i+<span class="number">1</span>)%<span class="keyword">int</span>(size))</span><br><span class="line"><span class="keyword">if</span> x, _ := l.shared.popTail(); x != <span class="literal">nil</span> {</span><br><span class="line"><span class="keyword">return</span> x</span><br><span class="line">}</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line"> <span class="comment">//3. 从其他 process 也没有偷到</span></span><br><span class="line"> <span class="comment">//那么判断是否 process数量发生了变化</span></span><br><span class="line">size = atomic.LoadUintptr(&p.victimSize)</span><br><span class="line"><span class="keyword">if</span> <span class="keyword">uintptr</span>(pid) >= size {</span><br><span class="line"><span class="keyword">return</span> <span class="literal">nil</span></span><br><span class="line">}</span><br><span class="line"> <span class="comment">//4. 从 victim 中获取</span></span><br><span class="line">locals = p.victim</span><br><span class="line">l := indexLocal(locals, pid)</span><br><span class="line"><span class="keyword">if</span> x := l.private; x != <span class="literal">nil</span> {</span><br><span class="line">l.private = <span class="literal">nil</span></span><br><span class="line"><span class="keyword">return</span> x</span><br><span class="line">}</span><br><span class="line"><span class="keyword">for</span> i := <span class="number">0</span>; i < <span class="keyword">int</span>(size); i++ {</span><br><span class="line">l := indexLocal(locals, (pid+i)%<span class="keyword">int</span>(size))</span><br><span class="line"><span class="keyword">if</span> x, _ := l.shared.popTail(); x != <span class="literal">nil</span> {</span><br><span class="line"><span class="keyword">return</span> x</span><br><span class="line">}</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line"> <span class="comment">//到这里说明没有了，那么就将 victimSize 设置为 0后续不会访问</span></span><br><span class="line">atomic.StoreUintptr(&p.victimSize, <span class="number">0</span>)</span><br><span class="line"></span><br><span class="line"><span class="keyword">return</span> <span class="literal">nil</span></span><br><span class="line">}</span><br></pre></td></tr></table></figure><h3 id="put"><a class="markdownIt-Anchor" href="#put"></a> Put</h3><p>Put 的操作如下</p><img src="https://images-1307117474.cos.ap-guangzhou.myqcloud.com/blogs/Blog_3/syncpool5.png" alt="syncpool5" style="zoom:50%;" /><p>存放策略是：</p><ol><li>如果存放 <code>nil</code> 直接返回</li><li>获取当前 <code>Process</code> 的 <code>poolLocal</code></li><li>如果 <code>private == nil</code> 则放到 <code>private</code> 中</li><li>如果<code>private != nil</code> 则将起放入到 链表头部</li></ol><figure class="highlight go"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br></pre></td><td class="code"><pre><span class="line"><span class="function"><span class="keyword">func</span> <span class="params">(p *Pool)</span> <span class="title">Put</span><span class="params">(x any)</span></span> {</span><br><span class="line"><span class="keyword">if</span> x == <span class="literal">nil</span> {</span><br><span class="line"><span class="keyword">return</span></span><br><span class="line">}</span><br><span class="line"><span class="comment">//...race</span></span><br><span class="line"> <span class="comment">// 获得一个 localPool</span></span><br><span class="line">l, _ := p.pin()</span><br><span class="line"> <span class="comment">// 优先放入private</span></span><br><span class="line"><span class="keyword">if</span> l.private == <span class="literal">nil</span> {</span><br><span class="line">l.private = x</span><br><span class="line">} <span class="keyword">else</span> {</span><br><span class="line">l.shared.pushHead(x)</span><br><span class="line">}</span><br><span class="line">runtime_procUnpin()</span><br><span class="line"><span class="comment">//...race</span></span><br><span class="line">}</span><br></pre></td></tr></table></figure><h3 id="总结"><a class="markdownIt-Anchor" href="#总结"></a> 总结</h3><ol><li>Pool 本质是为了提高<strong>临时对象的复用率</strong>；</li><li>Pool 使用两层回收策略（local + victim）避免性能波动；</li><li>Pool 本质是一个杂货铺属性，啥都可以放，Pool 池本身不做限制；</li><li>Pool 池里面 cache 对象也是分层的，一层层的 cache，取用方式从最热的数据到最冷的数据递进；</li><li>Pool 是并发安全的，但是内部是无锁结构，原理是对每个 P 都分配 cache 数组（ <code>poolLocalInternal</code> 数组），这样 cache 结构就不会导致并发；</li><li>永远不要 copy 一个 Pool，明确禁止，不然会导致内存泄露和程序并发逻辑错误；</li><li>代码编译之前用 <code>go vet</code> 做静态检查，能减少非常多的问题；</li><li>每轮 GC 开始都会清理一把 Pool 里面 cache 的对象，注意流程是分两步，当前 Pool 池 local 数组里的元素交给 victim 数组句柄，victim 里面 cache 的元素全部清理。换句话说，引入 victim 机制之后，对象的缓存时间变成两个 GC 周期；</li><li>不要对 Pool 里面的对象做任何假定，有两种方案：要么就归还的时候 memset 对象之后，再调用 <code>Pool.Put</code> ，要么就 <code>Pool.Get</code> 取出来的时候 memset 之后再使用；</li></ol><h3 id="参考文档"><a class="markdownIt-Anchor" href="#参考文档"></a> 参考文档</h3><ol><li><a href="https://mp.weixin.qq.com/s/dLzWAqM9lCln83jhkvmtMw" target="_blank" rel="noopener">https://mp.weixin.qq.com/s/dLzWAqM9lCln83jhkvmtMw</a></li><li><a href="https://geektutu.com/post/hpg-sync-pool.html" target="_blank" rel="noopener">https://geektutu.com/post/hpg-sync-pool.html</a></li><li><a href="https://golang.design/under-the-hood/zh-cn/part1basic/ch05sync/pool/" target="_blank" rel="noopener">https://golang.design/under-the-hood/zh-cn/part1basic/ch05sync/pool/</a></li><li><a href="https://xie.infoq.cn/article/55f28d278cccf0d8195459263" target="_blank" rel="noopener">https://xie.infoq.cn/article/55f28d278cccf0d8195459263</a></li></ol>]]></content>
<summary type="html">
<p>前面一篇讲解了<strong>Sync.Pool</strong>的底层数据结构 <a href="https://www.yuankang.top/2023/07/01/Go/Go%E5%85%A5%E9%97%A825-PoolDequeue/">poolDequeue
</summary>
<category term="Go" scheme="https://www.yuankang.top/categories/Go/"/>
<category term="Go" scheme="https://www.yuankang.top/tags/Go/"/>
</entry>
<entry>
<title>Go-26-Issue</title>
<link href="https://www.yuankang.top/2023/07/13/Go/Go-26-Issue/"/>
<id>https://www.yuankang.top/2023/07/13/Go/Go-26-Issue/</id>
<published>2023-07-13T14:59:45.000Z</published>
<updated>2023-07-19T15:34:46.967Z</updated>
<content type="html"><![CDATA[<h3 id="issue-23199"><a class="markdownIt-Anchor" href="#issue-23199"></a> Issue-23199</h3><p>在阅读 <code>sync.Pool</code> 在 <code>fmt</code> 中的使用时候看到了这样一段代码</p><figure class="highlight go"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br></pre></td><td class="code"><pre><span class="line"><span class="function"><span class="keyword">func</span> <span class="params">(p *pp)</span> <span class="title">free</span><span class="params">()</span></span> {</span><br><span class="line"><span class="comment">// Proper usage of a sync.Pool requires each entry to have approximately</span></span><br><span class="line"><span class="comment">// the same memory cost. To obtain this property when the stored type</span></span><br><span class="line"><span class="comment">// contains a variably-sized buffer, we add a hard limit on the maximum</span></span><br><span class="line"><span class="comment">// buffer to place back in the pool. If the buffer is larger than the</span></span><br><span class="line"><span class="comment">// limit, we drop the buffer and recycle just the printer.</span></span><br><span class="line"><span class="comment">//</span></span><br><span class="line"><span class="comment">// See .https://golang.org/issue/23199</span></span><br><span class="line"><span class="keyword">if</span> <span class="built_in">cap</span>(p.buf) > <span class="number">64</span>*<span class="number">1024</span> {</span><br><span class="line">p.buf = <span class="literal">nil</span></span><br><span class="line">} <span class="keyword">else</span> {</span><br><span class="line">p.buf = p.buf[:<span class="number">0</span>]</span><br><span class="line">}</span><br><span class="line"><span class="comment">//...</span></span><br><span class="line"><span class="comment">//存入内存池</span></span><br><span class="line">ppFree.Put(p)</span><br><span class="line">}</span><br></pre></td></tr></table></figure><p>上述英文解释的大致意思是</p><blockquote><p><code>sync.Pool</code>在使用的时候<strong>每个对象的内存消耗应该大致相同</strong>。当存储的类型包含可变大小的缓冲区时，需要对最大缓冲区设置一个硬限制，以确保如果缓冲区的大小超过限制，将丢弃缓冲区，并仅回收</p></blockquote><p>因为 <code>fmt</code> 使用内存池分配的对象大小不是固定的，如下 <code>buf</code> 其实是一个缓冲区</p><figure class="highlight go"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">var</span> ppFree = sync.Pool{</span><br><span class="line">New: <span class="function"><span class="keyword">func</span><span class="params">()</span> <span class="title">any</span></span> { <span class="keyword">return</span> <span class="built_in">new</span>(pp) },</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line"><span class="keyword">type</span> pp <span class="keyword">struct</span> {</span><br><span class="line">buf buffer</span><br><span class="line"> <span class="comment">//...</span></span><br><span class="line">}</span><br><span class="line"></span><br><span class="line"><span class="keyword">type</span> buffer []<span class="keyword">byte</span></span><br></pre></td></tr></table></figure><p>这里做一个测试就会明白这样的处理</p><figure class="highlight go"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br></pre></td><td class="code"><pre><span class="line">a := <span class="built_in">make</span>([]<span class="keyword">int</span>, <span class="number">100</span>)</span><br><span class="line">b := <span class="built_in">make</span>([]<span class="keyword">int</span>, <span class="number">100</span>)</span><br><span class="line">fmt.Println(<span class="built_in">cap</span>(a), <span class="built_in">len</span>(a))<span class="comment">//100 100</span></span><br><span class="line">fmt.Println(<span class="built_in">cap</span>(b), <span class="built_in">len</span>(b)) <span class="comment">//100 100</span></span><br><span class="line"></span><br><span class="line">a = <span class="literal">nil</span></span><br><span class="line">b = b[:<span class="number">0</span>]</span><br><span class="line">fmt.Println(<span class="built_in">cap</span>(a), <span class="built_in">len</span>(a))<span class="comment">//0 0</span></span><br><span class="line">fmt.Println(<span class="built_in">cap</span>(b), <span class="built_in">len</span>(b)) <span class="comment">///100 0</span></span><br></pre></td></tr></table></figure><p>使用 b[:0] 之后只是切片长度变化，容量并不会变，也就是内存仍然占用那么多</p><p>所以，如果内存池大小不固定的时候注意主动释放，防止额外占用空间而不被释放。再来看看它的测试结果</p><figure class="highlight go"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br><span class="line">45</span><br><span class="line">46</span><br><span class="line">47</span><br><span class="line">48</span><br><span class="line">49</span><br><span class="line">50</span><br><span class="line">51</span><br><span class="line">52</span><br><span class="line">53</span><br><span class="line">54</span><br><span class="line">55</span><br><span class="line">56</span><br><span class="line">57</span><br><span class="line">58</span><br><span class="line">59</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">package</span> main</span><br><span class="line"></span><br><span class="line"><span class="keyword">import</span> (</span><br><span class="line"><span class="string">"bytes"</span></span><br><span class="line"><span class="string">"fmt"</span></span><br><span class="line"><span class="string">"runtime"</span></span><br><span class="line"><span class="string">"sync"</span></span><br><span class="line"><span class="string">"time"</span></span><br><span class="line">)</span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">func</span> <span class="title">finalizer</span><span class="params">(buffer *bytes.Buffer)</span></span> {</span><br><span class="line">fmt.Printf(<span class="string">"GC %d buffer \n"</span>, buffer.Cap())</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">func</span> <span class="title">main</span><span class="params">()</span></span> {</span><br><span class="line"><span class="keyword">var</span> stats runtime.MemStats</span><br><span class="line">runtime.ReadMemStats(&stats)</span><br><span class="line">fmt.Printf(<span class="string">"start %d B\n"</span>, stats.Alloc)</span><br><span class="line"></span><br><span class="line">pool := sync.Pool{New: <span class="function"><span class="keyword">func</span><span class="params">()</span> <span class="title">interface</span></span>{} {</span><br><span class="line">buf := <span class="built_in">new</span>(bytes.Buffer)</span><br><span class="line">runtime.SetFinalizer(buf, finalizer)</span><br><span class="line"><span class="keyword">return</span> buf</span><br><span class="line">}}</span><br><span class="line"></span><br><span class="line">processRequest := <span class="function"><span class="keyword">func</span><span class="params">(size <span class="keyword">int</span>)</span></span> {</span><br><span class="line">b := pool.Get().(*bytes.Buffer)</span><br><span class="line">time.Sleep(<span class="number">500</span> * time.Millisecond) <span class="comment">// Simulate processing time</span></span><br><span class="line">b.Grow(size)</span><br><span class="line">pool.Put(b)</span><br><span class="line">time.Sleep(<span class="number">1</span> * time.Millisecond) <span class="comment">// Simulate idle time</span></span><br><span class="line">}</span><br><span class="line"></span><br><span class="line"><span class="comment">// 模拟一组大规模写入</span></span><br><span class="line"><span class="keyword">for</span> i := <span class="number">0</span>; i < <span class="number">10</span>; i++ {</span><br><span class="line"><span class="keyword">go</span> <span class="function"><span class="keyword">func</span><span class="params">()</span></span> {</span><br><span class="line">processRequest(<span class="number">1</span> << <span class="number">20</span>) <span class="comment">// 256MiB</span></span><br><span class="line">}()</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line">time.Sleep(time.Second) <span class="comment">// Let the initial set finish</span></span><br><span class="line"></span><br><span class="line"><span class="comment">// 模拟小规模写入且不会停</span></span><br><span class="line"><span class="keyword">for</span> i := <span class="number">0</span>; i < <span class="number">10</span>; i++ {</span><br><span class="line"><span class="keyword">go</span> <span class="function"><span class="keyword">func</span><span class="params">()</span></span> {</span><br><span class="line"><span class="keyword">for</span> {</span><br><span class="line">processRequest(<span class="number">1</span> << <span class="number">10</span>) <span class="comment">// 1KiB</span></span><br><span class="line">}</span><br><span class="line">}()</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line"><span class="comment">// 每次GC之后查看分配的内存</span></span><br><span class="line"><span class="keyword">for</span> i := <span class="number">0</span>; ; i++ {</span><br><span class="line">runtime.ReadMemStats(&stats)</span><br><span class="line">fmt.Printf(<span class="string">"Cycle %d: %dB\n"</span>, i, stats.Alloc)</span><br><span class="line">time.Sleep(time.Second)</span><br><span class="line">runtime.GC()</span><br><span class="line">}</span><br><span class="line">}</span><br></pre></td></tr></table></figure><p>通过 <a href="https://www.yuankang.top/2023/01/04/Go/Go-21-SetFinalizer/">runtime.SetFinalizer</a> 大概说明对象的回收时间，会发现大对象并不是立即回收的，而是经过了一段时间，内存才趋于稳定。高并发场景下，比如处理网络请求的时候，可能导致大量内存的占用而没有及时释放</p><p><strong>解决办法</strong>: 就在在判断超过一定大小的时候直接丢弃</p><h3 id="参考链接"><a class="markdownIt-Anchor" href="#参考链接"></a> 参考链接</h3><ol><li><a href="https://golang.org/issue/23199" target="_blank" rel="noopener">https://golang.org/issue/23199</a></li><li><a href="https://go-review.googlesource.com/c/go/+/136035/1/src/encoding/json/encode.go" target="_blank" rel="noopener">https://go-review.googlesource.com/c/go/+/136035/1/src/encoding/json/encode.go</a></li></ol>]]></content>
<summary type="html">
<h3 id="issue-23199"><a class="markdownIt-Anchor" href="#issue-23199"></a> Issue-23199</h3>
<p>在阅读 <code>sync.Pool</code> 在 <code>fmt</code>
</summary>
<category term="Go" scheme="https://www.yuankang.top/categories/Go/"/>
<category term="Go" scheme="https://www.yuankang.top/tags/Go/"/>
</entry>
</feed>