总计约 97.00K 字
所有文章 56 -
总计约 96.12K 字
diff --git a/content/posts/sysprog/c-compiler-construction.md b/content/posts/sysprog/c-compiler-construction.md index 878574b5..a45945e7 100644 --- a/content/posts/sysprog/c-compiler-construction.md +++ b/content/posts/sysprog/c-compiler-construction.md @@ -54,6 +54,11 @@ repost: - [ ] [用十分鐘 向 jserv 學習作業系統設計](https://www.slideshare.net/ccckmit/jserv#22) - [x] [用1500 行建構可自我編譯的 C 編譯器](https://hackmd.io/coscup18-source-c-compiler) / [投影片](https://drive.google.com/file/d/1-0QGf2JSni-CwYigaEORW6JUehS8LS79/view) + +[AMaCC](https://github.com/jserv/amacc) 是由成功大學師生開發的 self-compiling 的 C 語言編譯器,可產生 Arm 架構的執行檔 (ELF 格式,運作在 GNU/Linux)、也支援 just-in-time (JIT) 編譯和執行,原始程式碼僅 1500 行,在這次講座中,我們就來揭開 AMaCC 背後的原理和實作議題。 + +預期會接觸到 IR (Intermediate representation), dynamic linking, relocation, symbol table, parsing tree, language frontend, Arm 指令編碼和 ABI 等等。 + - Wikipedia: [Executable and Linkable Format](https://en.wikipedia.org/wiki/Executable_and_Linkable_Format) ## 编译器和软件工业强度息息相关 @@ -84,15 +89,85 @@ repost: {{< image src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEifha5yJnrvK51Mpal4CZX5hkw3F1LAQO5XCUEBhyphenhyphenvDfGYEFH2x5XBIVGps49SszNN5QoP1BBbtiAYdKvVQtLvsfoCNvtPtwc9czkkRc8Iz2Q2uG1n_G_xZZs4XTdKO4lK_LUaGVNkAF3NU/s1600/compiler.jpg" >}} -### 手把手教你构建 C 语言编译器 +## 手把手教你构建 C 语言编译器 [原文地址](https://github.com/lotabout/write-a-C-interpreter) -### 延伸阅读 +编译原理课程教导的是如何完成一个「编译器的编译器」 即 [Compiler-compiler](https://en.wikipedia.org/wiki/Compiler-compiler),这个难度比较大,因为需要考虑通用性,但是实作一个简单的编译器并没有这么难。 -- Wikipedia: [Stack machine](https://en.wikipedia.org/wiki/Stack_machine) - Wikipedia: [Compiler-compiler](https://en.wikipedia.org/wiki/Compiler-compiler) +### 设计 + +一般而言,编译器的编写分为 3 个步骤: +1. 词法分析器,用于将字符串转化成内部的表示结构。 +2. 语法分析器,将词法分析得到的标记流(token)生成一棵语法树。 +3. 目标代码的生成,将语法树转化成目标代码。 + +### 虚拟机 + +在该项目的虚拟机设计中,函数调用时 callee 的参数位于 caller 的栈帧 (frame) 内,并且函数调用需要使用到 4 条指令: + +1. `CALL` 指令将 callee 的返回地址压入栈,然后跳转到 callee 的入口处 +2. `ENT` 指令保存 ebp 寄存器的值并为 callee 的栈帧设置 esp 和 ebp 寄存器,在栈中给 callee 的局部变量分配空间 +3. `ADJ` 指令在 callee 逻辑执行完毕后,释放之前分配给局部变量的栈空间 +4. `LEV` 指令将 ebp 和 esp 寄存器恢复为对应 caller 栈帧,并跳转到之前保存的 callee 的返回地址处 + +除此之外,在 callee 执行期间,可能需要通过 `LEA` 指令来访问函数参数和局部变量。 + +``` +sub_function(arg1, arg2, arg3); + +| .... | high address ++---------------+ +| arg: 1 | new_bp + 4 ++---------------+ +| arg: 2 | new_bp + 3 ++---------------+ +| arg: 3 | new_bp + 2 ++---------------+ +|return address | new_bp + 1 ++---------------+ +| old BP | <- new BP ++---------------+ +| local var 1 | new_bp - 1 ++---------------+ +| local var 2 | new_bp - 2 ++---------------+ +| .... | low address +``` + +- Wikipedia: [Stack machine](https://en.wikipedia.org/wiki/Stack_machine) +- Wikipedia: [x86 calling conventions](https://en.wikipedia.org/wiki/X86_calling_conventions) + +{{< admonition question "问题 `PRTF` means?" false >}} +```c +else if (op == PRTF) { + tmp = sp + pc[1]; + ax = printf((char *)tmp[-1], tmp[-2], tmp[-3], tmp[-4], tmp[-5], tmp[-6]); +} +``` + +这里 [c4](https://github.com/rswier/c4) 对于 `PRTF` 指令的处理暂时没看明白... +{{< /admonition >}} + +{{< admonition question "问题 `-m32` error" false >}} +gcc 通过 `-m32` 参数编译本节代码时可能会遇到以下报错: + +```bash +fatal error: bits/wordsize.h: No such file or directory +``` + +这是因为当前安装的 gcc 只有 64 位的库而没有 32 位的库,通过以下命令安装 32 位库解决问题: + +```bash +$ sudo apt install gcc-multilib +``` + +Stack Overflow: +- ["fatal error: bits/libc-header-start.h: No such file or directory" while compiling HTK](https://stackoverflow.com/questions/54082459/fatal-error-bits-libc-header-start-h-no-such-file-or-directory-while-compili) +{{< /admonition >}} + ## IR (Intermediate representation) - [ ] [Interpreter, Compiler, JIT from scratch](https://www.slideshare.net/jserv/jit-compiler) @@ -110,6 +185,8 @@ JIT (Just in time) 表示“即时”,形象描述就是“及时雨” :rofl: {{< image src="/images/c/shell.drawio.svg" >}} +系统编程 (System Programming) 的入门项目,阅读过程需要查询搭配 man 手册,以熟悉库函数和系统调用的原型和作用。 + Linux manual page: [fflush](https://man7.org/linux/man-pages/man3/fflush.3.html) / [elf](https://man7.org/linux/man-pages/man5/elf.5.html) diff --git a/docs/friends/index.html b/docs/friends/index.html index 61653d50..7979b742 100644 --- a/docs/friends/index.html +++ b/docs/friends/index.html @@ -9,14 +9,14 @@ - + - + @@ -36,7 +36,7 @@ "@type": "WebPage", "@id": "https:\/\/ccrysisa.github.io\/friends\/" },"genre": "friends","wordcount": 74 , - "url": "https:\/\/ccrysisa.github.io\/friends\/","datePublished": "2024-04-29T23:46:50+08:00","dateModified": "2024-04-30T00:49:33+08:00","publisher": { + "url": "https:\/\/ccrysisa.github.io\/friends\/","datePublished": "2024-04-29T23:46:50+08:00","dateModified": "2024-04-30T00:50:51+08:00","publisher": { "@type": "Organization", "name": ""},"author": { "@type": "Person", diff --git a/docs/images/c/shell.drawio.svg b/docs/images/c/shell.drawio.svg new file mode 100644 index 00000000..d2159f00 --- /dev/null +++ b/docs/images/c/shell.drawio.svg @@ -0,0 +1,4 @@ + + + + \ No newline at end of file diff --git a/docs/index.json b/docs/index.json index 68b701b6..60e77818 100644 --- a/docs/index.json +++ b/docs/index.json @@ -1 +1 @@ -[{"categories":["draft"],"content":"This post is used to record the process of my English learning. ","date":"2024-03-30","objectID":"/posts/english/:0:0","tags":["draft"],"title":"English Everyday","uri":"/posts/english/"},{"categories":["draft"],"content":"Preface 工欲善其事,必先利其器 If I had eight hours to chop down a tree, I’d spend six hours sharpening my axe. 单词书: Merriam-Webster’s Vocabulary Builder 写作书: The Elements of Style 语法书: https://grammar.codeyu.com/ 发音教学: 一些 YouTube channels: https://www.youtube.com/@LearnEnglishWithTVSeries https://www.youtube.com/@letstalk https://www.youtube.com/@bbclearningenglish https://www.youtube.com/@coachshanesesl 一些 B 站 UP 主: 妈妈不用担心我的英语 英语兔 一些 GitHub 仓库: https://github.com/byoungd/English-level-up-tips https://github.com/xiaolai/everyone-can-use-english https://github.com/IammyselfBOOKS/New_concept_English https://github.com/protogenesis/NewConceptEnglish 仓库中关于新概念英语的网址,录音是正确的,但是有一些正文不太准确,可以下载书籍进行对比 ","date":"2024-03-30","objectID":"/posts/english/:1:0","tags":["draft"],"title":"English Everyday","uri":"/posts/english/"},{"categories":["draft"],"content":"New Concept English ","date":"2024-03-30","objectID":"/posts/english/:2:0","tags":["draft"],"title":"English Everyday","uri":"/posts/english/"},{"categories":["draft"],"content":"NCE 1 001: Excuse me! 003: Sorry, sir! 005: Nice to meet you! 007: Are you a teacher? 009: How are you today? 011: Is this your shirt? 013: A new dress 015: Your passports, please 017: How do you do 019: Tired and thirsty 021: Which book? 023: Which glasses? 025: Mrs. Smith’s kitchen 027: Mrs. Smith’s living room 029: Come in, Amy 031: Where’s Sally 033: A fine day 035: Our village 037: Making a bookcase 039: Don’t drop it! 041: Penny’s bag 043: Hurry up! 045: The boss’s letter 047: A cup of coffee 049: At the butcher’s 051: A pleasant climate 053: An interesting climate 055: The Sawyer family 057: An unusual day 059: Is that all? 063: Thank you, doctor. 065: Not a baby. 067: The weekend 069: The car race 071: He’s awful 073: The way to King Street 075: Uncomfortable shoes 077: Terrible toothache 079: Peggy’s shopping list 081: Roast beef and potato 083: Going on a holiday Source: Cambridge Dictionary handbag n. a small bag used by a woman to carry everyday personal items. umbrella n. a device consisting of a circular canopy of cloth on a folding metal frame supported by a central rod, used as protection against rain. nationality n. the official right to belong to a particular country. engineer n. a person whose job is to design or build machines, engines, or electrical equipment, or things such as roads, railways, or bridges, using scientific principles. perhaps adv. used to show that something is possible or that you are not certain about something. refrigerator n. a piece of kitchen equipment that uses electricity to preserve food at a cold temperature. armchair n. a comfortable chair with sides that support your arms. stereo n. wardrobe n. a tall cupboard in which you hang your clothes. dust v. to use a cloth to remove dust from the surface of something. sweep v. to clean something, especially a floor by using a brush to collect the dirt into one place from which it can be removed. aeroplane n. vase n. tobacco n. kettle n. cupboard n. beef n. lamb n. steak n. mince n. mild pad n. chalk n. greengrocer n. phrase n. jam n. grocer n. bear n. wine n. cinema n. ","date":"2024-03-30","objectID":"/posts/english/:2:1","tags":["draft"],"title":"English Everyday","uri":"/posts/english/"},{"categories":["Linux Kernel Internals"],"content":" 在「Linux 核心设计/实作」Spring 2023 课程进度页面的原始档案的基础上,稍作修改以记录我的学习进度 原始页面 | PDF 成功 如果你学习时感到挫折,感到进度推进很慢,这很正常,因为 Jserv 的一个讲座,需要我们花费一个星期去消化 🤣 并且 Jserv 也提到前 6 周课程的密度是比较大的 所以没必要为此焦虑,如果你觉得某个内容不太理解,可以尝试先去看其他讲座,将原先不懂的知识交给大脑隐式消化,过段时间再回来看,你的理解会大有不同。 Instructor: Jim Huang (黃敬群) \u003cjserv.tw@gmail.com\u003e 往年課程進度 Linux 核心設計 (線上講座) 注意: 下方課程進度表標註有 * 的項目,表示內附錄影的教材 注意: 新開的「Linux 核心實作」課程內容幾乎與「Linux 核心設計」一致,採線上為主的進行方式 ","date":"2024-02-28","objectID":"/posts/linux2023/:0:0","tags":["Sysprog","Linux"],"title":"Linux 核心设计/实作 (Linux Kernel Internals)","uri":"/posts/linux2023/"},{"categories":["Linux Kernel Internals"],"content":"Linux 核心設計/實作 (Spring 2023) 課程進度表暨線上資源 ","date":"2024-02-28","objectID":"/posts/linux2023/:1:0","tags":["Sysprog","Linux"],"title":"Linux 核心设计/实作 (Linux Kernel Internals)","uri":"/posts/linux2023/"},{"categories":["Linux Kernel Internals"],"content":"第 1 週: 誠實面對自己 (Feb 13, 14, 16) 教材解說* (僅止於概況,請詳閱下方教材及個別的對應解說錄影) 課程簡介和注意須知 / 課程簡介解說錄影* 每週均安排隨堂測驗,採計其中最高分的 9 次 學期評分方式: 隨堂測驗 (20%) + 個人作業+報告及專題 (30%) + 自我評分 (50%) 歷屆修課學生心得: 向景亘, 張家榮, 蕭奕凱, 方鈺學 分組報告示範: ARM-Linux, Xvisor GNU/Linux 開發工具共筆*: 務必 自主 學習 Linux 操作, Git, HackMD, LaTeX 語法 (特別是數學式), GNU make, perf, gnuplot 確認 Ubuntu Linux 22.04-LTS (或更新的版本) 已順利安裝到你的電腦中 透過 Computer Systems: A Programmer’s Perspective 學習系統軟體*: 本課程指定的教科書 (請及早購買: 天瓏書店) 軟體缺失導致的危害 1970 年代推出的首款廣體民航客機波音 747 軟體由大約 40 萬行程式碼構成,而 2011 年引進的波音 787 的軟體規模則是波音 747 的 16 倍,約 650 萬行程式碼。換言之,你我的性命緊繫於一系列極為複雜的軟體系統之中,能不花點時間了解嗎? 軟體開發的安全性設計和測試驗證應獲得更高的重視 The adoption of Rust in Business (2022) 搭配觀看短片: Rust in 100 Seconds 解讀計算機編碼 人們對數學的加減運算可輕易在腦中辨識符號並理解其結果,但電腦做任何事都受限於實體資料儲存及操作方式,換言之,電腦硬體實際只認得 0 和 1,卻不知道符號 + 和 - 在數學及應用場域的意義,於是工程人員引入「補數」以表達人們認知上的正負數 您有沒有想過,為何「二補數」(2’s complement) 被電腦廣泛採用呢?背後的設計考量是什麼?本文嘗試從數學觀點去解讀編碼背後的原理 你所不知道的 C 語言:指標篇* linked list 和非連續記憶體操作* 安排 linked list 作為第一份作業及隨堂測驗的考量點: 檢驗學員對於 C 語言指標操作的熟悉程度 (附帶思考:對於 Java 程式語言來說,該如何實作 linked list 呢?) linked list 本質上就是對非連續記憶體的操作,乍看僅是一種單純的資料結構,但對應的演算法變化多端,像是「如何偵測 linked list 是否存在環狀結構?」和「如何對 linked list 排序並確保空間複雜度為 O(1) 呢?」 linked list 的操作,例如走訪 (traverse) 所有節點,反映出 Locality of reference (cache 用語) 的表現和記憶體階層架構 (memory hierarchy) 高度相關,學員很容易從實驗得知系統的行為,從而思考其衝擊和效能改進方案 無論是作業系統核心、C 語言函式庫內部、應用程式框架,到應用程式,都不難見到 linked list 的身影,包含多種針對效能和安全議題所做的 linked list 變形,又還要考慮到應用程式的泛用性 (generic programming),是很好的進階題材 題目 1 + 分析* 題目2 / 參考題解1, 參考題解2 題目3 / 參考題解 題目4 / 參考題解 題目5 / 參考題解 佳句偶得:「大部分的人一輩子洞察力不彰,原因之一是怕講錯被笑。想了一點點就不敢繼續也沒記錄或分享,時間都花在讀書查資料看別人怎麼想。看完就真的沒有自己的洞察了」(出處) 作業: 截止繳交日: Feb 28, 2023 lab0* quiz1 第 1 週隨堂測驗: 題目 (內含作答表單) 課堂問答簡記 ","date":"2024-02-28","objectID":"/posts/linux2023/:1:1","tags":["Sysprog","Linux"],"title":"Linux 核心设计/实作 (Linux Kernel Internals)","uri":"/posts/linux2023/"},{"categories":["Linux Kernel Internals"],"content":"第 2 週: C 語言程式設計 (Feb 20, 21, 23) 教材解說* (僅止於概況,請詳閱下方教材及個別的對應解說錄影) Linux v6.2 發布: 接下來會是讓學員眼花撩亂的主版號/次版號的飛快跳躍 / kernel.org Linux: 作業系統術語及概念* 系統軟體開發思維 C 語言: 數值系統* 儘管數值系統並非 C 語言所特有,但在 Linux 核心大量存在 u8/u16/u32/u64 這樣透過 typedef 所定義的型態,伴隨著各式 alignment 存取,若學員對數值系統的認知不夠充分,可能立即就被阻擋在探索 Linux 核心之外 —— 畢竟你完全搞不清楚,為何在 Linux 核心存取特定資料需要繞一大圈。 C 語言: Bitwise 操作* Linux 核心原始程式碼存在大量 bit(-wise) operations (簡稱 bitops),頗多乍看像是魔法的 C 程式碼就是 bitops 的組合 類神經網路的 ReLU 及其常數時間複雜度實作 從 √2 的存在談開平方根的快速運算 Linux 核心的 hash table 實作 為什麼要深入學習 C 語言?* C 語言發明者 Dennis M. Ritchie 說:「C 很彆扭又缺陷重重,卻異常成功。固然有歷史的巧合推波助瀾,可也的確是因為它能滿足於系統軟體實作的程式語言期待:既有相當的效率來取代組合語言,又可充分達到抽象且流暢,能用於描述在多樣環境的演算法。」 Linux 核心作為世界上最成功的開放原始碼計畫,也是 C 語言在工程領域的瑰寶,裡頭充斥各式「藝術」,往往會嚇到初次接觸的人們,但總是能夠用 C 語言標準和開發工具提供的擴展 (主要來自 gcc 的 GNU extensions) 來解釋。 基於 C 語言標準研究與系統程式安全議題 藉由研讀漏洞程式碼及 C 語言標準,討論系統程式的安全議題 透過除錯器追蹤程式碼實際運行的狀況,了解其運作原理; 取材自 dangling pointer, CWE-416 Use After Free, CVE-2017-16943 以及 integer overflow 的議題; C 語言:記憶體管理、對齊及硬體特性* 搭配閱讀: The Lost Art of Structure Packing 從虛擬記憶體談起,歸納出現代銀行和虛擬記憶體兩者高度相似: malloc 給出 valid pointer 不要太高興,等你要開始用的時候搞不好作業系統給個 OOM ——簡單來說就是一張支票,能不能拿來開等到兌現才知道。 探討 heap (動態配置產生,系統會存放在另外一塊空間)、data alignment,和 malloc 實作機制等議題。這些都是理解 Linux 核心運作的關鍵概念。 C 語言: bit-field bit field 是 C 語言一個很被忽略的特徵,但在 Linux 和 gcc 這類系統軟體很常出現,不僅是精準規範每個 bit 的作用,甚至用來「擴充」C 語言 參考題目 / 參考題目* / 參考題解 1, 參考題解 2, 參考題解 3 作業: 截止繳交日 Mar 7 quiz2 第 2 週隨堂測驗: 題目 (內含作答表單) 課堂問答簡記 ","date":"2024-02-28","objectID":"/posts/linux2023/:1:2","tags":["Sysprog","Linux"],"title":"Linux 核心设计/实作 (Linux Kernel Internals)","uri":"/posts/linux2023/"},{"categories":["Linux Kernel Internals"],"content":"第 3 週: 並行和 C 語言程式設計 (Feb 27, 28, Mar 2) 教材解說* (僅止於概況,請詳閱下方教材及個別的對應解說錄影) 公告 2 月 28 日沒有實體課程,但安排線上測驗 (「Linux 核心設計」課程的學員務必參加),在 15:20-23:59 之間依據 Google Calendar 進行作答 第二次作業已指派,可在 2 月 28 日晚間起開始繳交,截止繳交日 Mar 7 3 月 1 日晚間安排第一次作業的檢討直播 (事後有錄影),請參見 Google Calendar Linux: 發展動態回顧* 從 Revolution OS 看作業系統生態變化* 並行和多執行緒程式設計*: 應涵蓋 Part 1 到 Part 4 Part 1: 概念、执行顺序 Part 2 Part 3 Part 4 C 語言: 函式呼叫* 著重在計算機架構對應的支援和行為分析 C 語言: 遞迴呼叫* 或許跟你想像中不同,Linux 核心的原始程式碼裡頭也用到遞迴函式呼叫,特別在較複雜的實作,例如檔案系統,善用遞迴可大幅縮減程式碼,但這也導致追蹤程式運作的難度大增 C 語言: 前置處理器應用* C 語言之所以不需要時常發佈新的語言特徵又可以保持活力,前置處理器 (preprocessor) 是很重要的因素,有心者可逕行「擴充」C 語言 C 語言: goto 和流程控制* goto 在 C 語言被某些人看做是妖魔般的存在,不過實在不用這樣看待,至少在 Linux 核心原始程式碼中,goto 是大量存在 (跟你想像中不同吧)。有時不用 goto 會寫出更可怕的程式碼 C 語言程式設計技巧* 作業: 截止繳交日: Mar 21 fibdrv* quiz3 review* Week3 隨堂測驗: 題目 (內含作答表單) ","date":"2024-02-28","objectID":"/posts/linux2023/:1:3","tags":["Sysprog","Linux"],"title":"Linux 核心设计/实作 (Linux Kernel Internals)","uri":"/posts/linux2023/"},{"categories":["Linux Kernel Internals"],"content":"第 4 週: 數值系統 + 編譯器 (Mar 6, 7, 9) 教材解說* (僅止於概況,請詳閱下方教材及個別的對應解說錄影) 公告: 請填寫 Google 表單,以利後續追蹤 《Demystifying the Linux CPU Scheduler》的書稿已寄送給成功大學的選課學生,旁聽的學員預計在 3 月 13 日取得 (第 5 週進度) 貢獻程式碼到 Linux 核心 第一次給 Linux Kernel 發 patch 提交第一份 Patch 到 Linux Kernel 第一次發 patch 到 LKML 追求神乎其技的程式設計之道 「可以看出抄襲風氣在台灣並不只是小時候在學校抄抄作業而已;媒體工作者在報導中任意抄襲及轉載是種不尊重自己專業的表現,不但隱含著一種應付了事的心態,更代表著這些人對於自己的工作沒有熱情,更沒有著一點堅持。如果要說我在美國看到這邊和台灣有什麼最大的不同,我想關鍵的差異就在對自己的工作有沒有熱情和堅持而已了。」 「程式藝術家也不過是在『簡潔』、『彈性』、『效率』這三大目標上進行一連串的取捨 (trade-off) 和最佳化。」 Linux 核心的紅黑樹 CS:APP 第 2 章重點提示和練習* 核心開發者當然要熟悉編譯器行為 Linus Torvalds 教你分析 gcc 行為 Pointers are more abstract than you might expect in C / HackerNews 討論 C 編譯器原理和案例分析* C 語言: 未定義行為*: C 語言最初為了開發 UNIX 和系統軟體而生,本質是低階的程式語言,在語言規範層級存在 undefined behavior,可允許編譯器引入更多最佳化 C 語言: 編譯器和最佳化原理* 《Demystifying the Linux CPU Scheduler》第 1 章 作業: 截止繳交日: Mar 30 quiz4 Week4 隨堂測驗: 題目 (內含作答表單) 課堂問答簡記 第 5 週 (Mar 13, 14, 16): Linux CPU scheduler 教材解說* (僅止於概況,請詳閱下方教材及個別的對應解說錄影) 公告: 本週導入客製化作業,讓學員選擇改進前四週的作業或自訂題目 (例如貢獻程式碼到 Linux 核心),隨後安排授課教師和學員的線上一對一討論 浮點數運算*: 工程領域往往是一系列的取捨結果,浮點數更是如此,在軟體發開發有太多失誤案例源自工程人員對浮點數運算的掌握不足,本議程希望藉由探討真實世界的血淋淋案例,帶著學員思考 IEEE 754 規格和相關軟硬體考量點,最後也會探討在深度學習領域為了改善資料處理效率,而引入的 BFloat16 這樣的新標準 float16 vs. bfloat16 記憶體配置器涉及 bitwise 操作及浮點數運算。傳統的即時系統和該領域的作業系統 (即 RTOS) 為了讓系統行為更可預測,往往捨棄動態記憶體配置的能力,但這顯然讓系統的擴充能力大幅受限。後來研究人員提出 TLSF (Two-Level Segregated Fit) 嘗試讓即時系統也能享用動態記憶體管理,其關鍵訴求是 “O(1) cost for malloc, free, realloc, aligned_alloc” Benchmarking Malloc with Doom 3 tlsf-bsd TLSF: Part 1: Background, Part 2: The floating point Linux 核心模組運作原理 Linux: 不只挑選任務的排程器*: 排程器 (scheduler) 是任何一個多工作業系統核心都具備的機制,但彼此落差極大,考量點不僅是演算法,還有當應用規模提昇時 (所謂的 scalability) 和涉及即時處理之際,會招致不可預知的狀況 (non-determinism),不僅即時系統在意,任何建構在 Linux 核心之上的大型服務都會深受衝擊。是此,Linux 核心的排程器經歷多次變革,需要留意的是,排程的難度不在於挑選下一個可執行的行程 (process),而是讓執行完的行程得以安插到合適的位置,使得 runqueue 依然依據符合預期的順序。 C 語言: 動態連結器* C 語言: 連結器和執行檔資訊* C 語言: 執行階段程式庫 (CRT)* 作業: 截止繳交 Apr 10 assessment Week5 隨堂測驗: 題目 (內含作答表單) 課堂問答簡記 第 6 週 (Mar 20, 21, 23): System call + CPU Scheduler 教材解說* (僅止於概況,請詳閱下方教材及個別的對應解說錄影) 公告 自 3 月 22 日起,開放讓學員 (選課的學生 + 完成前二次作業過半要求的旁聽者) 跟授課教師預約一對一線上討論,請參照課程行事曆裡頭標注 “Office hour” 的時段,發訊息到 Facebook 粉絲專頁,簡述你的學習狀況並選定偏好的時段 (建議是 30 分鐘)。留意課程發送的公告信件 選修課程的學員在本學期至少要安排一次一對一討論,否則授課教師難以評估學習狀況,從而會影響評分,請重視自己的權益。 coroutine Linux: 賦予應用程式生命的系統呼叫 vDSO: 快速的 Linux 系統呼叫機制 UNIX 作業系統 fork/exec 系統呼叫的前世今生 《Demystifying the Linux CPU Scheduler》 1.2.1 System calls 1.2.2 A different kind of software 1.2.3 User and kernel stacks 1.3 Process management 2.1 Introduction 2.2 Prior to CFS 2.3 Completely Fair Scheduler (CFS) 3.1 Structs and their role 作業: 截止繳交 Apr 17 quiz5, quiz6 Week6 隨堂測驗: 題目 (內含作答表單) 課堂問答簡記 第 7 週 (Mar 27, 28, 30): Process, 並行和多執行緒 教材解說-1*, 教材解說-2* (僅止於概況,請詳閱下方教材及個別的對應解說錄影) 公告: 第 5 次作業 和 第 6 次作業 作業已指派 本週測驗順延到 4 月 4 日和 4 月 6 日,3 月 30 日晚間安排課程講解 4 月 3 日晚間依舊講課 (事後有錄影)、4 月 4 日下午到晚間安排在家測驗,4 月 6 日晚間安排測驗 Linux: 不僅是個執行單元的 Process*: Linux 核心對於 UNIX Process 的實作相當複雜,不僅蘊含歷史意義 (幾乎每個欄位都值得講古),更是反映出資訊科技產業的變遷,核心程式碼的 task_struct 結構體更是一絕,廣泛涵蓋 process 狀態、處理器、檔案系統、signal 處理、底層追蹤機制等等資訊,更甚者,還很曖昧地保存著 thread 的必要欄位,好似這兩者天生就脫不了干係 探討 Linux 核心設計的特有思維,像是如何透過 LWP 和 NPTL 實作執行緒,又如何透過行程建立記憶體管理的一種抽象層,再者回顧行程間的 context switch 及排程機制,搭配 signal 處理 測試 Linux 核心的虛擬化環境 建構 User-Mode Linux 的實驗環境* 〈Concurrency Primer〉導讀 The C11 and C++11 Concurrency Model Time to move to C11 atomics? C11 atomic variables and the kernel C11 atomics part 2: “consume” semantics An introduction to lockless algorithms 並行和多執行緒程式設計* CS:APP 第 12 章 Concurrency / 錄影* Synchronization: Basic / 錄影* Synchronization: Advanced / 錄影* Thread-Level Parallelism / 錄影* 課堂問答簡記 第 8 週 (Apr 3, 4, 6): 並行程式設計, lock-free, Linux 同步機制 教材解說* (僅止於概況,請詳閱下方教材及個別的對應解說錄影) 公告: 4 月 4 日下午到晚間安排在家測驗,請在當日 15:00 刷新課程進度表/行事曆,以得知測驗方式 4 月 6 日晚間安排測驗 並行和多執行緒程式設計,涵蓋 Atomics 操作 POSIX Threads (請對照 CS:APP 第 12 章自行學習) Lock-free 程式設計 案例: Hazard pointer 案例: Ring buffer 案例: Thread Pool Linux: 淺談同步機制* 利用 lkm 來變更特定 Linux 行程的內部狀態 Week8 隨堂測驗: 題目 (內含作答表單) 第 9 週 (Apr 10, 11, 13): futex, RCU, 伺服器開發與 Linux 核心對應的系統呼叫 教材解說* (僅止於概況,請詳閱下方教材及個別的對應解說錄影) 第二次作業檢討 公告: 請於 4 月 14 日 10:00PM 刷新本頁面,以得知新指派的作業 4 月 13 日晚間安排課程測驗和","date":"2024-02-28","objectID":"/posts/linux2023/:1:4","tags":["Sysprog","Linux"],"title":"Linux 核心设计/实作 (Linux Kernel Internals)","uri":"/posts/linux2023/"},{"categories":null,"content":"ccrysisa's friends","date":"2024-04-29","objectID":"/friends/","tags":null,"title":"友情链接","uri":"/friends/"},{"categories":null,"content":" Reminder ","date":"2024-04-29","objectID":"/friends/:0:0","tags":null,"title":"友情链接","uri":"/friends/"},{"categories":null,"content":"Base info - nickname: Lruihao avatar: https://lruihao.cn/images/avatar.jpg url: https://lruihao.cn description: Lruihao's Note ","date":"2024-04-29","objectID":"/friends/:1:0","tags":null,"title":"友情链接","uri":"/friends/"},{"categories":null,"content":"Friendly Reminder Notice If you want to exchange link, please leave a comment in the above format. (personal non-commercial blogs / websites only) Website failure, stop maintenance and improper content may be unlinked! Those websites that do not respect other people’s labor achievements, reprint without source, or malicious acts, please do not come to exchange. ","date":"2024-04-29","objectID":"/friends/:2:0","tags":null,"title":"友情链接","uri":"/friends/"},{"categories":["Rust"],"content":" Rust is a statically compiled, fast language with great tooling and a rapidly growing ecosystem. That makes it a great fit for writing command line applications: They should be small, portable, and quick to run. Command line applications are also a great way to get started with learning Rust; or to introduce Rust to your team! 整理自 Command line apps in Rust ","date":"2024-04-29","objectID":"/posts/rust-cli/:0:0","tags":["Rust","CLI"],"title":"Command Line Applications in Rust","uri":"/posts/rust-cli/"},{"categories":["Rust"],"content":"重点提示 ","date":"2024-04-29","objectID":"/posts/rust-cli/:1:0","tags":["Rust","CLI"],"title":"Command Line Applications in Rust","uri":"/posts/rust-cli/"},{"categories":["Rust"],"content":"Arguments C 语言的 CLI 程序处理参数的逻辑是过程式的,即每次执行都会通过 argv 来获取本次执行的参数并进行相应的处理 (Rust 的 std::env::args() 处理 CLI 程序的参数方式也类似,都是对每次执行实例进行过程式的处理),而 Clap 不同,它类似于面向对象的思想,通过定义一个结构体 (object),每次运行时通过 clap::Parser::parse 获取并处理本次运行的参数 (即实例化 object),这样开发的 CLI 程序扩展性会更好。 ","date":"2024-04-29","objectID":"/posts/rust-cli/:1:1","tags":["Rust","CLI"],"title":"Command Line Applications in Rust","uri":"/posts/rust-cli/"},{"categories":["Rust"],"content":"BufReader Struct std::io::BufReader 中关于系统调用 (syscall) 的开销,以及如何使用 buffer 这一机制减少 syscall 调用以此提高效能,进行了比较直观的描述: It can be excessively inefficient to work directly with a Read instance. For example, every call to read on TcpStream results in a system call. A BufReader performs large, infrequent reads on the underlying Read and maintains an in-memory buffer of the results. BufReader can improve the speed of programs that make small and repeated read calls to the same file or network socket. It does not help when reading very large amounts at once, or reading just one or a few times. It also provides no advantage when reading from a source that is already in memory, like a Vec. When the BufReader is dropped, the contents of its buffer will be discarded. Creating multiple instances of a BufReader on the same stream can cause data loss. Reading from the underlying reader after unwrapping the BufReader with BufReader::into_inner can also cause data loss. ","date":"2024-04-29","objectID":"/posts/rust-cli/:1:2","tags":["Rust","CLI"],"title":"Command Line Applications in Rust","uri":"/posts/rust-cli/"},{"categories":["Rust"],"content":"Documentations 这里列举视频中一些概念相关的 documentation 学习的一手资料是官方文档,请务必自主学会阅读规格书之类的资料 ","date":"2024-04-29","objectID":"/posts/rust-cli/:2:0","tags":["Rust","CLI"],"title":"Command Line Applications in Rust","uri":"/posts/rust-cli/"},{"categories":["Rust"],"content":"Crate std 可以使用这里提供的搜素栏进行搜索 (BTW 不要浪费时间在 Google 搜寻上!) Function std::fs::read_to_string Function std::env::args Struct std::path::PathBuf Struct std::io::BufReader method std::iter::Iterator::nth Primitive Type str: method str::lines method str::contains expect: method std::option::Option::expect method std::result::Result::expect ","date":"2024-04-29","objectID":"/posts/rust-cli/:2:1","tags":["Rust","CLI"],"title":"Command Line Applications in Rust","uri":"/posts/rust-cli/"},{"categories":["Rust"],"content":"Crate clap method clap::Parser::parse ","date":"2024-04-29","objectID":"/posts/rust-cli/:2:2","tags":["Rust","CLI"],"title":"Command Line Applications in Rust","uri":"/posts/rust-cli/"},{"categories":["Rust"],"content":"References","date":"2024-04-29","objectID":"/posts/rust-cli/:3:0","tags":["Rust","CLI"],"title":"Command Line Applications in Rust","uri":"/posts/rust-cli/"},{"categories":["C","Linux Kernel Internals"],"content":" AMaCC 是由成功大學師生開發的 self-compiling 的 C 語言編譯器,可產生 Arm 架構的執行檔 (ELF 格式,運作在 GNU/Linux)、也支援 just-in-time (JIT) 編譯和執行,原始程式碼僅 1500 行,在這次講座中,我們就來揭開 AMaCC 背後的原理和實作議題。 預期會接觸到 IR (Intermediate representation), dynamic linking, relocation, symbol table, parsing tree, language frontend, Arm 指令編碼和 ABI 等等。 原文地址 ","date":"2024-04-23","objectID":"/posts/c-compiler-construction/:0:0","tags":["Sysprog","C","Compiler"],"title":"你所不知道的 C 语言: 编译器原理和案例分析","uri":"/posts/c-compiler-construction/"},{"categories":["C","Linux Kernel Internals"],"content":"如何打造一个具体而微的 C 语言编译器 用十分鐘 向 jserv 學習作業系統設計 用1500 行建構可自我編譯的 C 編譯器 / 投影片 Wikipedia: Executable and Linkable Format ","date":"2024-04-23","objectID":"/posts/c-compiler-construction/:1:0","tags":["Sysprog","C","Compiler"],"title":"你所不知道的 C 语言: 编译器原理和案例分析","uri":"/posts/c-compiler-construction/"},{"categories":["C","Linux Kernel Internals"],"content":"编译器和软件工业强度息息相关 形式化驗證 (Formal Verification) ","date":"2024-04-23","objectID":"/posts/c-compiler-construction/:2:0","tags":["Sysprog","C","Compiler"],"title":"你所不知道的 C 语言: 编译器原理和案例分析","uri":"/posts/c-compiler-construction/"},{"categories":["C","Linux Kernel Internals"],"content":"背景知识 你所不知道的 C 语言: 编译器和最佳化原理篇 你所不知道的 C 语言: 函数呼叫篇 你所不知道的 C 语言: 动态连链接器和执行时期篇 虚拟机器设计与实作 ","date":"2024-04-23","objectID":"/posts/c-compiler-construction/:3:0","tags":["Sysprog","C","Compiler"],"title":"你所不知道的 C 语言: 编译器原理和案例分析","uri":"/posts/c-compiler-construction/"},{"categories":["C","Linux Kernel Internals"],"content":"C 程序的解析和语意 手把手教你构建 C 语言编译器 descent 點評幾本編譯器設計書籍 desent 教你逐步開發編譯器 c4 是很好的切入點,原作者 Robert Swierczek 還又另一個 更完整的 C 編譯器實作,这个实作支持 preprocessor AMaCC 在 Robert Swierczek 的基礎上,額外實作 C 語言的 struct, switch-case, for, C-style comment 支援,並且重寫了 IR 執行程式碼,得以輸出合法 GNU/Linux ELF 執行檔 (支援 armhf ABI) 和 JIT 編譯 徒手写一个 RISC-V 编译器!初学者友好的实战课程 注意 上面的第一个链接是关于 c4 的教程,非常值得一看和一做 (Make your hands dirty!),同时它也是 AMaCC 的基础 (AMaCC 在这个基础上进行了重写和扩展)。 最后一个关于虚拟机器的讲座也不错,对于各类虚拟机器都进行了介绍和说明,并搭配了相关实作 RTMux 进行了讲解。 ","date":"2024-04-23","objectID":"/posts/c-compiler-construction/:4:0","tags":["Sysprog","C","Compiler"],"title":"你所不知道的 C 语言: 编译器原理和案例分析","uri":"/posts/c-compiler-construction/"},{"categories":["C","Linux Kernel Internals"],"content":"手把手教你构建 C 语言编译器 原文地址 ","date":"2024-04-23","objectID":"/posts/c-compiler-construction/:4:1","tags":["Sysprog","C","Compiler"],"title":"你所不知道的 C 语言: 编译器原理和案例分析","uri":"/posts/c-compiler-construction/"},{"categories":["C","Linux Kernel Internals"],"content":"延伸阅读 Wikipedia: Stack machine Wikipedia: Compiler-compiler ","date":"2024-04-23","objectID":"/posts/c-compiler-construction/:4:2","tags":["Sysprog","C","Compiler"],"title":"你所不知道的 C 语言: 编译器原理和案例分析","uri":"/posts/c-compiler-construction/"},{"categories":["C","Linux Kernel Internals"],"content":"IR (Intermediate representation) Interpreter, Compiler, JIT from scratch How to JIT - an introduction How to write a very simple JIT compiler How to write a UNIX shell, with a lot of background 注意 JIT (Just in time) 表示“即时”,形象描述就是“及时雨” 🤣 原理是将解释执行的“热点“编译成位于一个内存区域的 machine code,从而减轻内存的压力 (解释执行时会在内存中跳来跳去,而一个区域的 machine code 是连续执行,内存压力没这么大并且可以充分利用 cache 从而提高效能)。 最后两个链接对于提高系统编程 (System programming) 能力非常有益,Just do it! ","date":"2024-04-23","objectID":"/posts/c-compiler-construction/:5:0","tags":["Sysprog","C","Compiler"],"title":"你所不知道的 C 语言: 编译器原理和案例分析","uri":"/posts/c-compiler-construction/"},{"categories":["C","Linux Kernel Internals"],"content":"延伸阅读 Linux manual page: bsearch ","date":"2024-04-23","objectID":"/posts/c-compiler-construction/:5:1","tags":["Sysprog","C","Compiler"],"title":"你所不知道的 C 语言: 编译器原理和案例分析","uri":"/posts/c-compiler-construction/"},{"categories":["C","Linux Kernel Internals"],"content":"程序语言设计和编译器考量 YouTube: Brian Kernighan on successful language design ","date":"2024-04-23","objectID":"/posts/c-compiler-construction/:6:0","tags":["Sysprog","C","Compiler"],"title":"你所不知道的 C 语言: 编译器原理和案例分析","uri":"/posts/c-compiler-construction/"},{"categories":["Toolkit"],"content":"今日闲来无事,刷了会 B 站,突发奇想将发灰了一年多的 WSL2 找回来折腾一下 (之前为了在 VMware 中开启嵌套虚拟化,不得以将 WSL2 打入冷宫 🤣)。由于这一年内功功力大涨,很快就完成了 WLS2 的再召集启用,下面列出配置的主要步骤和相关材料。 操作系统: Windows 10 ","date":"2024-04-20","objectID":"/posts/wsl2/:0:0","tags":["Windows","WSL","Ubuntu"],"title":"Windows 10 WSL2 Ubuntu 22.04 配置指南","uri":"/posts/wsl2/"},{"categories":["Toolkit"],"content":"安装 WSL2 和 Linux 发行版 启用或关闭 Windows 功能 适用于 Linux 的 Windows 子系统 虚拟机平台 以管理员身份运行 PowerShell \u003e bcdedit /v ... hypervisorlaunchtype Auto # 保证上面这个虚拟化选项是 Auto,如果是 Off 则使用下面命令设置 \u003e bcdedit /set hypervisorlaunchtype auto 在 PowerShell 中安装 wsl2 \u003e wsl --update \u003e wsl --set-default-version 2 \u003e wsl -l -o NAME FRIENDLY NAME ... Ubuntu-18.04 Ubuntu 18.04 LTS Ubuntu-20.04 Ubuntu 20.04 LTS Ubuntu-22.04 Ubuntu 22.04 LTS ... \u003e wsl --install Ubuntu-22.04 # 后面需要创建用户和密码,自行设置 上面以 Ubuntu2 22.04 发行版为例,你也可以安装其它的发行版。安装过程中可能会出现无法访问源 / 仓库的问题,这个是网络问题,请自行通过魔法/科学方法解决 ","date":"2024-04-20","objectID":"/posts/wsl2/:1:0","tags":["Windows","WSL","Ubuntu"],"title":"Windows 10 WSL2 Ubuntu 22.04 配置指南","uri":"/posts/wsl2/"},{"categories":["Toolkit"],"content":"迁移至非系统盘 以管理员身份运行 PowerShell # 查看已安装的 Linux 发行版 \u003e wsl -l- v # 停止正在运行的发行版 \u003e wsl --shutdown # 导出发行版的镜像 (以 Ubuntu 22.04 为例) \u003e wsl --export Ubuntu-22.04 D:/ubuntu.tar # 导出镜像后,卸载原有发行版以释放 C 盘空间 \u003e wsl --unregister Ubuntu-22.04 # 重新导入发行版镜像。并指定该子系统储存的目录 (即进行迁移) \u003e wsl --import Ubuntu-22.04 D:\\Ubuntu\\ D:\\ubuntu.tar --version 2 # 上面命令完成后,在目录 D:\\Ubuntu 下会出现名为 ext4.vhdx 的文件,这个就是子系统的虚拟磁盘 # 设置启用子系统时的默认用户 (建议使用迁移前创建的用户),否则启动子系统时进入的是 root 用户 \u003e ubuntu-22.04.exe config --default-user \u003cusername\u003e ","date":"2024-04-20","objectID":"/posts/wsl2/:2:0","tags":["Windows","WSL","Ubuntu"],"title":"Windows 10 WSL2 Ubuntu 22.04 配置指南","uri":"/posts/wsl2/"},{"categories":["Toolkit"],"content":"Windows Terminal 美化 ","date":"2024-04-20","objectID":"/posts/wsl2/:3:0","tags":["Windows","WSL","Ubuntu"],"title":"Windows 10 WSL2 Ubuntu 22.04 配置指南","uri":"/posts/wsl2/"},{"categories":["Toolkit"],"content":"其它 目前 Windows 10 上的 WSL2 应该都支持 WSLg (如果你一直更新的话),可以使用 gedit 来测试一下 WLSg 的功能,可以参考微软的官方文档: https://github.com/microsoft/wslg ","date":"2024-04-20","objectID":"/posts/wsl2/:4:0","tags":["Windows","WSL","Ubuntu"],"title":"Windows 10 WSL2 Ubuntu 22.04 配置指南","uri":"/posts/wsl2/"},{"categories":["Toolkit"],"content":"效果展示 ","date":"2024-04-20","objectID":"/posts/wsl2/:5:0","tags":["Windows","WSL","Ubuntu"],"title":"Windows 10 WSL2 Ubuntu 22.04 配置指南","uri":"/posts/wsl2/"},{"categories":["CSAPP","Linux Kernel Internals"],"content":" 千万不要小看数值系统,史上不少有名的 软体缺失案例 就因为开发者未能充分掌握相关议题,而导致莫大的伤害与损失。 原文地址 搭配 CMU: 15-213: Intro to Computer Systems: Schedule for Fall 2015 可以在 这里 找到相关的投影片和录影 B 站上有一个汉化版本的 录影 ","date":"2024-04-19","objectID":"/posts/csapp-ch2/:0:0","tags":["Sysprog","CSAPP"],"title":"CS:APP 第 2 章重点提示和练习","uri":"/posts/csapp-ch2/"},{"categories":["CSAPP","Linux Kernel Internals"],"content":"数值系统 ","date":"2024-04-19","objectID":"/posts/csapp-ch2/:1:0","tags":["Sysprog","CSAPP"],"title":"CS:APP 第 2 章重点提示和练习","uri":"/posts/csapp-ch2/"},{"categories":["CSAPP","Linux Kernel Internals"],"content":"导读 YouTube: 十进制,十二进制,六十进制从何而来? YouTube: 老鼠和毒药问题怎么解?二进制和易经八卦有啥关系? YouTube: 小精靈遊戲中的幽靈是怎麼追蹤人的? 鮮為人知的 bug 解读计算机编码 你所不知道的 C 语言: 未定义/未指定行为篇 你所不知道的 C 语言: 数值系统篇 基于 C 语言标准研究与系统程式安全议题 熟悉浮点数每个位的表示可以获得更大的最佳化空间 Faster arithmetic by flipping signs Faster floating point arithmetic with Exclusive OR 看了上面的第 3 个影片后,对 pac-man 256 莫名感兴趣 🤣 ","date":"2024-04-19","objectID":"/posts/csapp-ch2/:1:1","tags":["Sysprog","CSAPP"],"title":"CS:APP 第 2 章重点提示和练习","uri":"/posts/csapp-ch2/"},{"categories":["CSAPP","Linux Kernel Internals"],"content":"Bits, Bytes \u0026 Integers 信息 第一部分录影 ✅ / 投影片 ✅ / 阅读章节: 2.1 ✅ 信息 第二部分录影 ✅ / 投影片 ✅ / 阅读章节: 2.2-2.3 计算乘法至多需要多少位可以从无符号数和二补数的编码方式来思考。无符号数乘法最大值为 $2^{2w}-2^{2+1}+1$ 不超过 $2^{2w}$,依据无符号数编码方式至多需要 $2w$ bits 表示;二补数乘法最小值为 $-2^{2w-2}+2^{w-1}$,依据而二补数编码 MSB 表示值 $-2^{2w-2}$,所以 MSB 为第 $2w-2$ 位,至多需要 $2w-1$ bits 表示二补数乘法的最小值;二补数乘法最大值为 $2^{2w-2}$,因为 MSB 为符号位,所以 MSB 的右一位表示值 $2^{2w-2}$,即第 $2w-2$ 位,所以至多需要 $2w$ 位来表示该值 (因为还需要考虑一个符号位)。 ","date":"2024-04-19","objectID":"/posts/csapp-ch2/:1:2","tags":["Sysprog","CSAPP"],"title":"CS:APP 第 2 章重点提示和练习","uri":"/posts/csapp-ch2/"},{"categories":["Linux","Linux Kernel Internals"],"content":" Linux 核心原始程式碼中,許多地方出現紅黑樹的蹤影,例如:hr_timer 使用紅黑樹來記錄計時器 (timer) 端發出的要求、ext3 檔案系統使用紅黑樹來追蹤目錄內容變更,以及 CFS (Completely Fair Scheduler) 這個 Linux 預設 CPU 排程器,由於需要頻繁地插入跟移除節點 (任務),因此開發者選擇用紅黑樹 (搭配一些效能調整)。VMA(Virtual Memory Area)也用紅黑樹來紀錄追蹤頁面 (page) 變更,因為後者不免存在頻繁的讀取 VMA 結構,如 page fault 和 mmap 等操作,且當大量的已映射 (mapped) 區域時存在時,若要尋找某個特定的虛擬記憶體地址,鏈結串列 (linked list) 的走訪成本過高,因此需要一種資料結構以提供更有效率的尋找,於是紅黑樹就可勝任。 原文地址 ","date":"2024-04-12","objectID":"/posts/linux-rbtree/:0:0","tags":["Sysprog","Linux","Red Black Tree"],"title":"Linux 核心的红黑树","uri":"/posts/linux-rbtree/"},{"categories":["Linux","Linux Kernel Internals"],"content":"简述红黑树 Left-Leaning Red-Black Trees: 论文 by Robert Sedgewick 投影片 by Robert Sedgewick 解说录影: Left Leaning Red Black Trees (Part 1) Left Leaning Red Black Trees (Part 2) 2-3-4 tree: Problem: Doesn’t work if parent is a 4-node 为解决该问题,投影片主要说明的是 Split 4-nodes on the way down 方法,这个方法的逻辑是:在插入节点前向下走访的过程中,如果发现某个节点是 4-nodes 则对该节点进行 split 操作,具体例子可以参考投影片的 P24 ~ P25。 LLRB tree: Problem: Doesn’t work if parent is a 4-node 为解决该问题,投影片主要说明的也是 Split 4-nodes on the way down 方法,其逻辑和之前相同,除此之外,在插入节点后向上返回的过程中,进行 rotate 操作,保证了 LLRB 节点的结构满足要求 (即红边的位置)。 技巧 在拆分 4-node 时 3-node 和 4-node 的孩子节点的大小关系不太直观,这时可以参考解说录影的老师的方法,使用数字或字母标识节点,进而可以直观看出 4-node 转换前后的等价性。 如果我们将 4-node 节点的拆分放在插入节点后向上返回的过程进行处理,则会将原本的树结构转换成 2-3 tree,因为这样插入节点后,不会保留有 4-node (插入产生的 4-node 立刻被拆分)。 注意 红黑树的 perfect-balance 的特性在于:它随着节点的增多而逐渐将 4-nodes (因为新增节点都是 red 边,所以叶节点很大概率在插入结束后会是 4-node) 从根节点方向移动 (on the way down 时 split 4-nodes 的效果),当 4-node 移动到根节点时,进行颜色反转并不会破坏树的平衡,只是树高加 1 (这很好理解,因为根节点是唯一的,只要保证 4-node 的根节点拆分操作保持平衡即可,显然成立)。 ","date":"2024-04-12","objectID":"/posts/linux-rbtree/:1:0","tags":["Sysprog","Linux","Red Black Tree"],"title":"Linux 核心的红黑树","uri":"/posts/linux-rbtree/"},{"categories":["Linux","Linux Kernel Internals"],"content":"Maple tree 解说录影: The Linux Maple Tree - Matthew Wilcox, Oracle ","date":"2024-04-12","objectID":"/posts/linux-rbtree/:2:0","tags":["Sysprog","Linux","Red Black Tree"],"title":"Linux 核心的红黑树","uri":"/posts/linux-rbtree/"},{"categories":["Concurrency","Linux Kernel Internals"],"content":" 原文地址 ","date":"2024-04-10","objectID":"/posts/posix-threads/:0:0","tags":["Sysprog","Linux","Concurrency"],"title":"并行程序设计: POSIX Threads","uri":"/posts/posix-threads/"},{"categories":["Concurrency","Linux Kernel Internals"],"content":"Process vs. Thread vs. Coroutines With threads, the operating system switches running tasks preemptively according to its scheduling algorithm. With coroutines, the programmer chooses, meaning tasks are cooperatively multitasked by pausing and resuming functions at set points. coroutine switches are cooperative, meaning the programmer controls when a switch will happen. The kernel is not involved in coroutine switches. 一图胜千语: 具体一点,从函数执行流程来看: $\\rightarrow$ 在使用 coroutinues 后执行流程变成 $\\rightarrow$ ","date":"2024-04-10","objectID":"/posts/posix-threads/:1:0","tags":["Sysprog","Linux","Concurrency"],"title":"并行程序设计: POSIX Threads","uri":"/posts/posix-threads/"},{"categories":["Concurrency","Linux Kernel Internals"],"content":"Thread \u0026 Process Wikipedia: Light-weight process On Linux, user threads are implemented by allowing certain processes to share resources, which sometimes leads to these processes to be called “light weight processes”. Wikipedia: Thread-local storage On a modern machine, where multiple threads may be modifying the errno variable, a call of a system function on one thread may overwrite the value previously set by a call of a system function on a different thread, possibly before following code on that different thread could check for the error condition. The solution is to have errno be a variable that looks as if it is global, but is physically stored in a per-thread memory pool, the thread-local storage. ","date":"2024-04-10","objectID":"/posts/posix-threads/:1:1","tags":["Sysprog","Linux","Concurrency"],"title":"并行程序设计: POSIX Threads","uri":"/posts/posix-threads/"},{"categories":["Concurrency","Linux Kernel Internals"],"content":"PThread (POSIX threads) POSIX 的全称是 Portable Operating System Interfaces,结合上图,所以你明白 pthread 的 P 代表的意义了吗? Answer 从 CPU 厂商群魔乱舞中诞生的标准,自然是要保证可移植 Portable 的啦 🤣 下面的这个由 Lawrence Livermore National Laboratory 撰写的教程文档写的非常棒,值得一读 (他们还有关于 HPC 高性能计算的相关教程文档): POSIX Threads Programming ","date":"2024-04-10","objectID":"/posts/posix-threads/:1:2","tags":["Sysprog","Linux","Concurrency"],"title":"并行程序设计: POSIX Threads","uri":"/posts/posix-threads/"},{"categories":["Concurrency","Linux Kernel Internals"],"content":"Synchronizing Threads 3 basic synchronization primitives (为什么是这 3 个?请从 synchronized-with 关系进行思考) mutex locks condition variables semaphores 取材自 Ching-Kuang Shene 教授的讲义: Part IV Other Systems: IIIPthreads: A Brief Review Conditions in Pthreads are usually used with a mutex to enforce mutual exclusion. mutex locks pthread_mutex_t mutex = PTHREAD_MUTEX_INITIALIZER; int pthread_mutex_init(pthread_mutex_t *mutex, pthread_mutexattr_t *attr); int pthread_mutex_destroy(pthread_mutex_t *mutex); int pthread_mutex_lock(pthread_mutex_t *mutex); int pthread_mutex_unlock(pthread_mutex_t *mutex); int pthread_mutex_trylock(pthread_mutex_t *mutex); Only the owner can unlock a mutex. Since mutexes cannot be copied, use pointers. If pthread_mutex_trylock() returns EBUSY, the lock is already locked. Otherwise, the calling thread becomes the owner of this lock. With pthread_mutexattr_settype(), the type of a mutex can be set to allow recursive locking or report deadlock if the owner locks again 注意 单纯的 Mutex 无法应对复杂情形的「生产者-消费者」问题,例如单生产者单消费者、多生产者单消费者、单生产者多消费者,甚至是多生产者多消费者 😵 需要配合 condition variables 我有用 Rust 写过一个「多生产者单消费者」的程序,相关的博客解说在 这里 condition variables int pthread_cond_init(pthread_cond_t *cond, const pthread_condattr_t *attr); int pthread_cond_destroy(pthread_cond_t *cond); int pthread_cond_wait(pthread_cond_t *cond, pthread_mutex_t *mutex); int pthread_cond_signal(pthread_cond_t *cond); int pthread_cond_broadcast(pthread_cond_t *cond); // all threads waiting on a condition need to be woken up Condition variables allow a thread to block until a specific condition becomes true blocked thread goes to wait queue for condition When the condition becomes true, some other thread signals the blocked thread(s) Conditions in Pthreads are usually used with a mutex to enforce mutual exclusion. the wait call should occur under the protection of a mutex 使用 condition variables 改写之前 mutex 部分的 producer 实作 (实作是单生产者单消费者模型,且缓冲区有 MAX_SIZE 个元素): void producer(char *buf) { for (;;) { pthread_mutex_lock(lock); while (count == MAX_SIZE) pthread_cond_wait(notFull, lock); buf[count] = getChar(); count++; pthread_cond_signal(notEmpty); pthread_mutex_unlock(lock); } } semaphores semaphores 是站在「资源的数量」的角度来看待问题,这与 condition variables 是不同的 sem_t semaphore; int sem_init(sem_t *sem, int pshared, unsigned int value); int sem_wait(sem_t *sem); int sem_post(sem_t *sem); Can do increments and decrements of semaphore value Semaphore can be initialized to any value Thread blocks if semaphore value is less than or equal to zero when a decrement is attempted As soon as semaphore value is greater than zero, one of the blocked threads wakes up and continues no guarantees as to which thread this might be ","date":"2024-04-10","objectID":"/posts/posix-threads/:1:3","tags":["Sysprog","Linux","Concurrency"],"title":"并行程序设计: POSIX Threads","uri":"/posts/posix-threads/"},{"categories":["Concurrency","Linux Kernel Internals"],"content":"POSIX Threads","date":"2024-04-10","objectID":"/posts/posix-threads/:2:0","tags":["Sysprog","Linux","Concurrency"],"title":"并行程序设计: POSIX Threads","uri":"/posts/posix-threads/"},{"categories":["C","Linux Kernel Internals"],"content":" 本次講座將選定幾個案例,藉此解說 C 語言程式設計的技巧,像是對矩陣操作進行包裝、初始化特定結構的成員、追蹤物件配置的記憶體、Smart Pointer 等等。 原文地址 ","date":"2024-04-10","objectID":"/posts/c-trick/:0:0","tags":["Sysprog","C"],"title":"你所不知道的 C 语言: 技巧篇","uri":"/posts/c-trick/"},{"categories":["C","Linux Kernel Internals"],"content":"从矩阵操作谈起 C 语言也可作实现 Object-oriented programming (需要搭配前置处理器扩充语法) GNU Manual 6.29 Designated Initializers Stack Overflow: Why does C++11 not support designated initializer lists as C99? 从 C99 (含) 以后,C 和 C++ 就分道扬镳了。相关差异可以参考: Incompatibilities Between ISO C and ISO C++ 结构体的成员函数实作时使用 static,并搭配 API gateway 可以获得一部分 namespace 的功能 Fun with C99 Syntax ","date":"2024-04-10","objectID":"/posts/c-trick/:1:0","tags":["Sysprog","C"],"title":"你所不知道的 C 语言: 技巧篇","uri":"/posts/c-trick/"},{"categories":["C","Linux Kernel Internals"],"content":"明确初始化特定结构的成员 静态空间初始化配置: 动态空间初始化配置: Initializing a heap-allocated structure in C ","date":"2024-04-10","objectID":"/posts/c-trick/:2:0","tags":["Sysprog","C"],"title":"你所不知道的 C 语言: 技巧篇","uri":"/posts/c-trick/"},{"categories":["C","Linux Kernel Internals"],"content":"追踪物件配置的记忆体 ","date":"2024-04-10","objectID":"/posts/c-trick/:3:0","tags":["Sysprog","C"],"title":"你所不知道的 C 语言: 技巧篇","uri":"/posts/c-trick/"},{"categories":["C","Linux Kernel Internals"],"content":"Smart Pointer ","date":"2024-04-10","objectID":"/posts/c-trick/:4:0","tags":["Sysprog","C"],"title":"你所不知道的 C 语言: 技巧篇","uri":"/posts/c-trick/"},{"categories":["C","Linux Kernel Internals"],"content":"C99 Variable Length Arrays (VLA) ","date":"2024-04-10","objectID":"/posts/c-trick/:5:0","tags":["Sysprog","C"],"title":"你所不知道的 C 语言: 技巧篇","uri":"/posts/c-trick/"},{"categories":["C","Linux Kernel Internals"],"content":"字符串和数值转换 Integer to string conversion ","date":"2024-04-10","objectID":"/posts/c-trick/:6:0","tags":["Sysprog","C"],"title":"你所不知道的 C 语言: 技巧篇","uri":"/posts/c-trick/"},{"categories":["C","Linux Kernel Internals"],"content":"GCC 支援 Plan 9 C Extension GCC 6.65 Unnamed Structure and Union Fields ","date":"2024-04-10","objectID":"/posts/c-trick/:7:0","tags":["Sysprog","C"],"title":"你所不知道的 C 语言: 技巧篇","uri":"/posts/c-trick/"},{"categories":["C","Linux Kernel Internals"],"content":"GCC transparent union GCC 6.35.1 Common Type Attributes ","date":"2024-04-10","objectID":"/posts/c-trick/:8:0","tags":["Sysprog","C"],"title":"你所不知道的 C 语言: 技巧篇","uri":"/posts/c-trick/"},{"categories":["C","Linux Kernel Internals"],"content":"高阶的 C 语言的「开发框架」 cello 是上面提到的技巧的集大成者,在 C 语言基础上,提供以下进阶特征: ","date":"2024-04-10","objectID":"/posts/c-trick/:9:0","tags":["Sysprog","C"],"title":"你所不知道的 C 语言: 技巧篇","uri":"/posts/c-trick/"},{"categories":["C","Linux Kernel Internals"],"content":"善用 GNU extension 的 typeof GCC 6.7 Referring to a Type with typeof typeof 在 C23 中已由 GNU extenison 转正为 C 语言标准 ","date":"2024-04-10","objectID":"/posts/c-trick/:10:0","tags":["Sysprog","C"],"title":"你所不知道的 C 语言: 技巧篇","uri":"/posts/c-trick/"},{"categories":["Rust"],"content":" 从基础到进阶讲解探讨 Rust 生命周期,不仅仅是 lifetime kata,还有更多的 lifetime 资料,都来讲解和探讨,从「入门 Rust」到「进阶 Rust」 整理自 B 站 UP 主 @这周你想干啥 的 教学影片合集 注意 学习 John Gjengset 的教学影片 Subtying and Variance 时发现自己对 Rust 生命周期 (lifetime) 还是不太理解,于是便前来补课 🤣 同时完成 LifetimeKata 的练习。 ","date":"2024-04-05","objectID":"/posts/rust-lifetime/:0:0","tags":["Rust","Lifetime"],"title":"Rust Lifetime: 由浅入深理解生命周期","uri":"/posts/rust-lifetime/"},{"categories":["Rust"],"content":"引用 \u0026 生命周期 ","date":"2024-04-05","objectID":"/posts/rust-lifetime/:1:0","tags":["Rust","Lifetime"],"title":"Rust Lifetime: 由浅入深理解生命周期","uri":"/posts/rust-lifetime/"},{"categories":["Rust"],"content":"生命周期标注 在 单线程 的程序中,通过函数参数传入的引用,无论其生命周期标注,它的生命周期在该函数的函数体范围内都是有效的。因为从 状态机 模型来考虑,该函数没有传入的引用的所有权 (因为是通过参数传入的),所以该函数不可能在其函数体范围内某一节点,就结束传入的引用的生命周期。但是在 多线程 的程序,上述规则就不一定成立了。 fn split\u003c'a, 'b\u003e(text: \u0026'a str, delimiter: \u0026'b str) {...} 单线程情况下,参数 text 和 delimiter 在函数 split 范围内都是有效的。 也因为这个状态机模型,Rust 的生命周期对于参数在函数体内的使用的影响并不大,主要影响的是涉及生命周期的参数或返回值在 函数调用后的生命周期使用约束,下面给出一些技巧: 技巧 最大共同生命周期: 从引用的当前共同使用开始,直到任一引用对应的 object 消亡,即其生命周期结束。 当以相同的生命周期标注,来对函数参数的生命周期进行标注时,其表示参数的最大共同生命周期,例如: fn f\u003c'a\u003e(x: \u0026'a i32, y: \u0026'a i32, z: \u0026'a i32) {...} 生命周期 'a 表示参数 x, y, z 的最大共同生命周期,即 x, y, z 可以共同存活的最大生命周期。 当以相同的生命周期标注,来对函数参数和返回值的生命周期进行标注时,其表示返回值的生命周期必须在参数的生命周期内,例如: fn f\u003c'a\u003e(x: \u0026'a i32) -\u003e \u0026'a i32 {...} 生命周期 'a 表示返回值的生命周期必须在参数 x 的生命周期内,即返回值的生命周期是参数和返回值的最大共同生命周期。所以在返回值可以使用的地方,参数都必须存活,这也是常出现问题的地方。 最后看一下将这两个技巧结合起来的一个例子: fn f\u003c'a\u003e(x: \u0026'a i32, y: \u0026'a i32) -\u003e \u0026'a i32 {...} 参数取最大生命周期在容器情况下会被另一个技巧取代,即容器和容器内部元素都被标注为相同的生命周期,这种情况会让容器的生命周期和容器内的元素的生命周期保持一致。这是因为隐式规则: 容器的生命周期 $\\leq$ 容器内元素的生命周期,这显然已经满足了最大生命周期的要求,而此时标注一样的生命周期,会被编译器推导为二者的生命周期相同,即 容器的生命周期和容器内的元素的生命周期一致: fn strtok(x: \u0026'a mut \u0026'a str, y: char) {...} ","date":"2024-04-05","objectID":"/posts/rust-lifetime/:2:0","tags":["Rust","Lifetime"],"title":"Rust Lifetime: 由浅入深理解生命周期","uri":"/posts/rust-lifetime/"},{"categories":["Rust"],"content":"生命周期在函数上的省略规则 The Rust Reference: Lifetime elision Each elided lifetime in the parameters becomes a distinct lifetime parameter. If there is exactly one lifetime used in the parameters (elided or not), that lifetime is assigned to all elided output lifetimes. If the receiver has type \u0026Self or \u0026mut Self, then the lifetime of that reference to Self is assigned to all elided output lifetime parameters. 正如投影片所说,虽然有生命周期的省略规则,但有时并不符合我们的预期,这时候需要我们手动标注。这种情况常出现于可变引用 \u0026mut self 的使用: struct S {} impl S { fn as_slice_mut\u003c'a\u003e(\u0026'a mut self) -\u003e \u0026'a [u8] {...} } let s: S = S{}; let x: \u0026[u8] = s.as_slice_mut(); //--- ... // | ... // | scope ... // | // end of s's lifetime //--- 在上面例子中,由于将方法 as_slice_mut 的可变引用参数和返回值的生命周期都标注为相同 'a,所以在范围 scope 中,在编译器看来 s 的可变引用仍然有效 (回想一下之前的参数和返回值的生命周期推导技巧),所以在这个范围内无法使用 s 的引用 (不管是可变还是不可变,回想一下 Rust 的引用规则),这是一个很常见的可变引用引起非预期的生命周期的例子,下一节会进一步介绍。 ","date":"2024-04-05","objectID":"/posts/rust-lifetime/:3:0","tags":["Rust","Lifetime"],"title":"Rust Lifetime: 由浅入深理解生命周期","uri":"/posts/rust-lifetime/"},{"categories":["Rust"],"content":"关注可变引用 fn insert_value\u003c'a\u003e(my_vec: \u0026'a mut Vec\u003c\u0026'a i32\u003e, value: \u0026'a i32) {...} 这个例子和之前的例子很类似,同样的,使用我们的参数生命周期推导技巧,调用函数 insert_value 后,当参数 vec 和 value 的最大共同生命周期的范围很广时,这时就需要注意,在这个范围内,我们无法使用 my_vec 对应的 object 的任何其它引用 (因为编译器会认为此时还存在可变引用 my_vec),从而编译错误。这就是容器和引用使用相同生命周期标注,而导致的强约束。为避免这种非预期的生命周期,应当将函数原型改写如下: fn insert_value\u003c'a, 'b\u003e(my_vec: \u0026'a mut Vec\u003c\u0026'b i32\u003e, value: \u0026'b i32) {...} 这样改写会包含一个隐式的生命周期规则: 'a $\\leq$ 'b,这很好理解,容器的生命周期应该比所引用的 object 短,这个隐式规则在下一节的 struct/enum 的生命周期标注非常重要。 ","date":"2024-04-05","objectID":"/posts/rust-lifetime/:4:0","tags":["Rust","Lifetime"],"title":"Rust Lifetime: 由浅入深理解生命周期","uri":"/posts/rust-lifetime/"},{"categories":["Rust"],"content":"struct / enum 生命周期标注 struct / enum 的生命周期标注也可以通过之前所提的 状态机 模型来进行理解,因为 struct / enum 本身不具备引用对应的 object 的所有权,在进行方法 (method) 调用时并不能截断引用对应的 object 的生命周期。 struct / enum 生命周期标注主要需要特别注意一点,就是 struct / enum 本身的可变引用的生命周期标注,最好不要和为引用的成员的生命周期的标注,标注为相同,因为这极大可能会导致 生命周期强约束,例如: fn strtok(x: \u0026mut 'a Vec\u003c'a i32\u003e, y: \u0026'a i32) {...} 如果参数 Vec\u003c'a i32\u003e 的 vector 里的 i32 引用的生命周期是 static 的话,依据我们之前所提的技巧,会将可变引用 \u0026'a mut 的生命周期也推导为 static,这就导致再也无法借用 x 对应的 object。 ","date":"2024-04-05","objectID":"/posts/rust-lifetime/:5:0","tags":["Rust","Lifetime"],"title":"Rust Lifetime: 由浅入深理解生命周期","uri":"/posts/rust-lifetime/"},{"categories":["Rust"],"content":"'static 和 '_ fn foo(_input: \u0026'a str) -\u003e 'static str { \"abc\" } 如果不进行 static 生命周期标注,依据省略规则,编译器会把返回值的生命周期推导为 'a,即和输入参数一样,这就不符合我们预期使用了。 如果使用 static 标注 struct / enum 里的成员,则无需标注 struct / enum 的生命周期,因为 static 表示在整个程序运行起见都有效,没必要对容器进行生命周期标注。 struct UniqueWords { sentence: \u0026'static str, unique_words: Vec\u003c\u0026'static str\u003e, } impl UniqueWords {...} 在没有使用到 struct 的生命周期标注时,impl 可以不显式指明生命周期,而是通过 '_ 让编译器自行推导: struct Counter\u003c'a\u003e { inner: \u0026'a mut i32, } impl Counter\u003c'_\u003e { fn increment(\u0026mut self) {...} } // is the same as impl\u003c'a\u003e Counter\u003c'a\u003e { fn increment(\u0026mut self) {...} } 函数返回值不是引用,但是返回值类型里有成员是引用,依据省略规则,编译器无法自行推导该成员的生命周期,此时可以通过 '_ 来提示编译器依据省略规则,对返回值的成员的生命周期进行推导: struct StrWrap\u003c'a\u003e(\u0026'a str); fn make_wrapper(string: \u0026str) -\u003e StrWrap\u003c'_\u003e {...} // is the same as fn make_wrapper\u003c'a\u003e(string: \u0026'a str) -\u003e StrWrap\u003c'a\u003e {...} ","date":"2024-04-05","objectID":"/posts/rust-lifetime/:6:0","tags":["Rust","Lifetime"],"title":"Rust Lifetime: 由浅入深理解生命周期","uri":"/posts/rust-lifetime/"},{"categories":["Rust"],"content":"生命周期型变和绑定 因为 Rust 是没有继承的概念,所以下面以 scala 来对类型的型变举例子进行讲解 (但是不需要你对 Scala 有任何了解): class A class B extends A // B is subclass of A private def f1(a: A): Unit = { println(\"f1 works!\") } def main(args: Array[STring]): Unit = { val a = new A val b = new B f1(a) // succeed f1(b) // succeed } 这个例子很好理解,因为 B 是 A 的子类,所以作为参数传入函数 f1 显然是可以接受的。但是当泛型 (generic) 和子类 (subclass) 结合起来时,就变得复杂了: class A class B extends A // B is subclass of A class Foo[T] // generic private def f1(a: Foo[A]): Unit = { println(\"f1 works!\") } def main(args: Array[STring]): Unit = { val foo_a = new Foo[A] val foo_b = new Foo[B] f1(a) // succeed f1(b) // error } 在编译器看来,虽然 B 是 A 的子类,但是编译器认为 Foo[A] 和 Foo[B] 是两个独立的类型,这个被称为 不变 (invariant)。而我们的直觉是,这种情况 Foo[B] 应该是 Foo[A] 的子类,这就引出了 协变 (covariant)。将上面例子的第 3 行的 Foo 定义修改如下: class Foo[+T] // covariant 就可以让编译器推导出 Foo[B] 是 Foo[A] 的子类,进而让第 14 行编译通过。 除此之外,还有 逆变 (contra-variant),它会将子类关系反转。将上面例子的第 3 行的 Foo 定义修改如下: class Foo[-T] // contra-variant 编译器就会推导出关系: Foo[A] 是 Foo[B] 的子类,这个关系刚好是 A 和 B 的反转。 在 Scala 中,函数之间的关系也体现了协变和逆变,即 参数是逆变的,返回值是协变的: class A class B extends A // B is subclass of A class C extends B // C is subclass of B /* * `A =\u003e C` is subclass of `B =\u003e B` */ 为什么 A =\u003e C 是 B =\u003e B 的子类呢?其实也很好理解,B =\u003e B 的返回值是 B,这个返回值可以用 C 来代替,但不能用 A 来代替,这显然满足协变。B =\u003e B 的参数是 B,这个参数可以用 A 来代替而不能用 C 来代替 (因为有一部分 B 不一定是 C,而 B 则一定是 A),这满足逆变。 T 可以表示所有情况: ownership, immutable reference, mutable reference,例如 T 可以表示 i32, \u0026i32, \u0026mut i32 (如果你使用过 into_iterator 的话应该不陌生) T: 'a 是说:如果 T 里面含有引用,那么这个引用的生命周期必须是 'a 的子类,即比 'a 长或和 'a 相等。T: 'static 也类似,表示 T 里面的引用 (如果有的话),要么比 'static 长或和 'static 相等,因为不可能有比 'static 更长的生命周期,所以这个标注表示 要么 T 里面的引用和 'static 一样长,要么 T 里面没有引用只有所有权 (owneship)。 The Rust Reference: Subtyping and Variance The Rustonomicon: Subtyping and Variance 基本和我们之前所说的一致,这里需要注意一点: 凡是涉及可变引用的 T,都是不变 (invariant)。这也很好理解,因为可变引用需要保证所引用的类型 T 是一致并且是唯一的,否则会扰乱 Rust 的引用模型。因为可变引用不仅可以改变所指向的 object 的内容,还可以改变自身,即改变指向的 object,如果此时 T 不是不变 (invariant) 的,那么可以将这个可变引用指向 T 的子类,这会导致该可变引用指向的 object 被可变借用一次后无法归还,从而导致后续再也无法引用该 object。此外还会导致原本没有生命周期约束的两个独立类型,被生命周期约束,具体见后面的例子。 struct Foo\u003c'a\u003e { _phantom: PhantomData\u003c\u0026'a i32\u003e, } fn foo\u003c'short, 'long: 'short\u003e( // long is subclass of short mut short_foo: Foo\u003c'short\u003e, mut long_foo: Foo\u003c'long\u003e, ) { short_foo = long_foo; // succeed long_foo = short_foo; // error } 下面是一个可变引用的例子。参数 short_foo 和 long_foo 没有关系,是两个独立的类型,所以无法相互赋值,这保证了可变引用的模型约束。除此之外,如果可变引用的型变规则不是不变 (inariant) 则会导致 short_foo 和 long_foo 在函数 foo 调用后的生命周期约束为: short_foo $\\leq$ long_foo (协变) long_foo $\\leq$ short_foo (逆变) 而它们本身可能并没有这种约束,生命周期是互相独立的。 struct Foo\u003c'a\u003e { _phantom: PhantomData\u003c\u0026'a i32\u003e, } fn foo\u003c'short, 'long: 'short\u003e( // long is subclass of short mut short_foo: \u0026mut Foo\u003c'short\u003e, mut long_foo: \u0026mut Foo\u003c'long\u003e, ) { short_foo = long_foo; // error long_foo = short_foo; // error } ","date":"2024-04-05","objectID":"/posts/rust-lifetime/:7:0","tags":["Rust","Lifetime"],"title":"Rust Lifetime: 由浅入深理解生命周期","uri":"/posts/rust-lifetime/"},{"categories":["Rust"],"content":"Documentations 这里列举视频中一些概念相关的 documentation 学习的一手资料是官方文档,请务必自主学会阅读规格书之类的资料 ","date":"2024-04-05","objectID":"/posts/rust-lifetime/:8:0","tags":["Rust","Lifetime"],"title":"Rust Lifetime: 由浅入深理解生命周期","uri":"/posts/rust-lifetime/"},{"categories":["Rust"],"content":"Crate std 可以使用这里提供的搜素栏进行搜索 (BTW 不要浪费时间在 Google 搜寻上!) ","date":"2024-04-05","objectID":"/posts/rust-lifetime/:8:1","tags":["Rust","Lifetime"],"title":"Rust Lifetime: 由浅入深理解生命周期","uri":"/posts/rust-lifetime/"},{"categories":["Rust"],"content":"References LifetimeKata The Rust Reference 泛型中的型变 (协变,逆变,不可变) Variant Types and Polymorphism ","date":"2024-04-05","objectID":"/posts/rust-lifetime/:9:0","tags":["Rust","Lifetime"],"title":"Rust Lifetime: 由浅入深理解生命周期","uri":"/posts/rust-lifetime/"},{"categories":["C","Linux Kernel Internals"],"content":" goto 在 C 語言被某些人看做是妖魔般的存在,不過實在不用這樣看待,至少在 Linux 核心原始程式碼中,goto 是大量存在 (跟你想像中不同吧)。有時不用 goto 會寫出更可怕的程式碼。 原文地址 Stack Overflow: GOTO still considered harmful? ","date":"2024-04-05","objectID":"/posts/c-control-flow/:0:0","tags":["Sysprog","C","Control flow"],"title":"你所不知道的 C 语言: goto 和流程控制篇","uri":"/posts/c-control-flow/"},{"categories":["C","Linux Kernel Internals"],"content":"MISRA C MISRA-C:2004 Guidelines for the use of the C language in critical systems MISRA C 禁用 goto 和 continue,但可用 break: Rule 14.4 (required): The goto statement shall not be used. Rule 14.5 (required): The continue statement shall not be used. Rule 14.6 (required): For any iteration statement there shall be at most one break statement used for loop termination. These rules are in the interests of good structured programming. One break statement is allowed in a loop since this allows, for example, for dual outcome loops or for optimal coding. Stack Overflow 上的相关讨论: Why “continue” is considered as a C violation in MISRA C:2004? 使用 goto 可能会混淆静态分析的工具 (当然使用 goto 会极大可能写出 ugly 的程式码): Case in point: MISRA C forbids goto statements primarily because it can mess up static analysis. Yet this rule is gratuitously followed even when no static analysis tools are used, thus yielding none of the gains that you trade off for occasionally writing ugly code. ","date":"2024-04-05","objectID":"/posts/c-control-flow/:1:0","tags":["Sysprog","C","Control flow"],"title":"你所不知道的 C 语言: goto 和流程控制篇","uri":"/posts/c-control-flow/"},{"categories":["C","Linux Kernel Internals"],"content":"GOTO 没有想象中那么可怕 虽然 MISRA C 这类规范都明确禁止了使用 goto,但 goto 并没有想像中的那么可怕,在一些领域还是极具活力的。 在 C 语言中 goto 语句是实作错误处理的极佳选择 (如果你看过 xv6 应该不陌生),有时不用 goto 可能会写出更可怕的程式码: Using goto for error handling in C Wikipedia: RAII C requires significant administrative code since it doesn’t support exceptions, try-finally blocks, or RAII at all. A typical approach is to separate releasing of resources at the end of the function and jump there with gotos in the case of error. This way the cleanup code need not be duplicated. 相关实作: goto 在 Linux 核心广泛应用 OpenBSD’s httpd Linux kernel 里 NFS inode 验证的函数: fs/nfs/inode.c 以 goto 为关键字进行检索 Wikipedia: Common usage patterns of Goto To make the code more readable and easier to follow Error handling (in absence of exceptions), particularly cleanup code such as resource deallocation. ","date":"2024-04-05","objectID":"/posts/c-control-flow/:2:0","tags":["Sysprog","C","Control flow"],"title":"你所不知道的 C 语言: goto 和流程控制篇","uri":"/posts/c-control-flow/"},{"categories":["C","Linux Kernel Internals"],"content":"switch \u0026 goto switch 通过 jump table 的内部实作可以比大量的 if-else 效率更高。 GCC: 6.3 Labels as Values You can get the address of a label defined in the current function (or a containing function) with the unary operator ‘\u0026\u0026’. The value has type void *. To use these values, you need to be able to jump to one. This is done with the computed goto statement6, goto *exp;. 下面这篇文章以 VM 为例子对 computed goto 和 switch 的效能进行了对比 (之前学的 RISC-V 模拟器派上用场了hhh): Computed goto for efficient dispatch tables the condition serves as an offset into a lookup table that says where to jump next. To anyone with a bit of experience with assembly language programming, the computed goto immediately makes sense because it just exposes a common instruction that most modern CPU architectures have - jump through a register (aka. indirect jump). computed goto 比 switch 效能更高的原因: The switch does a bit more per iteration because of bounds checking. The effects of hardware branch prediction. C99: If no converted case constant expression matches and there is no default label, no part of the switch body is executed. 因为标准的这个要求,所以编译器对于 switch 会生成额外的 safe 检查代码,以符合上面情形的 “no part of the switch body is executed” 的要求。 Since the switch statement has a single “master jump” that dispatches all opcodes, predicting its destination is quite difficult. On the other hand, the computed goto statement is compiled into a separate jump per opcode, so given that instructions often come in pairs it’s much easier for the branch predictor to “home in” on the various jumps correctly. 作者提到,ta 个人认为分支预测是导致效能差异的主要因素: I can’t say for sure which one of the two factors weighs more in the speed difference between the switch and the computed goto, but if I had to guess I’d say it’s the branch prediction. 除此之外,有这篇文章的 disassembly 部分可以得知,switch 的底层是通过所谓的 jump table 来实作的: 引用 How did I figure out which part of the code handles which opcode? Note that the “table jump” is done with: jmpq *0x400b20(,%rdx,8) bounds checking 是在 switch 中執行的一個環節,每次迴圈中檢查是否有 default case 的狀況,即使程式中的 switch 沒有用到 default case,編譯期間仍會產生強制檢查的程式,所以 switch 會較 computed goto 多花一個 bounds checking 的步驟 branch prediction 的部分,switch 需要預測接下來跳到哪個分支 case,而 computed goto 則是在每個 instruction 預測下一個 instruction,這之中比較直覺的想法是 computed goto 的prediction可以根據上個指令來預測,但是 switch 的prediction每次預測沒辦法根據上個指令,因此在 branch prediction accuracy 上 computed goto 會比較高。 所以在实际中大部分也是采取 computed goto 来实作 VM: Ruby 1.9 (YARV): also uses computed goto. Dalvik (the Android Java VM): computed goto Lua 5.2: uses a switch ","date":"2024-04-05","objectID":"/posts/c-control-flow/:3:0","tags":["Sysprog","C","Control flow"],"title":"你所不知道的 C 语言: goto 和流程控制篇","uri":"/posts/c-control-flow/"},{"categories":["C","Linux Kernel Internals"],"content":"do {…} while (0) 宏 用于避免 dangling else,即 if 和 else 未符合预期的配对 (常见于未使用 {} 包裹) Stack Overflow: C multi-line macro: do/while(0) vs scope block 我写了 相关笔记 记录在前置处理器篇。 ","date":"2024-04-05","objectID":"/posts/c-control-flow/:4:0","tags":["Sysprog","C","Control flow"],"title":"你所不知道的 C 语言: goto 和流程控制篇","uri":"/posts/c-control-flow/"},{"categories":["C","Linux Kernel Internals"],"content":"用 goto 实作 RAII 开发风格 RAII in C If you have functions or control flows that allocate resources and a failure occurs, then goto turns out to be one of the nicest ways to unwind all resources allocated before the point of failure. Linux 核心中的实作: shmem.c 以 goto 为关键字进行检索 ","date":"2024-04-05","objectID":"/posts/c-control-flow/:5:0","tags":["Sysprog","C","Control flow"],"title":"你所不知道的 C 语言: goto 和流程控制篇","uri":"/posts/c-control-flow/"},{"categories":["C","Linux Kernel Internals"],"content":"检阅 C 语言规格书 ISO/IEC 9899:201x Committee Draft 6.8.6 Jump statements A jump statement causes an unconditional jump to another place. The identifier in a goto statement shall name a label located somewhere in the enclosing function. A goto statement shall not jump from outside the scope of an identifier having a variably modified type to inside the scope of that identifier. 规格书后面的例子也值得一看 (特别是当你看不懂规格书严格的英语语法想表达什么的时候 🤣) 从规格书中也可以得知,goto 虽然是无条件跳转 (对应汇编语言的 jmp 这类无条件跳转指令),但它的跳转范围是有限制的 (jump to another place),而不是可以跳转到任意程式码 (这也是为什么 setjmp/longjmp 被称为「长跳转」的原因,与 goto 这类「短跳转」相对应)。 相关实作: CPython 的 Modules/_asynciomodule.c 以 goto 为关键字进行检索 Modern C 作者也总结了 3 项和 goto 相关的规范 (大可不必视 goto 为洪水猛兽,毕竟我们有规范作为指导是不是): Rule 2.15.0.1: Labels for goto are visible in the whole function that contains them. Rule 2.15.0.2: goto can only jump to a label inside the same function. Rule 2.15.0.3: goto should not jump over variable initializations. ","date":"2024-04-05","objectID":"/posts/c-control-flow/:6:0","tags":["Sysprog","C","Control flow"],"title":"你所不知道的 C 语言: goto 和流程控制篇","uri":"/posts/c-control-flow/"},{"categories":["C","Linux Kernel Internals"],"content":"和想象中不同的 switch-case switch-case 语句中的 case 部分本质上是 label,所以使用其它语句 (例如 if) 将其包裹起来并不影响 switch 语句的跳转。 Something You May Not Know About the Switch Statement in C/C++ How to Get Fired Using Switch Statements \u0026 Statement Expressions ","date":"2024-04-05","objectID":"/posts/c-control-flow/:7:0","tags":["Sysprog","C","Control flow"],"title":"你所不知道的 C 语言: goto 和流程控制篇","uri":"/posts/c-control-flow/"},{"categories":["C","Linux Kernel Internals"],"content":"Duff’s Device 这个技巧常用于内存数据的复制,类似于 memcpy Wikipedia: Duff’s Device Duff’s Device 的详细解释 Tom Duff 本人的解释 ","date":"2024-04-05","objectID":"/posts/c-control-flow/:8:0","tags":["Sysprog","C","Control flow"],"title":"你所不知道的 C 语言: goto 和流程控制篇","uri":"/posts/c-control-flow/"},{"categories":["C","Linux Kernel Internals"],"content":"co-routine 应用 Wikipedia: Coroutine 不借助操作系统也可以实作出多工交执行的 illusion (通过 switch-case 黑魔法来实现 🤣) ","date":"2024-04-05","objectID":"/posts/c-control-flow/:9:0","tags":["Sysprog","C","Control flow"],"title":"你所不知道的 C 语言: goto 和流程控制篇","uri":"/posts/c-control-flow/"},{"categories":["RISC-V"],"content":" pretask 作为社区入门探索,目的是帮助实习生一起搭建工作环境,熟悉 oerv 的工作流程和合作方式。pretask 分为三个步骤: 任务一:通过 QEMU 仿真 RISC-V 环境并启动 openEuler RISC-V 系统,设法输出 neofetch 结果并截图提交 任务二:在 openEuler RISC-V 系统上通过 obs 命令行工具 osc,从源代码构建 RISC-V 版本的 rpm 包,比如 pcre2。提示首先需要在 openEuler 的 OBS 上注册账号 任务三:尝试使用 qemu user \u0026 nspawn 或者 docker 加速完成任务二 ","date":"2024-03-28","objectID":"/posts/oerv-pretask/:0:0","tags":["RISC-V","openEuler","QEMU","Neofetch","Container","chroot","nspawn"],"title":"OERV 之 Pretask","uri":"/posts/oerv-pretask/"},{"categories":["RISC-V"],"content":"Pretask 1: Neofetch 任务一:通过 QEMU 仿真 RISC-V 环境并启动 openEuler RISC-V 系统,设法输出 neofetch 结果并截图提交 由于工作内容是对软件包进行: 编译 -\u003e 失败 -\u003e 定位问题 -\u003e 修复 -\u003e 重新编译,所以我们倾向于直接从源码编译,根据 neofetch wiki 从 git 拉取最新数据进行构建: # enter into euler openEuler RISC-V QEMU $ git clone https://github.com/dylanaraps/neofetch $ cd neofetch $ make install $ neofetch ","date":"2024-03-28","objectID":"/posts/oerv-pretask/:1:0","tags":["RISC-V","openEuler","QEMU","Neofetch","Container","chroot","nspawn"],"title":"OERV 之 Pretask","uri":"/posts/oerv-pretask/"},{"categories":["RISC-V"],"content":"Pretask 2: Open Build Service (OBS) 任务二:在 openEuler RISC-V 系统上通过 obs 命令行工具 osc,从源代码构建 RISC-V 版本的 rpm 包,比如 pcre2。提示首先需要在 openEuler 的 OBS 上注册账号 观看教学影片: openEuler构建之OBS使用指导 - bilibili 并对比阅读 Beginnerʼs Guide openSUSE:Build Service 新手入门 如何通过OpenSUSE Open Build Service(OBS)构建Debian包 for RISCV-64 了解掌握 OBS 的基本概念、OBS 网页 以及 OSC 命令行工具 的使用方法。 这部分内容很重要,和后续工作内容息息相关,在这里不要图快,打牢基础比较好。 OBS 的 Package 中 _service 配置文件,revision 字段是对应与 Git 仓库的 commit id (如果你使用的 Source Code Management (SCM) 方式是 Git 托管的话) 参考仓库: https://gitee.com/zxs-un/doc-port2riscv64-openEuler 内的相关文档 osc命令工具的安装与~/.oscrc配置文件 在 openEuler 上安装 osc build 本地构建工具 使用 osc build 在本地构建 openEuler OBS 服务端的内容 在 openEuler RISC-V QEMU 虚拟机内完成 OBS、OSC 相关基础设施的安装: # install osc and build $ sudo yum install osc build # configure osc in ~/.oscrc [general] apiurl = https://build.openeuler.openatom.cn no_verify = 1 # 未配置证书情况下不验证 [https://build.openeuler.openatom.cn] user=username # 用户名 pass=password # 明文密码 trusted_prj=openEuler:selfbuild:function # 此项目为openEuler:Mailine:RISC-V项目的依赖库 在 openEuler RISC-V QEMU 虚拟机内完成 pcre2 的本地编译构建: # 选定 pcre2 包 $ osc co openEuler:Mainline:RISC-V/pcre2 $ cd openEuler\\:Mainline\\:RISC-V/pcre2/ # 更新并下载相关文件到本地 $ osc up -S # 重命名刚刚下载的文件 $ rm -f _service;for file in `ls | grep -v .osc`;do new_file=${file##*:};mv $file $new_file;done # 查看一下仓库信息,方便后续构建 $ osc repos standard_riscv64 riscv64 mainline_gcc riscv64 # 指定仓库和架构并进行本地构建 $ osc build standard_riscv64 riscv64 总计用时 1301s ","date":"2024-03-28","objectID":"/posts/oerv-pretask/:2:0","tags":["RISC-V","openEuler","QEMU","Neofetch","Container","chroot","nspawn"],"title":"OERV 之 Pretask","uri":"/posts/oerv-pretask/"},{"categories":["RISC-V"],"content":"Pretask 3: 容器加速构建 任务三:尝试使用 qemu user \u0026 nspawn 或者 docker 加速完成任务二 参考 文档 由于 deepin 20.9 的 Python3 版本仅为 3.7,构建 osc 和 qemu 显然不太够,所以我通过 KVM 构建了一个 openEuler 22.03 LTS SP3 的虚拟机,在上面进行这项任务。 Deepin 20.9 KVM 安装和管理 openEuler 22.03 LTS SP3 编译 QEMU 时常见错误修正: ERROR: Python package 'sphinx' was not found nor installed. $ sudo yum install python3-sphinx ERROR: cannot find ninja $ sudo yum install ninja-build openEuler 22.03 LTS SP3 没有预先安装好 nspawn,所以需要手动安装: $ sudo yum install systemd-container systemd-nspawn 其余同任务二。 尝试使用 nspawn 来构建 pcre2: $ osc build standard_riscv64 riscv64 --vm-type=nspawn 会遇到以下报错 (且经过相当多时间排错,仍无法解决该问题,个人猜测是平台问题): can't locate file/copy.pm: /usr/lib64/perl5/vendor_perl/file/copy.pm: permission denied at /usr/bin/autoreconf line 49. 所以退而求其次,使用 chroot 来构建: $ osc build standard_riscv64 riscv64 --vm-type=chroot 总计用时 749s,比 qemu-system-riscv64 快了将近 2 倍,效能提升相当可观。 ","date":"2024-03-28","objectID":"/posts/oerv-pretask/:3:0","tags":["RISC-V","openEuler","QEMU","Neofetch","Container","chroot","nspawn"],"title":"OERV 之 Pretask","uri":"/posts/oerv-pretask/"},{"categories":["RISC-V"],"content":"References https://gitee.com/zxs-un/doc-port2riscv64-openEuler/blob/master/doc/build-osc-config-oscrc.md https://gitee.com/zxs-un/doc-port2riscv64-openEuler/blob/master/doc/build-osc-build-tools.md https://gitee.com/zxs-un/doc-port2riscv64-openEuler/blob/master/doc/build-osc-obs-service.md https://gitee.com/openeuler/RISC-V/blob/master/doc/tutorials/qemu-user-mode.md https://stackoverflow.com/questions/5308816/how-can-i-merge-multiple-commits-onto-another-branch-as-a-single-squashed-commit ","date":"2024-03-28","objectID":"/posts/oerv-pretask/:4:0","tags":["RISC-V","openEuler","QEMU","Neofetch","Container","chroot","nspawn"],"title":"OERV 之 Pretask","uri":"/posts/oerv-pretask/"},{"categories":["Toolkit"],"content":"本篇主要介绍在 deepin20.9 操作系统平台下,使用 KVM 虚拟化技术来创建和安装 Linux 发行版,并以创建安装 openEuler 22.03 LTS SP3 的 KVM 虚拟机作为示范,让学员领略 KVM 虚拟化技术的强大魅力。 ","date":"2024-03-28","objectID":"/posts/deepin-kvm/:0:0","tags":["Linux","Deepin","KVM","QEMU","openEuler"],"title":"Deepin 20.9 KVM 安装和管理","uri":"/posts/deepin-kvm/"},{"categories":["Toolkit"],"content":"什么是虚拟化? 什么是虚拟化技术?KVM 虚拟化和 Virtual Box、VMware 这类虚拟机软件的区别是什么?请阅读下面的这篇文章。 KVM 与 VMware 的区别盘点 ","date":"2024-03-28","objectID":"/posts/deepin-kvm/:1:0","tags":["Linux","Deepin","KVM","QEMU","openEuler"],"title":"Deepin 20.9 KVM 安装和管理","uri":"/posts/deepin-kvm/"},{"categories":["Toolkit"],"content":"配置虚拟化环境 首先需要检查 CPU 是否支持虚拟化 (以 Intel 处理器为例): # intel vmx,amd svm $ egrep '(vmx|svm)' /proc/cpuinfo ...vmx... $ lscpu | grep Virtualization Virtualization: VT-x 检查 KVM 模块是否已加载: $ lsmod | grep -i kvm kvm_intel 278528 11 kvm 901120 1 kvm_intel 确保 CPU 支持虚拟化并且 KVM 模块已被加载,接下来是安装 QEMU 和 virt-manager (虚拟系统管理器)。直接通过 apt 安装的 QEMU 版本过低,而通过 GitHub 下载最新的 QEMU 源码编译安装需要Python3.9,而 deepin 20.9 的 Python 3 版本是 3.7 (保险起见不要随便升级),所以折中一下,编译安装 QEMU 7.2.0 🤣 安装 QEMU: $ wget https://download.qemu.org/qemu-7.2.0.tar.xz $ tar xvJf qemu-7.2.0.tar.xz $ mv qemu-7.2.0 qemu $./configure $ sudo make -j$(nproc) # in ~/.bashrc export PATH=$PATH:/path/to/qemu/build 安装 virt-manager: $ sudo apt install virt-manager ","date":"2024-03-28","objectID":"/posts/deepin-kvm/:2:0","tags":["Linux","Deepin","KVM","QEMU","openEuler"],"title":"Deepin 20.9 KVM 安装和管理","uri":"/posts/deepin-kvm/"},{"categories":["Toolkit"],"content":"安装 openEuler KVM 虚拟机 可以在启动器看到一个虚拟机管理应用图标,如下: 点击打开 (需要输入密码认证,以下图片中的 “本地” 可能会显示为 “QEMU/KVM”): 接下来创建虚拟机: 下图的操作系统选择对应的类型 (可以在 这里 下载 openEuler 22.03 LTS SP3 镜像,对于 openEuler 这类未被收录的类型,选择 Generic): 这里选择 iso 镜像后可能会显示路径搜索问题,选择 “是” 将该路径加入存储池即可 接下来是处理器和内存配置,建议配置 8 核 8G 内存,根据自己物理机配置选择即可: 接下来是虚拟磁盘的大小设置和存放位置,建议选择自定义存储路径,并搭配 更改 KVM 虚拟机默认存储路径,特别是如果你的根目录空间不太够的情况: 在对应的存储卷中创建虚拟磁盘 (注意: 如果你更改了默认存储路径,请选择对应的存储池而不是 default): 创建虚拟磁盘 (名称可以自定义,分配默认初始为 0,它会随着虚拟机使用而增大,当然也可以直接将分配等于最大容量,这样就会直接分配相应的磁盘空间,玩过虚拟机的学员应该很熟悉): 接下来自定义虚拟机名称并生成虚拟机即可: 最后就是熟悉的安装界面: 参考 这里 安装 openEuler 即可。 ","date":"2024-03-28","objectID":"/posts/deepin-kvm/:3:0","tags":["Linux","Deepin","KVM","QEMU","openEuler"],"title":"Deepin 20.9 KVM 安装和管理","uri":"/posts/deepin-kvm/"},{"categories":["Toolkit"],"content":"透过 SSH 连接 KVM 虚拟机 首先先检查 Guest OS 上 ssh 服务是否开启 (一般是开启的): $ sudo systemctl status sshd sshd.service - OpenSSH server daemon Loaded: loaded (/usr/lib/systemd/system/sshd.service; enabled; vendor preset: enabled) Active: active (running) since Thu 2024-03-28 14:40:15 CST; 20min ago ... 然后在 Guest OS 上获取其 IP 地址 (ens3 的 inet 后的数字即是,openEuler 启动时也会输出一下 IP 地址): $ ip addr 在 Host OS 上通过 ssh 连接登录 GuestOS: $ ssh user@ip # user: user name in the guest os # ip ip addr of guest os ","date":"2024-03-28","objectID":"/posts/deepin-kvm/:4:0","tags":["Linux","Deepin","KVM","QEMU","openEuler"],"title":"Deepin 20.9 KVM 安装和管理","uri":"/posts/deepin-kvm/"},{"categories":["Toolkit"],"content":"Development Tools 由于是最小安装,很多趁手的工具都没有,俗话说“工欲善其事,必先利其器”,所以先安装必要的开发工具。幸好 openEuler 提供了整合包 Development Tools,直接安装即可: $ sudo yum group install -y \"Development Tools\" ","date":"2024-03-28","objectID":"/posts/deepin-kvm/:5:0","tags":["Linux","Deepin","KVM","QEMU","openEuler"],"title":"Deepin 20.9 KVM 安装和管理","uri":"/posts/deepin-kvm/"},{"categories":["Toolkit"],"content":"Neofetch 安装 neofetch 来酷炫地输出一下系统信息: $ git clone https://github.com/dylanaraps/neofetch $ cd neofetch $ make install $ neofetch ","date":"2024-03-28","objectID":"/posts/deepin-kvm/:6:0","tags":["Linux","Deepin","KVM","QEMU","openEuler"],"title":"Deepin 20.9 KVM 安装和管理","uri":"/posts/deepin-kvm/"},{"categories":["Toolkit"],"content":"References 使用 KVM 安装和管理 deepin Linux 下使用 KVM 虚拟机安装 OpenEuler 系统 KVM 更改虚拟机默认存储路径 实践 KVM on Deepin ","date":"2024-03-28","objectID":"/posts/deepin-kvm/:7:0","tags":["Linux","Deepin","KVM","QEMU","openEuler"],"title":"Deepin 20.9 KVM 安装和管理","uri":"/posts/deepin-kvm/"},{"categories":["C","Linux Kernel Internals"],"content":" C 语言之所以不需要时常发布新的语言特性又可以保持活力,前置处理器 (preprocessor) 是很重要的因素,有心者可进行「扩充」C 语言。 原文地址 ","date":"2024-03-25","objectID":"/posts/c-preprocessor/:0:0","tags":["Sysprog","C","Preprocessor"],"title":"你所不知道的 C 语言: 前置处理器应用篇","uri":"/posts/c-preprocessor/"},{"categories":["C","Linux Kernel Internals"],"content":"不要小看 preprocessor man gcc -D name Predefine name as a macro, with definition 1. -D name=definition The contents of definition are tokenized and processed as if they appeared during translation phase three in a #define directive. In particular, the definition is truncated by embedded newline characters. 在 Makefile 中往 CFLAGS 加入 -D’;’=’;’ 这类搞怪信息,会导致编译时出现一些不明所以的编译错误 (恶搞专用 🤣) 早期的 C++ 是和 C 语言兼容的,那时候的 C++ 相当于 C 语言的一种 preprocessor,将 C++ 代码预编译为对应的 C 语言代码,具体可以参考 C with Classes。事实上现在的 C++ 和 C 语言早已分道扬镳,形同陌路,虽然语法上有相似的地方,但请把这两个语言当成不同的语言看待 🤣 体验一下 C++ 模版 (template) 的威力 ❌ 丑陋 ✔️ : C 语言: 大道至简 ✅ ","date":"2024-03-25","objectID":"/posts/c-preprocessor/:1:0","tags":["Sysprog","C","Preprocessor"],"title":"你所不知道的 C 语言: 前置处理器应用篇","uri":"/posts/c-preprocessor/"},{"categories":["C","Linux Kernel Internals"],"content":"Object Oriented Programming 面向对象编程时,善用前置处理器可大幅简化和开发 #: Stringizing convert a macro argument into a string constant ##: Concatenation merge two tokens into one while expanding macros. 宏的实际作用: generate (产生/生成) 程式码 Rust 的过程宏 (procedural macros) 进一步强化了这一目的,可以自定义语法树进行代码生成。 可以 gcc -E -P 来观察预处理后的输出: man gcc -E Stop after the preprocessing stage; do not run the compiler proper. The output is in the form of preprocessed source code, which is sent to the standard output. Input files that don't require preprocessing are ignored. -P Inhibit generation of linemarkers in the output from the preprocessor. This might be useful when running the preprocessor on something that is not C code, and will be sent to a program which might be confused by the linemarkers. 可以依据不同时期的标准来对 C 源程序编译生成目标文件: Feature Test Macros The exact set of features available when you compile a source file is controlled by which feature test macros you define. 使用 gcc -E -P 观察 objects.h 预处理后的输出,透过 make 和 make check 玩一下这个最简单光线追踪引擎 GitHub: raytracing object oriented programming 不等于 class based programming, 只需要满足 Object-oriented programming (OOP) is a computer programming model that organizes software design around data, or objects, rather than functions and logic. 这个概念的就是 OOP。 Source ","date":"2024-03-25","objectID":"/posts/c-preprocessor/:2:0","tags":["Sysprog","C","Preprocessor"],"title":"你所不知道的 C 语言: 前置处理器应用篇","uri":"/posts/c-preprocessor/"},{"categories":["C","Linux Kernel Internals"],"content":"C11: _Generic 阅读 C11 规格书 6.5.1.1 Generic selection The controlling expression of a generic selection is not evaluated. If a generic selection has a generic association with a type name that is compatible with the type of the controlling expression, then the result expression of the generic selection is the expression in that generic association. Otherwise, the result expression of the generic selection is the expression in the default generic association. None of the expressions from any other generic association of the generic selection is evaluated. #define cbrt(X) \\ _Generic((X), \\ long double: cbrtl, \\ default: cbrt, \\ const float: cbrtf, \\ float: cbrtf \\ )(X) 经过 func.c/func.cpp 的输出对比,C++ 模版在字符类型的的判定比较准确,C11 的 _Generic 会先将 char 转换成 int 导致结果稍有瑕疵,这是因为在 C 语言中字符常量 (例如 ‘a’) 的类型是 int 而不是 char。 Stack Overflow: What to do to make ‘_Generic(‘a’, char : 1, int : 2) == 1’ true ","date":"2024-03-25","objectID":"/posts/c-preprocessor/:3:0","tags":["Sysprog","C","Preprocessor"],"title":"你所不知道的 C 语言: 前置处理器应用篇","uri":"/posts/c-preprocessor/"},{"categories":["C","Linux Kernel Internals"],"content":"Block Wikipedia: Blocks (C language extension) Blocks are a non-standard extension added by Apple Inc. to Clang’s implementations of the C, C++, and Objective-C programming languages that uses a lambda expression-like syntax to create closures within these languages. Like function definitions, blocks can take arguments, and declare their own variables internally. Unlike ordinary C function definitions, their value can capture state from their surrounding context. A block definition produces an opaque value which contains both a reference to the code within the block and a snapshot of the current state of local stack variables at the time of its definition. The block may be later invoked in the same manner as a function pointer. The block may be assigned to variables, passed to functions, and otherwise treated like a normal function pointer, although the application programmer (or the API) must mark the block with a special operator (Block_copy) if it’s to be used outside the scope in which it was defined. 使用 BLock 可以减少宏展开时的重复计算次数。目前 clang 是支持 Block 这个扩展的,但是在编译时需要加上参数 -fblocks: $ clang -fblocks blocks-test.c -lBlocksRuntime 同时还需要 BlocksRuntime 这个库,按照仓库 README 安装即可: # clone repo $ git clone https://github.com/mackyle/blocksruntime.git $ cd blocksruntime/ # building $ ./buildlib # testing $ ./checktests # installing $ sudo ./installlib 除了 Block 之外,常见的避免 double evaluation 的方法还有利用 typeof 提前计算: #define DOUBLE(a) ((a) + (a)) #define DOUBLE(a) ({ \\ __typeof__(a) _x_in_DOUBLE = (a); \\ _x_in_DOUBLE + _x_in_DOUBLE; \\ }) ","date":"2024-03-25","objectID":"/posts/c-preprocessor/:4:0","tags":["Sysprog","C","Preprocessor"],"title":"你所不知道的 C 语言: 前置处理器应用篇","uri":"/posts/c-preprocessor/"},{"categories":["C","Linux Kernel Internals"],"content":"ARRAY_SIZE 宏 // get the number of elements in array #define ARRAY_SIZE(arr) (sizeof(arr) / sizeof((arr)[0])) 这样实作的 ARRAY_SIZE 宏有很大的隐患,例如它无法对传入的 arr 进行类型检查,如果碰上不合格的 C 程序员,在数组隐式转换成指针后使用 ARRAY_SIZE 宏会得到非预期的结果,我们需要在编译器就提醒程序员不要错用这个宏。 注意 阅读以下博客以理解 Linux 核心的 ARRAY_SIZE 原理机制和实作手法: Linux Kernel: ARRAY_SIZE() Linux 核心的 ARRAY_SIZE 宏在上面那个简陋版的宏的基础上,加上了类型检查,保证传入的是数组而不是指针: #define ARRAY_SIZE(arr) (sizeof(arr) / sizeof((arr)[0]) + __must_be_array(arr)) /* \u0026a[0] degrades to a pointer: a different type from an array */ #define __must_be_array(a) BUILD_BUG_ON_ZERO(__same_type((a), \u0026(a)[0])) /* Are two types/vars the same type (ignoring qualifiers)? */ #ifndef __same_type # define __same_type(a, b) __builtin_types_compatible_p(typeof(a), typeof(b)) #endif 6.54 Other built-in functions provided by GCC You can use the built-in function __builtin_types_compatible_p to determine whether two types are the same. This built-in function returns 1 if the unqualified versions of the types type1 and type2 (which are types, not expressions) are compatible, 0 otherwise. The result of this built-in function can be used in integer constant expressions. 6.6 Referring to a Type with typeof Another way to refer to the type of an expression is with typeof. The syntax of using of this keyword looks like sizeof, but the construct acts semantically like a type name defined with typedef. 所以 Linux 核心的 ARRAY_SIZE 宏额外加上了 __must_be_array 宏,但是这个宏在编译成功时会返回 0,编译失败自然就不需要考虑返回值了 🤣 所以它起到的作用是之前提到的类型检查,透过 BUILD_BUG_ON_ZERO 宏和 __same_type 宏。 从 Linux 核心 「提炼」 出的 array_size _countof Macro ","date":"2024-03-25","objectID":"/posts/c-preprocessor/:5:0","tags":["Sysprog","C","Preprocessor"],"title":"你所不知道的 C 语言: 前置处理器应用篇","uri":"/posts/c-preprocessor/"},{"categories":["C","Linux Kernel Internals"],"content":"do { … } while (0) 宏 用于避免 dangling else,即 if 和 else 未符合预期的配对 (常见于未使用 {} 包裹) 考虑以下情形: #define handler(cond) if (cond) foo() if (\u003ccondition1\u003e) handler(\u003cconditional2\u003e) else bar() 这个写法乍一看没什么问题,但是我们把它展开来看一下: if (\u003ccondition1\u003e) if (\u003cconditional2\u003e) foo() else bar() 显然此时由于未使用 {} 区块进行包裹,导致 else 部分与 handler 宏的 if 逻辑进行配对了。do {...} while (0) 宏的作用就是提供类似于 {} 区块的隔离性 (因为它的循环体只能执行一遍 🤣) 注意 下面的讨论是关于为什么要使用 do {...} while(0) 而不是 {},非常值得一读: Stack Overflow: C multi-line macro: do/while(0) vs scope block The more elegant solution is to make sure that macro expand into a regular statement, not into a compound one. 主要是考虑到对包含 {} 的宏,像一般的 statement 一样加上 ; 会导致之前的 if 语句结束,从而导致后面的 else 语句无法配对进而编译失败,而使用 do {...} while (0) 后面加上 ; 并不会导致这个问题。 ","date":"2024-03-25","objectID":"/posts/c-preprocessor/:6:0","tags":["Sysprog","C","Preprocessor"],"title":"你所不知道的 C 语言: 前置处理器应用篇","uri":"/posts/c-preprocessor/"},{"categories":["C","Linux Kernel Internals"],"content":"应用: String switch in C 这篇博文展示了如何在 C 语言中对 string 使用 switch case: String switch in C #define STRING_SWITCH_L(s) switch (*((int32_t *)(s)) | 0x20202020) #define MULTICHAR_CONSTANT(a,b,c,d) ((int32_t)((a) | (b) \u003c\u003c 8 | (c) \u003c\u003c 16 | (d) \u003c\u003c 24)) Note that STRING_SWITCH_L performs a bitwise OR with the 32-bit integral value – this is a fast means of lowering the case of four characters at once. 这里有一个 | 0x20202020 的位运算操作,这个运算的作用是将对应的字符转换成对应小写字符,具体可以参考本人于数值系统篇的 记录 (提示: 字符 ' ' 对应的 ASCII 编码为 0x20)。 然后 MULTICHAR_CONSTANT 则是将参数按小端字节序计算出对应的数值。 这篇博文说明了在 C 语言中对 string 使用 switch case 提升效能的原理 (除此之外还讲解了内存对齐相关的效能问题): More on string switch in C ","date":"2024-03-25","objectID":"/posts/c-preprocessor/:7:0","tags":["Sysprog","C","Preprocessor"],"title":"你所不知道的 C 语言: 前置处理器应用篇","uri":"/posts/c-preprocessor/"},{"categories":["C","Linux Kernel Internals"],"content":"应用: Linked List 的各式变种 宏和函数调用的效能对比: Simple code for checking the speed difference between function call and macro 在進行函式呼叫時,我們除了需要把參數推入特定的暫存器或是堆疊,還要儲存目前暫存器的值到堆疊。在函式呼叫數量少的狀況,影響不顯著,但隨著數量增長,就會導致程式運作比用 macro 實作時慢。 这也是为什么 Linux 核心对于 linked list 的功能大量采用宏来实现。 静态的 linked list 初始化需要使用到 compound literal: C99 6.5.2.5 Compound literals The type name shall specify an object type or an array of unknown size, but not a variable length array type. A postfix expression that consists of a parenthesized type name followed by a braceenclosed list of initializers is a compound literal. It provides an unnamed object whose value is given by the initializer list. If the type name specifies an array of unknown size, the size is determined by the initializer list as specified in 6.7.8, and the type of the compound literal is that of the completed array type. Otherwise (when the type name specifies an object type), the type of the compound literal is that specified by the type name. In either case, the result is an lvalue. C99 6.7.8 Initialization Each brace-enclosed initializer list has an associated current object. When no designations are present, subobjects of the current object are initialized in order according to the type of the current object: array elements in increasing subscript order, structure members in declaration order, and the first named member of a union. In contrast, a designation causes the following initializer to begin initialization of the subobject described by the designator. Initialization then continues forward in order, beginning with the next subobject after that described by the designator. ","date":"2024-03-25","objectID":"/posts/c-preprocessor/:8:0","tags":["Sysprog","C","Preprocessor"],"title":"你所不知道的 C 语言: 前置处理器应用篇","uri":"/posts/c-preprocessor/"},{"categories":["C","Linux Kernel Internals"],"content":"其它应用 ","date":"2024-03-25","objectID":"/posts/c-preprocessor/:9:0","tags":["Sysprog","C","Preprocessor"],"title":"你所不知道的 C 语言: 前置处理器应用篇","uri":"/posts/c-preprocessor/"},{"categories":["C","Linux Kernel Internals"],"content":"Unit Test 测试框架本质是提供一个框架模版,让程序员将精力放在测试逻辑的编写上。使用 C 语言的宏配合前置处理器,可以很方便地实现这个功能。 unity/unity_fixture.h Google Test ","date":"2024-03-25","objectID":"/posts/c-preprocessor/:9:1","tags":["Sysprog","C","Preprocessor"],"title":"你所不知道的 C 语言: 前置处理器应用篇","uri":"/posts/c-preprocessor/"},{"categories":["C","Linux Kernel Internals"],"content":"Object Model 同样的,使用 C 语言的宏和前置处理器,可以让 C 语言拥有 OOP 的表达能力: ObjectC: use as a superset of the C language adding a lot of modern concepts missing in C ","date":"2024-03-25","objectID":"/posts/c-preprocessor/:9:2","tags":["Sysprog","C","Preprocessor"],"title":"你所不知道的 C 语言: 前置处理器应用篇","uri":"/posts/c-preprocessor/"},{"categories":["C","Linux Kernel Internals"],"content":"Exception Handling 通过宏和 setjmp/longjmp 可以很轻松地实作出 C 语言的异常机制: ExtendedC library extends the C programming language through complex macros and other tricks that increase productivity, safety, and code reuse without needing to use a higher-level language such as C++, Objective-C, or D. include/exception.h ","date":"2024-03-25","objectID":"/posts/c-preprocessor/:9:3","tags":["Sysprog","C","Preprocessor"],"title":"你所不知道的 C 语言: 前置处理器应用篇","uri":"/posts/c-preprocessor/"},{"categories":["C","Linux Kernel Internals"],"content":"ADT 与之前所提的 Linux 核心的 linked list 类似,使用宏取代函数调用可以降低 抽象数据类型 (ADT) 的相关操作的效能损失: pearldb: A Lightweight Durable HTTP Key-Value Pair Database in C klib/ksort.h 通过宏展开实作的排序算法 成功 Linux 核心原始程式码也善用宏来扩充 ","date":"2024-03-25","objectID":"/posts/c-preprocessor/:9:4","tags":["Sysprog","C","Preprocessor"],"title":"你所不知道的 C 语言: 前置处理器应用篇","uri":"/posts/c-preprocessor/"},{"categories":["C","Linux Kernel Internals"],"content":"Linux 核心宏: BUILD_BUG_ON_ZERO 原文地址 简单来说就是编译时期就进行检查的 assert,我写了 相关笔记 来说明它的原理。 ","date":"2024-03-25","objectID":"/posts/c-preprocessor/:10:0","tags":["Sysprog","C","Preprocessor"],"title":"你所不知道的 C 语言: 前置处理器应用篇","uri":"/posts/c-preprocessor/"},{"categories":["C","Linux Kernel Internals"],"content":"Linux 核心原始程式码宏: max, min 原文地址 ","date":"2024-03-25","objectID":"/posts/c-preprocessor/:11:0","tags":["Sysprog","C","Preprocessor"],"title":"你所不知道的 C 语言: 前置处理器应用篇","uri":"/posts/c-preprocessor/"},{"categories":["C","Linux Kernel Internals"],"content":"Linux 核心原始程式码宏: contain_of 原文地址","date":"2024-03-25","objectID":"/posts/c-preprocessor/:12:0","tags":["Sysprog","C","Preprocessor"],"title":"你所不知道的 C 语言: 前置处理器应用篇","uri":"/posts/c-preprocessor/"},{"categories":["Systems"],"content":"操作系统使用正确的抽象使构造庞大的计算机软件/硬件生态从不可能变为可能。这门课围绕操作系统是 如何设计 (应用程序视角)、怎样实现 (硬件视角) 两个角度展开,分为两个主要部分: 原理课 (并发/虚拟化/持久化):以教科书内容为主,介绍操作系统的原理性内容。课程同时注重讲解操作系统相关的代码实现和编程技巧,包括操作系统中常用的命令行/代码工具、教学操作系统 xv6 的代码讲解等 理解操作系统最重要的实验部分: Mini labs (应用程序视角;设计):通过实现一系列有趣的 (黑科技) 代码理解操作系统中对象存在的意义和操作系统 API 的使用方法、设计理念 OS labs (计算机硬件视角;实现):基于一个简化的硬件抽象层实现多处理器操作系统内核,向应用程序提供一些基础操作系统 API 时隔一年,在跟随 B 站 up 主 @踌躇月光 从零编写一个基于 x86 架构的内核 Txics 后,终于可以跟得上 @绿导师 的课程了 🤣 这次以 2022 年的 OS 课程 作为主线学习,辅以 2023 年课程 和 2024 年课程 的内容加以补充、扩展,并搭配南大的 ICS 课程进行作业,后期可能会加入清华大学的 rCore 实验 (待定)。 tux 问题 JYY 2022 年的 OSDI 课程讲义和阅读材料是分开的,2023 年和 2024 年进行了改进,讲义和阅读材料合并成类似于共笔的材料,所以下面有一些 lectures 是没有阅读材料链接的。 ","date":"2024-03-24","objectID":"/posts/nju-osdi/:0:0","tags":["Linux","QEMU","RISC-V"],"title":"南京大学 操作系统: 设计与实现 重点提示","uri":"/posts/nju-osdi/"},{"categories":["Systems"],"content":"第 1 周: 绪论 ","date":"2024-03-24","objectID":"/posts/nju-osdi/:1:0","tags":["Linux","QEMU","RISC-V"],"title":"南京大学 操作系统: 设计与实现 重点提示","uri":"/posts/nju-osdi/"},{"categories":["Systems"],"content":"操作系统概述 (为什么要学操作系统) 信息 直播录影 / 讲义页面 一个 Talk 的经典三段式结构: Why? What? How? (这个真是汇报的大杀器 🤣) 1950s 的计算机 I/O 设备的速度已经严重低于处理器的速度,中断机制出现 (1953) 希望使用计算机的人越来越多;希望调用 API 而不是直接访问设备 批处理系统 = 程序的自动切换 (换卡) + 库函数 API 操作系统中开始出现 设备、文件、任务 等对象和 API 1960s 的计算机 可以同时载入多个程序而不用 “换卡” 了 能载入多个程序到内存且灵活调度它们的管理程序,包括程序可以调用的 API 既然操作系统已经可以在程序之间切换,为什么不让它们定时切换呢? 操作系统机制出现和发展的原因,不需要死记硬背,这些机制都是应需求而诞生、发展的,非常的自然。 什么是操作系统? 程序视角:对象 + API 硬件视角:一个 C 程序 实验环境: deepin 20.9 $ uname -a Linux cai-PC 5.15.77-amd64-desktop #2 SMP Thu Jun 15 16:06:18 CST 2023 x86_64 GNU/Linux 安装 tldr: $ sudo apt install tldr 有些系统可能没有预装 man 手册: $ sudo apt install manpages manpages-de manpages-de-dev manpages-dev manpages-posix manpages-posix-dev glibc-doc ","date":"2024-03-24","objectID":"/posts/nju-osdi/:1:1","tags":["Linux","QEMU","RISC-V"],"title":"南京大学 操作系统: 设计与实现 重点提示","uri":"/posts/nju-osdi/"},{"categories":["Systems"],"content":"操作系统上的程序 (什么是程序和编译器) 信息 直播录影 / 讲义页面 / 阅读材料 UNIX 哲学: Make each program do one thing well Expect the output of every program to become the input to another 什么是程序 计算机是构建在状态机 (数字电路) 之上的,所以运行在计算机之上的程序 (不管是操作系统还是应用,无论是源代码还是二进制) 都是状态机。C程序的状态机模型中,状态是由堆栈确定的,所以函数调用是状态迁移,因为它改变了堆栈,即改变了状态机的状态。明确这一点之后,我们可以通过模拟堆栈的方式,来将任意的递归程序改写为非递归程序,例如经典的汉诺塔程序。 程序 = 状态机 源代码 $S$ (状态机): 状态迁移 = 执行语句 二进制代码 $C$ (状态机): 状态迁移 = 执行指令 注意 jyy 所给的非递归汉诺塔程序也是通过模拟堆栈状态转移实现的,但是比较晦涩的一点是,对于每一个堆栈状态,都有可能需要执行最多 4 条语句 (对应 for 循环和 pc),这一点比较难懂。 只使用纯\"计算\"的指令 (无论是 deterministic 还是 non-deterministic) 无法使程序停下来,因为将程序本质是状态机,而状态机通过“计算”的指令只能从一个状态迁移到另一个状态,无法实现销毁状态机的操作 (对应退出/停下程序),要么死循环,要么 undefined behavior。这时需要程序对应的状态机之外的另一个东西来控制、管理该状态机,以实现程序的停下/退出操作,这就是 OS 的 syscall 存在的意义,它可以游离在程序对应的状态机之外,并修改状态机的内容 (因为程序呼叫 syscall 时已经全权授予 OS 对其状态内容进行修改)。 空的 _start 函数可以成功编译并链接,但是由于函数是空的,它会编译生成 retq 指令,这会导致 pc 跳转到不合法的区域,而正确的做法应该是使用 syscall exit 来结束该程序 (熟悉 C 语言函数调用的同学应该能看懂这段描述)。 // start.c int _start() {} // start.o 0000000000000000 \u003c_start\u003e: 0: 55 push %rbp 1: 48 89 e5 mov %rsp,%rbp 4: 90 nop 5: 5d pop %rbp 6: c3 retq 通过 syscall 实现了和 mininal.S 功能一致的最小 C 语言 hello, world 程序 mininal.c: #include \u003csys/syscall.h\u003e #include \u003cunistd.h\u003e int main() { char buf[] = \"\\033[01;31mHello, OS World\\033[0m\\n\"; syscall(SYS_write, 1, buf, sizeof(buf)); syscall(SYS_exit, 42); } System Calls Manual 如何在程序的两个视角之间切换? 从“状态机”的角度可以帮我们解决一个重要的基本问题: 什么是编译器??? 编译器: 源代码 S (状态机) $\\rightarrow$ 二进制代码 C (状态机) $$C=compile(S)$$ 即编译器的功能是将源代码对应的状态机 $S$ 转换成二进制代码对应的状态机 $C$。但是这里需要注意,这两个状态机不需要完全等价,只需要满足 $S$ 与 $C$ 的可观测行为严格一致 即可,这也是编译优化的理论基础:在保证观测一致性 (sound) 的前提下改写代码 (rewriting)。 Jserv 的讲座 並行程式設計: 執行順序 对这个有更清晰的讲解 可以通过以下指令来观察编译器的优化情况,以理解什么是观测一致性: $ gcc -On -c a.c # n couldbe 0, 1, 2, 3 $ objdump -d a.o 操作系统中的一般程序 对于操作系统之上的程序,它们看待操作系统的视角是 API (syscall),所以这门课中有一个很重要的工具:strace (system call trace 追踪程序运行时使用的系统调用,可以查看程序和操作系统的交互): $ sudo apt install strace $ strace ./hello-goodbye Linux manual page: strace 技巧 可以通过 apt-file 来检索文件名可能在那些 package 里,例如: $ sudo apt install apt-file $ sudo apt-file update $ sudo apt-file search \u003cfilename\u003e ","date":"2024-03-24","objectID":"/posts/nju-osdi/:1:2","tags":["Linux","QEMU","RISC-V"],"title":"南京大学 操作系统: 设计与实现 重点提示","uri":"/posts/nju-osdi/"},{"categories":["Systems"],"content":"第 2 周: 并发","date":"2024-03-24","objectID":"/posts/nju-osdi/:2:0","tags":["Linux","QEMU","RISC-V"],"title":"南京大学 操作系统: 设计与实现 重点提示","uri":"/posts/nju-osdi/"},{"categories":["Rust"],"content":" In this episode of Crust of Rust, we go over subtyping and variance — a niche part of Rust that most people don’t have to think about, but which is deeply ingrained in some of Rust’s borrow ergonomics, and occasionally manifests in confusing ways. In particular, we explore how trying to implement the relatively straightforward strtok function from C/C++ in Rust quickly lands us in a place where the function is more or less impossible to call due to variance! 整理自 John Gjengset 的影片 ","date":"2024-03-17","objectID":"/posts/subtying-and-variance/:0:0","tags":["Rust","Subtying","Variance"],"title":"Crust of Rust: Subtying and Variance","uri":"/posts/subtying-and-variance/"},{"categories":["Rust"],"content":"影片注解 ","date":"2024-03-17","objectID":"/posts/subtying-and-variance/:1:0","tags":["Rust","Subtying","Variance"],"title":"Crust of Rust: Subtying and Variance","uri":"/posts/subtying-and-variance/"},{"categories":["Rust"],"content":"strtok A sequence of calls to this function split str into tokens, which are sequences of contiguous characters separated by any of the characters that are part of delimiters. cplusplus: strtok cppreference: strtok ","date":"2024-03-17","objectID":"/posts/subtying-and-variance/:1:1","tags":["Rust","Subtying","Variance"],"title":"Crust of Rust: Subtying and Variance","uri":"/posts/subtying-and-variance/"},{"categories":["Rust"],"content":"shortening lifetimes 影片大概 19 分时给出了为何 cargo test 失败的推导,个人觉得非常巧妙 pub fn strtok\u003c'a\u003e(s: \u0026'a mut \u0026'a str, delimiter: char) { ... } let mut x = \"hello world\"; strtok(\u0026mut x, ' '); 为了更直观地表示和函数 strtok 的返回值 lifetime 无关,这里将返回值先去掉了。在调用 strtok 时,编译器对于参数 s 的 lifetime 推导如下: \u0026'a mut \u0026'a str \u0026 mut x \u0026'a mut \u0026'a str \u0026 mut \u0026'static str \u0026'a mut \u0026'static str \u0026 mut \u0026'static str \u0026'static mut \u0026'static str \u0026 mut \u0026'static str \u0026'static mut \u0026'static str \u0026'static mut \u0026'static str 所以 strtok 在接收参数 s 后 (通过传入 \u0026mut x),会推导其 lifetime 为 static,这就会导致后面使用 x 的不可变引用 (\u0026x) 时发生冲突。 ","date":"2024-03-17","objectID":"/posts/subtying-and-variance/:1:2","tags":["Rust","Subtying","Variance"],"title":"Crust of Rust: Subtying and Variance","uri":"/posts/subtying-and-variance/"},{"categories":["Rust"],"content":"Documentations 这里列举视频中一些概念相关的 documentation 学习的一手资料是官方文档,请务必自主学会阅读规格书之类的资料 ","date":"2024-03-17","objectID":"/posts/subtying-and-variance/:2:0","tags":["Rust","Subtying","Variance"],"title":"Crust of Rust: Subtying and Variance","uri":"/posts/subtying-and-variance/"},{"categories":["Rust"],"content":"Crate std 可以使用这里提供的搜素栏进行搜索 (BTW 不要浪费时间在 Google 搜寻上!) method str::find ","date":"2024-03-17","objectID":"/posts/subtying-and-variance/:2:1","tags":["Rust","Subtying","Variance"],"title":"Crust of Rust: Subtying and Variance","uri":"/posts/subtying-and-variance/"},{"categories":["Rust"],"content":"References The Rust Reference: Subtyping and Variance The Rustonomicon: Subtyping and Variance ","date":"2024-03-17","objectID":"/posts/subtying-and-variance/:3:0","tags":["Rust","Subtying","Variance"],"title":"Crust of Rust: Subtying and Variance","uri":"/posts/subtying-and-variance/"},{"categories":["C","Linux Kernel Internals"],"content":" 在许多应用程序中,递归 (recursion) 可以简单又优雅地解决貌似繁琐的问题,也就是不断地拆解原有问题为相似的子问题,直到无法拆解为止,并且定义最简化状况的处理机制,一如数学思维。递归对 C 语言程序开发者来说,绝对不会陌生,但能掌握者却少,很多人甚至难以讲出汉诺塔之外的使用案例。 究竟递归是如何优雅地解决真实世界的问题,又如何兼顾执行效率呢》我们从运作原理开始探讨,搭配若干 C 程序解说,并且我们将以简化过的 UNIX 工具为例,分析透过递归来大幅缩减程式码。 或许跟你想象中不同,Linux 核心的原始程式码里头也用到递归函数呼叫,特别在较复杂的实作,例如文件系统,善用递归可大幅缩减程式码,但这也导致追踪程序运作的难度大增。 原文地址 ","date":"2024-03-16","objectID":"/posts/c-recursion/:0:0","tags":["Sysprog","C","Recursion"],"title":"你所不知道的 C 语言: 递归调用篇","uri":"/posts/c-recursion/"},{"categories":["C","Linux Kernel Internals"],"content":"Recursion To Iterate is Human, to Recurse, Divine. http://coder.aqualuna.me/2011/07/to-iterate-is-human-to-recurse-divine.html 注意 笔者的递归 (Recursion) 是通过 UC Berkeley 的 CS61A: Structure and Interpretation of Computer Programs CS70: Discrete Mathematics and Probability Theory 学习的,这个搭配式的学习模式使得我在实作——递归 (cs61a) 和理论——归纳法 (cs70) 上相互配合理解,从而对递归在实作和理论上都有了充分认知。 ","date":"2024-03-16","objectID":"/posts/c-recursion/:1:0","tags":["Sysprog","C","Recursion"],"title":"你所不知道的 C 语言: 递归调用篇","uri":"/posts/c-recursion/"},{"categories":["C","Linux Kernel Internals"],"content":"递归并没有想象的那么慢 以最大公因数 (Greatest Common Divisor, GCD) 为例,分别以循环和递归进行实作: unsigned gcd_rec(unsigned a, unsigned b) { if (!b) return a; return gcd_rec(b, a % b); } unsigned gcd_itr(unsigned a, unsigned b) { while (b) { unsigned t = b; b = a % b; a = t; } return a; } 这两个函数在 clang/llvm 优化后的编译输出 (clang -S -O2 gcd.c) 的汇编是一样的: .LBB0_2: movl %edx, %ecx xorl %edx, %edx divl %ecx movl %ecx, %eax testl %edx, %edx jne .LBB1_2 技巧 遞迴 (Recursion) Tail recursion 可以被编译器进行k空间利用最优化,从而达到和循环一样节省空间,但这需要编译器支持,有些编译器并不支持 tail recursion 优化 🤣 虽然如此,将一般的递归改写为 tail recursion 还是可以获得极大的效能提升。 Source ","date":"2024-03-16","objectID":"/posts/c-recursion/:2:0","tags":["Sysprog","C","Recursion"],"title":"你所不知道的 C 语言: 递归调用篇","uri":"/posts/c-recursion/"},{"categories":["C","Linux Kernel Internals"],"content":"案例分析: 等效电阻 r ----------###------------- A -------- A | | | # # # R(r, n - 1) # r # ==\u003e # R(r, n) # # # | | | --------------------------- B -------- B $$ R(r,n)= \\begin{cases} r \u0026 \\text{if n = 1}\\\\ 1 / (\\frac1r + \\frac1{R(r, n - 1) + r}) \u0026 \\text{if n \u003e 1} \\end{cases} $$ def circuit(n, r): if n == 1: return r else: return 1 / (1 / r + 1 / (circuit(n - 1, r) + r)) ","date":"2024-03-16","objectID":"/posts/c-recursion/:3:0","tags":["Sysprog","C","Recursion"],"title":"你所不知道的 C 语言: 递归调用篇","uri":"/posts/c-recursion/"},{"categories":["C","Linux Kernel Internals"],"content":"案例分析: 数列输出 man 3 printf RETURN VALUE Upon successful return, these functions return the number of characters printed (excluding the null byte used to end output to strings). 可以通过 ulimit -s 来改 stack size,预设为 8MB ulimit User limits - limit the use of system-wide resources. -s The maximum stack size. 现代编译器的最优化可能会造成递归实作的非预期改变,因为编译器可能会对递归实作在编译时期进行一些优化,从而提高效能和降低内存使用。 ","date":"2024-03-16","objectID":"/posts/c-recursion/:4:0","tags":["Sysprog","C","Recursion"],"title":"你所不知道的 C 语言: 递归调用篇","uri":"/posts/c-recursion/"},{"categories":["C","Linux Kernel Internals"],"content":"递归程序设计 Recursive Programming ","date":"2024-03-16","objectID":"/posts/c-recursion/:5:0","tags":["Sysprog","C","Recursion"],"title":"你所不知道的 C 语言: 递归调用篇","uri":"/posts/c-recursion/"},{"categories":["C","Linux Kernel Internals"],"content":"Fibonacci sequence 使用矩阵配合快速幂算法,可以将时间复杂度从 $O(n)$ 降低到 $O(\\log n)$ 方法 时间复杂度 空间复杂度 Rcursive $O(2^n)$ $O(n)$ Iterative $O(n)$ $O(1)$ Tail recursion $O(n)$ $O(1)$ Q-Matrix $O(\\log n)$ $O(n)$ Fast doubling $O(\\log n)$ $O(1)$ 原文的 Q-Matrix 实作挺多漏洞的,下面为修正后的实作 (注意矩阵乘法的 memset 是必须的,否则会使用到栈上超出生命周期的 obeject): void matrix_multiply(int a[2][2], int b[2][2], int t[2][2]) { memset(t, 0, sizeof(int) * 2 * 2); for (int i = 0; i \u003c 2; i++) for (int j = 0; j \u003c 2; j++) for (int k = 0; k \u003c 2; k++) t[i][j] += a[i][k] * b[k][j]; } void matrix_pow(int a[2][2], int n, int t[2][2]) { if (n == 1) { t[0][0] = a[0][0]; t[0][1] = a[0][1]; t[1][0] = a[1][0]; t[1][1] = a[1][1]; return; } if (n % 2 == 0) { int t1[2][2]; matrix_pow(a, n \u003e\u003e 1, t1); matrix_multiply(t1, t1, t); return; } else { int t1[2][2], t2[2][2]; matrix_pow(a, n \u003e\u003e 1, t1); matrix_pow(a, (n \u003e\u003e 1) + 1, t2); matrix_multiply(t1, t2, t); return; } } int fib(int n) { if (n \u003c= 0) return 0; int A1[2][2] = {{1, 1}, {1, 0}}; int result[2][2]; matrix_pow(A1, n, result); return result[0][1]; } Fast doubling 公式: $$ \\begin{split} F(2k) \u0026= F(k)[2F(k+1) - F(k)] \\\\ F(2k+1) \u0026= F(k+1)^2+F(k)^2 \\end{split} $$ 具体推导: $$ \\begin{split} \\begin{bmatrix} F(2n+1) \\\\ F(2n) \\end{bmatrix} \u0026= \\begin{bmatrix} 1 \u0026 1 \\\\ 1 \u0026 0 \\end{bmatrix}^{2n} \\begin{bmatrix} F(1) \\\\ F(0) \\end{bmatrix}\\\\ \\\\ \u0026= \\begin{bmatrix} 1 \u0026 1 \\\\ 1 \u0026 0 \\end{bmatrix}^n \\begin{bmatrix} 1 \u0026 1 \\\\ 1 \u0026 0 \\end{bmatrix}^n \\begin{bmatrix} F(1) \\\\ F(0) \\end{bmatrix}\\\\ \\\\ \u0026= \\begin{bmatrix} F(n+1) \u0026 F(n) \\\\ F(n) \u0026 F(n-1) \\end{bmatrix} \\begin{bmatrix} F(n+1) \u0026 F(n) \\\\ F(n) \u0026 F(n-1) \\end{bmatrix} \\begin{bmatrix} 1 \\\\ 0 \\end{bmatrix}\\\\ \\\\ \u0026= \\begin{bmatrix} F(n+1)^2 + F(n)^2\\\\ F(n)F(n+1) + F(n-1)F(n) \\end{bmatrix} \\end{split} $$ 然后根据 $F(k + 1) = F(k) + F(k - 1)$ 可得 $F(2k)$ 情况的公式。 原文中非递增情形比较晦涩,但其本质是通过累加来逼近目标值: else { t0 = t3; // F(n-2); t3 = t4; // F(n-1); t4 = t0 + t4; // F(n) i++; } ","date":"2024-03-16","objectID":"/posts/c-recursion/:6:0","tags":["Sysprog","C","Recursion"],"title":"你所不知道的 C 语言: 递归调用篇","uri":"/posts/c-recursion/"},{"categories":["C","Linux Kernel Internals"],"content":"案例分析: 字符串反转 原文对于时间复杂度的分析貌似有些问题,下面给出本人的见解。第一种方法的时间复杂度为: $$ T(n) = 2T(n-1) + T(n-2) $$ 所以第一种方法的时间复杂度为 $O(2^n)$。 第二种方法只是列出了程式码,而没有说明递归函数的作用,在本人看来,递归函数一定要明确说明其目的,才能比较好理解递归的作用,所以下面给出递归函数 rev_core 的功能说明: // 返回字符串 head 的最大下标 (下标相对于 idx 偏移),并且将字符串 head 相对于 // 整条字符串的中间对称点进行反转 int rev_core(char *head, int idx) { if (head[idx] != '\\0') { int end = rev_core(head, idx + 1); if (idx \u003e end / 2) swap(head + idx, head + end - idx); return end; } return idx - 1; } char *reverse(char *s) { rev_core(s, 0); return s; } 时间复杂度显然为 $O(n)$ ","date":"2024-03-16","objectID":"/posts/c-recursion/:7:0","tags":["Sysprog","C","Recursion"],"title":"你所不知道的 C 语言: 递归调用篇","uri":"/posts/c-recursion/"},{"categories":["C","Linux Kernel Internals"],"content":"案例分析: 建立目录 mkdir [Linux manual page (2)] DESCRIPTION Create the DIRECTORY(ies), if they do not already exist. 补充一下递归函数 mkdir_r 的功能描述: // 从路径 `path` 的第 `level` 层开始创建目录 int mkdir_r(const char *path, int level); ","date":"2024-03-16","objectID":"/posts/c-recursion/:8:0","tags":["Sysprog","C","Recursion"],"title":"你所不知道的 C 语言: 递归调用篇","uri":"/posts/c-recursion/"},{"categories":["C","Linux Kernel Internals"],"content":"案例分析: 类似 find 的程序 opendir [Linux manual page (3)] RETURN VALUE The opendir() and fdopendir() functions return a pointer to the directory stream. On error, NULL is returned, and errno is set to indicate the error. readdir [Linux manual page (3)] RETURN VALUE On success, readdir() returns a pointer to a dirent structure. (This structure may be statically allocated; do not attempt to free(3) it.) If the end of the directory stream is reached, NULL is returned and errno is not changed. If an error occurs, NULL is returned and errno is set to indicate the error. To distinguish end of stream from an error, set errno to zero before calling readdir() and then check the value of errno if NULL is returned. 练习: 连同文件一起输出 练习: 将输出的 . 和 .. 过滤掉 ","date":"2024-03-16","objectID":"/posts/c-recursion/:9:0","tags":["Sysprog","C","Recursion"],"title":"你所不知道的 C 语言: 递归调用篇","uri":"/posts/c-recursion/"},{"categories":["C","Linux Kernel Internals"],"content":"案例分析: Merge Sort Program for Merge Sort in C MapReduce with POSIX Thread ","date":"2024-03-16","objectID":"/posts/c-recursion/:10:0","tags":["Sysprog","C","Recursion"],"title":"你所不知道的 C 语言: 递归调用篇","uri":"/posts/c-recursion/"},{"categories":["C","Linux Kernel Internals"],"content":"函数式程序开发 Toward Concurrency Functional programming in C Functional Programming 风格的 C 语言实作 ","date":"2024-03-16","objectID":"/posts/c-recursion/:11:0","tags":["Sysprog","C","Recursion"],"title":"你所不知道的 C 语言: 递归调用篇","uri":"/posts/c-recursion/"},{"categories":["C","Linux Kernel Internals"],"content":"递归背后的理论 YouTube: Lambda Calculus - Computerphile YouTube: Essentials: Functional Programming’s Y Combinator - Computerphile 第一个影片相对还蛮好懂,第二个影片对于非 PL 背景的人来说完全是看不懂,所以暂时先放弃了 第一个影片主要介绍函数式编程的核心概念: 函数可以像其它 object 一样被传递使用,没有额外的限制,并且 object 是可以由函数来定义、构建的,例如我们可以定义 true 和 false: TRUE: $\\lambda x.\\ \\lambda y.\\ x$ FALSE: $\\lambda x.\\ \\lambda y.\\ y$ 因为 true 和 false 就是用来控制流程的,为 true 时我们 do somthing,为 false 我们 do other,所以上面这种定义是有意义的,当然你也可以定义为其它,毕竟函数式编程让我们可以定义任意我们想定义的东西 🤣 接下来我们就可以通过先前定义的 TRUE 和 FALSE 来实现 NOT, AND, OR 这类操作了: NOT: $\\lambda b.\\ b.$ FALSE TRUE AND: $\\lambda x.\\ \\lambda y.\\ x.\\ y.$ FALSE OR: $\\lambda x.\\ \\lambda y.\\ x$ TRUE $y.$ 乍一看这个挺抽象的,其实上面的实现正体现了函数式编程的威力,我们以 NOT TRUE 的推导带大家体会一下: NOT TRUE $\\ \\ \\ \\ $ $b.$ FALSE TRUE $\\ \\ \\ \\ $ TRUE FALSE TRUE $\\ \\ \\ \\ $ TRUE (FALSE TRUE) $\\ \\ \\ \\ $ FALSE 其余推导同理 ","date":"2024-03-16","objectID":"/posts/c-recursion/:12:0","tags":["Sysprog","C","Recursion"],"title":"你所不知道的 C 语言: 递归调用篇","uri":"/posts/c-recursion/"},{"categories":["Linux","Linux Kernel Internals"],"content":" Linux 核心如同其它复杂的资讯系统,也提供 hash table 的实作,但其原始程式码中却藏有间接指针 (可参见 你所不知道的 C 语言: linked list 和非连续内存) 的巧妙和数学奥秘。 原文地址 ","date":"2024-03-16","objectID":"/posts/linux-hashtable/:0:0","tags":["Sysprog","Linux","Hash table"],"title":"Linux 核心的 hash table 实作","uri":"/posts/linux-hashtable/"},{"categories":["Linux","Linux Kernel Internals"],"content":"间接指针 Linux 核心的 hashtable 结构示意图: 不难看出,pprev 是指向上一个节点 next 的指针,即是指向 hlist_node * 的指针,而不是指向上一个节点 (hlist_node) 的指针,因为 hashtable 的数组中存放的是 hlist_node *,所以这样也简化了表示方法,将拉链和数组元素相互联系了起来。需要使用间接指针来实现 doubly linked 本质上是因为:拉链节点和数组节点在表示和操作上的不等价。 当然也可以将数组元素和拉链元素都统一为带有两个指针 prev 和 next 的 doubly linked list node,这样解决了之前所提的不等价,可以消除特判,但这样会导致存取数组元素时内存开销增大,进而降低 cache 的利用率。 信息 List, HList, and Hash Table 内核基础设施——hlist_head/hlist_node 结构解析 hlist数据结构图示说明 ","date":"2024-03-16","objectID":"/posts/linux-hashtable/:1:0","tags":["Sysprog","Linux","Hash table"],"title":"Linux 核心的 hash table 实作","uri":"/posts/linux-hashtable/"},{"categories":["Linux","Linux Kernel Internals"],"content":"hash 函数 Wikipedia: Hash function A hash function is any function that can be used to map data of arbitrary size to fixed-size values, though there are some hash functions that support variable length output. ","date":"2024-03-16","objectID":"/posts/linux-hashtable/:2:0","tags":["Sysprog","Linux","Hash table"],"title":"Linux 核心的 hash table 实作","uri":"/posts/linux-hashtable/"},{"categories":["Linux","Linux Kernel Internals"],"content":"常见 hash 策略 Division method $$ h(k) = k % N $$ Mid-square $$ h(k) = bits_{i,i+r-1}(k^2) $$ Folding addition $$ key = 3823749374 \\\\ 382\\ |\\ 374\\ |\\ 937\\ |\\ 4 \\\\ index = 382 + 374 + 937 + 4 = 1697 \\\\ $$ 先将 key 切成片段后再相加,也可以对相加后的结果做其他运算 Multiplication Method ","date":"2024-03-16","objectID":"/posts/linux-hashtable/:2:1","tags":["Sysprog","Linux","Hash table"],"title":"Linux 核心的 hash table 实作","uri":"/posts/linux-hashtable/"},{"categories":["Linux","Linux Kernel Internals"],"content":"Linux 核心的 hash 函数 Linux 核心的 hash.h 使用的是 Multiplication Method 策略,但是是通过整数和位运算实现的,没有使用到浮点数。 $$ \\begin{split} h(K) \u0026= \\lfloor m \\cdot (KA - \\lfloor KA \\rfloor) \\rfloor \\\\ h(K) \u0026= K \\cdot 2^w \\cdot A \u003e\u003e (w - p) \\end{split} $$ 上面两条式子的等价关键在于,使用 二进制编码 表示的整数和小数配合进行推导,进而只使用整数来实现,具体推导见原文。 $(\\sqrt{5} - 1 ) / 2 = 0.618033989$ $2654435761 / 4294967296 = 0.618033987$ $2^{32} = 4294967296$ 因此 val * GOLDEN_RATIO_32 \u003e\u003e (32 - bits) $\\equiv K \\times A \\times 2^w \u003e\u003e (w - p)$,其中 GOLDEN_RATIO_32 等于 $2654435761$ Linux 核心的 64 bit 的 hash 函数: #ifndef HAVE_ARCH_HASH_64 #define hash_64 hash_64_generic #endif static __always_inline u32 hash_64_generic(u64 val, unsigned int bits) { #if BITS_PER_LONG == 64 /* 64x64-bit multiply is efficient on all 64-bit processors */ return val * GOLDEN_RATIO_64 \u003e\u003e (64 - bits); #else /* Hash 64 bits using only 32x32-bit multiply. */ return hash_32((u32)val ^ __hash_32(val \u003e\u003e 32), bits); #endif } Linux 核心采用 golden ratio 作为 $A$,这是因为这样碰撞较少,且分布均匀: ","date":"2024-03-16","objectID":"/posts/linux-hashtable/:3:0","tags":["Sysprog","Linux","Hash table"],"title":"Linux 核心的 hash table 实作","uri":"/posts/linux-hashtable/"},{"categories":["C","Linux Kernel Internals"],"content":" 本讲座将带着学员重新探索函数呼叫背后的原理,从程序语言和计算机结构的发展简史谈起,让学员自电脑软硬件演化过程去掌握 calling convention 的考量,伴随着 stack 和 heap 的操作,再探讨 C 程序如何处理函数呼叫、跨越函数间的跳跃 (如 setjmp 和 longjmp),再来思索资讯安全和执行效率的议题。着重在计算机架构对应的支援和行为分析。 原文地址 ","date":"2024-03-15","objectID":"/posts/c-function/:0:0","tags":["Sysprog","C","Function"],"title":"你所不知道的 C 语言: 函数调用篇","uri":"/posts/c-function/"},{"categories":["C","Linux Kernel Internals"],"content":"function prototype Very early C compilers and language 一个小故事,可以解释 C 语言的一些设计理念,例如 switch-case 中每个 case 都需要 break The Development of the C Language Dennis M. Ritchie 讲述 C 语言漫长的发展史,并搭配程式码来说明当初为何如此设计、取舍考量。了解这些历史背景可以让我们成为更专业的 C 语言 Programmer Rationale for International Standard – Programming Languages – C 讲述 C 语言标准的变更,并搭配程式码解释变更的原理和考量 在早期的 C 语言中,并不需要 function prototype,因为当编译器发现一个函数名出现在表达式并且后面跟着左括号 (,例如 a = func(...),就会将该函数解读为:返回值类型预设为 int,参数类型和个数由调用者提供来决定,按照这样规则编写程式码,可以在无需事先定义函数即可先写调用函数的逻辑。但是这样设计也会造成潜在问题:程序员在调用函数时需要谨慎处理,需要自己检查调用时的参数类型和个数符合函数定义 (因为当时的编译器无法正确判断调用函数时的参数是否符合预期的类型和个数,当时编译器的能力与先前提到的规则是一体两面),并且返回值类型预设为 int (当时还没有 void 类型),所以对于函数返回值,也需要谨慎处理。 显然 function prototype 的缺失导致程式码编写极其容易出错,所以从 C99 开始就规范了 function prototype,这个规范除了可以降低 programmer 心智负担之外,还可以提高程序效能。编译器的最佳化阶段 (optimizer) 可以通过 function prototype 来得知内存空间的使用情形,从而允许编译器在函数调用表达式的上下文进行激进的最佳化策略,例如 const 的使用可以让编译器知道只会读取内存数据而不会修改内存数据,从而没有 side effect,可以进行激进的最优化。 int compare(const char *string1, const char *string2); void func2(int x) { char *str1, *str2; // ... x = compare(str1, str2); // ... } Rust 的不可变引用也是编译器可以进行更激进的最优化处理的一个例子 注意 为什么早期的 C 语言没有 function prototype 呢?因为早期的 C 语言,不管有多少个源程序文件,都是先通过 cat 合并成一个单元文件,在进行编译链接生成目标文件。这样就导致了就算写了 function prototye,使用 cat 合并时,这些 prototype 不一定会出现在我们期望的程序开始处,即无法利用 prototype 对于函数调用进行检查,所以干脆不写 prototype。 在 preprocessor 出现后,通过 #include 这类语法并搭配 preprocessor 可以保证对于每个源文件,都可以通过 function prototype 对函数调用进行参数个数、类型检查,因为 #include 语句位于源文件起始处,并且此时 C 语言程序的编译过程改变了: 对单一源文件进行预处理、编译,然后再对得到的目标文件进行链接。所以此时透过 preprocessor 可以保证 function prototype 位于函数调用之前,可以进行严格地检查。 ","date":"2024-03-15","objectID":"/posts/c-function/:1:0","tags":["Sysprog","C","Function"],"title":"你所不知道的 C 语言: 函数调用篇","uri":"/posts/c-function/"},{"categories":["C","Linux Kernel Internals"],"content":"编程语言的 function C 语言不允许 nested function 以简化编译器的设计 (当然现在的 gcc 提供 nested funtion 的扩展),即 C 语言的 function 是一等公民,位于语法最顶层 (top-level),因为支持 nested function 需要 staic link 机制来确认外层函数。 编程语言中的函数,与数学的函数不完全一致,编程语言的函数隐含了状态机的转换过程 (即有 side effect),只有拥有 Referential Transparency 特性的函数,才能和数学上的函数等价。 ","date":"2024-03-15","objectID":"/posts/c-function/:2:0","tags":["Sysprog","C","Function"],"title":"你所不知道的 C 语言: 函数调用篇","uri":"/posts/c-function/"},{"categories":["C","Linux Kernel Internals"],"content":"Process 与 C 程序 程序存放在磁盘时叫 Program,加载到内存后叫 “Process” Wikipedia: Application binary interface In computer software, an application binary interface (ABI) is an interface between two binary program modules. Often, one of these modules is a library or operating system facility, and the other is a program that is being run by a user. 在 Intel x86 架构中,当返回值可以放在寄存器时就放在寄存器中返回,以提高效能,如果放不下,则将返回值的起始地址放在寄存器中返回。 ","date":"2024-03-15","objectID":"/posts/c-function/:3:0","tags":["Sysprog","C","Function"],"title":"你所不知道的 C 语言: 函数调用篇","uri":"/posts/c-function/"},{"categories":["C","Linux Kernel Internals"],"content":"stack ","date":"2024-03-15","objectID":"/posts/c-function/:4:0","tags":["Sysprog","C","Function"],"title":"你所不知道的 C 语言: 函数调用篇","uri":"/posts/c-function/"},{"categories":["C","Linux Kernel Internals"],"content":"Layout System V Application Binary Interface AMD64 Architecture Processor Supplement [PDF] ","date":"2024-03-15","objectID":"/posts/c-function/:4:1","tags":["Sysprog","C","Function"],"title":"你所不知道的 C 语言: 函数调用篇","uri":"/posts/c-function/"},{"categories":["C","Linux Kernel Internals"],"content":"PEDA 实验需要使用到 GDB 的 PEDA 扩展: Enhance the display of gdb: colorize and display disassembly codes, registers, memory information during debugging. $ git clone https://github.com/longld/peda.git ~/peda $ echo \"source ~/peda/peda.py\" \u003e\u003e ~/.gdbinit 技巧 动态追踪 Stack 实验的 call funcA 可以通过 GDB 指令 stepi 或 si 来实现 ","date":"2024-03-15","objectID":"/posts/c-function/:4:2","tags":["Sysprog","C","Function"],"title":"你所不知道的 C 语言: 函数调用篇","uri":"/posts/c-function/"},{"categories":["C","Linux Kernel Internals"],"content":"从递归观察函数调用 infinite.c int func(int x) { static int count = 0; int y = x; // local var return ++count \u0026\u0026 func(x++); } int main() { return func(0); } func 函数在调用时,一个栈帧的内容包括: x (parameter), y (local variable), return address。这些数据的类型都是 int,即占据空间相同,这也是为什么计时器 count 的变化大致呈现 $x : \\frac{x}{2} : \\frac{x}{3}$ 的比例。 ","date":"2024-03-15","objectID":"/posts/c-function/:5:0","tags":["Sysprog","C","Function"],"title":"你所不知道的 C 语言: 函数调用篇","uri":"/posts/c-function/"},{"categories":["C","Linux Kernel Internals"],"content":"stack-based buffer overflow ","date":"2024-03-15","objectID":"/posts/c-function/:5:1","tags":["Sysprog","C","Function"],"title":"你所不知道的 C 语言: 函数调用篇","uri":"/posts/c-function/"},{"categories":["C","Linux Kernel Internals"],"content":"ROP ","date":"2024-03-15","objectID":"/posts/c-function/:6:0","tags":["Sysprog","C","Function"],"title":"你所不知道的 C 语言: 函数调用篇","uri":"/posts/c-function/"},{"categories":["C","Linux Kernel Internals"],"content":"heap 使用 malloc 时操作系统可能会 overcommit,而正因为这个 overcommit 的特性,malloc 返回有效地址也不见得是安全的。除此之外,因为 overcommit,使用 malloc 后立即搭配使用 memset 代价也很高 (因为操作系统 overcommit 可能会先分配一个小空间而不是一下子分配全部,因为它优先重复使用之前已使用过的小块空间),并且如果是设置为 0,则有可能会对原本为 0 的空间进行重复设置,降低效能。此时可以应该善用 calloc,虽然也会 overcommit,但是会保证分配空间的前面都是 0 (因为优先分配的是需要操作系统参与的大块空间),无需使用 memset 这类操作而降低效能。 ","date":"2024-03-15","objectID":"/posts/c-function/:7:0","tags":["Sysprog","C","Function"],"title":"你所不知道的 C 语言: 函数调用篇","uri":"/posts/c-function/"},{"categories":["C","Linux Kernel Internals"],"content":"malloc / free ","date":"2024-03-15","objectID":"/posts/c-function/:7:1","tags":["Sysprog","C","Function"],"title":"你所不知道的 C 语言: 函数调用篇","uri":"/posts/c-function/"},{"categories":["C","Linux Kernel Internals"],"content":"RAII ","date":"2024-03-15","objectID":"/posts/c-function/:8:0","tags":["Sysprog","C","Function"],"title":"你所不知道的 C 语言: 函数调用篇","uri":"/posts/c-function/"},{"categories":["C","Linux Kernel Internals"],"content":"setjmp \u0026 longjmp setjmp(3) — Linux manual page The functions described on this page are used for performing “nonlocal gotos”: transferring execution from one function to a predetermined location in another function. The setjmp() function dynamically establishes the target to which control will later be transferred, and longjmp() performs the transfer of execution. 具体解说可以阅读 lab0-c 的「自動測試程式」部分 ","date":"2024-03-15","objectID":"/posts/c-function/:9:0","tags":["Sysprog","C","Function"],"title":"你所不知道的 C 语言: 函数调用篇","uri":"/posts/c-function/"},{"categories":["Concurrency","Linux Kernel Internals"],"content":" 多執行緒環境下,程式會出問題,往往在於執行順序的不確定性。一旦顧及分散式系統 (distributed systems),執行順序和衍生的時序 (timing) 問題更加複雜。 我們將從如何定義程式執行的順序開始說起,為了簡單起見,我們先從單執行緒的觀點來看執行順序這件事,其中最關鍵知識就是 Sequenced-before,你將會發現就連單執行緒的程式,也可能會產生不確定的執行順序。 原文地址 ","date":"2024-03-11","objectID":"/posts/concurrency-ordering/:0:0","tags":["Sysprog","Linux","Concurrency"],"title":"并行程序设计: 执行顺序","uri":"/posts/concurrency-ordering/"},{"categories":["Concurrency","Linux Kernel Internals"],"content":"Evaluation 所謂求值 (Evaluation),其實指二件事情,一是 value computations,對一串運算式計算的結果;另一是 side effect,亦即修改物件狀態,像是修改記憶體內變數的值、呼叫函式庫的 I/O 處理函式之類的操作。 ","date":"2024-03-11","objectID":"/posts/concurrency-ordering/:1:0","tags":["Sysprog","Linux","Concurrency"],"title":"并行程序设计: 执行顺序","uri":"/posts/concurrency-ordering/"},{"categories":["Concurrency","Linux Kernel Internals"],"content":"Sequenced-before sequenced-before 是種對 同一個執行緒 下,求值順序關係的描述。 若 A is sequenced-before B,代表 A 的求值會先完成,才進行對 B 的求值 若 A is not sequenced before B 而且 B is sequenced before A,代表 B 的求值會先完成,才開始對 A 的求值。 若 A is not sequenced before B 而且 B is not sequenced before A,代表兩種可能,一種是順序不定,甚至這兩者的求值過程可能會重疊(因為 CPU 重排指令交錯的關係)或不重疊。 而程式語言的工作,就是定義一連串關於 sequenced-before 的規範,舉例來說: 以下提到的先於、先進行之類的用詞,全部的意思都是 sequenced-before,也就是「先完成之後才開始進行」 i++ 這類的後置運算子,value computation 會先於 side effect 對於 assignment operator 而言 (=, +=, -= 一類),會先進行運算元的 value computation,之後才是 assignment 的 side effect,最後是整個 assignment expression 的 value computation。 虽然规格书定义了关于 sequenced-before 的规范,但不可能面面俱到,还是存在有些执行顺序是未定义的,例如 f1() + f2() + f3(),规格书只规定了 + 操作是在对 f1(), f2(), f3() 求值之后进行的,但是对于求值时的 f1() 这类函数呼叫,并没有规定哪个函数先进行调用求值,所以在求值时第一个调用的可能是 f1() 或 f2() 或 f3()。 sequenced-before 的规范缺失导致了 partial order 场景的出现,二这可能会导致未定义行为,例如经典的头脑体操 i = i++: 出现这个未定义行为的原因是,i++ 的 side effect 与 = 之间不存在 sequenced-bofore 关系 (因为 partial order),而这会导致该语句的执行结果是不确定的 (没想到吧,单线程的程序你也有可能不确定执行顺序 🤣) 警告 注意: 在 C++17 後,上方敘述不是未定義行為 假設 i 初始值為 0,由於 = 在 C++17 後為 sequenced,因此 i++ 的計算與 side effect 都會先完成,所以 i++ 得到 0,隨後 side-effect 導致 i 遞增 1,因此此時 i 為 1;之後執行 i = 這邊,所以利用右側表達式的值來指定數值,亦即剛才的 0,因此 i 最後結果為 0。 所以 i 值轉變的順序為 $0 \\rightarrow 1 \\rightarrow 0$,第一個箭頭為 side effect 造成的結果,第二個則是 = 造成的結果。 C++ sequenced-before graphs Order of evaluation from cppreference What are sequence points, and how do they relate to undefined behavior? ","date":"2024-03-11","objectID":"/posts/concurrency-ordering/:2:0","tags":["Sysprog","Linux","Concurrency"],"title":"并行程序设计: 执行顺序","uri":"/posts/concurrency-ordering/"},{"categories":["Concurrency","Linux Kernel Internals"],"content":"Happens-before 短片: Happened Before Relationship Happened Before Relation (cont) 从上图可以看出 happens-before 其实就是在 sequenced-before 基础上增加了多执行绪 communication 情形,可以理解为 happens-before 将 sequenced-before 扩大到涵盖多执行绪的情形了,即执行顺序有先后次序 (执行顺序其实不太准确,执行结果显现 的先后次序更加准确 🤣) 图中的 concurrent events 其实就是多执行绪下没有先后次序的情形 Java 规格书 17.4.5. Happens-before Order 也能佐证我们的观点: If one action happens-before another, then the first is visible to and ordered before the second. 引用 換言之,若望文生義說 “Happens-Before” 是「先行發生」,那就南轅北轍。Happens-Before 並非說前一操作發生在後續操作的前面,它真正要表達的是:「前面一個操作的效果對後續操作是 可見」。 這裡的關鍵是,Happens-before 強調 visible,而非實際執行的順序。 實際程式在執行時,只需要「看起來有這樣的效果」即可,編譯器有很大的空間可調整程式執行順序,亦即 compile-time memory ordering。 因此我們得知一個關鍵概念: A happens-before B 不代表實際上 A happening before B (注意時態,後者強調進行中,前者則是從結果來看),亦即只要 A 的效果在 B 執行之前,對於 B 是 visible 即可,實際的執行順序不用細究。 C11 正式将并行和 memory order 相关的规范引入到语言的标准: 5.1.2.4 Multi-threaded executions and data races All modifications to a particular atomic object M occur in some particular total order, called the modification order of M. If A and B are modifications of an atomic object M, and A happens before B, then A shall precede B in the modification order of M, which is defined below. cppreference std::memory_order Regardless of threads, evaluation A happens-before evaluation B if any of the following is true: A is sequenced-before B A inter-thread happens before B 引用 通常程式開發者逐行撰寫程式,期望前一行的效果會影響到後一行的程式碼。稍早已解釋何謂 Sequenced-before,現在可注意到,Sequenced-before 實際就是同一個執行緒內的 happens-before 關係。 在多执行绪情况下,如果没法确保 happens-before 关系,程序往往会产生意料之外的结果,例如: int counter = 0; 如果现在有两个执行绪在同时执行,执行绪 A 执行 counter++,执行绪 B 将 counter 的值打印出来。因为 A 和 B 两个执行绪不具备 happens-before 关系,没有保证 counter++ 后的效果对打印 counter 是可见的,导致打印出来的可能是 1 也可能是 0,这个也就是图中的 concurrent events 关系。 引用 因此,程式語言必須提供適當的手段,讓程式開發者得以建立跨越執行緒間的 happens-before 的關係,如此一來才能確保程式執行的結果正確。 ","date":"2024-03-11","objectID":"/posts/concurrency-ordering/:3:0","tags":["Sysprog","Linux","Concurrency"],"title":"并行程序设计: 执行顺序","uri":"/posts/concurrency-ordering/"},{"categories":["Concurrency","Linux Kernel Internals"],"content":"The Happens-Before Relation The Happens-Before Relation Let A and B represent operations performed by a multithreaded process. If A happens-before B, then the memory effects of A effectively become visible to the thread performing B before B is performed. No matter which programming language you use, they all have one thing in common: If operations A and B are performed by the same thread, and A’s statement comes before B’s statement in program order, then A happens-before B. Happens-Before Does Not Imply Happening Before In this case, though, the store to A doesn’t actually influence the store to B. (2) still behaves the same as it would have even if the effects of (1) had been visible, which is effectively the same as (1)’s effects being visible. Therefore, this doesn’t count as a violation of the happens-before rule. Happening Before Does Not Imply Happens-Before The happens-before relationship only exists where the language standards say it exists. And since these are plain loads and stores, the C++11 standard has no rule which introduces a happens-before relation between (2) and (3), even when (3) reads the value written by (2). 这里说的 happens-before 关系必须要在语言标准中有规定的才算,单执行绪的情况自然在标准内,多执行绪的情况,标准一般会制定相关的同步原语之间的 happens-before 关系,例如对 mutex 的连续两个操作必然是 happens-before 关系,更多的例子见后面的 synchronized-with 部分。 ","date":"2024-03-11","objectID":"/posts/concurrency-ordering/:3:1","tags":["Sysprog","Linux","Concurrency"],"title":"并行程序设计: 执行顺序","uri":"/posts/concurrency-ordering/"},{"categories":["Concurrency","Linux Kernel Internals"],"content":"Synchronized-with 引用 synchronized-with 是個發生在二個不同執行緒間的同步行為,當 A synchronized-with B 時,代表 A 對記憶體操作的效果,對於 B 是可見的。而 A 和 B 是二個不同的執行緒的某個操作。 不難發現,其實 synchronized-with 就是跨越多個執行緒版本的 happens-before。 ","date":"2024-03-11","objectID":"/posts/concurrency-ordering/:4:0","tags":["Sysprog","Linux","Concurrency"],"title":"并行程序设计: 执行顺序","uri":"/posts/concurrency-ordering/"},{"categories":["Concurrency","Linux Kernel Internals"],"content":"从 Java 切入 synchronized 关键字 引用 Mutual Exclusive 對同一個物件而言,不可能有二個前綴 synchronized 的方法同時交錯執行,當一個執行緒正在執行前綴 synchronized 的方法時,其他想執行 synchronized 方法的執行緒會被阻擋 (block)。 確立 Happens-before 關係 對同一個物件而言,當一個執行緒離開 synchronized 方法時,會自動對接下來呼叫 synchronized 方法的執行緒建立一個 Happens-before 關係,前一個 synchronized 的方法對該物件所做的修改,保證對接下來進入 synchronized 方法的執行緒可見。 volatile 关键字 引用 A write to a volatile field happens-before every subsequent read of that same volatile thread create/join ","date":"2024-03-11","objectID":"/posts/concurrency-ordering/:4:1","tags":["Sysprog","Linux","Concurrency"],"title":"并行程序设计: 执行顺序","uri":"/posts/concurrency-ordering/"},{"categories":["Concurrency","Linux Kernel Internals"],"content":"C++ 的观点 The library defines a number of atomic operations and operations on mutexes that are specially identified as synchronization operations. These operations play a special role in making assignments in one thread visible to another. 又是 visible 说明强调的还是 happens-before 这一关系 🤣 #include \u003ciostream\u003e // std::cout #include \u003cthread\u003e // std::thread #include \u003cmutex\u003e // std::mutex std::mutex mtx; // mutex for critical section int count = 0; void print_thread_id (int id) { // critical section (exclusive access to std::cout signaled by locking mtx): mtx.lock(); std::cout \u003c\u003c \"thread #\" \u003c\u003c id \u003c\u003c \" count:\" \u003c\u003c count \u003c\u003c '\\n'; count++; mtx.unlock(); } int main () { std::thread threads[10]; // spawn 10 threads: for (int i=0; i\u003c10; ++i) threads[i] = std::thread(print_thread_id,i+1); for (auto\u0026 th : threads) th.join(); return 0; } 这段程序里每个执行的 thread 之间都是 happens-before / synchronized-with 关系,因为它们的执行体都被 mutex 包裹了,而对 mutex 的操作是 happens-before 关系的。如果没有使用 mutex,那么 thread 之间不存在 happens-before 关系,打印出来的内容也是乱七八糟的。 cppreference std::mutex cplusplus std::mutex::lock ","date":"2024-03-11","objectID":"/posts/concurrency-ordering/:4:2","tags":["Sysprog","Linux","Concurrency"],"title":"并行程序设计: 执行顺序","uri":"/posts/concurrency-ordering/"},{"categories":["Concurrency","Linux Kernel Internals"],"content":"深入 Synchronizes-with The Synchronizes-With Relation ","date":"2024-03-11","objectID":"/posts/concurrency-ordering/:4:3","tags":["Sysprog","Linux","Concurrency"],"title":"并行程序设计: 执行顺序","uri":"/posts/concurrency-ordering/"},{"categories":["Concurrency","Linux Kernel Internals"],"content":"Memory Consistency Models 技巧 相关论文 / 技术报告 (可以用来参考理解): Shared Memory Consistency Models: A Tutorial 1995 Sarita V. Adve, Kourosh Gharachorloo ","date":"2024-03-11","objectID":"/posts/concurrency-ordering/:5:0","tags":["Sysprog","Linux","Concurrency"],"title":"并行程序设计: 执行顺序","uri":"/posts/concurrency-ordering/"},{"categories":["Concurrency","Linux Kernel Internals"],"content":" 透过建立 Concurrency 和 Parallelism、Mutex 与 Semaphore 的基本概念,本讲座将透过 POSIX Tread 探讨 thread pool, Lock-Free Programming, lock-free 使用的 atomic 操作, memory ordering, M:N threading model 等进阶议题。 原文地址 ","date":"2024-03-08","objectID":"/posts/concurrency-concepts/:0:0","tags":["Sysprog","Linux","Concurrency"],"title":"并行程序设计: 概念","uri":"/posts/concurrency-concepts/"},{"categories":["Concurrency","Linux Kernel Internals"],"content":"Mutex 与 Semaphore Mutex 和 Semaphore 在实作上可能是没有差异的 (例如早期的 Linux),但是 Mutex 与 Semaphore 在使用上是有显著差异的: process 使用 Mutex 就像使用一把锁,谁先跑得快就能先获得锁,释放锁 “解铃还须系铃人”,并且释放锁后不一定能立即调度到等待锁的 process (如果想立即调度到等待锁的 process 需要进行显式调度) process 使用 Semaphore 就如同它的名字类似 “信号枪”,process 要么是等待信号的选手,要么是发出信号的裁判,并且裁判在发出信号后,选手可以立即收到信号并调度 (无需显式调度)。并不是你跑得快就可以先获得,如果你是选手,跑得快你也得停下来等裁判到场发出信号 🤣 注意 关于 Mutex 与 Semphore 在使用手法上的差异,可以参考我使用 Rust 实现的 Channel,里面的 Share\u003cT\u003e 结构体包含了 Mutex 和 Semphore,查看相关方法 (send 和 recv) 来研究它们在使用手法的差异。 除此之外,Semaphore 的选手和裁判的数量比例不一定是 $1:1$,可以是 $m:n$ ","date":"2024-03-08","objectID":"/posts/concurrency-concepts/:1:0","tags":["Sysprog","Linux","Concurrency"],"title":"并行程序设计: 概念","uri":"/posts/concurrency-concepts/"},{"categories":["Concurrency","Linux Kernel Internals"],"content":"CTSS Fernando J. Corbato: 1963 Timesharing: A Solution to Computer Bottlenecks ","date":"2024-03-08","objectID":"/posts/concurrency-concepts/:2:0","tags":["Sysprog","Linux","Concurrency"],"title":"并行程序设计: 概念","uri":"/posts/concurrency-concepts/"},{"categories":["Concurrency","Linux Kernel Internals"],"content":"可重入性 (Reentrancy) 一個可再進入 (reentrancy) 的函式是可被多個工作同時呼叫,而不會有資料不一致的問題。簡單來說,一個可再進入的函式,會避免在函式中使用任何共享記憶區 (global memory),所有的變數與資料均存在呼叫者的資料區或函式本身的堆疊區 (stack memory)。 ","date":"2024-03-08","objectID":"/posts/concurrency-concepts/:3:0","tags":["Sysprog","Linux","Concurrency"],"title":"并行程序设计: 概念","uri":"/posts/concurrency-concepts/"},{"categories":["Concurrency","Linux Kernel Internals"],"content":"经典的 Fork-join 模型 $\\rightarrow$ $\\rightarrow$ $\\rightarrow$ Fork-join Parallelism Fork/join model 从图也可以看出,设定一个 join 点是非常必要的 (通常是由主执行绪对 join 点进行设置),因为 fork 之后新增的执行绪有可能立刻就执行完毕了,然后当主执行绪到达 join 点时,即可 join 操作进行下一步,也有可能 fork 之后新增的执行绪是惰性的,它们只有当主执行绪到达 join 点时,才会开始执行直到完毕,即主执行绪先抵达 join 点等待其它执行绪完成执行,从而完成 join 操作接着进行下一步。 因为 fork 操作时分叉处的执行绪的执行流程,对于主执行绪是无法预测的 (立刻执行、惰性执行、…),所以设定一个 join 点可以保证在这个 join 点时主执行绪和其它分叉的执行绪的执行预期行为一致,即在这个 join 点,不管是主执行绪还是分叉执行绪都完成了相应的执行流程。 ","date":"2024-03-08","objectID":"/posts/concurrency-concepts/:4:0","tags":["Sysprog","Linux","Concurrency"],"title":"并行程序设计: 概念","uri":"/posts/concurrency-concepts/"},{"categories":["Concurrency","Linux Kernel Internals"],"content":"Concurrency 和 Parallelism Rob Pike: Concurrency Is Not Parallelism / slides Stack Overflow 上的相关讨论 Concurrency 是指程式架構,將程式拆開成多個可獨立運作的工作。案例: 裝置驅動程式,可獨立運作,但不需要平行化。 Parallelism 是指程式執行,同時執行多個程式。Concurrency 可能會用到 parallelism,但不一定要用 parallelism 才能實現 concurrency。案例: 向量內積計算 Concurrent, non-parallel execution Concurrent, parallel execution Tim Mattson (Intel): Introduction to OpenMP [YouTube] ","date":"2024-03-08","objectID":"/posts/concurrency-concepts/:5:0","tags":["Sysprog","Linux","Concurrency"],"title":"并行程序设计: 概念","uri":"/posts/concurrency-concepts/"},{"categories":["RISC-V"],"content":" 本文对通过 QEMU 仿真 RISC-V 环境并启动 OpenEuler RISC-V 系统的流程进行详细介绍,以及介绍如何通过 mugen 测试框架来对 RISC-V 版本的 openEuler 进行系统、软件等方面测试,并根据测试日志对错误原因进行分析。 ","date":"2024-03-07","objectID":"/posts/openeuler-riscv-qemu/:0:0","tags":["RISC-V","openEuler","QEMU","Mugen"],"title":"openEuler RISC-V 系统: QEMU 仿真","uri":"/posts/openeuler-riscv-qemu/"},{"categories":["RISC-V"],"content":"实验环境 操作系统: deepin 20.9 ","date":"2024-03-07","objectID":"/posts/openeuler-riscv-qemu/:1:0","tags":["RISC-V","openEuler","QEMU","Mugen"],"title":"openEuler RISC-V 系统: QEMU 仿真","uri":"/posts/openeuler-riscv-qemu/"},{"categories":["RISC-V"],"content":"安装支持 RISC-V 架构的 QEMU 模拟器 $ sudo apt install qemu-system-misc $ qemu-system-riscv64 --version QEMU emulator version 5.2.0 (Debian 1:5.2+dfsg-11+deb11u1) Copyright (c) 2003-2020 Fabrice Bellard and the QEMU Project developers 虽然 deepin 仓库提供的 QEMU 软件包版本比较低 (5.2.0),但是根据「引用文档」的说明,不低于 5.0 即可 通过上面安装的 QEMU 版本过低,无法支持 VGA 这类虚拟外设 (virtio),需要手动编译安装: 如果之前通过 apt 安装了 QEMU 的可以先进行卸载: $ sudo apt remove qemu-system-risc $ sudo apt autoremove 安装必要的构建工具: $ sudo apt install build-essential git libglib2.0-dev libfdt-dev libpixman-1-dev zlib1g-dev ninja-build libslirp-dev 下载 QEMU 源码包 (此处以 7.2 版本为例): $ wget https://download.qemu.org/qemu-7.2.0.tar.xz 解压源码包、修改名称: $ tar xvJf qemu-7.2.0.tar.xz $ mv qemu-7.2.0 qemu 进入 qemu 对应目录并配置编译选项: $./configure 编译安装: $ sudo make -j$(nproc) 在 ~/.bashrc 中添加环境变量: export PATH=$PATH:/path/to/qemu/build 刷新一下 ~/.bashrc (或新开一个终端) 查看一下 QEMU 是否安装成功: $ source ~/.bashrc $ qemu-system-riscv64 --version QEMU emulator version 7.2.0 Copyright (c) 2003-2022 Fabrice Bellard and the QEMU Project developers ","date":"2024-03-07","objectID":"/posts/openeuler-riscv-qemu/:2:0","tags":["RISC-V","openEuler","QEMU","Mugen"],"title":"openEuler RISC-V 系统: QEMU 仿真","uri":"/posts/openeuler-riscv-qemu/"},{"categories":["RISC-V"],"content":"下载 openEuler RISC-V 系统镜像 实验指定的测试镜像 (当然如果不是实验指定的话,你也可以使用其他的镜像): https://repo.tarsier-infra.com/openEuler-RISC-V/preview/openEuler-23.09-V1-riscv64/QEMU/ 由于我是直接使用 ssh 连接 openEuler RISC-V 的 QEMU 虚拟机,所以只下载了: fw_payload_oe_uboot_2304.bin 启动用内核 openEuler-23.09-V1-base-qemu-preview.qcow2.zst 不带有桌面镜像的根文件系统 start_vm.sh 启动不带有桌面镜像的根文件系统用脚本 $ wget https://repo.tarsier-infra.com/openEuler-RISC-V/preview/openEuler-23.09-V1-riscv64/QEMU/fw_payload_oe_uboot_2304.bin $ wget https://repo.tarsier-infra.com/openEuler-RISC-V/preview/openEuler-23.09-V1-riscv64/QEMU/openEuler-23.09-V1-base-qemu-preview.qcow2.zst $ wget https://repo.tarsier-infra.com/openEuler-RISC-V/preview/openEuler-23.09-V1-riscv64/QEMU/start_vm.sh 解压缩根文件系统的磁盘映像: # install unzip tool zstd for zst $ sudo apt install zstd $ unzstd openEuler-23.09-V1-base-qemu-preview.qcow2.zst ","date":"2024-03-07","objectID":"/posts/openeuler-riscv-qemu/:3:0","tags":["RISC-V","openEuler","QEMU","Mugen"],"title":"openEuler RISC-V 系统: QEMU 仿真","uri":"/posts/openeuler-riscv-qemu/"},{"categories":["RISC-V"],"content":"启动 openEuler RISC-V 系统并连接 确认当前在刚刚下载了内核、根文件系统、启动脚本的目录,然后在一个终端上执行启动脚本: $ bash start_vm.sh 安心等待输出完毕出现提示登录界面 (时间可能会有点长),然后输入账号和密码进行登录即可 或者启动 QEMU 虚拟机后,新开一个终端通过 ssh 进行登录: $ ssh -p 12055 root@localhost $ ssh -p 12055 openeuler@localhost 通过 exit 命令可以退出当前登录账号,通过快捷键 Ctrl + A, X 可以关闭 QEMU 虚拟机 (本质上是信号 signal 处理 🤣) 建议登录后修改账号的密码 (相关命令: passwd) ","date":"2024-03-07","objectID":"/posts/openeuler-riscv-qemu/:4:0","tags":["RISC-V","openEuler","QEMU","Mugen"],"title":"openEuler RISC-V 系统: QEMU 仿真","uri":"/posts/openeuler-riscv-qemu/"},{"categories":["RISC-V"],"content":"Mugen 测试框架 根据 「mugen」 README 的使用教程,在指定测试镜像上完成 「安装依赖软件」「配置测试套环境变量」「用例执行」这三个部分,并给出实验总结 ","date":"2024-03-07","objectID":"/posts/openeuler-riscv-qemu/:5:0","tags":["RISC-V","openEuler","QEMU","Mugen"],"title":"openEuler RISC-V 系统: QEMU 仿真","uri":"/posts/openeuler-riscv-qemu/"},{"categories":["RISC-V"],"content":"安装 git \u0026 克隆 mugen 仓库 登录普通用户 openeuler 然后发现此时没有安装 git 无法克隆 mugen 仓库,先安装 git: $ sudo dnf install git $ git clone https://gitee.com/openeuler/mugen.git 原始设定的 vim 配置不太优雅,我根据我的 vim 配置进行了设置,具体见 「Vim 配置」 ","date":"2024-03-07","objectID":"/posts/openeuler-riscv-qemu/:5:1","tags":["RISC-V","openEuler","QEMU","Mugen"],"title":"openEuler RISC-V 系统: QEMU 仿真","uri":"/posts/openeuler-riscv-qemu/"},{"categories":["RISC-V"],"content":"安装依赖软件 进入 mugen 目录执行安装依赖软件脚本 (因为我使用的是普通用户,需要使用 sudo 提高权级): $ sudo bash dep_install.sh ... Complete! ","date":"2024-03-07","objectID":"/posts/openeuler-riscv-qemu/:5:2","tags":["RISC-V","openEuler","QEMU","Mugen"],"title":"openEuler RISC-V 系统: QEMU 仿真","uri":"/posts/openeuler-riscv-qemu/"},{"categories":["RISC-V"],"content":"配置测试套环境变量 $ sudo bash mugen.sh -c --ip $ip --password $passwd --user $user --port $port 这部分仓库的文档对于本机测试没有很清楚地说明,参考文章 「基于openEuler虚拟机本地执行mugen测试脚本」完成配置 执行完成后会多出一个环境变量文件 ./conf/env.json ","date":"2024-03-07","objectID":"/posts/openeuler-riscv-qemu/:5:3","tags":["RISC-V","openEuler","QEMU","Mugen"],"title":"openEuler RISC-V 系统: QEMU 仿真","uri":"/posts/openeuler-riscv-qemu/"},{"categories":["RISC-V"],"content":"用例执行 \u0026 结果分析 我对于 openEuler RISC-V 是否支持了 binutils 比较感兴趣,便进行了测试: $ sudo bash mugen.sh -f binutils -x ... INFO - A total of 8 use cases were executed, with 8 successes 0 failures and 0 skips. 执行结果显示已正确支持 binutils 接下来对程序静态分析工具 cppcheck 的支持进行测试: $ sudo bash mugen.sh -f cppcheck -x ... INFO - A total of 2 use cases were executed, with 1 successes 1 failures and 0 skips. 根据文档 suite2cases 中 json文件的写法 的解释,分析刚刚执行的测试套 suit2cases/cppcheck.json,是测试用例 oe_test_cppcheck 失败了。 观察该用例对应的脚本 testcases/cli-test/cppcheck/oe_test_cppcheck/oe_test_cppcheck.sh,并打开对应的日志 logs/cppcheck/oe_test_cppcheck/$(date).log 在里面检索 LOG_ERROR,找到两处相关错误: + LOG_ERROR 'oe_test_cppcheck.sh line 70' + LOG_ERROR 'oe_test_cppcheck.sh line 95' 比照用例脚本,对应的测试逻辑是: cppcheck --std=c99 --std=posix test.cpp 70--\u003e CHECK_RESULT $? if [ $VERSION_ID != \"22.03\" ]; then cppcheck -DA --force file.c | grep \"A=1\" 95--\u003e CHECK_RESULT $? 1 else cppcheck -DA --force file.c | grep \"A=1\" CHECK_RESULT $? fi Cppcheck manual P11 The flag -D tells Cppcheck that a name is defined. There will be no Cppcheck analysis without this define. The flag –force and –max-configs is used to control how many combinations are checked. When -D is used, Cppcheck will only check 1 configuration unless these are used. 这里面 CHECK_RESULT 是一个自定义的 shell 函数,扫一下 mugen 的库目录 libs,在 locallibs/common_lib.sh 里找到该函数的定义,它的逻辑比较好懂 (类似于 assert),但是函数开头的变量定义让我有些迷糊,于是求教于 GPT: actual_result=$1 expect_result=${2-0} mode=${3-0} error_log=$4 GPT: actual_result 变量被赋值为第一个参数的值。 expect_result 变量被赋值为第二个参数的值,如果第二个参数不存在,则默认为 0。 mode 变量被赋值为第三个参数的值,如果第三个参数不存在,则默认为 0。 error_log 变量被赋值为第四个参数的值。 所以,涉及错误的两个测试逻辑都很好理解了: CHECK_RESULT $? 表示上一条命令返回值的预期是 0 CHECK_RESULT $? 1 表示上一条命令返回值的预期是 1 接下来我们就实际测试一下这两个用例: 安装 cppcheck: $ sudo dnf install cppcheck 执行测试脚本 70 行对应的上一条命令: $ cppcheck --std=c99 --std=posix test.cpp cppcheck: error: unknown --std value 'posix' $ echo $? 1 测试失败原因是 cppcheck risc-v 版本不支持指定 C/C++ 标准为 posix (同时查询了下 「cppcheck manual」目前 cppcheck 支持的标准里并未包括 posix) 执行测试脚本 95 行对应的上一条命令: $ cppcheck -DA --force file.c | grep \"A=1\" Checking file.c: A=1... file.c:5:6: error: Array 'a[10]' accessed at index 10, which is out of bounds. [arrayIndexOutOfBounds] a[10] = 0; ^ $ echo $? 0 测试失败原因是 grep 在之前的 cppcheck 的输出里匹配到 A=1,所以导致返回值为 0。这部分测试的逻辑是: 仅对于 22.03 版本 openEuler 上的 cppcheck 在以参数 -DA 执行时才会输出包含 A=1 的信息,但是个人猜测是在比 22.03 及更高版本的 openEuler 上使用 cppcheck 搭配 -DA 都可以输出包含 A=1 的信息 ","date":"2024-03-07","objectID":"/posts/openeuler-riscv-qemu/:5:4","tags":["RISC-V","openEuler","QEMU","Mugen"],"title":"openEuler RISC-V 系统: QEMU 仿真","uri":"/posts/openeuler-riscv-qemu/"},{"categories":["RISC-V"],"content":"实验总结和讨论 初步体验了使用 QEMU 构建 openEuler RISC-V 系统虚拟机的流程,以及使用 ssh 连接 QEMU 虚拟机的技巧。实验过程中最大感触是 mugen 的文档,相对于 cppcheck 这类产品的文档,不够详细,很多内容需要阅读源码来理解 (好处是精进了我对 shell 脚本编程的理解 🤣)。 我个人比较期待 RISC-V 配合 nommu 在嵌入式这类低功耗领域的发展,同时也对 RISC-V Hypervisor Extension 在虚拟化方面的发展感兴趣。 ","date":"2024-03-07","objectID":"/posts/openeuler-riscv-qemu/:5:5","tags":["RISC-V","openEuler","QEMU","Mugen"],"title":"openEuler RISC-V 系统: QEMU 仿真","uri":"/posts/openeuler-riscv-qemu/"},{"categories":["RISC-V"],"content":"References openEuler RISC-V: 通过 QEMU 仿真 RISC-V 环境并启动 OpenEuler RISC-V 系统 openEuler RISC-V: 使用 QEMU 安装 openEuler RISC-V 23.03 Ariel Heleneto: 通过 QEMU 仿真 RISC-V 环境并启动 OpenEuler RISC-V 系统 openEuler: mugen openEuler Docs: 使用 DNF 管理软件包 基于 openEuler 虚拟机本地执行 mugen 测试脚本 Video: Mugen 框架的使用 https://openbuildservice.org/help/manuals/obs-user-guide/ https://gitee.com/openEuler/RISC-V#/openeuler/RISC-V/ https://gitee.com/zxs-un/doc-port2riscv64-openEuler ","date":"2024-03-07","objectID":"/posts/openeuler-riscv-qemu/:6:0","tags":["RISC-V","openEuler","QEMU","Mugen"],"title":"openEuler RISC-V 系统: QEMU 仿真","uri":"/posts/openeuler-riscv-qemu/"},{"categories":["C","Linux Kernel Internals"],"content":" 借由阅读 C 语言标准理解规范是研究系统安全最基础的步骤,但很多人都忽略阅读规范这点,而正因对于规范的不了解、撰写程序的不严谨,导致漏洞的产生的案例比比皆是,例如 2014 年的 OpenSSL Heartbleed Attack1 便是便是因为使用 memcpy 之际缺乏对应内存范围检查,造成相当大的危害。本文重新梳理 C 语言程序设计的细节,并借由调试器帮助理解程序的运作。 原文地址 ","date":"2024-03-05","objectID":"/posts/c-std-security/:0:0","tags":["Sysprog","C","Security"],"title":"基于 C 语言标准研究与系统程序安全议题","uri":"/posts/c-std-security/"},{"categories":["C","Linux Kernel Internals"],"content":"目标 借由研读漏洞程序及 C 语言标准,讨论系统程序的安全议题 通过调试器追踪程序实际运行的状况,了解其运作原理 取材自 dangling pointer, CWE-416 Use After Free, CVE-2017-16943 以及 integer overflow 的议题 ","date":"2024-03-05","objectID":"/posts/c-std-security/:1:0","tags":["Sysprog","C","Security"],"title":"基于 C 语言标准研究与系统程序安全议题","uri":"/posts/c-std-security/"},{"categories":["C","Linux Kernel Internals"],"content":"实验环境 编译器版本: gcc 11 调试器: GDB 操作系统: Ubuntu Linux 22.04 ","date":"2024-03-05","objectID":"/posts/c-std-security/:2:0","tags":["Sysprog","C","Security"],"title":"基于 C 语言标准研究与系统程序安全议题","uri":"/posts/c-std-security/"},{"categories":["C","Linux Kernel Internals"],"content":"主题 (一): Integer type 资料处理 ","date":"2024-03-05","objectID":"/posts/c-std-security/:3:0","tags":["Sysprog","C","Security"],"title":"基于 C 语言标准研究与系统程序安全议题","uri":"/posts/c-std-security/"},{"categories":["C","Linux Kernel Internals"],"content":"I. Integer Conversion \u0026 Integer Promotion #include \u003cstdint.h\u003e #include \u003cstdio.h\u003e unsigned int ui = 0; unsigned short us = 0; signed int si = -1; int main() { int64_t r1 = ui + si; int64_t r2 = us + si; printf(\"%lld %lld\\n\", r1, r2); } 上述程式码执行结果为: r1 输出为十进制的 4294967295,r2 输出为十进制的 -1。这个结果和 C11 规格书中提到的 Integer 的两个特性有关: Integer Conversion 和 Integer Promotion。 (1) Integer Conversion C11 6.3.1.1 Boolean, characters, and integers Every integer type has an integer conversion rank defined as follows: No two signed integer types shall have the same rank, even if they hav e the same representation. The rank of a signed integer type shall be greater than the rank of any signed integer type with less precision. The rank of long long int shall be greater than the rank of long int, which shall be greater than the rank of int, which shall be greater than the rank of short int, which shall be greater than the rank of signed char. The rank of any unsigned integer type shall equal the rank of the corresponding signed integer type, if any. The rank of any standard integer type shall be greater than the rank of any extended integer type with the same width. The rank of any extended signed integer type relative to another extended signed integer type with the same precision is implementation-defined, but still subject to the other rules for determining the integer conversion rank. 依据上述标准可排出 integer 的 rank: long long int \u003e long int \u003e int \u003e short int \u003e signed char unsigned int == signed int, if they are both in same precision and same size (2) Integer Promotion 当 integer 进行通常的算数运算 (Usual arithmetic) 时,会先进行 integer promotions 转换成 int 或 unsigned int 或者保持不变 (转换后的运算子被称为 promoted operands),然后 promoted operands 再根据自身类型以及对应的 rank 进行 arithmetic conversions,最终得到结果的类型。 C11 6.3.1.1 Boolean, characters, and integers If an int can represent all values of the original type (as restricted by the width, for a bit-field), the value is converted to an int; otherwise, it is converted to an unsigned int. These are called the integer promotions. All other types are unchanged by the integer promotions. C11 6.3.1.8 Usual arithmetic conversions Otherwise, the integer promotions are performed on both operands. Then the following rules are applied to the promoted operands: If both operands have the same type, then no further conversion is needed. Otherwise, if both operands have signed integer types or both have unsigned integer types, the operand with the type of lesser integer conversion rank is converted to the type of the operand with greater rank. Otherwise, if the operand that has unsigned integer type has rank greater or equal to the rank of the type of the other operand, then the operand with signed integer type is converted to the type of the operand with unsigned integer type. Otherwise, if the type of the operand with signed integer type can represent all of the values of the type of the operand with unsigned integer type, then the operand with unsigned integer type is converted to the type of the operand with signed integer type. Otherwise, both operands are converted to the unsigned integer type corresponding to the type of the operand with signed integer type. /* In the case that the rank is smaller than int */ char c1, c2; // Both of them are char c1 = c1 + c2; // Both are promoted to int, thus result of c1 becomes to integer /* In the case that the rank is same as int */ signed int si = -1; /* si \u0026 ui are at the same rank both are unchanged by the integer promotions */ unsigned int ui = 0; int result = si + ui; // si is converted to unsigned int, result is unsigned ","date":"2024-03-05","objectID":"/posts/c-std-security/:3:1","tags":["Sysprog","C","Security"],"title":"基于 C 语言标准研究与系统程序安全议题","uri":"/posts/c-std-security/"},{"categories":["C","Linux Kernel Internals"],"content":"II. 衍生的安全议题: Integer Overflow Stack Overflow: What is an integer overflow error? ","date":"2024-03-05","objectID":"/posts/c-std-security/:3:2","tags":["Sysprog","C","Security"],"title":"基于 C 语言标准研究与系统程序安全议题","uri":"/posts/c-std-security/"},{"categories":["C","Linux Kernel Internals"],"content":"主题 (二): Object 的生命周期 ","date":"2024-03-05","objectID":"/posts/c-std-security/:4:0","tags":["Sysprog","C","Security"],"title":"基于 C 语言标准研究与系统程序安全议题","uri":"/posts/c-std-security/"},{"categories":["C","Linux Kernel Internals"],"content":"I. Dangling Pointer C11 6.2.4 Storage durations of objects (2) The lifetime of an object is the portion of program execution during which storage is guaranteed to be reserved for it. An object exists, has a constant address, and retains its last-stored value throughout its lifetime. If an object is referred to outside of its lifetime, the behavior is undefined. The value of a pointer becomes indeterminate when the object it points to (or just past) reaches the end of its lifetime. Stack Overflow: What is a dangling pointer? When a pointer is pointing at the memory address of a variable but after some time that variable is deleted from that memory location while the pointer is still pointing to it, then such a pointer is known as a dangling pointer and this problem is known as the dangling pointer problem. 所以在 object 的生命周期结束后,应将指向 object 原本处于的内存空间的指针置为 NULL,避免 dangling pointer。 ","date":"2024-03-05","objectID":"/posts/c-std-security/:4:1","tags":["Sysprog","C","Security"],"title":"基于 C 语言标准研究与系统程序安全议题","uri":"/posts/c-std-security/"},{"categories":["C","Linux Kernel Internals"],"content":"II. CWE-416 Use After Free OWASP: Using freed memory Referencing memory after it has been freed can cause a program to crash. The use of heap allocated memory after it has been freed or deleted leads to undefined system behavior and, in many cases, to a write-what-where condition. ","date":"2024-03-05","objectID":"/posts/c-std-security/:4:2","tags":["Sysprog","C","Security"],"title":"基于 C 语言标准研究与系统程序安全议题","uri":"/posts/c-std-security/"},{"categories":["C","Linux Kernel Internals"],"content":"III. 案例探讨: CVE-2017-16943 Abusing UAF leads to Exim RCE Road to Exim RCE - Abusing Unsafe Memory Allocator in the Most Popular MTA ","date":"2024-03-05","objectID":"/posts/c-std-security/:4:3","tags":["Sysprog","C","Security"],"title":"基于 C 语言标准研究与系统程序安全议题","uri":"/posts/c-std-security/"},{"categories":["C","Linux Kernel Internals"],"content":"实验结果与验证 Source ","date":"2024-03-05","objectID":"/posts/c-std-security/:5:0","tags":["Sysprog","C","Security"],"title":"基于 C 语言标准研究与系统程序安全议题","uri":"/posts/c-std-security/"},{"categories":["C","Linux Kernel Internals"],"content":"(ㄧ) Integer Promotion 验证 测试程式码: #include \u003cstdint.h\u003e #include \u003cstdio.h\u003e unsigned int ui = 0; unsigned short us = 0; signed int si = -1; int main() { int64_t r1 = ui + si; int64_t r2 = us + si; printf(\"%lld %lld\\n\", r1, r2); } 验证结果: $ gcc -g -o integer-promotion.o integer-promotion.c $ ./integer-promotion.o 4294967295 -1 ","date":"2024-03-05","objectID":"/posts/c-std-security/:5:1","tags":["Sysprog","C","Security"],"title":"基于 C 语言标准研究与系统程序安全议题","uri":"/posts/c-std-security/"},{"categories":["C","Linux Kernel Internals"],"content":"(二) Object 生命周期 测试程式码: #include \u003cinttypes.h\u003e #include \u003cstdint.h\u003e #include \u003cstdio.h\u003e #include \u003cstdlib.h\u003e int main(int argc, char *argv[]) { char *p, *q; uintptr_t pv, qv; { char a = 3; p = \u0026a; pv = (uintptr_t) p; } { char b = 4; q = \u0026b; qv = (uintptr_t) q; } if (p != q) { printf(\"%p is different from %p\\n\", (void *) p, (void *) q); printf(\"%\" PRIxPTR \" is not the same as %\" PRIxPTR \"\\n\", pv, qv); } else { printf(\"Surprise!\\n\"); } return 0; } 验证结果: $ gcc -g -o uaf.o uaf.c $ ./uaf.o Surprise! $ gcc -g -o uaf.o uaf.c -fsanitize-address-use-after-scope $ ./uaf.o 0x7ffca405c596 is different from 0x7ffca405c597 7ffca405c596 is not the same as 7ffca405c597 $ clang -g -o uaf.o uaf.c $ ./uaf.o 0x7fff86b298ff is different from 0x7fff86b298fe 7fff86b298ff is not the same as 7fff86b298fe gcc 可以通过显式指定参数 -fsanitize-address-use-after-scope 来避免 Use-After-Scope 的问题,否则在 scope 结束后,接下来的其他 scope 会使用之前已结束的 scope 的内存空间,从而造成 Use-After-Scope 问题 (使用 GDB 在上面两种不同的情况下,查看变量 a, b 所在的地址),而 clang 则是默认开启相关保护。 “OpenSSL Heartbleed”, Synopsys ↩︎ ","date":"2024-03-05","objectID":"/posts/c-std-security/:5:2","tags":["Sysprog","C","Security"],"title":"基于 C 语言标准研究与系统程序安全议题","uri":"/posts/c-std-security/"},{"categories":["Rust"],"content":" In this Crust of Rust episode, we implement some common sorting algorithms in Rust. This episode doesn't aim to explain any single concept, but rather showcase what writing “normal” Rust code is like, and explaining various “odd bits” we come across along the way. The thinking here is that sorting algorithms are both familiar and easy to compare across languages, so this might serve as a good bridge into Rust if you are familiar with other languages. 整理自 John Gjengset 的影片 问题 You may note that the url of this posy is “orst”. Why was it given this name? Since “sort” when sorted becomes “orst”. 🤣 ","date":"2024-03-04","objectID":"/posts/orst/:0:0","tags":["Rust","Sort","Algorithm"],"title":"Crust of Rust: Sorting Algorithms","uri":"/posts/orst/"},{"categories":["Rust"],"content":"影片注解 ","date":"2024-03-04","objectID":"/posts/orst/:1:0","tags":["Rust","Sort","Algorithm"],"title":"Crust of Rust: Sorting Algorithms","uri":"/posts/orst/"},{"categories":["Rust"],"content":"Total order vs Partial order Wikipedia: Total order Wikipedia: Partial order Stack Overflow: What does it mean by “partial ordering” and “total ordering” in the discussion of Lamport's synchronization Algorithm? This definition says that in a total order any two things are comparable. Wheras in a partial order a thing needs neither to be “smaller” than an other nor the other way around, in a total order each thing is either “smaller” than an other or the other way around. 简单来说,在 total order 中任意两个元素都可以进行比较,而在 partial order 中则不一定满足。例如对于集合 $$ S = \\{a,\\ b,\\ c\\} $$ 在 total order 中,$a, b, c$ 任意两个元素之间都必须能进行比较,而在 partial order 中没有怎么严格的要求,可能只有 $a \u003c b, b \u003c c$ 这两条比较规则。 在 Rust 中,浮点数 (f32, f64) 只实现了 PartialOrd 这个 Trait 而没有实现 Ord,因为根据 IEEE 754,浮点数中存在一些特殊值,例如 NaN,它们是没法进行比较的。出于相同原因,浮点数也只实现了 PartialEq 而没有实现 Eq trait。 ","date":"2024-03-04","objectID":"/posts/orst/:1:1","tags":["Rust","Sort","Algorithm"],"title":"Crust of Rust: Sorting Algorithms","uri":"/posts/orst/"},{"categories":["Rust"],"content":"Trait \u0026 Generic pub fn sort\u003cT, S\u003e(slice: \u0026mut [T]) where T: Ord, S: Sorter\u003cT\u003e, { S::sort(slice); } sort::\u003c_, StdSorter\u003e(\u0026mut things); 这段代码巧妙地利用泛型 (generic) 来传递了\"参数\",当然这种技巧只限于可以通过类型来调用方法的情况 (上面代码段的 S::sort(...) 以及 sort::\u003c_, StdSorter\u003e(...) 片段)。 思考以下代码表示的意义: pub trait Sorter\u003cT\u003e { fn sort(slice: \u0026mut [T]) where T: Ord; } pub trait Sorter { fn sort\u003cT\u003e(slice: \u0026mut [T]) where T: Ord; } 第一个表示的是有多个 tait,例如 Sorter\u003ci32\u003e, Sorter\u003ci64\u003e 等,第二个表示只有一个 trait Sorter,但是实现这个 trait 需要实现多个方法,例如 sort\u003ci32\u003e, sort\u003ci64\u003e 等,所以第一种写法更加普适和使用 (因为未必能完全实现第二种 trait 要求的所有方法)。 ","date":"2024-03-04","objectID":"/posts/orst/:1:2","tags":["Rust","Sort","Algorithm"],"title":"Crust of Rust: Sorting Algorithms","uri":"/posts/orst/"},{"categories":["Rust"],"content":"Bubble sort Wikipedia: Bubble sort n := length(A) repeat swapped := false for i := 1 to n-1 inclusive do { if this pair is out of order } if A[i-1] \u003e A[i] then { swap them and remember something changed } swap(A[i-1], A[i]) swapped := true end if end for until not swapped ","date":"2024-03-04","objectID":"/posts/orst/:1:3","tags":["Rust","Sort","Algorithm"],"title":"Crust of Rust: Sorting Algorithms","uri":"/posts/orst/"},{"categories":["Rust"],"content":"Insertion sort Wikipedia: Insertion sort i ← 1 while i \u003c length(A) j ← i while j \u003e 0 and A[j-1] \u003e A[j] swap A[j] and A[j-1] j ← j - 1 end while i ← i + 1 end while 使用 Binary search algorithm 可以将 insertion sort 的 comparsion 次数降到 $O(nlogn)$,但是 swap 次数仍然是 $O(n^2)$ 🤣 // use binary search to find index // then use .insert to splice in i let i = match slice[..unsorted].binary_search(\u0026slice[unsorted]) { // [ a, c, e].binary_search(c) =\u003e Ok(1) Ok(i) =\u003e i, // [ a, c, e].binary_search(b) =\u003e Err(1) Err(i) =\u003e i, }; slice[i..=unsorted].rotate_right(1); match 的内部逻辑也可以改写为 OK(i) | Err(i) =\u003e i ","date":"2024-03-04","objectID":"/posts/orst/:1:4","tags":["Rust","Sort","Algorithm"],"title":"Crust of Rust: Sorting Algorithms","uri":"/posts/orst/"},{"categories":["Rust"],"content":"Selection sort Wikipedia: Selection sort 引用 There are many different ways to sort the cards. Here’s a simple one, called selection sort, possibly similar to how you sorted the cards above: Find the smallest card. Swap it with the first card. Find the second-smallest card. Swap it with the second card. Find the third-smallest card. Swap it with the third card. Repeat finding the next-smallest card, and swapping it into the correct position until the array is sorted. source 使用函数式编程可以写成相当 readable 的程式码,以下为获取 slice 最小值对应的 index: let smallest_in_rest = slice[unsorted..] .iter() .enumerate() .min_by_key(|\u0026(_, v)| v) .map(|(i, _)| unsorted + i) .expect(\"slice is not empty\"); ","date":"2024-03-04","objectID":"/posts/orst/:1:5","tags":["Rust","Sort","Algorithm"],"title":"Crust of Rust: Sorting Algorithms","uri":"/posts/orst/"},{"categories":["Rust"],"content":"Quicksort Wikipedia: Quicksort 可以通过 extra allocation 和 in-place 两种方式来实现 quicksort,其中 extra allocation 比较好理解,in-place 方式的 pseudocode 如下: Quicksort(A,p,r) { if (p \u003c r) { q \u003c- Partition(A,p,r) Quicksort(A,p,q) Quicksort(A,q+1,r) } } Partition(A,p,r) x \u003c- A[p] i \u003c- p-1 j \u003c- r+1 while (True) { repeat { j \u003c- j-1 } until (A[j] \u003c= x) repeat { i \u003c- i+1 } until (A[i] \u003e= x) if (i \u003c j) swap(A[i], A[j]) else return(j) } } source method slice::split_at_mut 实现 Quick sort 时使用了 split_at_mut 来绕开引用检查,因为如果你此时拥有一个指向 pivot 的不可变引用,就无法对 slice 剩余的部分使用可变引用,而 split_at_mut 则使得原本的 slice 被分为两个可变引用,从而绕开了之前的单一引用检查。 后面发现可以使用更符合语义的 split_first_mut,当然思路还是一样的 注意 我个人认为实现 Quick sort 的关键在于把握以下两个 invariants: left: current checking index for element which is equal or less than the pivot right: current checking index for element which is greater than the pivot 即这两个下标对应的元素只是当前准备检查的,不一定符合元素的排列规范,如下图所示: [ \u003c= pivot ] [ ] [ ... ] [ ] [ \u003e pivot ] ^ ^ | | left right 所以当 left == right 时两边都没有对所指向的元素进行检查,分情况讨论 (该元素是 $\u003c= pivot$ 或 $\u003e pivot$) 可以得出: 当 left \u003e right 时,right 指向的是 $\u003c= pivot$ 的元素,将其与 pivot 进行 swap 即可实现 partition 操作。(其实此时 left 指向的是 $\u003e pivot$ 部分的第一个元素,right 指向的是 $\u003c= pivot$ 部分的最后一个元素,但是需要注意 rest 与 slice 之间的下标转换) ","date":"2024-03-04","objectID":"/posts/orst/:1:6","tags":["Rust","Sort","Algorithm"],"title":"Crust of Rust: Sorting Algorithms","uri":"/posts/orst/"},{"categories":["Rust"],"content":"Benchmark 通过封装类型 SortEvaluator 及实现 trait PartialEq, Eq, PartialOrd, Ord 来统计排序过程中的比较操作 (eq, partial_cmp, cmp) 的次数。 Stack Overflow: Why can't the Ord trait provide default implementations for the required methods from the inherited traits using the cmp function? ","date":"2024-03-04","objectID":"/posts/orst/:1:7","tags":["Rust","Sort","Algorithm"],"title":"Crust of Rust: Sorting Algorithms","uri":"/posts/orst/"},{"categories":["Rust"],"content":"R and ggplot2 # install R $ sudo apt install r-base # install ggplot2 by R $ R \u003e install.packages(\"ggplot2\") Are there Unix-like binaries for R? https://ggplot2.tidyverse.org/ 问题 deepin 软件源下载的 R 语言包可能版本过低 (3.5),可以通过添加库源的方式来下载高版本的 R 语言包: 1.添加 Debian buster (oldstable) 库源到 /etc/apt/sourcelist 里: # https://mirrors.tuna.tsinghua.edu.cn/CRAN/ deb http://cloud.r-project.org/bin/linux/debian buster-cran40/ 2.更新软件,可能会遇到没有公钥的问题 (即出现下方的 NO_PUBKEY): $ sudo apt update ... NO_PUBKEY XXXXXX ... 此时可以 NO_PUBKEY 后的 XXXXXX 就是公钥,我们只需要将其添加一下即可: $ sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys XXXXXX 添加完公钥后再重新更新一次软件源 3.通过指定库源的方式来安装 R (如果未指定库源则还是从默认源进行下载 3.5 版本): $ sudo apt install buster-cran40 r-base $ R --version R version 4.3.3 (2024-02-29) 大功告成,按照上面安装 ggplot2 即可 ","date":"2024-03-04","objectID":"/posts/orst/:1:8","tags":["Rust","Sort","Algorithm"],"title":"Crust of Rust: Sorting Algorithms","uri":"/posts/orst/"},{"categories":["Rust"],"content":"Homework 信息 实作说明: 添加标准库的 sort_unstable 进入基准测试 将交换操作 (swap) 纳入基准测试 尝试实现 Merge sort 尝试实现 Heapsort 参考资料: Wikipedia: Merge sort Wikipedia: Heapsort ","date":"2024-03-04","objectID":"/posts/orst/:2:0","tags":["Rust","Sort","Algorithm"],"title":"Crust of Rust: Sorting Algorithms","uri":"/posts/orst/"},{"categories":["Rust"],"content":"Documentations 这里列举视频中一些概念相关的 documentation 学习的一手资料是官方文档,请务必自主学会阅读规格书之类的资料 ","date":"2024-03-04","objectID":"/posts/orst/:3:0","tags":["Rust","Sort","Algorithm"],"title":"Crust of Rust: Sorting Algorithms","uri":"/posts/orst/"},{"categories":["Rust"],"content":"Crate std 可以使用这里提供的搜素栏进行搜索 (BTW 不要浪费时间在 Google 搜寻上!) Module std::cmp Trait std::cmp::Ord Trait std::cmp::PartialOrd Trait std::cmp::Eq Trait std::cmp::PartialEq Primitive Type slice method slice::sort method slice::sort_unstable method slice::sort_by method slice::sort_by_key method slice::swap method slice::binary_search method slice::rotate_right method slice::split_at_mut method slice::split_first_mut method slice::to_vec Trait std::iter::Iterator method std::iter::Iterator::min method std::iter::Iterator::min_by_key method std::iter::Iterator::enumerate Enum std::option::Option method std::option::Option::expect method std::option::Option::map Enum std::result::Result method std::result::Result::expect method std::result::Result::map Module std::time method std::time::Instant::now method std::time::Instant::elapsed method std::time::Duration::as_secs_f64 ","date":"2024-03-04","objectID":"/posts/orst/:3:1","tags":["Rust","Sort","Algorithm"],"title":"Crust of Rust: Sorting Algorithms","uri":"/posts/orst/"},{"categories":["Rust"],"content":"Crate rand Function rand::thread_rng method rand::seq::SliceRandom::shuffle ","date":"2024-03-04","objectID":"/posts/orst/:3:2","tags":["Rust","Sort","Algorithm"],"title":"Crust of Rust: Sorting Algorithms","uri":"/posts/orst/"},{"categories":["Rust"],"content":"References orst [Github] Sorting algorithm [Wikipedia] Timsort [Wikipedia] Difference between Benchmarking and Profiling [Stack Overflow] ","date":"2024-03-04","objectID":"/posts/orst/:4:0","tags":["Rust","Sort","Algorithm"],"title":"Crust of Rust: Sorting Algorithms","uri":"/posts/orst/"},{"categories":["Linux","Linux Kernel Internals"],"content":" 本講座將以 Thorsten Leemhuis 在 FOSDEM 2020 開場演說 “Linux kernel – Solving big problems in small steps for more than 20 years” (slides) 為主軸,嘗試歸納自 21 世紀第一年開始的 Linux 核心 2.4 版到如今的 5.x 版,中間核心開發者如何克服 SMP (Symmetric multiprocessing), scalability, 及各式硬體架構和周邊裝置支援等難題,過程中提出全面移除 BKL (Big kernel lock)、實作虛擬化技術 (如 Xen 和 KVM)、提出 namespace 和 cgroups 從而確立容器化 (container) 的能力,再來是核心發展的明星技術 eBPF 會在既有的基礎之上,帶來 XDP 和哪些令人驚豔的機制呢?又,Linux 核心終於正式納入發展十餘年的 PREEMPT_RT,使得 Linux 核心得以成為硬即時的作業系統,對內部設計有哪些衝擊?AIO 後繼的 io_uring 讓 Linux 有更優雅且高效的非同步 I/O 存取,我們該如何看待? 原文地址 ","date":"2024-03-03","objectID":"/posts/linux-dev-review/:0:0","tags":["Sysprog","Linux"],"title":"Linux 核心设计: 发展动态回顾","uri":"/posts/linux-dev-review/"},{"categories":["Linux","Linux Kernel Internals"],"content":"开篇点题 前置知识: Linux 核心设计: 操作系统术语及概念 FOSDEM 2020, T. Leemhuis: YouTube: Linux kernel – Solving big problems in small steps for more than 20 years slides (这个投影片共有 248 页,所以加载时可能会比较慢 🤣) 以上面的讲座为主轴,回顾 Linux 的发展动态,由此展望 Linux 未来的发展方向。 SMP (Symmetric multiprocessing) scalability BKL (Big kernel lock) Xen, KVM namespace, cgroups, container - 云服务 eBPF, XDP - 网络封包的高效过滤 (在内核即可处理封包的过滤,无需在用户态制定规则) PREEMPT_RT - 硬即时操作系统 (hard real time os) io_uring - 高效的非同步 I/O (Linux 大部分系统调用都是非同步的) nommu - 用于嵌入式降低功耗 Linux 相关人物 (可在 YouTube 上找到他们的一些演讲): Jonathan Corbet ","date":"2024-03-03","objectID":"/posts/linux-dev-review/:1:0","tags":["Sysprog","Linux"],"title":"Linux 核心设计: 发展动态回顾","uri":"/posts/linux-dev-review/"},{"categories":["Linux","Linux Kernel Internals"],"content":"Linux 2.4 Version 2.4 of the LINUX KERNEL–Why Should a System Administrator Upgrade? 自 2004 年開始,釋出過程發生變化,新核心每隔 2-3 個月定期釋出,編號為 2.6.0, 2.6.1,直到 2.6.39 这件事对于操作系统的开发有很大的影响,是一个巨大的变革。透过这种发行机制,CPU 厂商可以直接在最新的 Linux kernel 上适配正在开发的 CPU 及相关硬体,而无需拿到真正的 CPU 硬体再进行相应的开发,这使得 Linux 获得了更多厂商的支持和投入,进而进入了飞速发展期。 LInux 核心的道路: 只提供机制不提供策略。例如 khttp (in-kernel httpd) 的弃用,通过提供更高效的系统调用来提高网页服务器的效能,而不是像 Windows NT 一样用户态性能不够就把程式搬进 kernel 🤣 ","date":"2024-03-03","objectID":"/posts/linux-dev-review/:2:0","tags":["Sysprog","Linux"],"title":"Linux 核心设计: 发展动态回顾","uri":"/posts/linux-dev-review/"},{"categories":["Linux","Linux Kernel Internals"],"content":"SMP 支援 相关故事: Digital Domain and TITANIC (泰坦尼克号) Red Hat Sinks Titanic Linux Helps Bring Titanic to Life Digital Domain: TITANIC Industrial Light and Magic MaterialX Joins the Academy Software Foundation as a Hosted Project 制作《泰坦尼克号》的特效时,使用了安装 Linux 操作系统的 Alpha 处理器,而 Alpha 是多核处理器,所以当年将 Linux 安装到 Alpha 上需要支援 SMP,由此延伸出了 BLK (Big kernel lock)。 Linux 2.4 在 SMP 的效率问题也正是 BLK 所引起的: BLK 用于锁定整个 Linux kernel,而整个 Linux kernel 只有一个 BLK 实作机制: 在执行 schedule 时当前持有 BLK 的 process 需要释放 BLK 以让其他 process 可以获得 BLK,当轮到该 process 执行时,可以重新获得 BLK 从上面的实作机制可以看出,这样的机制效率是很低的,虽然有多核 (core),但是当一个 process 获得 BLK 时,只有该 process 所在的 core 可以执行,其他 core 只能等待 BLK 已于 v.6.39 版本中被彻底去除 Linux 5.5’s Scheduler Sees A Load Balancing Rework For Better Perf But Risks Regressions ✅ When testing on a dual quad-core ARM64 system they found the performance ranged from less than 1% to upwards of 10% for the Hackbench scheduler test. With a 224-core ARM64 server, the performance ranged from less than 1% improvements to 12% better performance with Hackbench and up to 33% better performance with Dbench. More numbers and details via the v4 patch revision. ","date":"2024-03-03","objectID":"/posts/linux-dev-review/:3:0","tags":["Sysprog","Linux"],"title":"Linux 核心设计: 发展动态回顾","uri":"/posts/linux-dev-review/"},{"categories":["Linux","Linux Kernel Internals"],"content":"虚拟化 Cloud Hypervisor Xen and the Art of Virtualization ","date":"2024-03-03","objectID":"/posts/linux-dev-review/:4:0","tags":["Sysprog","Linux"],"title":"Linux 核心设计: 发展动态回顾","uri":"/posts/linux-dev-review/"},{"categories":["Linux","Linux Kernel Internals"],"content":"DPDK (Data Plane Development Kit) 一言以蔽之: Kernel-bypass networking,即略过 kernel 直接让 User programs 处理网络封包,以提升效能。一般实作于高频交易的场景。 YouTube: Kernel-bypass networking for fun and profit Stack Overflow“zero copy networking” vs “kernel bypass”? ","date":"2024-03-03","objectID":"/posts/linux-dev-review/:5:0","tags":["Sysprog","Linux"],"title":"Linux 核心设计: 发展动态回顾","uri":"/posts/linux-dev-review/"},{"categories":["Linux","Linux Kernel Internals"],"content":"XDP: eXpress Data Path 常和 eBPF 配合实现在 kernel 进行定制化的封包过滤,从而减少 cop to/from kernel/user 这类操作的效能损失。 LPC2018 - Path to DPDK speeds for AF XDP / slides ","date":"2024-03-03","objectID":"/posts/linux-dev-review/:6:0","tags":["Sysprog","Linux"],"title":"Linux 核心设计: 发展动态回顾","uri":"/posts/linux-dev-review/"},{"categories":["Linux","Linux Kernel Internals"],"content":"AIO Synchronous / Asynchronous I/O:在從/向核心空間讀取/寫入資料 (i.e. 實際進行 I/O 操作) 的過程,使用者層級的行程是否會被 blocked。 AIO 在某些情景下处理不当,性能甚至低于 blocked 的 I/O 方法,这也引导出了 io_uring 技巧 UNIX 哲学: Everything is a file. Linux 不成文规范: Everything is a file descriptor. Kernel Recipes 2019 - Faster IO through io_uring / slides io_uring ","date":"2024-03-03","objectID":"/posts/linux-dev-review/:7:0","tags":["Sysprog","Linux"],"title":"Linux 核心设计: 发展动态回顾","uri":"/posts/linux-dev-review/"},{"categories":["Linux","Linux Kernel Internals"],"content":"Container Container 构建在 Linux 核心的基础建设上: namespace, cgroups, capabilities, seccomp +----------------------+ | +------------------+ | | | cgroup | | | | namespace | | | | union-capable fs | | | | | | | | Container | | | +------------------+ | | | | +------------------+ | | | Container | | | +------------------+ | | | | +------------------+ | | | Container | | | +------------------+ | | | | Linux kernel (host) | +----------------------+ YouTube: Containers: cgroups, Linux kernel namespaces, ufs, Docker, and intro to Kubernetes pods Stack Overflow: difference between cgroups and namespaces cgroup: Control Groups provide a mechanism for aggregating/partitioning sets of tasks, and all their future children, into hierarchical groups with specialized behaviour. namespace: wraps a global system resource in an abstraction that makes it appear to the processes within the namespace that they have their own isolated instance of the global resource. Wikipedia: UnionFS Wikipedia: Microservices ","date":"2024-03-03","objectID":"/posts/linux-dev-review/:8:0","tags":["Sysprog","Linux"],"title":"Linux 核心设计: 发展动态回顾","uri":"/posts/linux-dev-review/"},{"categories":["Linux","Linux Kernel Internals"],"content":"BPF/cBPF/eBPF 技巧 run small programs in kernel mode 20 years ago, this idea would likely have been shot down immediately Netflix talks about Extended BPF - A new software type / slides ","date":"2024-03-03","objectID":"/posts/linux-dev-review/:9:0","tags":["Sysprog","Linux"],"title":"Linux 核心设计: 发展动态回顾","uri":"/posts/linux-dev-review/"},{"categories":["Linux","Linux Kernel Internals"],"content":"Real-Time Linux 核心设计: PREEMPT_RT 作为迈向硬即时操作系统的机制 ","date":"2024-03-03","objectID":"/posts/linux-dev-review/:10:0","tags":["Sysprog","Linux"],"title":"Linux 核心设计: 发展动态回顾","uri":"/posts/linux-dev-review/"},{"categories":["Linux","Linux Kernel Internals"],"content":"printk Why printk() is so complicated (and how to fix it) ","date":"2024-03-03","objectID":"/posts/linux-dev-review/:11:0","tags":["Sysprog","Linux"],"title":"Linux 核心设计: 发展动态回顾","uri":"/posts/linux-dev-review/"},{"categories":["Linux","Linux Kernel Internals"],"content":"ZFS, BtrFS, RAID ZFS versus RAID: Eight Ironwolf disks, two filesystems, one winner ","date":"2024-03-03","objectID":"/posts/linux-dev-review/:12:0","tags":["Sysprog","Linux"],"title":"Linux 核心设计: 发展动态回顾","uri":"/posts/linux-dev-review/"},{"categories":["Linux","Linux Kernel Internals"],"content":"Rust Linux 核心采纳 Rust 的状况 ","date":"2024-03-03","objectID":"/posts/linux-dev-review/:13:0","tags":["Sysprog","Linux"],"title":"Linux 核心设计: 发展动态回顾","uri":"/posts/linux-dev-review/"},{"categories":["RISC-V"],"content":" The intention is to give specific actionable optimization recommendations for software developers writing code for RISC-V application processors. 近日 RISE 基金会发布了一版 《RISC-V Optimization Guide》,其目的是为给 RISC-V 应用处理器编写代码的软件开发人员提供具体可行的优化建议。本次活动的主要内容是解读和讨论该文档内容。 原文地址 原文 PDF 解说录影 ","date":"2024-02-29","objectID":"/posts/riscv-optimization-guide/:0:0","tags":["RISC-V","Optimization","Architecture"],"title":"RISC-V Optimization Guide 重点提示","uri":"/posts/riscv-optimization-guide/"},{"categories":["RISC-V"],"content":"相关知识 RISC-V ISA 规格书: https://riscv.org/technical/specifications/ 推荐参考 体系结构如何作用于编译器后端-邱吉 [bilibili] 这个讲座是关于微架构、指令集是怎样和编译器、软件相互协作、相互影响的 Overview 这个讲座介绍的是通用 CPU 并不仅限于 RISC-V 上 ","date":"2024-02-29","objectID":"/posts/riscv-optimization-guide/:1:0","tags":["RISC-V","Optimization","Architecture"],"title":"RISC-V Optimization Guide 重点提示","uri":"/posts/riscv-optimization-guide/"},{"categories":["RISC-V"],"content":"Detecting RISC-V Extensions on Linux 参考以下文章构建 Linux RISC-V 然后进行原文的 riscv_hwprobe 系统调用实验: How To Set Up The Environment for RISCV-64 Linux Kernel Development In Ubuntu 20.04 Running 64- and 32-bit RISC-V Linux on QEMU ","date":"2024-02-29","objectID":"/posts/riscv-optimization-guide/:2:0","tags":["RISC-V","Optimization","Architecture"],"title":"RISC-V Optimization Guide 重点提示","uri":"/posts/riscv-optimization-guide/"},{"categories":["RISC-V"],"content":"Multi-versioning 最新进展: https://reviews.llvm.org/D151730 相关介绍: https://maskray.me/blog/2023-02-05-function-multi-versioning ","date":"2024-02-29","objectID":"/posts/riscv-optimization-guide/:2:1","tags":["RISC-V","Optimization","Architecture"],"title":"RISC-V Optimization Guide 重点提示","uri":"/posts/riscv-optimization-guide/"},{"categories":["RISC-V"],"content":"Optimizing Scalar Integer ","date":"2024-02-29","objectID":"/posts/riscv-optimization-guide/:3:0","tags":["RISC-V","Optimization","Architecture"],"title":"RISC-V Optimization Guide 重点提示","uri":"/posts/riscv-optimization-guide/"},{"categories":["RISC-V"],"content":"Materializing Constants RV64I 5.2 Integer Computational Instructions Additional instruction variants are provided to manipulate 32-bit values in RV64I, indicated by a ‘W’ suffix to the opcode. These “*W” instructions ignore the upper 32 bits of their inputs and always produce 32-bit signed values, i.e. bits XLEN-1 through 31 are equal. ADDIW is an RV64I instruction that adds the sign-extended 12-bit immediate to register rs1 and produces the proper sign-extension of a 32-bit result in rd. 原文 Prefer idiomatic LUI/ADDI sequence for 32 bit constants 部分使用 lui 和 addiw 构建 0x1fffff 的说明比较晦涩难懂 (说实话我没看懂原文的 addiw 为什么需要减去 4096 😇) 注意 根据下面的参考文章,如果 addiw 的立即数的 MSB 被置为 1 时,只需在 lui 时多加一个 1 即可构建我们想要的 32-bit 数值。而原文中除了对 lui 加 1 外,还对 addiw 进行减去 4096 的操作: addiw a0, a0, (0xfff - 4096) ; addiw a0, a0, -1 这乍一看不知道为何需要减去 4096,其实本质很简单,根据上面的 ISA manual addiw 的立即数是 12-bit 的 signed number,即应该传入的是数值。但是直接使用 0xfff 表示传入的仅仅是 0xfff 这个编码对应的数值 (可以表示 12-bit signed 下的数值 -1,也可以表示 unsigned 编码下 0xfff 对应的数值 4095,在 12-bit signed 下 integer overflow),为了保证 addiw 的立即数的数值符合我们的预期 (即 0xfff 在 12-bit signed 下数值是 -1) 以及避免 integer overflow,所以需要将 0xfff - 4096 得到 12-bit signed 数值 -1 (虽然这个编码和 0xfff 是一样的…)。 addiw a0, a0, -1 ; right addiw a0, a0, 4095 ; integer overflow 解读计算机编码 C 语言: 数值系统篇 RV32G 下 lui/auipc 和 addi 结合加载立即数时的补值问题 [zhihu] RISC-V build 32-bit constants with LUI and ADDI [Stack Overflow] 原文 Fold immediates into consuming instructions where possible 部分,相关的 RISC-V 的 imm 优化: Craig Topper: 2022 LLVM Dev Mtg: RISC-V Sign Extension Optimizations 改进RISC-V的代码生成-廖春玉 [bilibili] ","date":"2024-02-29","objectID":"/posts/riscv-optimization-guide/:3:1","tags":["RISC-V","Optimization","Architecture"],"title":"RISC-V Optimization Guide 重点提示","uri":"/posts/riscv-optimization-guide/"},{"categories":["RISC-V"],"content":"Avoid branches using conditional moves Zicond extension 提供了我们在 RISC-V 上实作常数时间函数 (contant-time function) 的能力,用于避免分支预测,从而减少因分支预测失败带来的高昂代价。 $$ a0 = \\begin{cases} constant1 \u0026 \\text{if } x \\neq 0 \\newline constant2 \u0026 \\text{if } x = 0 \\end{cases} $$ 原文使用了 CZERO.NEZ,下面我们使用 CZERO.EQZ 来实作原文的例子: li t2, constant2 li t3, (constant1 - constant2) CZERO.EQZ t3, t3, a0 add a0, t3, t2 原文也介绍了如何使用 seqz 来实作 constant-time function,下面使用 snez 来实作原文的例子: li t2, constant1 li t3, constant2 snez t0, a0 addi t0, t0, -1 xor t1, t2, t3 and t1, t1, t0 xor a0, t1, t2 如果有 \\‘M\\’ 扩展可以通过 mul 指令进行简化 (通过 snez 来实作原文例子): li t2, constant1 li t3, constant2 xor t1, t2, t3 snez t0, a0 mul t1, t1, t0 xor a0, t1, t3 ","date":"2024-02-29","objectID":"/posts/riscv-optimization-guide/:3:2","tags":["RISC-V","Optimization","Architecture"],"title":"RISC-V Optimization Guide 重点提示","uri":"/posts/riscv-optimization-guide/"},{"categories":["RISC-V"],"content":"Padding Use canonical NOPs, NOP ( ADDI X0, X0, 0 ) and C.NOP ( C.ADDI X0, 0 ), to add padding within a function. Use the canonical illegal instruction ( either 2 or 4 bytes of zeros depending on whether the C extension is supported ) to add padding between functions. 因为在函数内部的执行频率高,使用合法的 NOPs 进行对齐 padding,防止在乱序执行时,流水线在遇见非法指令后就不再执行后续指令,造成效能损失 如果控制流被传递到两个函数之间,那么加大可能是程序执行出错了,使用非法的指令进行对齐 padding 可以帮助我们更好更快地 debug ","date":"2024-02-29","objectID":"/posts/riscv-optimization-guide/:3:3","tags":["RISC-V","Optimization","Architecture"],"title":"RISC-V Optimization Guide 重点提示","uri":"/posts/riscv-optimization-guide/"},{"categories":["RISC-V"],"content":"Align char array to greater alignment Why use wider load/store usage for memory copy? C 语言: 内存管理、对齐及硬体特性 ","date":"2024-02-29","objectID":"/posts/riscv-optimization-guide/:3:4","tags":["RISC-V","Optimization","Architecture"],"title":"RISC-V Optimization Guide 重点提示","uri":"/posts/riscv-optimization-guide/"},{"categories":["RISC-V"],"content":"Use shifts to clear leading/trailing bits 实作 64-bit 版本的原文例子 (retain the highest 12 bits): slli x6, x5, 52 slri x7, x5, 52 RV64I 5.2 Integer Computational Instructions LUI (load upper immediate) uses the same opcode as RV32I. LUI places the 20-bit U-immediate into bits 31–12 of register rd and places zero in the lowest 12 bits. The 32-bit result is sign-extended to 64 bits. ","date":"2024-02-29","objectID":"/posts/riscv-optimization-guide/:3:5","tags":["RISC-V","Optimization","Architecture"],"title":"RISC-V Optimization Guide 重点提示","uri":"/posts/riscv-optimization-guide/"},{"categories":["RISC-V"],"content":"Optimizing Scalar Floating Point ","date":"2024-02-29","objectID":"/posts/riscv-optimization-guide/:4:0","tags":["RISC-V","Optimization","Architecture"],"title":"RISC-V Optimization Guide 重点提示","uri":"/posts/riscv-optimization-guide/"},{"categories":["RISC-V"],"content":"Optimizing Vector What about vector instructions? YouTube: Introduction to SIMD Introduction to the RISC-V Vector Extension [PDF] 2020 RISC-V Summit: Tutorial: RISC-V Vector Extension Demystified ","date":"2024-02-29","objectID":"/posts/riscv-optimization-guide/:5:0","tags":["RISC-V","Optimization","Architecture"],"title":"RISC-V Optimization Guide 重点提示","uri":"/posts/riscv-optimization-guide/"},{"categories":["Rust"],"content":" In this (fifth) Crust of Rust video, we cover multi-produce/single-consumer (mpsc) channels, by re-implementing some of the std::sync::mpsc types from the standard library. As part of that, we cover what channels are used for, how they work at a high level, different common channel variants, and common channel implementations. In the process, we go over some common Rust concurrency primitives like Mutex and Condvar. 整理自 John Gjengset 的影片 ","date":"2024-02-29","objectID":"/posts/channels/:0:0","tags":["Rust","Channel"],"title":"Crust of Rust: Channels","uri":"/posts/channels/"},{"categories":["Rust"],"content":"Channel Wikipedia: Channel 引用 In computing, a channel is a model for interprocess communication and synchronization via message passing. A message may be sent over a channel, and another process or thread is able to receive messages sent over a channel it has a reference to, as a stream. YouTube: Channels in Rust Source ","date":"2024-02-29","objectID":"/posts/channels/:1:0","tags":["Rust","Channel"],"title":"Crust of Rust: Channels","uri":"/posts/channels/"},{"categories":["Rust"],"content":"Concurrency vs Parallelism What is the difference between concurrency and parallelism? Concurrency vs. Parallelism — A brief view ","date":"2024-02-29","objectID":"/posts/channels/:1:1","tags":["Rust","Channel"],"title":"Crust of Rust: Channels","uri":"/posts/channels/"},{"categories":["Rust"],"content":"影片注解 ","date":"2024-02-29","objectID":"/posts/channels/:2:0","tags":["Rust","Channel"],"title":"Crust of Rust: Channels","uri":"/posts/channels/"},{"categories":["Rust"],"content":"Sender \u0026 Receiver multi-produce/single-consumer (mpsc) Why does the recevier type need to have an arc protected by mutex if the channel may only have a single consumer thread? Because a send and a recevie might happen at the same time, and they need to be mutually exclusive to each other as well. Why not use a boolean semaphore over the implementation in mutex? A boolean semaphore is basically a boolean flag that you check and atomically update. The problem there is if the flag is currently set (someone else is in the critical section), with a boolean semaphore, you have to spin, you have to repeatedly check it. Whereas with a mutex, the operating system can put the thread to sleep and wake it back up when the mutex is available, which is generally more efficient although adds a little bit of latency. ","date":"2024-02-29","objectID":"/posts/channels/:2:1","tags":["Rust","Channel"],"title":"Crust of Rust: Channels","uri":"/posts/channels/"},{"categories":["Rust"],"content":"Condition Variable method std::sync::Condvar::wait This function will atomically unlock the mutex specified (represented by guard) and block the current thread. This means that any calls to notify_one or notify_all which happen logically after the mutex is unlocked are candidates to wake this thread up. When this function call returns, the lock specified will have been re-acquired. method std::sync::Condvar::notify_one If there is a blocked thread on this condition variable, then it will be woken up from its call to wait or wait_timeout. Calls to notify_one are not buffered in any way. wait \u0026 notify ","date":"2024-02-29","objectID":"/posts/channels/:2:2","tags":["Rust","Channel"],"title":"Crust of Rust: Channels","uri":"/posts/channels/"},{"categories":["Rust"],"content":"Clone 对 struct Sender\u003cT\u003e 标注属性宏 #[derive(clone)] 会实现以下的 triat: impl\u003cT: Clone\u003e Clone for Sender\u003cT\u003e { ... } 但是对于 Sender\u003cT\u003e 的成员 Arc\u003cInner\u003cT\u003e\u003e 来说,Arc 可以 clone 无论内部类型 T 是否实现了 Clone 这个 trait,所以我们需要手动实现 Clone 这个 trait。这也是 #[derive(clone)] 和手动实现 impl Clone 的一个细小差别。 impl\u003cT\u003e Clone for Sender\u003cT\u003e { ... } 为了防止调用 clone 产生的二义性 (因为编译器会自动解引用),建议使用 explict 方式来调用 Arc::clone(),这样编译器就会知道调用的是 Arc 的 clone 方法,而不是 Arc 内部 object 的 clone 方法。 let inner = Arc\u003cInner\u003cT\u003e\u003e; inner.clone(); // Inner\u003cT\u003e's clone method? or Arc::clone method? Arc::clone(\u0026inner); // explict Arc::clone ! ","date":"2024-02-29","objectID":"/posts/channels/:2:3","tags":["Rust","Channel"],"title":"Crust of Rust: Channels","uri":"/posts/channels/"},{"categories":["Rust"],"content":"dbg Macro std::dbg 引用 Prints and returns the value of a given expression for quick and dirty debugging. let a = 2; let b = dbg!(a * 2) + 1; // ^-- prints: [src/main.rs:2] a * 2 = 4 assert_eq!(b, 5); The macro works by using the Debug implementation of the type of the given expression to print the value to stderr along with the source location of the macro invocation as well as the source code of the expression. 调试的大杀器,作用类似于 kernel 中的 debugk 宏 🤣 常用于检测程序运行时是否执行了某些语句,以及这些语句的值如何。 ","date":"2024-02-29","objectID":"/posts/channels/:2:4","tags":["Rust","Channel"],"title":"Crust of Rust: Channels","uri":"/posts/channels/"},{"categories":["Rust"],"content":"Performance optimization Every operation takes the lock and that's fine if you have a channel that is not very high performance, but if you wanted like super high performance, like you have a lot of sends that compete with each other, then you might not want the sends to contend with one another. Image that you have 10 threads are trying to send at the same time, you could perhaps write an implementation that allows them to do that. The only thing that really needs to be synchronized is the senders with the receivers, as opposed to the senders with one another, whereas we're actually locking all of them. 使用 VecDeque 作为缓冲区,会导致 send 时的效能问题。因为 send 是使用 push_back 方法来将 object 加入到 VecDeque 中,这个过程 VecDeque 可能会发生 resize 操作,这会花费较长时间并且在这个过程时 sender 仍然持有 Mutex,所以导致其他 sender 和 recevier 并不能使用 VecDeque,所以在实作中并不使用 VecDeque 以避免相应的效能损失。 因为只有一个 receiver,所以可以通过缓冲区来提高效能,一次性接受大批数据并进行缓存,而不是每次只接收一个数据就放弃 Mutex (Batch recv optimization)。当然这个如果使用 VecDeque 依然会在 recv 时出现上面的 resize 效能问题。 ","date":"2024-02-29","objectID":"/posts/channels/:2:5","tags":["Rust","Channel"],"title":"Crust of Rust: Channels","uri":"/posts/channels/"},{"categories":["Rust"],"content":"Synchronous channels Module std::sync::mpsc These channels come in two flavors: An asynchronous, infinitely buffered channel. The channel function will return a (Sender, Receiver) tuple where all sends will be asynchronous (they never block). The channel conceptually has an infinite buffer. A synchronous, bounded channel. The sync_channel function will return a (SyncSender, Receiver) tuple where the storage for pending messages is a pre-allocated buffer of a fixed size. All sends will be synchronous by blocking until there is buffer space available. Note that a bound of 0 is allowed, causing the channel to become a “rendezvous” channel where each sender atomically hands off a message to a receiver. ","date":"2024-02-29","objectID":"/posts/channels/:2:6","tags":["Rust","Channel"],"title":"Crust of Rust: Channels","uri":"/posts/channels/"},{"categories":["Rust"],"content":"Channel flavors Synchronous channels: Channel where send() can block. Limited capacity. Mutex + Condvar + VecDeque Atomic VecDeque (atomic queue) + thread::park + thread::Thread::notify Asynchronous channels: Channel where send() cannot block. Unbounded. Mutex + Condvar + VecDeque Mutex + Condvar + LinkedList Atomic linked list, linked list of T Atomic block linked list, linked list of atomic VecDeque Rendezvous channels: Synchronous with capacity = 0. Used for thread synchronization. Oneshot channels: Any capacity. In practice, only one call to send(). ","date":"2024-02-29","objectID":"/posts/channels/:2:7","tags":["Rust","Channel"],"title":"Crust of Rust: Channels","uri":"/posts/channels/"},{"categories":["Rust"],"content":"async/await Module std::future Keyword async Keyword await ","date":"2024-02-29","objectID":"/posts/channels/:2:8","tags":["Rust","Channel"],"title":"Crust of Rust: Channels","uri":"/posts/channels/"},{"categories":["Rust"],"content":"Homework 信息 实作说明: 尝试实现 Synchronous channels 使用 Atomic 存储 senders 以提高效能 使用两个 ConVar 来指示 sender 和 receiver 进行 block 和 wake up receiver 被 drop 时需要通知所有 senders 以释放资源 使用 linked list 来取代 VecDeque 以避免 resize 的效能损失 尝试阅读 std 中 mpsc 的实现 Module std::sync::mpsc 对比阅读其他库关于 channel 的实现: crossbeam, flume 参考资料: Module std::sync::atomic Module std::sync::mpsc Crate crossbeam Crate flume ","date":"2024-02-29","objectID":"/posts/channels/:3:0","tags":["Rust","Channel"],"title":"Crust of Rust: Channels","uri":"/posts/channels/"},{"categories":["Rust"],"content":"Documentations 这里列举视频中一些概念相关的 documentation 学习的一手资料是官方文档,请务必自主学会阅读规格书之类的资料 ","date":"2024-02-29","objectID":"/posts/channels/:4:0","tags":["Rust","Channel"],"title":"Crust of Rust: Channels","uri":"/posts/channels/"},{"categories":["Rust"],"content":"Crate std 可以使用这里提供的搜素栏进行搜索 (BTW 不要浪费时间在 Google 搜寻上!) Module std::sync::mpsc Function std::sync::mpsc::channel Struct std::sync::mpsc::Sender Struct std::sync::mpsc::Receiver Module std::sync Struct std::sync::Arc Struct std::sync::Mutex Struct std::sync::Condvar method std::sync::Condvar::wait method std::sync::Condvar::notify_one Module std::sync::atomic Trait std::marker::Send Struct std::collections::VecDeque Function std::mem::take Function std::mem::swap Macro std::dbg ","date":"2024-02-29","objectID":"/posts/channels/:4:1","tags":["Rust","Channel"],"title":"Crust of Rust: Channels","uri":"/posts/channels/"},{"categories":["Rust"],"content":"References Go 语言也有 channel: 解说 Go channel 底层原理 [bilibili] 可能不是你看过最无聊的 Rust 入门喜剧 102 (3) 多线程并发 [bilibili] ","date":"2024-02-29","objectID":"/posts/channels/:5:0","tags":["Rust","Channel"],"title":"Crust of Rust: Channels","uri":"/posts/channels/"},{"categories":["C","Linux Kernel Internals"],"content":" Linux 核心作为世界上最成功的开放原始码计划,也是 C 语言在工程领域的瑰宝,里头充斥则各种“艺术”,往往会吓到初次接触的人们,但总是能够使用 C 语言标准和开发工具提供的扩展 (主要是来自 gcc 的 GNU extensions) 来解释。 工欲善其事,必先利其器 原文地址 If I had eight hours to chop down a tree, I’d spend six hours sharpening my axe. —— Abraham Lincoln 语言规格: C89/C90 -\u003e C99 -\u003e C11 -\u003e C17/C18 -\u003e C2x ","date":"2024-02-28","objectID":"/posts/c-standards/:0:0","tags":["Sysprog","C","Standard"],"title":"你所不知道的 C 语言: 开发工具和规格标准","uri":"/posts/c-standards/"},{"categories":["C","Linux Kernel Internals"],"content":"C vs C++ C is quirky, flawed, and an enormous success. Although accidents of history surely helped, it evidently satisfied a need for a system implementation language efficient enough to displace assembly language, yet sufficiently abstract and fluent to describe algorithms and interactions in a wide variety of environments. —— Dennis M. Ritchie David Brailsford: Why C is so Influential - Computerphile Linus Torvalds: c++ in linux kernel And I really do dislike C++. It’s a really bad language, in my opinion. It tries to solve all the wrong problems, and does not tackle the right ones. The things C++ “solves” are trivial things, almost purely syntactic extensions to C rather than fixing some true deep problem. Bjarne Stroustrup: Learning Standard C++ as a New Language [PDF] C++ 标准更新飞快: C++11, C++14, C++17, … 从 C99, C++98 开始,C 语言和 C++ 分道扬镳 in C, everything is a representation (unsigned char [sizeof(TYPE)]). —— Rich Rogers 第一個 C 語言編譯器是怎樣編寫的? 介绍了自举 (sel-hosting/compiling) 以及 C0, C1, C2, C3, … 等的演化过程 ","date":"2024-02-28","objectID":"/posts/c-standards/:1:0","tags":["Sysprog","C","Standard"],"title":"你所不知道的 C 语言: 开发工具和规格标准","uri":"/posts/c-standards/"},{"categories":["C","Linux Kernel Internals"],"content":"C 语言规格书 ","date":"2024-02-28","objectID":"/posts/c-standards/:2:0","tags":["Sysprog","C","Standard"],"title":"你所不知道的 C 语言: 开发工具和规格标准","uri":"/posts/c-standards/"},{"categories":["C","Linux Kernel Internals"],"content":"main 阅读 C 语言规格书可以让你洞察本质,不在没意义的事情上浪费时间,例如在某乎大肆讨论的 void main() 和 int main() 问题 🤣 C99/C11 5.1.2.2.1 Program startup The function called at program startup is named main. The implementation declares no prototype for this function. It shall be defined with a return type of int and with no parameters: int main(void) { /* ... */ } or with two parameters (referred to here as argc and argv, though any names may be used, as they are local to the function in which they are declared): int main(int argc, char *argv[]) { /* ... */ } or equivalent; or in some other implementation-defined manner. Thus, int can be replaced by a typedef name defined as int, or the type of argv can be written as char ** argv, and so on. ","date":"2024-02-28","objectID":"/posts/c-standards/:2:1","tags":["Sysprog","C","Standard"],"title":"你所不知道的 C 语言: 开发工具和规格标准","uri":"/posts/c-standards/"},{"categories":["C","Linux Kernel Internals"],"content":"incomplete type C99 6.2.5 Types incomplete types (types that describe objects but lack information needed to determine their sizes). ","date":"2024-02-28","objectID":"/posts/c-standards/:2:2","tags":["Sysprog","C","Standard"],"title":"你所不知道的 C 语言: 开发工具和规格标准","uri":"/posts/c-standards/"},{"categories":["C","Linux Kernel Internals"],"content":"规格不仅要看最新的,过往的也要熟悉 因为很多 (嵌入式) 设备上运行的 Linux 可能是很旧的版本,那时 Linux 使用的是更旧的 C 语言规格。例如空中巴士 330 客机的娱乐系统里执行的是十几年前的 Red Hat Linux,总有人要为这些“古董”负责 🤣 ","date":"2024-02-28","objectID":"/posts/c-standards/:2:3","tags":["Sysprog","C","Standard"],"title":"你所不知道的 C 语言: 开发工具和规格标准","uri":"/posts/c-standards/"},{"categories":["C","Linux Kernel Internals"],"content":"GDB 使用 GDB 这类调试工具可以大幅度提升我们编写代码、除错的能力 🐶 video: Linux basic anti-debug video: C Programming, Disassembly, Debugging, Linux, GDB rr (Record and Replay Framework) video: Quick demo video: Record and replay debugging with “rr” ","date":"2024-02-28","objectID":"/posts/c-standards/:3:0","tags":["Sysprog","C","Standard"],"title":"你所不知道的 C 语言: 开发工具和规格标准","uri":"/posts/c-standards/"},{"categories":["C","Linux Kernel Internals"],"content":"C23 上一个 C 语言标准是 C17,正式名称为 ISO/IEC 9899:2018,是 2017 年准备,2018年正式发布的标准规范。C23 则是目前正在开发的规格,其预计新增特性如下: typeof: 由 GNU extension 转正,用于实作 container_of 宏 call_once: 保证在 concurrent 环境中,某段程式码只会执行 1 次 char8_t: Unicode friendly u8\"💣\"[0] unreachable(): 由 GNU extension 转正,提示允许编译器对某段程式码进行更激进的最佳化 = {}: 取代 memset 函数调用 ISO/IEC 60559:2020: 最新的 IEEE 754 浮点数运算标准 _Static_assert: 扩充 C11 允许单一参数 吸收 C++11 风格的 attribute 语法,例如 nodiscard, maybe_unused, deprecated, fallthrough 新的函数: memccpy(), strdup(), strndup() ——— 类似于 POSIX、SVID中 C 函数库的扩充 强制规范使用二补数表示整数 不支援 K\u0026R 风格的函数定义 二进制表示法: 0b10101010 以及对应 printf() 的 %b (在此之前 C 语言是不支援二进制表示法的 🤣) Type generic functions for performing checked integer arithmetic (Integer overflow) _BitInt(N) and UnsignedBitInt(N) types for bit-precise integers #elifdef and #elifndef 支持在数值中间加入分隔符,易于阅读,例如 0xFFFF'FFFF 信息 Ever Closer - C23 Draws Nearer C23 is Finished: Here is What is on the Menu ","date":"2024-02-28","objectID":"/posts/c-standards/:4:0","tags":["Sysprog","C","Standard"],"title":"你所不知道的 C 语言: 开发工具和规格标准","uri":"/posts/c-standards/"},{"categories":["C","Linux Kernel Internals"],"content":" 不少 C/C++ 开发者听过 “内存对齐” (memory alignment),但不易掌握概念及规则,遑论其在执行时期的冲击。内存管理像是 malloc/free 函数的使用,是每个 C 语言程序设计开发者都会接触到,但却难保充分排除错误的难题。本讲座尝试从硬体的行为开始探讨,希望消除观众对于 alignment, padding, memory allocator 的误解,并且探讨高效能 memory pool 的设计,如何改善整体程序的效能和可靠度。也会探讨 C11 标准的 aligned_alloc。 原文地址 ","date":"2024-02-27","objectID":"/posts/c-memory/:0:0","tags":["Sysprog","C","Memory"],"title":"你所不知道的 C 语言: 记忆体管理、对齐及硬体特性","uri":"/posts/c-memory/"},{"categories":["C","Linux Kernel Internals"],"content":"背景知识 ","date":"2024-02-27","objectID":"/posts/c-memory/:1:0","tags":["Sysprog","C","Memory"],"title":"你所不知道的 C 语言: 记忆体管理、对齐及硬体特性","uri":"/posts/c-memory/"},{"categories":["C","Linux Kernel Internals"],"content":"你所不知道的 C 语言: 指针篇 C99/C11 6.2.5 Types (28) A pointer to void shall have the same representation and alignment requirements as a pointer to a character type. C99/C11 6.3.2.3 Pointers (1) A pointer to void may be converted to or from a pointer to any object type. A pointer to any object type may be converted to a pointer to void and back again; the result shall compare equal to the original pointer. 使用 void * 必须通过 explict (显式) 或强制转型,才能存取最终的 object,因为 void 无法判断 object 的大小信息。 ","date":"2024-02-27","objectID":"/posts/c-memory/:1:1","tags":["Sysprog","C","Memory"],"title":"你所不知道的 C 语言: 记忆体管理、对齐及硬体特性","uri":"/posts/c-memory/"},{"categories":["C","Linux Kernel Internals"],"content":"你所不知道的 C 语言: 函数呼叫篇 glibc 提供了 malloc_stats() 和 malloc_info() 这两个函数,可以查询 process 的 heap 空间使用情况信息。 ","date":"2024-02-27","objectID":"/posts/c-memory/:1:2","tags":["Sysprog","C","Memory"],"title":"你所不知道的 C 语言: 记忆体管理、对齐及硬体特性","uri":"/posts/c-memory/"},{"categories":["C","Linux Kernel Internals"],"content":"Memory 金字塔 这个金字塔的层级图提示我们,善用 Cache locality 可以有效提高程式效能。 技巧 What a C programmer should know about memory (简记) ","date":"2024-02-27","objectID":"/posts/c-memory/:2:0","tags":["Sysprog","C","Memory"],"title":"你所不知道的 C 语言: 记忆体管理、对齐及硬体特性","uri":"/posts/c-memory/"},{"categories":["C","Linux Kernel Internals"],"content":"Understanding virtual memory - the plot thickens The virtual memory allocator (VMA) may give you a memory it doesn’t have, all in a vain hope that you’re not going to use it. Just like banks today 虚拟内存的管理类似于银行,返回的分配空间未必可以立即使用。memory allocator 和银行类似,可用空间就类似于银行的现金储备金,银行可以开很多支票,但是这些支票可以兑现的前提是这些支票不会在同一时间来兑现,虚拟内存管理也类似,分配空间也期望用户不会立即全部使用。 ","date":"2024-02-27","objectID":"/posts/c-memory/:2:1","tags":["Sysprog","C","Memory"],"title":"你所不知道的 C 语言: 记忆体管理、对齐及硬体特性","uri":"/posts/c-memory/"},{"categories":["C","Linux Kernel Internals"],"content":"Understanding stack allocation This is how variable-length arrays (VLA), and also alloca() work, with one difference - VLA validity is limited by the scope, alloca’d memory persists until the current function returns (or unwinds if you’re feeling sophisticated). VLA 和 alloca 分配的都是栈 (stack) 空间,只需将栈指针 (sp) 按需求加减一下即可实现空间分配。因为 stack 空间是有限的,所以 Linux 核心中禁止使用 VLA,防止 Stack Overflow 🤣 ","date":"2024-02-27","objectID":"/posts/c-memory/:2:2","tags":["Sysprog","C","Memory"],"title":"你所不知道的 C 语言: 记忆体管理、对齐及硬体特性","uri":"/posts/c-memory/"},{"categories":["C","Linux Kernel Internals"],"content":"Slab allocator The principle of slab allocation was described by Bonwick for a kernel object cache, but it applies for the user-space as well. Oh-kay, we’re not interested in pinning slabs to CPUs, but back to the gist — you ask the allocator for a slab of memory, let’s say a whole page, and you cut it into many fixed-size pieces. Presuming each piece can hold at least a pointer or an integer, you can link them into a list, where the list head points to the first free element. 在使用 alloc 的内存空间时,这些空间很有可能是不连续的。所以此时对于系统就会存在一些问题,一个是内存空间碎片 fragment,因为分配的空间未必会全部使用到,另一个是因为不连续,所以无法利用 Cache locality 来提升效能。 ","date":"2024-02-27","objectID":"/posts/c-memory/:2:3","tags":["Sysprog","C","Memory"],"title":"你所不知道的 C 语言: 记忆体管理、对齐及硬体特性","uri":"/posts/c-memory/"},{"categories":["C","Linux Kernel Internals"],"content":"Demand paging explained Linux 系统会提供一些内存管理的 API 和机制: mlock() - lock/unlock memory 禁止某个区域的内存被 swapped out 到磁盘 (只是向 OS 建议,OS 可能不会理会) madvise() - give advice about use of memory (同样只是向 OS 建议,OS 可能不会理会) lazy loading - 利用缺页异常 (page-fault) 来实现 copy on write 信息 現代處理器設計: Cache 原理和實際影響 Cache 原理和實際影響: 進行 CPU caches 中文重點提示並且重現對應的實驗 針對多執行緒環境設計的 Memory allocator rpmalloc 探討 ","date":"2024-02-27","objectID":"/posts/c-memory/:2:4","tags":["Sysprog","C","Memory"],"title":"你所不知道的 C 语言: 记忆体管理、对齐及硬体特性","uri":"/posts/c-memory/"},{"categories":["C","Linux Kernel Internals"],"content":"堆 Heap Stack Overflow: Why are two different concepts both called “heap”? Several authors began about 1975 to call the pool of available memory a “heap.” ","date":"2024-02-27","objectID":"/posts/c-memory/:3:0","tags":["Sysprog","C","Memory"],"title":"你所不知道的 C 语言: 记忆体管理、对齐及硬体特性","uri":"/posts/c-memory/"},{"categories":["C","Linux Kernel Internals"],"content":"Data alignment 一个 data object 具有两个特性: value storage location (address) ","date":"2024-02-27","objectID":"/posts/c-memory/:4:0","tags":["Sysprog","C","Memory"],"title":"你所不知道的 C 语言: 记忆体管理、对齐及硬体特性","uri":"/posts/c-memory/"},{"categories":["C","Linux Kernel Internals"],"content":"alignment vs unalignment 假设硬体要求 4 Bytes alignment,CPU 存取数据时的操作如下: alignment unalignment Source 除此之外,unalignment 也可能会无法充分利用 cache 效能,即存取的数据一部分 cache hit,另一部分 cache miss。当然对于这种情况,cache 也是采用类似上面的 merge 机制来进行存取,只是效能低下。 GCC: 6.60.8 Structure-Packing Pragmas The n value below always is required to be a small power of two and specifies the new alignment in bytes. #pragma pack(push[,n]) pushes the current alignment setting on an internal stack and then optionally sets the new alignment. #pragma pack(pop) restores the alignment setting to the one saved at the top of the internal stack (and removes that stack entry). Note that #pragma pack([n]) does not influence this internal stack; thus it is possible to have #pragma pack(push) followed by multiple #pragma pack(n) instances and finalized by a single #pragma pack(pop). alignment 与 unalignment 的效能分布: ","date":"2024-02-27","objectID":"/posts/c-memory/:4:1","tags":["Sysprog","C","Memory"],"title":"你所不知道的 C 语言: 记忆体管理、对齐及硬体特性","uri":"/posts/c-memory/"},{"categories":["C","Linux Kernel Internals"],"content":"malloc malloc 分配的空间是 alignment 的: man malloc The malloc() and calloc() functions return a pointer to the allocated memory, which is suitably aligned for any built-in type. The GNU C Library - Malloc Example The block that malloc gives you is guaranteed to be aligned so that it can hold any type of data. On GNU systems, the address is always a multiple of eight on 32-bit systems, and a multiple of 16 on 64-bit systems. 使用 GDB 进行测试,确定在 Linux x86_64 上 malloc 分配的内存以 16 Bytes 对齐,即地址以 16 进制显示时最后一个数为 0。 ","date":"2024-02-27","objectID":"/posts/c-memory/:4:2","tags":["Sysprog","C","Memory"],"title":"你所不知道的 C 语言: 记忆体管理、对齐及硬体特性","uri":"/posts/c-memory/"},{"categories":["C","Linux Kernel Internals"],"content":"unalignment get \u0026 set 如上面所述,在 32-bit 架构上进行 8 bytes 对齐的存取效能比较高 (远比单纯访问一个 byte 高),所以原文利用这一特性实作了 unaligned_get8 这一函数。 csrc \u0026 0xfffffffc 向下取整到最近的 8 bytes alignment 的地址 v \u003e\u003e (((uint32_t) csrc \u0026 0x3) * 8) 将获取的 alignment 的 32-bit 进行位移以获取我们想要的那个字节 而在 你所不知道的 C 语言: 指针篇 中实作的 16-bit integer 在 unalignment 情况下的存取,并没有考虑到上面利用 alignment 来提升效能。 参考原文 32-bit integer 存取,实作 64-bit integer 的 get \u0026 set: ","date":"2024-02-27","objectID":"/posts/c-memory/:4:3","tags":["Sysprog","C","Memory"],"title":"你所不知道的 C 语言: 记忆体管理、对齐及硬体特性","uri":"/posts/c-memory/"},{"categories":["C","Linux Kernel Internals"],"content":"oncurrent-II 源码: concurrent-ll 论文: A Pragmatic Implementation of Non-Blocking Linked Lists ","date":"2024-02-27","objectID":"/posts/c-memory/:5:0","tags":["Sysprog","C","Memory"],"title":"你所不知道的 C 语言: 记忆体管理、对齐及硬体特性","uri":"/posts/c-memory/"},{"categories":["C","Linux Kernel Internals"],"content":" Linux 核心原始程式码存在大量 bit(-wise) operations (简称 bitops),颇多乍看像是魔法的 C 程式码就是 bitops 的组合。 原文地址 ","date":"2024-02-23","objectID":"/posts/c-bitwise/:0:0","tags":["Sysprog","C","Bitwise"],"title":"你所不知道的 C 语言: bitwise 操作","uri":"/posts/c-bitwise/"},{"categories":["C","Linux Kernel Internals"],"content":"复习数值系统 YouTube: 十进制,十二进制,六十进制从何而来?阿拉伯人成就了文艺复兴?[数学大师] 你所不知道的 C 语言: 数值系统 解读计算机编码 ","date":"2024-02-23","objectID":"/posts/c-bitwise/:1:0","tags":["Sysprog","C","Bitwise"],"title":"你所不知道的 C 语言: bitwise 操作","uri":"/posts/c-bitwise/"},{"categories":["C","Linux Kernel Internals"],"content":"位元组合 一些位元组合表示特定的意义,而不是表示数值,这些组合被称为 trap representation C11 6.2.6.2 Integer types For unsigned integer types other than unsigned char, the bits of the object representation shall be divided into two groups: value bits and padding bits (there need not be any of the latter). If there are N value bits, each bit shall represent a different power of 2 between 1 and 2N−1, so that objects of that type shall be capable of representing values from 0 to 2N−1 using a pure binary representation; this shall be known as the value representation. The values of any padding bits are unspecified. uintN_t 和 intN_t 保证没有填充位元 (padding bits),且 intN_t 是二补数编码,所以对这两种类型进行位操作是安全的。 C99 7.18.1.1 Exact-width integer types The typedef name intN_t designates a signed integer type with width N, no padding bits, and a two’s complement representation. 信息 有符号整数上也有可能产生陷阱表示法 (trap representation) 补充资讯: CS:APP Web Aside DATA:TMIN: Writing TMin in C ","date":"2024-02-23","objectID":"/posts/c-bitwise/:1:1","tags":["Sysprog","C","Bitwise"],"title":"你所不知道的 C 语言: bitwise 操作","uri":"/posts/c-bitwise/"},{"categories":["C","Linux Kernel Internals"],"content":"位移运算 位移运算的未定义情况: C99 6.5.7 Bitwise shift operators 左移超过变量长度,则运算结果未定义 If the value of the right operand is negative or is greater than or equal to the width of the promoted left operand, the behavior is undefined. 对一个负数进行右移,C 语言规格未定义,作为 implementation-defined,GCC 实作为算术位移 (arithmetic shift) If E1 has a signed type and a negative value, the resulting value is implementation-defined. ","date":"2024-02-23","objectID":"/posts/c-bitwise/:1:2","tags":["Sysprog","C","Bitwise"],"title":"你所不知道的 C 语言: bitwise 操作","uri":"/posts/c-bitwise/"},{"categories":["C","Linux Kernel Internals"],"content":"Signed \u0026 Unsigned 当 Unsigned 和 Signed 混合在同一表达式时,Signed 会被转换成 Unsigned,运算结果可能不符合我们的预期 (这里大赞 Rust,这种情况会编译失败🤣)。案例请参考原文,这里举一个比较常见的例子: int n = 10; for (int i = n - 1 ; i - sizeof(char) \u003e= 0; i--) printf(\"i: 0x%x\\n\",i); 这段程式码会导致无限循环,因为条件判断语句 i - sizeof(char) \u003e= 0 恒为真 (变量 i 被转换成 Unsigned 了)。 6.5.3.4 The sizeof operator The value of the result is implementation-defined, and its type (an unsigned integer type) is size_t, defined in \u003cstddef.h\u003e (and other headers). 7.17 Common definitions \u003cstddef.h\u003e size_t which is the unsigned integer type of the result of the sizeof operator ","date":"2024-02-23","objectID":"/posts/c-bitwise/:1:3","tags":["Sysprog","C","Bitwise"],"title":"你所不知道的 C 语言: bitwise 操作","uri":"/posts/c-bitwise/"},{"categories":["C","Linux Kernel Internals"],"content":"Sign Extension 将 w bit signed integer 扩展为 w+k bit signed integer,只需将 sign bit 补充至扩展的 bits。 数值等价性推导: positive: 显然是正确的,sign bit 为 0,扩展后数值仍等于原数值 negitive: 将 w bit 情形时的除开 sign bit 的数值设为 U,则原数值为 $2^{-(w-1)} + U$,则扩展为 w+k bit 后数值为 $2^{-(w+k-1)} + 2^{w+k-2} + … + 2^{-(w-1)} + U$,因为 $2^{-(w+k-1)} + 2^{w+k-2} + … + 2^{w-1} = 2^{-(w-1)}$,所以数值依然等价。 $2^{-(w+k-1)} + 2^{w+k-2} + … + 2^{w-1}$ 可以考虑从左往右的运算,每次都是将原先的数值减半,所以最后的数值为 $2^{-(w+k-1)}$ 所以如果 n 是 signed 32-bit,则 n \u003e\u003e 31 等价于 n == 0 ? 0 : -1。在这个的基础上,请重新阅读 解读计算机编码 中的 abs 和 min/max 的常数时间实作。 ","date":"2024-02-23","objectID":"/posts/c-bitwise/:1:4","tags":["Sysprog","C","Bitwise"],"title":"你所不知道的 C 语言: bitwise 操作","uri":"/posts/c-bitwise/"},{"categories":["C","Linux Kernel Internals"],"content":"Bitwise Operator Bitwise Operators Quiz Answers Practice with bit operations Bitwise Practice Each lowercase letter is 32 + uppercase equivalent. This means simply flipping the bit at position 5 (counting from least significant bit at position 0) inverts the case of a letter. The gdb print command (shortened p) defaults to decimal format. Use p/format to instead select other formats such as x for hex, t for binary, and c for char. // unsigned integer `mine`, `yours` remove yours from mine mine = mine \u0026 ~yours test if mine has both of two lowest bits on (mine \u0026 0x3) == 0x3 n least significant bits on, all others off (1 \u003c\u003c n) - 1 k most significant bits on, all others off (~0 \u003c\u003c (32 - k)) or ~(~0U \u003e\u003e k) // unsigned integer `x`, `y` (right-shift: arithmetic shift) x \u0026= (x - 1) clears lowest \"on\" bit in x (x ^ y) \u003c 0 true if x and y have opposite signs 程序语言只提供最小粒度为 Byte 的操作,但是不直接提供 Bit 粒度的操作,这与字节顺序相关。假设提供以 Bit 为粒度的操作,这就需要在编程时考虑 大端/小端模式,极其繁琐。 ","date":"2024-02-23","objectID":"/posts/c-bitwise/:2:0","tags":["Sysprog","C","Bitwise"],"title":"你所不知道的 C 语言: bitwise 操作","uri":"/posts/c-bitwise/"},{"categories":["C","Linux Kernel Internals"],"content":"bitwise \u0026 logical 位运算满足交换律,但逻辑运算并不满足交换律,因为短路机制。考虑 Linked list 中的情形: // list_head *head if (!head || list_empty(head)) if (list_empty(head) || !head) 第二条语句在执行时会报错,因为 list_empty 要求传入的参数不为 NULL。 逻辑运算符 ! 相当有效,C99 并没有完全支持 bool 类型,对于整数,它是将非零整数视为 true,零视为 false。所以如果你需要保证某一表达式的结果不仅是 true of false,还要求对应 0 or 1 时,可以使用 !!(expr) 来实现。 C99 6.5.3.3 Unary arithmetic operators The result of the logical negation operator ! is 0 if the value of its operand compares unequal to 0, 1 if the value of its operand compares equal to 0. The result has type int. The expression !E is equivalent to (0==E). 所以 !!(expr) 的结果为 int 并且数值只有 0 或 1。 ","date":"2024-02-23","objectID":"/posts/c-bitwise/:2:1","tags":["Sysprog","C","Bitwise"],"title":"你所不知道的 C 语言: bitwise 操作","uri":"/posts/c-bitwise/"},{"categories":["C","Linux Kernel Internals"],"content":"Right Shifts 对于 Unsigned 或 positive sign integer 做右移运算时 x \u003e\u003e n,其最终结果值为 $\\lfloor x / 2^n \\rfloor$。 因为这种情况的右移操作相当于对每个 bit 表示的 power 加上 $-n$,再考虑有些 bit 表示的 power 加上 $-n$ 后会小于 0,此时直接将这些 bit 所表示的值去除即可 (因为在 integer 中 bit 的 power 最小为 0,如果 power 小于 0 表示的是小数值),这个操作对应于向下取整。 00010111 \u003e\u003e 2 (23 \u003e\u003e 4) -\u003e 000101.11 (5.75) -\u003e 000101 (5) ","date":"2024-02-23","objectID":"/posts/c-bitwise/:2:2","tags":["Sysprog","C","Bitwise"],"title":"你所不知道的 C 语言: bitwise 操作","uri":"/posts/c-bitwise/"},{"categories":["C","Linux Kernel Internals"],"content":"bitwise 实作 Vi/Vim 为什么使用 hjkl 作为移动字符? 當我們回顧 1967 年 ASCII 的編碼規範,可發現前 32 個字元都是控制碼,讓人們得以透過這些特別字元來控制畫面和相關 I/O,早期鍵盤的 “control” 按鍵就搭配這些特別字元使用。“control” 組合按鍵會將原本字元的第 1 個 bit 進行 XOR,於是 H 字元對應 ASCII 編碼為 100_1000 (過去僅用 7 bit 編碼),組合 “control” 後 (即 Ctrl+H) 會得到 000_1000,也就是 backspace 的編碼,這也是為何在某些程式中按下 backspace 按鍵會得到 ^H 輸出的原因。相似地,當按下 Ctrl+J 時會得到 000_1010,即 linefeed 注意 where n is the bit number, and 0 is the least significant bit Source ","date":"2024-02-23","objectID":"/posts/c-bitwise/:3:0","tags":["Sysprog","C","Bitwise"],"title":"你所不知道的 C 语言: bitwise 操作","uri":"/posts/c-bitwise/"},{"categories":["C","Linux Kernel Internals"],"content":"Set a bit unsigned char a |= (1 \u003c\u003c n); ","date":"2024-02-23","objectID":"/posts/c-bitwise/:3:1","tags":["Sysprog","C","Bitwise"],"title":"你所不知道的 C 语言: bitwise 操作","uri":"/posts/c-bitwise/"},{"categories":["C","Linux Kernel Internals"],"content":"Clear a bit unsigned char a \u0026= ~(1 \u003c\u003c n); ","date":"2024-02-23","objectID":"/posts/c-bitwise/:3:2","tags":["Sysprog","C","Bitwise"],"title":"你所不知道的 C 语言: bitwise 操作","uri":"/posts/c-bitwise/"},{"categories":["C","Linux Kernel Internals"],"content":"Toggle a bit unsigned char a ^= (1 \u003c\u003c n); ","date":"2024-02-23","objectID":"/posts/c-bitwise/:3:3","tags":["Sysprog","C","Bitwise"],"title":"你所不知道的 C 语言: bitwise 操作","uri":"/posts/c-bitwise/"},{"categories":["C","Linux Kernel Internals"],"content":"Test a bit bool a = (val \u0026 (1 \u003c\u003c n)) \u003e 0; ","date":"2024-02-23","objectID":"/posts/c-bitwise/:3:4","tags":["Sysprog","C","Bitwise"],"title":"你所不知道的 C 语言: bitwise 操作","uri":"/posts/c-bitwise/"},{"categories":["C","Linux Kernel Internals"],"content":"The right/left most byte // assuming 16 bit, 2-byte short integer unsigned short right = val \u0026 0xff; /* right most (least significant) byte */ unsigned short left = (val \u003e\u003e 8) \u0026 0xff; /* left most (most significant) byte */ // assuming 32 bit, 4-byte int integer unsigned int right = val \u0026 0xff; /* right most (least significant) byte */ unsigned int left = (val \u003e\u003e 24) \u0026 0xff; /* left most (most significant) byte */ ","date":"2024-02-23","objectID":"/posts/c-bitwise/:3:5","tags":["Sysprog","C","Bitwise"],"title":"你所不知道的 C 语言: bitwise 操作","uri":"/posts/c-bitwise/"},{"categories":["C","Linux Kernel Internals"],"content":"Sign bit // assuming 16 bit, 2-byte short integer, two's complement bool sign = val \u0026 0x8000; // assuming 32 bit, 4-byte int integer, two's complement bool sign = val \u0026 0x80000000; ","date":"2024-02-23","objectID":"/posts/c-bitwise/:3:6","tags":["Sysprog","C","Bitwise"],"title":"你所不知道的 C 语言: bitwise 操作","uri":"/posts/c-bitwise/"},{"categories":["C","Linux Kernel Internals"],"content":"Uses of Bitwise Operations or Why to Study Bits Compression Set operations Encryption 最常见的就是位图 (bitmap),常用于文件系统 (file system),可以节省空间 (每个元素只用一个 bit 来表示),可以很方便的进行集合操作 (通过 bitwise operator)。 x ^ y = (~x \u0026 y) | (x \u0026 ~y) ","date":"2024-02-23","objectID":"/posts/c-bitwise/:3:7","tags":["Sysprog","C","Bitwise"],"title":"你所不知道的 C 语言: bitwise 操作","uri":"/posts/c-bitwise/"},{"categories":["C","Linux Kernel Internals"],"content":"影像处理 Stack Overflow: what (r+1 + (r » 8)) » 8 does? 在图形引擎中将除法运算 x / 255 用位运算 (x+1 + (x \u003e\u003e 8)) \u003e\u003e 8 来实作,可以大幅度提升计算效能。 ","date":"2024-02-23","objectID":"/posts/c-bitwise/:4:0","tags":["Sysprog","C","Bitwise"],"title":"你所不知道的 C 语言: bitwise 操作","uri":"/posts/c-bitwise/"},{"categories":["C","Linux Kernel Internals"],"content":"案例分析 实作程式码: RGBAtoBW 给定每个 pixel 为 32-bit 的 RGBA 的 bitmap,提升效能的方案: 建立表格加速浮点运算 减少位运算: 可以使用 pointer 的 offset 取代原本复杂的 bitwise operation bwPixel = table[rgbPixel \u0026 0x00ffffff] + rgbPixel \u0026 0xff000000; 只需对 RGB 部分建立浮点数表,因为 rgbPixel \u0026 0xff00000 获取的是 A,无需参与浮点运算。这样建立的表最大下标应为 0x00ffffff,所以这个表占用 $2^{24} Bytes = 16MB$,显然这个表太大了 not cache friendly bw = (uint32_t) mul_299[r] + (uint32_t) mul_587[g] + (uint32_t) mul_144[b]; bwPixel = (a \u003c\u003c 24) + (bw \u003c\u003c 16) + (bw \u003c\u003c 8) + bw; 分别对 R, G, B 建立对应的浮点数表,则这三个表总共占用 $3 \\times 2^8 Bytes \u003c 32KB$ cache friendly ","date":"2024-02-23","objectID":"/posts/c-bitwise/:4:1","tags":["Sysprog","C","Bitwise"],"title":"你所不知道的 C 语言: bitwise 操作","uri":"/posts/c-bitwise/"},{"categories":["C","Linux Kernel Internals"],"content":"案例探讨 信息 位元旋转实作和 Linux 核心案例 reverse bit 原理和案例分析 ","date":"2024-02-23","objectID":"/posts/c-bitwise/:5:0","tags":["Sysprog","C","Bitwise"],"title":"你所不知道的 C 语言: bitwise 操作","uri":"/posts/c-bitwise/"},{"categories":["C","Linux Kernel Internals"],"content":"类神经网络的 ReLU 极其常数时间复杂度实作 原文地址 ReLU 定义如下: $$ ReLU(x) = \\begin{cases} x \u0026 \\text{if } x \\geq 0 \\newline 0 \u0026 \\text{if } x \\lt 0 \\end{cases} $$ 显然如果 $x$ 是 32-bit 的二补数,可以使用上面提到的 x \u003e\u003e 31 的技巧来实作 constant-time function: int32_t ReLU(int32_t x) { return ~(x \u003e\u003e 31) \u0026 x; } 但是在深度学习中,浮点数使用更加常见,对于浮点数进行位移运算是不允许的 C99 6.5.7 Bitwise shift operators Each of the operands shall have integer type. 所以这里以 32-bit float 浮点数类型为例,利用 32-bit 二补数和 32-bit float 的 MSB 都是 sign bit,以及 C 语言类型 union 的特性 C99 6.5.2.3 (82) If the member used to access the contents of a union object is not the same as the member last used to store a value in the object, the appropriate part of the object representation of the value is reinterpreted as an object representation in the new type as described in 6.2.6 (a process sometimes called “type punning”). This might be a trap representation. 即 union 所有成员是共用一块内存的,所以访问成员时会将这块内存存储的 object 按照成员的类型进行解释。利用 int32_t 和 float 的 MSB 都是 sign bit 的特性,可以巧妙绕开对浮点数进行位移运算的限制,并且因为 union 成员内存的共用性质,保证结果的数值符合预期。 float ReLU(float x) { union { float f; int32_t i; } u = {.f = x}; u.i \u0026= ~(u.i \u003e\u003e 31); return u.f; } 同理可以完成 64-bit 浮点数的 ReLU 常数时间实作。 double ReLU(float x) { union { double f; int64_t i; } u = {.f = x}; u.i \u0026= ~(u.i \u003e\u003e 63); return u.f; } ","date":"2024-02-23","objectID":"/posts/c-bitwise/:6:0","tags":["Sysprog","C","Bitwise"],"title":"你所不知道的 C 语言: bitwise 操作","uri":"/posts/c-bitwise/"},{"categories":["C","Linux Kernel Internals"],"content":"从 $\\sqrt 2$ 的存在谈开平方根的快速运算 原文地址 注意 这一部分需要学员对现代数学观点有一些了解,强烈推荐修台大齐震宇老师的「数学潜水艇/新生营演讲」,齐老师的这些讲座难度和广度大致相当于其它院校开设的「数学导论」一课。 ","date":"2024-02-23","objectID":"/posts/c-bitwise/:7:0","tags":["Sysprog","C","Bitwise"],"title":"你所不知道的 C 语言: bitwise 操作","uri":"/posts/c-bitwise/"},{"categories":["C","Linux Kernel Internals"],"content":"$\\sqrt 2$ 的缘起和计算 YouTube: 第一次数学危机,天才是怎么被扼杀的? 可以通过「十分逼近法」来求得近似精确的 $\\sqrt 2$ 的数值,这也是「数值计算/分析」领域的一个应用,除此之外还可以使用「二分逼近法」进行求值。十分逼近法和二分逼近法的主要区别在于:十分逼近法的收敛速度比二分逼近法快很多,即会更快求得理想范围精度对应的数值。 在数组方法的分析中,主要关心两件事: 收敛速度 误差分析 由逼近法的过程不难看出,它们非常适合使用递归来实作: YouTube: 二分逼近法和十分逼近法求平方根 ","date":"2024-02-23","objectID":"/posts/c-bitwise/:7:1","tags":["Sysprog","C","Bitwise"],"title":"你所不知道的 C 语言: bitwise 操作","uri":"/posts/c-bitwise/"},{"categories":["C","Linux Kernel Internals"],"content":"固定点和牛顿法 固定点定理相关的证明挺有意思的 (前提是你修过数学导论这类具备现代数学观点和思维的课 🤣): 存在性 (分类讨论) 唯一性 (反证法) 固定点定理 (逻辑推理) 可以将求方程的根转换成求固定点的问题,然后利用收敛数列进行求解: $f(x) = 0$ $g(x) = p - f(x)$ and $g(p) = p$ 牛顿法公式: $$ p \\approx p_0 -\\dfrac{f(p_0)}{f’(p_0)} $$ 后面利用 $f(x) = x^2 - N = 0$ 求平方根的公式可以根据这个推导而来的,图形化解释 (切线) 也符合这个公式,自行推导: $$ \\begin{split} f(x) \u0026= x^2-N = 0 \\\\ b \u0026= a - \\frac{f(a)}{f'(a)} = a - \\frac{a^2 - N}{2a} \\\\ \u0026= \\frac{a^2+N}{2a} = (a+\\frac{N}{a}) \\div 2 \\end{split} $$ int mySqrt(int n) { if (n == 0) return 0; if (n \u003c 4) return 1; unsigned int ans = n / 2; if (ans \u003e 65535) // 65535 = 2^16 - 1 ans = 65535; while (ans * ans \u003e n || (ans + 1) * (ans + 1) \u003c= n) ans = (ans + n / ans) / 2; return ans; } 这个方法的流程是,选定一个不小于目标值 $x$ 的初始值 $a$,这样依据牛顿法,$a_i,\\ a_{i-1},\\ …$ 会递减逼近 $x$。因为是递减的,所以防止第 12 行的乘法溢出只需要考虑初始值 $a$ 即可,这也是第 9~10 行的逻辑。那么只剩下一个问题:如何保证初始值 $a$ 不小于目标值 $x$ 呢?其实很简单,只需要根据当 $n \\geq 2$ 时满足 $n=x^2 \\geq 2x$,即 $\\frac{n}{2} \\geq x$,便可推断出 $\\frac{n}{2}$ 在 $n \\geq 2$ 时必然是满足大等于目标值 $x$,所以可以使用其作为初始值 $a$,这也是第 8 行的逻辑。 因为求解的目标精度是整数,所以第 12 行的判断是否求得平方根的逻辑是合理的 (结合中间值 $a_i$ 递减的特性思考)。 LeetCode 69. Sqrt(x) ","date":"2024-02-23","objectID":"/posts/c-bitwise/:7:2","tags":["Sysprog","C","Bitwise"],"title":"你所不知道的 C 语言: bitwise 操作","uri":"/posts/c-bitwise/"},{"categories":["C","Linux Kernel Internals"],"content":"二进位的平方根 在使用位运算计算平方根的程式码中,我们又见到了的 union 和 do {...} whil (0) 的运用。位运算求解平方根的核心在于: $n$ 可以根据 IEEE 754 表示为 $S\\times1.Frac\\times2^{127+E}$,这种表示法下求解平方根只需计算 $\\sqrt{1.Frac}$ 和 $\\sqrt{2^{(127+E)}}$ 两部分 (只考虑非负数的平方根)。虽然描述起来简单,但由于 IEEE 754 编码的复杂性,需要考虑很多情况,例如 $E$ 全 0 或全 1,因为此时对应的数值就不是之前表示的那样了 (指 $S\\times1.Frac\\times2^{127+E}$),需要额外考量。 sign: 1 bit 0x80000000 exponent: 8 bits 0x7f800000 fraction: 23 bits 0x007fffff 原文给出的程式码是用于计算 $n$ 在 IEEE 754 编码下的指数部分在平方根的结果,虽然看起来只需要除以 2 即右移 1 位即可,但其实不然,例如上面所说的考虑指数部分全为 0 的情况,所以这个程式码是精心设计用于通用计算的。 在原始程式码的基础上,加上对 ix0 (对应 $1.Frac$) 使用牛顿法求平方根的逻辑,即可完成对 n 的平方根的求解。 当然这里要求和之前一样,平方根只需要整数精度即可,所以只需求出指数部分的平方根,然后通过二分法进行逼近即可满足要求 (因为剩余部分是 $1.Frac$ 并不影响平方根的整数精度,但是会导致一定误差,所以需要对指数部分进行二分逼近求值)。 先求出整數 n 開根號後結果的 $1.FRACTION \\times 2^{EXPONENT}$ 的 $EXPONENT$ 部份,則我們知道 n 開根號後的結果 $k$ 應滿足 $2^{EXPONENT} \\leq k \u003c 2^{EXPONENT+1}$,因此後續可以用二分搜尋法找出結果。 注意 这段程式码可以再一次看到 枚举 union 和 宏 do {...} while (0) 的应用之外,主要是根据 IEEE 754 编码规范进行求解,所以需要对浮点数的编码格式有一定认知,可以参考阅读: 你所不知道的 C 语言: 浮点数运算。 ","date":"2024-02-23","objectID":"/posts/c-bitwise/:7:3","tags":["Sysprog","C","Bitwise"],"title":"你所不知道的 C 语言: bitwise 操作","uri":"/posts/c-bitwise/"},{"categories":["C","Linux Kernel Internals"],"content":"Fast Inverse Square Root (平方根倒数) 下面的录影解说了程式码中的黑魔法 0x5f3759df 的原理,本质也是牛顿法,只不过是 选择一个接近目标值的初始值,从而只需要一次牛顿法即可求解出相对高精度的目标平方根 (例如将 $1.4$ 作为牛顿法求 $\\sqrt 2$ 的初始值,只需一次迭代求解出的精度已经相当惊人了),除此之外还运用到了对数求平方根倒数的技巧: YouTube: 没有显卡的年代,这群程序员用4行代码优化游戏 使用 IEEE 754 表示任意单精度浮点数为: $x = (1 + \\frac{M}{2^{23}}) \\times 2^{E-127}$,则该 $x$ 对应的对数为 $$ \\begin{split} \\log x \u0026= \\log{(1 + \\frac{M}{2^{23}}) \\times 2^{E-127}} \\\\ \u0026= \\log{(1 + \\frac{M}{2^{23}})} + \\log{2^{E-127}} \\\\ \u0026= \\frac{M}{2^{23}} + E - 127 \\\\ \u0026 = \\frac{1}{2^{23}}(2^{23} \\times E + M) - 127 \\\\ \u0026 = \\frac{1}{2^{23}}X - 127 \\end{split} $$ 注意上面处理 $\\log{(1 + \\frac{M}{2^{23}})}$ 部分时使用近似函数 $f(x) = x$ 代替了,当然会有一些误差,但由于我们后面计算的是平方根倒数的近似值,所以有一些误差没有关系。最后的 $2^{23} \\times E + M$ 部分只和浮点数的表示域相关,并且 这个运算的结果值和以二补数编码解释浮点数的数值相同 (参考上面的 IEEE 754 浮点数结构图,以及二补数的数值计算规则),我们用一个大写标识 $X$ 来标记其只与浮点数编码相关,并且对应二补数编码下的数值。 推导出对数的通用公式后,接下来就可以推导 平方根倒数的近似值 了,即求得对应数值的 $-\\frac{1}{2}$ 次方。假设 $a$ 是 $y$ 的平方根倒数,则有等式: $$ \\begin{split} \\log a \u0026= \\log{y^{-\\frac{1}{2}}} \\\\ \\log a \u0026= -\\frac{1}{2} \\log y \\\\ -\\frac{1}{2^{23}}A - 127 \u0026= -\\frac{1}{2}(-\\frac{1}{2^{23}} - 127) \\\\ A \u0026= 381 \\times 2^{22} - \\frac{1}{2} Y \\end{split} $$ 中间将数值由浮点数转换成二补数编码表示,并求得最终的浮点数表示为 $381 \\times 2^{22} - \\frac{1}{2} Y$,其中的 $381 \\times 2^{22}$ 对应的 16 进制恰好为 0x5f400000,这已经很接近我们看到的魔数了,但还有一点偏差。 这是因为在计算 $\\log{(1 + \\frac{M}{2^{23}})}$ 时直接使用 $f(x) = x$ 导致的总体误差还是较大,但是只需要将其稍微往 $y$ 轴正方向偏移一些就可以减少总体误差 (机器学习中常用的技巧 🤣),所以使用 $\\frac{M}{2^{23}} + \\lambda$ 代替原先的 $\\frac{M}{2^{23}}$ ($\\lambda$ 为修正的误差且 $\\lambda \u003e 0$),这会导致最终结果的 381 发生稍微一些变化 (因为二补数编码解释浮点数格式部分 $X$ 不能动,只能影响常数 $127$,而常数 $127$ 又直接影响最终结果的 $381$ 这类常数部分),进而产生魔数 0x5f3759df。 float InvSqrt(float x) { float xhalf = 0.5f * x; int i = *(int *) \u0026x; i = 0x5f3759df - (i \u003e\u003e 1); x = *(float *) \u0026i; x = x * (1.5f - xhalf * x * x); // only once newton iteration return x; } ","date":"2024-02-23","objectID":"/posts/c-bitwise/:7:4","tags":["Sysprog","C","Bitwise"],"title":"你所不知道的 C 语言: bitwise 操作","uri":"/posts/c-bitwise/"},{"categories":["C","Linux Kernel Internals"],"content":"C 语言的 bit-field 原文地址 #include \u003cstdbool.h\u003e #include \u003cstdio.h\u003e bool is_one(int i) { return i == 1; } int main() { struct { signed int a : 1; } obj = { .a = 1 }; puts(is_one(obj.a) ? \"one\" : \"not one\"); return 0; } C99 6.7.2.1 Structure and union specifiers A bit-field shall have a type that is a qualified or unqualified version of _Bool, signed int, unsigned int, or some other implementation-defined type. A bit-field is interpreted as a signed or unsigned integer type consisting of the specified number of bits. 将 a 这个 1-bit 的位域 (bit-field) 声明成 signed int,即将 a 视为一个 1-bit 的二补数,所以 a 的数值只有 0,-1。接下来将 1 赋值给 a 会使得 a 的数值为 -1,然后将 a 作为参数传入 is_one 时会进行符号扩展 (sign extension) 为 32-bit 的二补数 (假设编译器会将 int 视为 signed int),所以数值仍然为 -1。因此最终会输出 “not one”. ","date":"2024-02-23","objectID":"/posts/c-bitwise/:8:0","tags":["Sysprog","C","Bitwise"],"title":"你所不知道的 C 语言: bitwise 操作","uri":"/posts/c-bitwise/"},{"categories":["C","Linux Kernel Internals"],"content":"Linux 核心: BUILD_BUG_ON_ZERO() /* * Force a compilation error if condition is true, but also produce a * result (of value 0 and type size_t), so the expression can be used * e.g. in a structure initializer (or where-ever else comma expressions * aren't permitted). */ #define BUILD_BUG_ON_ZERO(e) (sizeof(struct { int:(-!!(e)); })) 这个宏运用了上面所说的 !! 技巧将 -!!(e) 的数值限定在 0 和 -1。 这个宏的功能是: 当 e 为 true 时,-!!(e) 为 -1,即 bit-field 的 size 为负数 当 e 为 false 时,-!!(e) 为 0,即 bit-field 的 size 为 0 C99 6.7.2.1 Structure and union specifiers The expression that specifies the width of a bit-field shall be an integer constant expression with a nonnegative value that does not exceed the width of an object of the type that would be specified were the colon and expression omitted. If the value is zero, the declaration shall have no declarator. A bit-field declaration with no declarator, but only a colon and a width, indicates an unnamed bit-field. As a special case, a bit-field structure member with a width of 0 indicates that no further bit-field is to be packed into the unit in which the previous bitfield, if any, was placed. (108) An unnamed bit-field structure member is useful for padding to conform to externally imposed layouts. 根据上面 C99 标准的说明,当 bit-feild 的 size 为负数时会编译失败 (只允许 integer constant expression with a nonnegative value),当 bit-field 为 0 时,会进行 alignment (以之前的 bit-field 成员所在的 unit 为单位)。 struct foo { int a : 3; int b : 2; int : 0; /* Force alignment to next boundary */ int c : 4; int d : 3; }; int main() { int i = 0xFFFF; struct foo *f = (struct foo *) \u0026i; printf(\"a=%d\\nb=%d\\nc=%d\\nd=%d\\n\", f-\u003ea, f-\u003eb, f-\u003ec, f-\u003ed); return 0; } 这里使用了 size 为 0 的 bit-field,其内存布局如下: i = 1111 1111 1111 1111 X stand for unknown value assume little endian padding \u0026 start from here ↓ 1111 1111 1111 1111XXXX XXXX XXXX XXXX b baaa ddd cccc |← int 32 bits →||← int 32 bits →| zero size bit-field 使得这里在 a, b 和 c, d 之间进行 sizeof(int) 的 alignment,所以 c, d 位于 i 这个 object 范围之外,因此 c, d 每次执行时的数值是不确定的,当然这也依赖于编译器,可以使用 gcc 和 clang 进行测试 🤣 C11 3.14 1 memory location (NOTE 2) A bit-field and an adjacent non-bit-field member are in separate memory locations. The same applies to two bit-fields, if one is declared inside a nested structure declaration and the other is not, or if the two are separated by a zero-length bit-field declaration, or if they are separated by a non-bit-field member declaration. It is not safe to concurrently update two non-atomic bit-fields in the same structure if all members declared between them are also (non-zero-length) bit-fields, no matter what the sizes of those intervening bit-fields happen to be. 所以 BUILD_BUG_ON_ZERO 宏相当于编译时期的 assert,因为 assert 是在执行时期才会触发的,对于 Linux 核心来说代价太大了 (想象一下核心运行着突然触发一个 assert 导致当掉 🤣),所以采用了 BUILD_BUG_ON_ZERO 宏在编译时期就进行检查 (莫名有一种 Rust 的风格 🤣) 对于 BUILD_BUG_ON_ZERO 这个宏,C11 提供了 _Static_assert 语法达到相同效果,但是 Linux kernel 自己维护了一套编译工具链 (这个工具链 gcc 版本可能还没接纳 C11 🤣),所以还是使用自己编写的 BUILD_BUG_ON_ZERO 宏。 ","date":"2024-02-23","objectID":"/posts/c-bitwise/:8:1","tags":["Sysprog","C","Bitwise"],"title":"你所不知道的 C 语言: bitwise 操作","uri":"/posts/c-bitwise/"},{"categories":["Rust"],"content":" In this fourth Crust of Rust video, we cover smart pointers and interior mutability, by re-implementing the Cell, RefCell, and Rc types from the standard library. As part of that, we cover when those types are useful, how they work, and what the equivalent thread-safe versions of these types are. In the process, we go over some of the finer details of Rust's ownership model, and the UnsafeCell type. We also dive briefly into the Drop Check rabbit hole (https://doc.rust-lang.org/nightly/nomicon/dropck.html) before coming back up for air. 整理自 John Gjengset 的影片 ","date":"2024-02-20","objectID":"/posts/smart-pointers-and-interior-mutability/:0:0","tags":["Rust","Smart pointer","Interior Mutability"],"title":"Crust of Rust: Smart Pointers and Interior Mutability","uri":"/posts/smart-pointers-and-interior-mutability/"},{"categories":["Rust"],"content":"影片注解 ","date":"2024-02-20","objectID":"/posts/smart-pointers-and-interior-mutability/:1:0","tags":["Rust","Smart pointer","Interior Mutability"],"title":"Crust of Rust: Smart Pointers and Interior Mutability","uri":"/posts/smart-pointers-and-interior-mutability/"},{"categories":["Rust"],"content":"Interior Mutability Module std::cell Rust memory safety is based on this rule: Given an object T, it is only possible to have one of the following: Having several immutable references (\u0026T) to the object (also known as aliasing). Having one mutable reference (\u0026mut T) to the object (also known as mutability). Values of the Cell\u003cT\u003e, RefCell\u003cT\u003e, and OnceCell\u003cT\u003e types may be mutated through shared references (i.e. the common \u0026T type), whereas most Rust types can only be mutated through unique (\u0026mut T) references. We say these cell types provide ‘interior mutability’ (mutable via \u0026T), in contrast with typical Rust types that exhibit ‘inherited mutability’ (mutable only via \u0026mut T). We can use (several) immutable references of a cell to mutate the thing inside of the cell. There is (virtually) no way for you to get a reference to the thing inside of a cell. Because if no one else has a pointer to it (the thing inside of a cell), the changing it is fine. Struct std::cell::UnsafeCell If you have a reference \u0026T, then normally in Rust the compiler performs optimizations based on the knowledge that \u0026T points to immutable data. Mutating that data, for example through an alias or by transmuting an \u0026T into an \u0026mut T, is considered undefined behavior. UnsafeCell\u003cT\u003e opts-out of the immutability guarantee for \u0026T: a shared reference \u0026UnsafeCell\u003cT\u003e may point to data that is being mutated. This is called “interior mutability”. The UnsafeCell API itself is technically very simple: .get() gives you a raw pointer *mut T to its contents. It is up to you as the abstraction designer to use that raw pointer correctly. ","date":"2024-02-20","objectID":"/posts/smart-pointers-and-interior-mutability/:1:1","tags":["Rust","Smart pointer","Interior Mutability"],"title":"Crust of Rust: Smart Pointers and Interior Mutability","uri":"/posts/smart-pointers-and-interior-mutability/"},{"categories":["Rust"],"content":"Cell Module std::cell Cell\u003cT\u003e Cell\u003cT\u003e implements interior mutability by moving values in and out of the cell. That is, an \u0026mut T to the inner value can never be obtained, and the value itself cannot be directly obtained without replacing it with something else. Both of these rules ensure that there is never more than one reference pointing to the inner value. This type provides the following methods: Cell 在 Rust 中对一个变量 (T),在已存在其 immutable references (\u0026T) 时使用 mutable reference (\u0026mut T) 是禁止的,因为这样会因为编译器优化而导致程序的行为不一定符合我们的预期。考虑以下的代码: let x = 3; let r1 = \u0026x, r2 = \u0026x; let r3 = \u0026mut x; println!(\"{}\", r1); r3 = 5; println!(\"{}\", r2); 假设以上的代码可以通过编译,那么程序执行到第 6 行打印出来的可能是 3 而不是我们预期的 5,这是因为编译器会对 immtuable references 进行激进的优化,例如进行预取,所以在第 6 行时对于 r2 使用的还是先前预取的值 3 而不是内存中最新的值 5。这也是 Rust 制定对 immutable reference 和 mutable reference 的规则的原因之一。 为了达到我们的预期行为,可以使用 UnsafeCell 来实现: let x = 3; let uc = UnsafeCell::new(x); let r1 = \u0026uc, r2 = \u0026uc, r3 = \u0026uc; unsafe { println!(\"{}\", *uc.get()); } unsafe { *uc.get() = 5; } unsafe { println!(\"{}\", *uc.get()); } 上面的代码可以通过编译,并且在第 6 行时打印出来的是预期的 5。这是因为编译器会对 UnsafeCell 进行特判,而避免进行一些激进的优化 (例如预取),从而使程序行为符合我们的预期。并且 UnsafeCell 的 get() 方法只需要接受 \u0026self 参数,所以可以对 UnsafeCell 进行多个 immutable references,这并不违反 Rust 的内存安全准则。同时每个对于 UnsafeCel 的 immutable references 都可以通过所引用的 UnsafeCell 来实现内部可变性 (interior mutability)。 上述代码片段存在大量的 unsafe 片段 (因为 UnsafeCell),将这些 unsafe 操作封装一下就实现了 Cell。但是因为 Cell 的方法 get() 和 set() 都需要转移所有权,所以 Cell 只能用于实现了 Copy trait 的类型的内部可变性。但是对于 concurrent 情形,UnsafeCell 就是一个临界区,无法保证内部修改是同步的,所以 Cell 不是 thread safe 的。 Cell is typically used for more simple types where copying or moving values isn’t too resource intensive (e.g. numbers) 注意 Cell 提供了这样一个“内部可变性”机制: 在拥有对一个 object 多个引用时,可以通过任意一个引用对 object 进行内部可变,并保证在此之后其他引用对于该 object 的信息是最新的。 ","date":"2024-02-20","objectID":"/posts/smart-pointers-and-interior-mutability/:1:2","tags":["Rust","Smart pointer","Interior Mutability"],"title":"Crust of Rust: Smart Pointers and Interior Mutability","uri":"/posts/smart-pointers-and-interior-mutability/"},{"categories":["Rust"],"content":"RefCell Module std::cell RefCell\u003cT\u003e RefCell\u003cT\u003e uses Rust\\’s lifetimes to implement “dynamic borrowing”, a process whereby one can claim temporary, exclusive, mutable access to the inner value. Borrows for RefCell\u003cT\u003e\\s are tracked at runtime, unlike Rust’s native reference types which are entirely tracked statically, at compile time. Runtime Borrow Check RefCell RefCell 也提供了之前所提的“内部可变性”机制,但是是通过提供 引用 而不是转移所有权来实现。所以它常用于 Tree, Graph 这类数据结构,因为这些数据结构的节点 “很大”,不大可能实现 Copy 的 Trait (因为开销太大了),所以一般使用 RefCell 来实现节点的相互引用关系。 ","date":"2024-02-20","objectID":"/posts/smart-pointers-and-interior-mutability/:1:3","tags":["Rust","Smart pointer","Interior Mutability"],"title":"Crust of Rust: Smart Pointers and Interior Mutability","uri":"/posts/smart-pointers-and-interior-mutability/"},{"categories":["Rust"],"content":"Rc method std::boxed::Box::into_raw After calling this function, the caller is responsible for the memory previously managed by the Box. In particular, the caller should properly destroy T and release the memory, taking into account the memory layout used by Box. The easiest way to do this is to convert the raw pointer back into a Box with the Box::from_raw function, allowing the Box destructor to perform the cleanup. Rc ","date":"2024-02-20","objectID":"/posts/smart-pointers-and-interior-mutability/:1:4","tags":["Rust","Smart pointer","Interior Mutability"],"title":"Crust of Rust: Smart Pointers and Interior Mutability","uri":"/posts/smart-pointers-and-interior-mutability/"},{"categories":["Rust"],"content":"Raw pointers vs references * mut and * const are not references, they are raw pointers. In Rust, there are a bunch of semantics you have to follow when you using references. Like if you use \u0026 symbol, an \u0026 alone means a shared reference, and you have to guarantee that there are no exclusive references to that thing. And similarly, if you have a \u0026mut, a exclusive reference, you know that there are not shared references. The * version of these, like * mut and * const, do not have these guarantees. If you have a * mut, there may be other * muts to the same thing. There might be * const to the same thing. You have no guarantee, but you also cann't do much with a *. If you have a raw pointer, the only thing you can really do to it is use an unsafe block to dereference it and turn it into reference. But that is unsafe, you need to document wht it is safe. You're not able to go from a const pointer to an exclusive reference. But you can go from a mutable pointer to an exclusive reference. To guarantee that you have to follow onwership semantics in Rust. ","date":"2024-02-20","objectID":"/posts/smart-pointers-and-interior-mutability/:1:5","tags":["Rust","Smart pointer","Interior Mutability"],"title":"Crust of Rust: Smart Pointers and Interior Mutability","uri":"/posts/smart-pointers-and-interior-mutability/"},{"categories":["Rust"],"content":"PhantomData \u0026 Drop check The Rustonomicon: Drop Check Medium: Rust Notes: PhantomData struct Foo\u003c'a, T: Default\u003e { v: \u0026'a mut T, } impl\u003cT\u003e Drop for Foo\u003c'_, T: Default\u003e { fn drop(\u0026mut self) { let _ = std::mem::replace(self.v, T::default()); } } fn main() { let mut t = String::from(\"hello\"); let foo = Foo { v: \u0026mut t }; drop(t); drop(foo); } 最后的 2 行 drop 语句会导致编译失败,因为编译器知道 foo 引用了 t,所以会进行 drop check,保证 t 的 lifetime 至少和 foo 一样长,因为 drop 时会按照从内到外的顺序对结构体的成员及其本身进行 drop。但是对于我们实现的 Rc 使用的是 raw pointer,如果不加 PhantomData,那么在对 Rc 进行 drop 时并不会检查 raw pointer 所指向的 RcInner 的 lifetime 是否满足要求,即在 drop Rc 之前 drop RcInner 并不会导致编译失败。简单来说,PhantomData 就是让编译器以为 Rc 拥有 RcInner 的所有权或引用,由此进行期望的 drop check 行为。 ","date":"2024-02-20","objectID":"/posts/smart-pointers-and-interior-mutability/:1:6","tags":["Rust","Smart pointer","Interior Mutability"],"title":"Crust of Rust: Smart Pointers and Interior Mutability","uri":"/posts/smart-pointers-and-interior-mutability/"},{"categories":["Rust"],"content":"Thread Safety Cell Because even though you're not giving out references to things, having two threads modify the same value at the same time is just not okay. There actually is o thread-safe version of Cell. (Think it as pointer in C 🤣) RefCell You could totally implement a thread-safe version of RefCell, one that uses an atomic counter instead of Cell for these numbers. So it turns out that the CPU has built-in instructions that can, in a thread-safe way, increment and decrement counters. Rc The thread-safe version of Rc is Arc, or Atomic Reference Count. ","date":"2024-02-20","objectID":"/posts/smart-pointers-and-interior-mutability/:1:7","tags":["Rust","Smart pointer","Interior Mutability"],"title":"Crust of Rust: Smart Pointers and Interior Mutability","uri":"/posts/smart-pointers-and-interior-mutability/"},{"categories":["Rust"],"content":"Copy-on-Write (COW) Struct std::borrow::Cow The type Cow is a smart pointer providing clone-on-write functionality: it can enclose and provide immutable access to borrowed data, and clone the data lazily when mutation or ownership is required. The type is designed to work with general borrowed data via the Borrow trait. ","date":"2024-02-20","objectID":"/posts/smart-pointers-and-interior-mutability/:1:8","tags":["Rust","Smart pointer","Interior Mutability"],"title":"Crust of Rust: Smart Pointers and Interior Mutability","uri":"/posts/smart-pointers-and-interior-mutability/"},{"categories":["Rust"],"content":"Homework 信息 实作说明: 尝试使用 RefCell 来实现 Linux kernel 风格的 linked list 数据结构为 circular doubly linked list 实现 insert_head, remove_head 方法 实现 insert_tail, remove_tail 方法 实现 list_size, list_empty, list_is_singular 方法 实现迭代器 (Iterator),支持双向迭代 (DoubleEndedIterator) 参考资料: sysprog21/linux-list linux/list.h ","date":"2024-02-20","objectID":"/posts/smart-pointers-and-interior-mutability/:2:0","tags":["Rust","Smart pointer","Interior Mutability"],"title":"Crust of Rust: Smart Pointers and Interior Mutability","uri":"/posts/smart-pointers-and-interior-mutability/"},{"categories":["Rust"],"content":"Documentations 这里列举视频中一些概念相关的 documentation 学习的一手资料是官方文档,请务必自主学会阅读规格书之类的资料 ","date":"2024-02-20","objectID":"/posts/smart-pointers-and-interior-mutability/:3:0","tags":["Rust","Smart pointer","Interior Mutability"],"title":"Crust of Rust: Smart Pointers and Interior Mutability","uri":"/posts/smart-pointers-and-interior-mutability/"},{"categories":["Rust"],"content":"Crate std 可以使用这里提供的搜素栏进行搜索 (BTW 不要浪费时间在 Google 搜寻上!) Module std::cell Struct std::cell::UnsafeCell Struct std::cell::Cell Struct std::cell::RefCell Module std::rc Module std::sync Struct std::sync::Mutex Struct std::sync::RwLock Struct std::sync::Arc Struct std::boxed::Box method std::boxed::Box::into_raw method std::boxed::Box::from_raw Struct std::ptr::NonNull method std::ptr::NonNull::new_unchecked method std::ptr::NonNull::as_ref method std::ptr::NonNull::as_ptr Struct std::marker::PhantomData Struct std::borrow::Cow Trait std::ops::Drop Trait std::ops::Deref Trait std::ops::DerefMut Trait std::marker::Sized Function std::thread::spawn Function std::mem::replace Function std::mem::drop ","date":"2024-02-20","objectID":"/posts/smart-pointers-and-interior-mutability/:3:1","tags":["Rust","Smart pointer","Interior Mutability"],"title":"Crust of Rust: Smart Pointers and Interior Mutability","uri":"/posts/smart-pointers-and-interior-mutability/"},{"categories":["Rust"],"content":"References 可能不是你看过最无聊的 Rust 入门喜剧102 (2) 智能指针 [bilibili] ","date":"2024-02-20","objectID":"/posts/smart-pointers-and-interior-mutability/:4:0","tags":["Rust","Smart pointer","Interior Mutability"],"title":"Crust of Rust: Smart Pointers and Interior Mutability","uri":"/posts/smart-pointers-and-interior-mutability/"},{"categories":["C","Linux Kernel Internals"],"content":" 尽管数值系统并非 C 语言所持有,但在 Linux 核心大量存在 u8/u16/u32/u64 这样通过 typedef 所定义的类型,伴随着各种 alignment 存取,如果对数值系统的认知不够充分,可能立即就被阻拦在探索 Linux 核心之外——毕竟你完全搞不清楚,为何 Linux 核心存取特定资料需要绕一大圈。 原文地址 ","date":"2024-02-20","objectID":"/posts/c-numerics/:0:0","tags":["Sysprog","C","Numerics"],"title":"你所不知道的 C 语言: 数值系统篇","uri":"/posts/c-numerics/"},{"categories":["C","Linux Kernel Internals"],"content":"Balanced ternary balanced ternary 三进制中 -, 0, + 在数学上具备对称性质。它相对于二进制编码的优势在于,其本身就可以表示正负数 (通过 +-, 0, +),而二进制需要考虑 unsigned 和 signed 的情况,从而决定最高位所表示的数值。 相关的运算规则: + add - = 0 0 add + = + 0 add - = - 以上运算规则都比较直观,这也决定了 balanced ternary 在编码上的对称性 (减法等价于加上逆元,逆元非常容易获得)。但是需要注意,上面的运算规则并没有涉及到相同位运算的规则,例如 $+\\ (add)\\ +$,这种运算也是 balanced ternary 相对于二进制编码的劣势,可以自行推导一下这种运算的规则。 The Balanced Ternary Machines of Soviet Russia ","date":"2024-02-20","objectID":"/posts/c-numerics/:1:0","tags":["Sysprog","C","Numerics"],"title":"你所不知道的 C 语言: 数值系统篇","uri":"/posts/c-numerics/"},{"categories":["C","Linux Kernel Internals"],"content":"数值编码与阿贝尔群 阿贝尔群也用于指示为什么使用二补数编码来表示整数: 存在唯一的单位元 (二补数中单位元 0 的编码是唯一的) 每个元素都有逆元 (在二补数中几乎每个数都有逆元) 浮点数 IEEE 754: An example of a layout for 32-bit floating point is Conversión de un número binario a formato IEEE 754 单精度浮点数相对于整数 在某些情況下不满足結合律和交换律,所以不构成 阿贝尔群,在编写程序时需要注意这一点。即使编写程序时谨慎处理了单精度浮点数运算,但是编译器优化可能会将我们的处理破划掉。所以涉及到单精度浮点数,都需要注意其运算。 信息 你所不知道的 C 语言: 浮点数运算 你所不知道的 C 语言: 编译器和最佳化原理篇 ","date":"2024-02-20","objectID":"/posts/c-numerics/:2:0","tags":["Sysprog","C","Numerics"],"title":"你所不知道的 C 语言: 数值系统篇","uri":"/posts/c-numerics/"},{"categories":["C","Linux Kernel Internals"],"content":"Integer Overflow ","date":"2024-02-20","objectID":"/posts/c-numerics/:3:0","tags":["Sysprog","C","Numerics"],"title":"你所不知道的 C 语言: 数值系统篇","uri":"/posts/c-numerics/"},{"categories":["C","Linux Kernel Internals"],"content":"2002 年 FreeBSD [53] #define KSIZE 1024 char kbuf[KSIZE]; int copy_from_kernel(void *user_dest, int maxlen) { int len = KSIZE \u003c maxlen ? KSIZE : maxlen; memcpy(user_dest, kbuf, len); return len; } 假设将“负”的数值带入 maxlen,那么在上述的程式码第 4 行时 len 会被赋值为 maxlen,在第 5 行中,根据 memcpy 的原型声明 void *memcpy(void *dest, const void *src, size_t n); 会将 len (=maxlen) 解释为 size_t 类型,关于 size_t 类型 C99 [7.17 Common definitions \u003cstddef.h\u003e] size_t which is the unsigned integer type of the result of the sizeof operator; 所以在 5 行中 memcpy 会将 len 这个“负“的数值按照无符号数的编码进行解释,这会导致将 len 解释为一个超级大的无符号数,可能远比 KSIZE 这个限制大。copy_from_kernel 这个函数是运行在 kernel 中的,这样可能会造成潜在的 kernel 信息数据泄露问题。 ","date":"2024-02-20","objectID":"/posts/c-numerics/:3:1","tags":["Sysprog","C","Numerics"],"title":"你所不知道的 C 语言: 数值系统篇","uri":"/posts/c-numerics/"},{"categories":["C","Linux Kernel Internals"],"content":"2002 年 External data representation (XDR) [62] void *copy_elements(void *ele_src[], int ele_cnt, int ele_size) { void *result = malloc(ele_cnt * ele_size); if (result==NULL) return NULL; void *next = result; for (int i = 0; i \u003c ele_cnt; i++) { memcpy(next, ele_src[i], ele_size); next += ele_size; } return result; } 假设将 ele_cnt = $2^{20}+1$, ele_size=$2^{12}$ 代入,显然在第 2 行的 ele_cnt * ele_size 会超出 32 位整数表示的最大值,导致 overflow。又因为 malloc 的原型声明 void *malloc(size_t size); malloc 会将 ele_cnt * ele_size 溢出后保留的值解释为 size_t,这会导致 malloc 分配的内存空间远小于 ele_cnt * ele_size Bytes (这是 malloc 成功的情况,malloc 也有可能会失败,返回 NULL)。 因为 malloc 成功分配空间,所以会通过第 3 行的测试。在第 5~8 行的 for 循环,根据 ele_cnt 和 ele_size 的值进行 memcpy,但是因为分配的空间远远小于 ele_cnt * ele_size,所以这样会覆写被分配空间外的内存区域,可能会造成 kernel 的信息数据被覆盖。 ","date":"2024-02-20","objectID":"/posts/c-numerics/:3:2","tags":["Sysprog","C","Numerics"],"title":"你所不知道的 C 语言: 数值系统篇","uri":"/posts/c-numerics/"},{"categories":["C","Linux Kernel Internals"],"content":"Bitwise 3Blue1Brown: How to count to 1000 on two hands [YouTube] 本质上是使用无符号数的二进制编码来进行计数,将手指/脚趾视为数值的 bit 信息 解读计算机编码 ","date":"2024-02-20","objectID":"/posts/c-numerics/:4:0","tags":["Sysprog","C","Numerics"],"title":"你所不知道的 C 语言: 数值系统篇","uri":"/posts/c-numerics/"},{"categories":["C","Linux Kernel Internals"],"content":"Power of two 通过以下程式码可以判断 x 是否为 2 的次方 x \u0026 (x - 1) == 0 通过值为 1 的最低位来进行归纳法证明,例如,对 0b00000001, 0b00000010, 0b00000100, … 来进行归纳证明 (还需要证明 x 中只能有一个 bit 为值 1,不过这个比较简单)。另一种思路,通过 LSBO 以及 $X$ 和 $-X$ 的特性来证明。 LSBO: Least Significant bit of value One $-X = ~(X - 1)$ $-X$ 的编码等价于 $X$ 的编码中比 LSBO 更高的 bits 进行反转,LSBO 及更低的 bits 保持不变 ","date":"2024-02-20","objectID":"/posts/c-numerics/:4:1","tags":["Sysprog","C","Numerics"],"title":"你所不知道的 C 语言: 数值系统篇","uri":"/posts/c-numerics/"},{"categories":["C","Linux Kernel Internals"],"content":"ASCII table 通过 ASCII table 中对 ASCII 编码的分布规律,可以实现大小写转换的 constant-time function 也可以通过命令 man ascii 来输出精美的 ASCII table // 字符转小写 (x | ' ') // 字符转大写 (x \u0026 ' ') // 大小写互转 (x ^ ' ') Each lowercase letter is 32 + uppercase equivalent. This means simply flipping the bit at position 5 (counting from least significant bit at position 0) inverts the case of a letter. ","date":"2024-02-20","objectID":"/posts/c-numerics/:4:2","tags":["Sysprog","C","Numerics"],"title":"你所不知道的 C 语言: 数值系统篇","uri":"/posts/c-numerics/"},{"categories":["C","Linux Kernel Internals"],"content":"XOR swap 通过 xor 运算符可以实现无需临时变量的,交换两个数值的程式码 void xorSwap(int *x, int *y) { *x ^= *y; *y ^= *x; *x ^= *y; } 第 3 行的 *y ^= *x 的结果等价于 *y ^ *x ^ *y,整数满足交换律和结合律,所以结果为 *x 第 4 行的 *x ^= *y 的结果等价于 *x ^ *y ^ *x,整数满足交换律和结合律,所以结果为 *y 这个实作方法常用于没有额外空间的情形,例如 Bootloader ","date":"2024-02-20","objectID":"/posts/c-numerics/:4:3","tags":["Sysprog","C","Numerics"],"title":"你所不知道的 C 语言: 数值系统篇","uri":"/posts/c-numerics/"},{"categories":["C","Linux Kernel Internals"],"content":"避免 overflow 整数运算 (x + y) / 2 可能会导致 overflow (如果 x, y 数值都接近 UINT32_MAX),可以改写为以下不会导致 overflow 的程式码 (x \u0026 y) + (x ^ y) \u003e\u003e 1 使用加法器来思考: 对于 x + y,x \u0026 y 表示进位,x ^ y 表示位元和,所以 x + y 等价于 (x \u0026 y) \u003c\u003c 1 + (x ^ y) 这个运算不会导致 overflow (因为使用了 bitwise 运算)。因此 (x + y) / 2 等价于 ((x \u0026 y) \u003c\u003c 1 + (x ^ y)) \u003e\u003e 1 = ((x \u0026 y) \u003c\u003c 1) \u003e\u003e 1 + (x ^ y) \u003e\u003e 1 = (x \u0026 y) + (x ^ y) \u003e\u003e 1 整数满足交换律和结合律 ","date":"2024-02-20","objectID":"/posts/c-numerics/:4:4","tags":["Sysprog","C","Numerics"],"title":"你所不知道的 C 语言: 数值系统篇","uri":"/posts/c-numerics/"},{"categories":["C","Linux Kernel Internals"],"content":"macro DIRECT #if LONG_MAX == 2147483647L #define DETECT(X) \\ (((X) - 0x01010101) \u0026 ~(X) \u0026 0x80808080) #else #if LONG_MAX == 9223372036854775807L #define DETECT(X) \\ (((X) - 0x0101010101010101) \u0026 ~(X) \u0026 0x8080808080808080) #else #error long int is not a 32bit or 64bit type. #endif #endif DIRECT 宏的作用是侦测 32bit/64bit 中是否存在一个 Byte 为 NULL。我们以最简单的情况 1 个 Byte 时来思考这个实作的本质: ((X) - 0x01) \u0026 ~(X) \u0026 0x80 = ~(~((X) - 0x01) | X) \u0026 0x80 ~((X) - 0x01) 是 X 的取负值编码,即 -X,根据二补数编码中 -X 和 X 的特性,可得 (~((X) - 0x01) | X) 为: X 二补数编码中值为 1 的最低位 (后续称之为 LSBO) 及更低位保持不变,LSBO 更高位均为 1。则 ~(~((X) - 0x01) | X) 为: X 二补数编码中值为 1 的最低位 (后续称之为 LSBO) 的更低位翻转,LSBO 及更高位均为 0。 LSBO: Least Significant Bit with value of One X = 0x0080 (X) - 0x01 = 0xff80 ~((X) - 0x01) = 0x007f ~(~((X) - 0x01) | X) \u0026 0x80 = 0 可以自行归纳推导出: 对于任意不为 0 的数值,上述流程推导的最终值都为 0,但对于值为 0 的数值,最终值为 0x80。由此可以推导出最开始的实作 DIRECT 宏。 这个 DIRECT 宏相当实用,常用于加速字符串操作,将原先的以 1-byte 为单元的操作加速为以 32bit/64bit 为单位的操作。可以阅读相关实作并寻找其中的逻辑: newlib 的 strlen newlib 的 strcpy ","date":"2024-02-20","objectID":"/posts/c-numerics/:4:5","tags":["Sysprog","C","Numerics"],"title":"你所不知道的 C 语言: 数值系统篇","uri":"/posts/c-numerics/"},{"categories":["C","Linux Kernel Internals"],"content":"Count Leading Zero 计算 $\\log_2N$ 可以通过计算数值对应的编码,高位有多少连续的 0’bits,再用 31 减去即可。可以通过 0x0001, 0x0010, 0x0002, … 等编码来进行归纳推导出该结论。 iteration version int clz(uint32_t x) { int n = 32, c = 16; do { uint32_t y = x \u003e\u003e c; if (y) { n -= c; x = y; } c \u003e\u003e= 1; } while (c); return (n - x); } binary search technique int clz(uint32_t x) { if (x == 0) return 32; int n = 0; if (x \u003c= 0x0000FFFF) { n += 16; x \u003c\u003c= 16; } if (x \u003c= 0x00FFFFFF) { n += 8; x \u003c\u003c= 8; } if (x \u003c= 0x0FFFFFFF) { n += 4; x \u003c\u003c= 4; } if (x \u003c= 0x3FFFFFFF) { n += 2; x \u003c\u003c= 2; } if (x \u003c= 0x7FFFFFFF) { n += 1; x \u003c\u003c= 1; } return n; } byte-shift version int clz(uint32_t x) { if (x == 0) return 32; int n = 1; if ((x \u003e\u003e 16) == 0) { n += 16; x \u003c\u003c= 16; } if ((x \u003e\u003e 24) == 0) { n += 8; x \u003c\u003c= 8; } if ((x \u003e\u003e 28) == 0) { n += 4; x \u003c\u003c= 4; } if ((x \u003e\u003e 30) == 0) { n += 2; x \u003c\u003c= 2; } n = n - (x \u003e\u003e 31); return n; } 在这些实作中,循环是比较直观的,但是比较低效;可以利用编码的特性,使用二分法或位运算来加速实作。 ","date":"2024-02-20","objectID":"/posts/c-numerics/:5:0","tags":["Sysprog","C","Numerics"],"title":"你所不知道的 C 语言: 数值系统篇","uri":"/posts/c-numerics/"},{"categories":["C","Linux Kernel Internals"],"content":"避免循环 int func(unsigned int x) { int val = 0; int i = 0; for (i = 0; i \u003c 32; i++) { val = (val \u003c\u003c 1) | (x \u0026 0x1); x \u003e\u003e= 1; } return val; } 这段程式码的作用是,对一个 32bit 的数值进行逐位元反转。这个逐位元反转功能非常实用,常实作于加密算法,例如 DES、AES。 但是与上面的 Count Leading Zero 类似,上面程式码使用了循环,非常低效,可以通过位运算来加速。 int func(unsigned int x) { int val = 0; val = num; val = ((val \u0026 0xffff0000) \u003e\u003e 16) | ((val \u0026 0x0000ffff) \u003c\u003c 16); val = ((val \u0026 0xff00ff00) \u003e\u003e 8) | ((val \u0026 0x00ff00ff) \u003c\u003c 8); val = ((val \u0026 0xf0f0f0f0) \u003e\u003e 4) | ((val \u0026 0x0f0f0f0f) \u003c\u003c 4); val = ((val \u0026 0xcccccccc) \u003e\u003e 2) | ((val \u0026 0x33333333) \u003c\u003c 2); val = ((val \u0026 0xaaaaaaaa) \u003e\u003e 1) | ((val \u0026 0x55555555) \u003c\u003c 1); return val; } Reverse integer bitwise without using loop [Stack Overflow] 技巧 Bits Twiddling Hacks 解析: (一), (二), (三) ","date":"2024-02-20","objectID":"/posts/c-numerics/:6:0","tags":["Sysprog","C","Numerics"],"title":"你所不知道的 C 语言: 数值系统篇","uri":"/posts/c-numerics/"},{"categories":["C","Linux Kernel Internals"],"content":"加解密的应用 假設有一張黑白的相片是由很多個0 ~255 的 pixel 組成 (0 是黑色,255 是白色),這時候可以用任意的 KEY (000000002 - 111111112) 跟原本的每個 pixel 做運算,如果使用 AND (每個 bit 有 75% 機率會變成 0),所以圖會變暗。如果使用 OR (每個 bit 有 75% 機率會變 1),圖就會變亮。這兩種幾乎都還是看的出原本的圖片,但若是用 XOR 的話,每個 bit 變成 0 或 1 的機率都是 50%,所以圖片就會變成看不出東西的雜訊。 上圖左 1 是原圖,左 2 是用 AND 做運算之後,右 2 是用 OR 做運算之後,右 1 是用 XOR,可見使用 XOR 的加密效果最好。 这就是在密码学领域偏爱 XOR 的原因之一。除此之外,XOR 在概率统计上的优异特性也是另一个原因,具体证明推导请查看原文的说明。 ","date":"2024-02-20","objectID":"/posts/c-numerics/:7:0","tags":["Sysprog","C","Numerics"],"title":"你所不知道的 C 语言: 数值系统篇","uri":"/posts/c-numerics/"},{"categories":["Linux Kernel Internals"],"content":"程序分析工具 Cppcheck 是 静态 程序分析工具,即无需运行程序就可以分析出程序潜在的问题,当然会有一定的误差,类似的工具有 cargo-check Valgrind 是 动态 程序分析工具,即需要将程序运行起来再进行分析,通常用于检测内存泄漏 (memory leak) ","date":"2024-02-19","objectID":"/posts/linux2023-lab0/:1:0","tags":["Sysprog","Linux","C"],"title":"Linux 核心设计: C Programming Lab","uri":"/posts/linux2023-lab0/"},{"categories":["Linux Kernel Internals"],"content":"Queue ","date":"2024-02-19","objectID":"/posts/linux2023-lab0/:2:0","tags":["Sysprog","Linux","C"],"title":"Linux 核心设计: C Programming Lab","uri":"/posts/linux2023-lab0/"},{"categories":["Linux Kernel Internals"],"content":"leetcode 相关的 LeetCode 题目的实作情况: LeetCode 2095. Delete the Middle Node of a Linked List LeetCode 82. Remove Duplicates from Sorted List II LeetCode 24. Swap Nodes in Pairs LeetCode 25. Reverse Nodes in k-Group LeetCode 2487. Remove Nodes From Linked List / 参考题解 LeetCode 23. Merge k Sorted Lists ","date":"2024-02-19","objectID":"/posts/linux2023-lab0/:2:1","tags":["Sysprog","Linux","C"],"title":"Linux 核心设计: C Programming Lab","uri":"/posts/linux2023-lab0/"},{"categories":["Linux Kernel Internals"],"content":"q_new \u0026 q_free q_new 使用 malloc 分配空间,并使用 INIT_LIST_HEAD 进行初始化。 q_free 遍历 queue 进行逐个节点释放,所以需要使用 _safe 后缀的 for_each 宏,释放时需要先释放成员 value,再释放节点 (回想一下 C++ 的析构函数),可以直接使用 q_release_element 函数。 q_free 在遍历时需要释放当前节点所在元素的空间,所以需要使用 list_for_each_entry_safe,而 q_size 无需在遍历时修改当前节点,所以使用 list_for_each 就足够了。 ","date":"2024-02-19","objectID":"/posts/linux2023-lab0/:2:2","tags":["Sysprog","Linux","C"],"title":"Linux 核心设计: C Programming Lab","uri":"/posts/linux2023-lab0/"},{"categories":["Linux Kernel Internals"],"content":"q_insert \u0026 q_remove insert 时需要特判 head 是否为 NULL 以及 malloc 分配是否成功,接下来需要使用 strdup 对所给参数进行复制 (strdup 内部是通过 malloc 来实现的,所以之前 q_free 时也需要是否 value),最后根据插入的位置调用 list_add 或 list_add_tail 进行插入。 remove 时需要特判 head 是否为 NULL 以及 queue 是否为空,接下来根据需要 remove 的节点调用 list_first_entry 或 list_last_entry 获取节点对应的元素,通过 list_del_init 来清除出 queue,最后如果 value 字段不为 NULL,则通过 memcpy 将 value 字段对应的字符串复制到指定位置。 ","date":"2024-02-19","objectID":"/posts/linux2023-lab0/:2:3","tags":["Sysprog","Linux","C"],"title":"Linux 核心设计: C Programming Lab","uri":"/posts/linux2023-lab0/"},{"categories":["Linux Kernel Internals"],"content":"Valgrind 2007 年的论文: Valgrind: A Framework for Heavyweight Dynamic Binary Instrumentation 繁体中文版本的 论文导读 memory lost: definitely lost indirectly lost possibly lost still readchable 运行 valgrind 和 gdb 类似,都需要使用 -g 参数来编译 C/C++ 源程序以生成调试信息,然后还可以通过 -q 参数指示 valgrind 进入 quite 模式,减少启动时信息的输出。 $ valgrind -q --leak-check=full ./case1 --leak-check=full: 启用全面的内存泄漏检查,valgrind 将会报告所有的内存泄漏情况,包括详细的堆栈跟踪信息 --show-possibly-lost=no: 不输出 possibly lost 相关报告 --track-fds=yes: 侦测 file descriptor 开了没关的情况 valgrind 输出的报告 invalid write/read 这类的单位是 Byte,即 size of X (bytes) 程序运行时内存布局: 信息 Valgrind User Manual Massif: a heap profiler ","date":"2024-02-19","objectID":"/posts/linux2023-lab0/:3:0","tags":["Sysprog","Linux","C"],"title":"Linux 核心设计: C Programming Lab","uri":"/posts/linux2023-lab0/"},{"categories":["Linux Kernel Internals"],"content":"自动测试 ","date":"2024-02-19","objectID":"/posts/linux2023-lab0/:4:0","tags":["Sysprog","Linux","C"],"title":"Linux 核心设计: C Programming Lab","uri":"/posts/linux2023-lab0/"},{"categories":["Linux Kernel Internals"],"content":"追踪内存的分配和释放 Wikipedia: Hooking Wikipedia: Test harness GCC: Arrays of Length Zero The alignment of a zero-length array is the same as the alignment of its elements. 相关源代码阅读 (harness.h, harness.c): test_malloc() test_free() test_calloc() find_footer() find_header() ","date":"2024-02-19","objectID":"/posts/linux2023-lab0/:4:1","tags":["Sysprog","Linux","C"],"title":"Linux 核心设计: C Programming Lab","uri":"/posts/linux2023-lab0/"},{"categories":["Linux Kernel Internals"],"content":"qtest 命令解释器 新增指令 hello,用于打印 Hello, world\" 的信息。调用流程: main → run_console → cmd_select → interpret_cmd → parse_args → interpret_cmda → do_hello 相关源代码阅读 (console.h, console.c): init_cmd() ADD_COMMADN add_cmd() ","date":"2024-02-19","objectID":"/posts/linux2023-lab0/:4:2","tags":["Sysprog","Linux","C"],"title":"Linux 核心设计: C Programming Lab","uri":"/posts/linux2023-lab0/"},{"categories":["Linux Kernel Internals"],"content":"Signal signal(2) — Linux manual page signal() sets the disposition of the signal signum to handler, which is either SIG_IGN, SIG_DFL, or the address of a programmer-defined func‐ tion (a “signal handler”). setjmp(3) — Linux manual page The functions described on this page are used for performing “nonlocal gotos”: transferring execution from one function to a predetermined location in another function. The setjmp() function dynamically establishes the target to which control will later be transferred, and longjmp() performs the transfer of execution. Why use sigsetjmp()/siglongjmp() instead of setjmp()/longjmp()? The Linux Programming Interface The sa_mask field allows us to specify a set of signals that aren’t permitted to interrupt execution of this handler. In addition, the signal that caused the handler to be invoked is automatically added to the process signal mask. This means that a signal handler won’t recursively interrupt itself if a second instance of the same signal arrives while the handler is executing. However, there is a problem with using the standard longjmp() function to exit from a signal handler. We noted earlier that, upon entry to the signal handler, the kernel automatically adds the invoking signal, as well as any signals specified in the act.sa_mask field, to the process signal mask, and then removes these signals from the mask when the handler does a normal return. What happens to the signal mask if we exit the signal handler using longjmp()? The answer depends on the genealogy of the particular UNIX implementation. jmp_ready 技巧 (用于保证在 siglongjmp() 之前必然执行过一次 sigsetjmp()): Because a signal can be generated at any time, it may actually occur before the target of the goto has been set up by sigsetjmp() (or setjmp()). To prevent this possibility (which would cause the handler to perform a nonlocal goto using an uninitialized env buffer), we employ a guard variable, canJump, to indicate whether the env buffer has been initialized. If canJump is false, then instead of doing a nonlocal goto, the handler simply returns. 相关源代码阅读: qtest.c q_init() sigsegv_handler() sigalrm_handler() harness.c trigger_exception() exception_setup() ","date":"2024-02-19","objectID":"/posts/linux2023-lab0/:4:3","tags":["Sysprog","Linux","C"],"title":"Linux 核心设计: C Programming Lab","uri":"/posts/linux2023-lab0/"},{"categories":["Rust","Network"],"content":" In this stream, we started implementing the ubiquitous TCP protocol that underlies much of the traffic on the internet! In particular, we followed RFC 793 — https://tools.ietf.org/html/rfc793 — which describes the original protocol, with the goal of being able to set up and tear down a connection with a “real” TCP stack at the other end (netcat in particular). We’re writing it using a user-space networking interface (see https://www.kernel.org/doc/Documentation/networking/tuntap.txt and the Rust bindings at https://docs.rs/tun-tap/). 整理自 John Gjengset 的影片: Part 1 ","date":"2024-02-17","objectID":"/posts/rust-tcp/:0:0","tags":["Rust","TCP","Network"],"title":"Impl Rust: TCP/IP","uri":"/posts/rust-tcp/"},{"categories":["Rust","Network"],"content":"影片注解 Part 1 ","date":"2024-02-17","objectID":"/posts/rust-tcp/:1:0","tags":["Rust","TCP","Network"],"title":"Impl Rust: TCP/IP","uri":"/posts/rust-tcp/"},{"categories":["Rust","Network"],"content":"Raw socket vs TUN/TAP device Raw sockets [Wikipedia] TUN/TAP [Wikipedia] Raw socket vs TUN device [Stack Overflow] Universal TUN/TAP device driver [Linux kernel documentation] Raw socket: Internet –\u003e NIC –\u003e kernel –\u003e user space Internet \u003c– NIC \u003c– kernel \u003c– user space Host interact with other hosts in Internet. TUN/TAP device: kernel –\u003e | TUN/TAP | –\u003e user space kernel \u003c– | TUN/TAP | \u003c– user space Kernel interact with programs in user space in the same host. 和其他物理网卡一样,用户进程创建的 TUN/TAP 设备仍然是被 kernel 所拥有的 (kernel 可以使用设备进行发送/接收),只不过用户进程也可以像操作 管道 (pipe) 那样,操作所创建的 TUN/TAP 设备 (可以使用该设备进行发送/接收),从而与 kernel 的物理网卡进行通信。 Universal TUN/TAP device driver [Linux kernel documentation] 3.2 Frame format: If flag IFF_NO_PI is not set each frame format is: Flags [2 bytes] Proto [2 bytes] Raw protocol(IP, IPv6, etc) frame. 通过 TUN/TAP 设备接收的封包,会拥有 Flags 和 Proto 这两个字段 (共 4 个字节,这也是 iface 的 without_packet_info 和 recv 方法所描述的 prepended packet info),然后才是原始协议的 frame。其中的 Proto 字段是 EtherType [Wikipedia],可以根据里面的 values 来判断接受封包的协议类型 (0x0800 表示 IPv4,0x86DD 表示 IPv6)。 ","date":"2024-02-17","objectID":"/posts/rust-tcp/:1:1","tags":["Rust","TCP","Network"],"title":"Impl Rust: TCP/IP","uri":"/posts/rust-tcp/"},{"categories":["Rust","Network"],"content":"setcap setcap [Linux manual page] cap_from_text [Linux manual page] 因为 TUN/TAP 是由 kernel 提供的,所以需要赋予我们项目的可执行文件权限,使它能访问我们创建的 TUN/TAP 设备 (为了简单起见,下面只列出 release 版本的方法,debug 版本的方法类似)。 # 编译 $ cargo build --release # 设置文件权限 $ sudo setcap cap_net_admin=eip target/release/trust # 运行 $ cargo run --release 在另一终端输入命令 ip addr 就可以看到此时会多出一个名为 tun0 的设备,这正是我们创建的 TUN 设备。 ip-address [Linux manual page] ip-link [Linux manual page] 在另一个终端中输入: # 列出当前所有的网络设备 $ ip addr # 配置设备 tun0 的 IP 地址 $ sudo ip addr add 192.168.0.1/24 dev tun0 # 启动设备 tun0 $ sudo ip link set up dev tun0 每次编译后都需要执行一遍这个流程 (因为重新编译生成的可执行文件需要重新设置权限),我们将这些流程的逻辑写成一个脚本 run.sh。这个脚本为了输出的美观性增加了额外逻辑,例如将 trust 放在后台执行,将脚本设置为等待 trust 执行完成后才结束执行。 ","date":"2024-02-17","objectID":"/posts/rust-tcp/:1:2","tags":["Rust","TCP","Network"],"title":"Impl Rust: TCP/IP","uri":"/posts/rust-tcp/"},{"categories":["Rust","Network"],"content":"Endianness Endianness [Wikipedia] Why is network-byte-order defined to be big-endian? [Stack Overflow] Rust 提供了 Trait std::simd::ToBytes 用于大小端字节序之间的相互转换,方法 from_be_bytes 是将大端字节序的一系列字节转换成对应表示的数值。 ","date":"2024-02-17","objectID":"/posts/rust-tcp/:1:3","tags":["Rust","TCP","Network"],"title":"Impl Rust: TCP/IP","uri":"/posts/rust-tcp/"},{"categories":["Rust","Network"],"content":"IP 因为 TUN 只是在 Network layer 的虚拟设备 (TAP 则是 Data link layer 层),所以需要手动解析 IP 封包。 RFC 791 3.1. Internet Header Format List of IP protocol numbers [Wikipedia] 可以按照上面的格式来解析封包头,也可以引入 Crate etherparse 来解析 IP 封包头。 ping 命令使用的是 Network layer 上的 ICMP 协议,可以用于测试 TUN 是否成功配置并能接收封包。 $ ping -I tun0 192.168.0.2 ping (networking utility) [Wikipedia] ping [Linux man page] nc 命令用于发送 TCP 封包 $ nc 192.168.0.2 80 nc [Linux man page] 注意 ping, nc 这些命令使用的都是 kernel 的协议栈来实现,所以在创建虚拟设备 tun0 之后,使用以上 ping, nc 命令表示 kernel 发送相应的 ICMP, TCP 封包给创建 tun0 的进程 (process)。 可以使用 tshark (Terminal Wireshark) 工具来抓包,配合 ping,nc 命令可以分析 tun0 的封包传送。 $ sudo apt install tshark $ sudo tshark -i tun0 Wireshark [Wikipedia] tshark [Manual Page] ","date":"2024-02-17","objectID":"/posts/rust-tcp/:1:4","tags":["Rust","TCP","Network"],"title":"Impl Rust: TCP/IP","uri":"/posts/rust-tcp/"},{"categories":["Rust","Network"],"content":"TCP [RFC 793] 3.2 Terminology The state diagram in figure 6 illustrates only state changes, together with the causing events and resulting actions, but addresses neither error conditions nor actions which are not connected with state changes. 这里面提到的 Figure 6. TCP Connection State Diagram 在其中我们可以看到 TCP 的状态转换,非常有利于直观理解 TCP 建立连接时的三次握手过程。 警告 NOTE BENE: this diagram is only a summary and must not be taken as the total specification. Time to live [Wikipedia] In the IPv4 header, TTL is the 9th octet of 20. In the IPv6 header, it is the 8th octet of 40. The maximum TTL value is 255, the maximum value of a single octet. A recommended initial value is 64. SND.WL1 and SND.WL2 Note that SND.WND is an offset from SND.UNA, that SND.WL1 records the sequence number of the last segment used to update SND.WND, and that SND.WL2 records the acknowledgment number of the last segment used to update SND.WND. The check here prevents using old segments to update the window. ","date":"2024-02-17","objectID":"/posts/rust-tcp/:1:5","tags":["Rust","TCP","Network"],"title":"Impl Rust: TCP/IP","uri":"/posts/rust-tcp/"},{"categories":["Rust","Network"],"content":"Documentations 这里列举视频中一些概念相关的 documentation 学习的一手资料是官方文档,请务必自主学会阅读规格书之类的资料 ","date":"2024-02-17","objectID":"/posts/rust-tcp/:2:0","tags":["Rust","TCP","Network"],"title":"Impl Rust: TCP/IP","uri":"/posts/rust-tcp/"},{"categories":["Rust","Network"],"content":"Crate std Module std::io Type Alias std::io::Result Module std::collections::hash_map method std::collections::hash_map::HashMap::entry method std::collections::hash_map::Entry::or_default Trait std::default::Default Module std::net Macro std::eprintln method std::result::Result::expect method u16::from_be_bytes ","date":"2024-02-17","objectID":"/posts/rust-tcp/:2:1","tags":["Rust","TCP","Network"],"title":"Impl Rust: TCP/IP","uri":"/posts/rust-tcp/"},{"categories":["Rust","Network"],"content":"Crate tun_tap Enum tun_tap::Mode ","date":"2024-02-17","objectID":"/posts/rust-tcp/:2:2","tags":["Rust","TCP","Network"],"title":"Impl Rust: TCP/IP","uri":"/posts/rust-tcp/"},{"categories":["Rust","Network"],"content":"Crate etherparse Struct etherparse::Ipv4HeaderSlice Struct etherparse::Ipv4Header Struct etherparse::TcpHeaderSlice Struct etherparse::TcpHeader ","date":"2024-02-17","objectID":"/posts/rust-tcp/:2:3","tags":["Rust","TCP","Network"],"title":"Impl Rust: TCP/IP","uri":"/posts/rust-tcp/"},{"categories":["Rust","Network"],"content":"References https://datatracker.ietf.org/doc/html/rfc793 https://datatracker.ietf.org/doc/html/rfc1122 https://datatracker.ietf.org/doc/html/rfc7414#section-2 https://datatracker.ietf.org/doc/html/rfc2398 https://datatracker.ietf.org/doc/html/rfc2525 https://datatracker.ietf.org/doc/html/rfc791 https://www.saminiir.com/lets-code-tcp-ip-stack-3-tcp-handshake/ https://www.saminiir.com/lets-code-tcp-ip-stack-4-tcp-data-flow-socket-api/ https://www.saminiir.com/lets-code-tcp-ip-stack-5-tcp-retransmission/ 注意 RFC 793 描述了原始的 TCP 协议的内容 (重点阅读 3.FUNCTIONAL SPECIFICATION ) RFC 1122 则是对原始的 TCP 功能的一些扩展进行说明 RFC 7414 的 Section 2 则对 TCP 的核心功能进行了简要描述 RFC 2398 描述了对实现的 TCP 的一些测试方法和工具 RFC 2525 说明了在实现 TCP 过程中可能会出现的错误,并指出可能导致错误的潜在问题 RFC 791 描述了 IP 协议 的内容 最后 3 篇博客介绍了 TCP 协议相关术语和概念,可以搭配 RFC 793 阅读 ","date":"2024-02-17","objectID":"/posts/rust-tcp/:3:0","tags":["Rust","TCP","Network"],"title":"Impl Rust: TCP/IP","uri":"/posts/rust-tcp/"},{"categories":["Linux Kernel Internals"],"content":"Source ","date":"2024-02-16","objectID":"/posts/linux-quiz1/:0:0","tags":["Sysprog","Linux","C"],"title":"Linux 核心设计: 第 1 周测验题 linked list","uri":"/posts/linux-quiz1/"},{"categories":["Linux Kernel Internals"],"content":"2018q1 第 4 週測驗題 ","date":"2024-02-16","objectID":"/posts/linux-quiz1/:1:0","tags":["Sysprog","Linux","C"],"title":"Linux 核心设计: 第 1 周测验题 linked list","uri":"/posts/linux-quiz1/"},{"categories":["Linux Kernel Internals"],"content":"测验 1 FuncA 的作用是 (e) 建立新節點,內容是 value,並安插在結尾 FuncB 的作用是 (d) 建立新節點,內容是 value,並安插在開頭 FuncC 的作用是 (e) 找到節點內容為 value2 的節點,並在之後插入新節點,內容為 value1 在 main 函数调用 display 函数之前,链表分布为: 48 -\u003e 51 -\u003e 63 -\u003e 72 -\u003e 86 在程式輸出中,訊息 Traversal in forward direction 後依序印出哪幾個數字呢? (d) 48 (c) 51 (a) 63 (e) 72 (b) 86 在程式輸出中,訊息 Traversal in reverse direction 後依序印出哪幾個數字呢? (b) 86 (e) 72 (a) 63 (c) 51 (d) 48 技巧 延伸題目: 在上述 doubly-linked list 實作氣泡排序和合併排序,並提出需要額外實作哪些函示才足以達成目標 引入統計模型,隨機新增和刪除節點,然後評估上述合併排序程式的時間複雜度和效能分佈 (需要製圖和數學分析) ","date":"2024-02-16","objectID":"/posts/linux-quiz1/:1:1","tags":["Sysprog","Linux","C"],"title":"Linux 核心设计: 第 1 周测验题 linked list","uri":"/posts/linux-quiz1/"},{"categories":["Linux Kernel Internals"],"content":"测验 2 FuncX 的作用是 (涵蓋程式執行行為的正確描述最多者) (f) 判斷是否為 circular linked list,若為 circular 則回傳 0,其他非零值,過程中計算走訪的節點總數 K1 » 後面接的輸出為何 (b) Yes K2 » 後面接的輸出為何 (a) No K3 » 後面接的輸出為何 (a) No K4 » 後面接的輸出為何 (a) No K5 » 後面接的輸出為何 (f) 0 count » 後面接的輸出為何 (f) 0 ","date":"2024-02-16","objectID":"/posts/linux-quiz1/:1:2","tags":["Sysprog","Linux","C"],"title":"Linux 核心设计: 第 1 周测验题 linked list","uri":"/posts/linux-quiz1/"},{"categories":["Linux Kernel Internals"],"content":"2020q1 第 1 週測驗題 ","date":"2024-02-16","objectID":"/posts/linux-quiz1/:2:0","tags":["Sysprog","Linux","C"],"title":"Linux 核心设计: 第 1 周测验题 linked list","uri":"/posts/linux-quiz1/"},{"categories":["Linux Kernel Internals"],"content":"测验 1 本题使用的是单向 linked list typedef struct __list { int data; struct __list *next; } list; 一开始的 if 语句用于判断 start 是否为 NULL 或是否只有一个节点,如果是则直接返回无需排序 接下来使用 mergesort 来对 linked list 进行从小到大排序,并且每次左侧链表只划分一个节点,剩余节点全部划为右侧链表 list *left = start; list *right = left-\u003enext; left-\u003enext = NULL; // LL0; 再来就是归并操作,将 left 和 right 进行归并,如果 merge 为 NULL,则将对应的节点赋值给它和 start,否则需要迭代 left 或 right 以及 merge 以完成归并操作 for (list *merge = NULL; left || right; ) { if (!right || (left \u0026\u0026 left-\u003edata \u003c right-\u003edata)) { if (!merge) { start = merge = left; // LL1; } else { merge-\u003enext = left; // LL2; merge = merge-\u003enext; } left = left-\u003enext; // LL3; } else { if (!merge) { start = merge = right; // LL4; } else { merge-\u003enext = right; // LL5; merge = merge-\u003enext; } right = right-\u003enext; // LL6; } } 技巧 延伸問題: 解釋上述程式運作原理; 指出程式改進空間,特別是考慮到 Optimizing merge sort; 將上述 singly-linked list 擴充為 circular doubly-linked list 並重新實作對應的 sort; 依循 Linux 核心 include/linux/list.h 程式碼的方式,改寫上述排序程式; 嘗試將原本遞迴的程式改寫為 iterative 版本; ","date":"2024-02-16","objectID":"/posts/linux-quiz1/:2:1","tags":["Sysprog","Linux","C"],"title":"Linux 核心设计: 第 1 周测验题 linked list","uri":"/posts/linux-quiz1/"},{"categories":["Linux","Linux Kernel Internals"],"content":" 面對原始程式碼超越 3 千萬行規模的 Linux 核心 (2023 年),最令人感到挫折的,絕非缺乏程式註解,而是就算見到滿滿的註解,自己卻有如文盲,全然無從理解起。為什麼呢?往往是因為對作業系統的認知太侷限。 原文地址 ","date":"2024-02-15","objectID":"/posts/linux-concepts/:0:0","tags":["Sysprog","Linux"],"title":"Linux 核心设计: 操作系统术语及概念","uri":"/posts/linux-concepts/"},{"categories":["Linux","Linux Kernel Internals"],"content":"Linux 核心发展 虚拟化 (Virtualization) 技术分为 CPU 层级的虚拟化技术,例如 KVM 和 RVM,也有操作系统层级的虚拟化技术,例如 Docker。 Plan 9 from Bell Labs [Wikipedia] LXC [Wikipedia] 信息 從 Revolution OS 看作業系統生態變化 Linux 核心設計: 透過 eBPF 觀察作業系統行為 ","date":"2024-02-15","objectID":"/posts/linux-concepts/:1:0","tags":["Sysprog","Linux"],"title":"Linux 核心设计: 操作系统术语及概念","uri":"/posts/linux-concepts/"},{"categories":["Linux","Linux Kernel Internals"],"content":"看漫画学 Linux 原文地址 inside the linux kernel 整理上图,可以得到 自底向上 的 Linux 系统结构: 地下层: 文件系统 (File System) 中央大厅层: 进程表 (process table) 内存管理 (memory management) 信息安全 (security) 看门狗 (watchdog) httpd cron 管道 (pipe) FTP SSH Wine GNOME 最上层 tty / terminal wiki: Pipeline (Unix) [Wikipedia] Process identifier [Wikipedia] watchdog [Linux man page] init [Wikipedia] systemd [Wikipedia] fork [Linux man page] clone [Linux man page] Project Genie [Wikipedia] posix_spawn [Linux man page] Native POSIX Thread Library [Wikipedia] 极客漫画: 不要使用 SIGKILL 的原因 wait [Linux man page] signal [Linux man page] TUX web server [Wikipedia] -[x] cron 技巧 Multics 采用了当时背景下的几乎所有的先进技术,可以参考该系统获取系统领域的灵感。 虚拟内存管理与现代银行的运行逻辑类似,通过 malloc 分配的有效虚拟地址并不能保证真正可用,类似于支票得去银行兑现时才知道银行真正的现金储备。但是根据统计学公式,虚拟地址和银行现金可以保证在大部分情况下,都可以满足需求,当然突发的大规模虚拟内存使用、现金兑现时就无法保证了。这部分的原理推导需要学习概率论、统计学等数理课程。 信息 Linux 核心设计: Linux 核心設計: 檔案系統概念及實作手法 Linux 核心設計: 不僅是個執行單元的 Process Linux 核心設計: 不只挑選任務的排程器 UNIX 作業系統 fork/exec 系統呼叫的前世今生 Linux 核心設計: 記憶體管理 Linux 核心設計: 發展動態回顧 Linux 核心設計: 針對事件驅動的 I/O 模型演化 Linux 核心設計: Scalability 議題 Effective System Call Aggregation (ESCA) 你所不知道的 C 語言: Stream I/O, EOF 和例外處理 Unix-like 工具使用技巧: Mastering UNIX pipes, Part 1 Mastering UNIX pipes, Part 2 ","date":"2024-02-15","objectID":"/posts/linux-concepts/:2:0","tags":["Sysprog","Linux"],"title":"Linux 核心设计: 操作系统术语及概念","uri":"/posts/linux-concepts/"},{"categories":["Linux","Linux Kernel Internals"],"content":"高阶观点 投影片: Linux Kernel: Introduction ✅ 对投影片的 重点描述 一些概念理解: 1963 Timesharing: A Solution to Computer Bottlenecks [YouTube] Supervisory program [Wikipedia] ","date":"2024-02-15","objectID":"/posts/linux-concepts/:3:0","tags":["Sysprog","Linux"],"title":"Linux 核心设计: 操作系统术语及概念","uri":"/posts/linux-concepts/"},{"categories":["Linux","Linux Kernel Internals"],"content":"Monolithic kernel vs Microkernel 淺談 Microkernel 設計和真實世界中的應用 Hybrid kernel [wikipedia] “As to the whole ‘hybrid kernel’ thing - it’s just marketing. It’s ‘oh, those microkernels had good PR, how can we try to get good PR for our working kernel? Oh, I know, let’s use a cool name and try to imply that it has all the PR advantages that that other system has’.” —— Linus Torvalds ","date":"2024-02-15","objectID":"/posts/linux-concepts/:3:1","tags":["Sysprog","Linux"],"title":"Linux 核心设计: 操作系统术语及概念","uri":"/posts/linux-concepts/"},{"categories":["Linux","Linux Kernel Internals"],"content":"虚拟化 MicroVM 和 Unikernel 都是使用 CPU 层级的虚拟化技术,在 Host OS 上面构建的 GuestOS: MicroVM 会减少硬件驱动方面的初始化,从而加快启动和服务速度 (在云服务器方面很常见,服务器端并不需要进行硬件驱动)。 Unikernel 则更激进,将 programs 和 kernel 一起进行动态编译,并且限制只能运行一个 process (例如只运行一个数据库进程,这样云服务器很常见),这样就减少了一些系统调用的呼叫,例如 fork (因为只能运行一个 process),提升了安全性 (因为 fork 系统调用可能会造成一些漏洞)。Unikernel 又叫 Library OS,可以理解为分时多人多工操作系统的另一个对立面,拥有极高的运行速度 (因为只有一个 process)。 Container Sandbox 使用的是 OS 层级的虚拟化技术,即它是将一组进程隔离起来构建为容器,这样可能会导致这一组进程就耗尽了系统的资源,其他进程无法使用系统的资源。同时因为是进程级的隔离,所以安全性不及 CPU 层级的 MicroVM 和 Unikernel。 信息 相关演讲、录影: YouTube: Inside the Mac OS X Kernel YouTube: What Are MicroVMs? And Why Should I Care? YouTube: From the Ground Up: How We Built the Nanos Unikernel 相关论文阅读: ","date":"2024-02-15","objectID":"/posts/linux-concepts/:3:2","tags":["Sysprog","Linux"],"title":"Linux 核心设计: 操作系统术语及概念","uri":"/posts/linux-concepts/"},{"categories":["Linux","Linux Kernel Internals"],"content":"Scalability Wikipedia: scalability A system whose performance improves after adding hardware, proportionally to the capacity added, is said to be a scalable system. lock-free sequence lock RCU algorithm complexity ","date":"2024-02-15","objectID":"/posts/linux-concepts/:3:3","tags":["Sysprog","Linux"],"title":"Linux 核心设计: 操作系统术语及概念","uri":"/posts/linux-concepts/"},{"categories":["Linux","Linux Kernel Internals"],"content":"eBPF 透过 eBPF 可将 Monolithic kernel 的 Linux 取得 microkernel 的特性 The Beginners Guide to eBPF Programming, Liza RIce (live programming + source code) A thorough introduction to eBPF (four articles in lwn.net), Matt FLeming, December 2017 ","date":"2024-02-15","objectID":"/posts/linux-concepts/:3:4","tags":["Sysprog","Linux"],"title":"Linux 核心设计: 操作系统术语及概念","uri":"/posts/linux-concepts/"},{"categories":["Linux","Linux Kernel Internals"],"content":"细节切入点 CPU 和 OS 的基本概念科普网站: Putting the “You” in CPU 相当于科普版 CSAPP 信息 UNSW COMP9242: Advanced Operating Systems (2023/T3) YouTube: 2022: UNSW’s COMP9242 Advanced Operating Systems 这门课可以作为辅助材料,讲得深入浅出,可以作为进阶材料阅读。 ","date":"2024-02-15","objectID":"/posts/linux-concepts/:4:0","tags":["Sysprog","Linux"],"title":"Linux 核心设计: 操作系统术语及概念","uri":"/posts/linux-concepts/"},{"categories":["Linux","Linux Kernel Internals"],"content":"系统软件开发思维 ","date":"2024-02-15","objectID":"/posts/linux-concepts/:5:0","tags":["Sysprog","Linux"],"title":"Linux 核心设计: 操作系统术语及概念","uri":"/posts/linux-concepts/"},{"categories":["Linux","Linux Kernel Internals"],"content":"Maslow’s pyramid of code review Maslow’s pyramid of code review ","date":"2024-02-15","objectID":"/posts/linux-concepts/:5:1","tags":["Sysprog","Linux"],"title":"Linux 核心设计: 操作系统术语及概念","uri":"/posts/linux-concepts/"},{"categories":["Linux","Linux Kernel Internals"],"content":"Benchmark / Profiling Benchmark / Profiling ","date":"2024-02-15","objectID":"/posts/linux-concepts/:5:2","tags":["Sysprog","Linux"],"title":"Linux 核心设计: 操作系统术语及概念","uri":"/posts/linux-concepts/"},{"categories":["Rust"],"content":" In this third Crust of Rust video, we cover iterators and trait bounds, by re-implementing the “flatten” Iterator method from the standard library. As part of that, we cover some of the weirder trait bounds that are required, including what’s needed to extend the implementation to support backwards iteration. 整理自 John Gjengset 的影片 ","date":"2024-02-05","objectID":"/posts/iterators/:0:0","tags":["Rust","Iterator"],"title":"Crust of Rust: Iterators","uri":"/posts/iterators/"},{"categories":["Rust"],"content":"影片注解 ","date":"2024-02-05","objectID":"/posts/iterators/:1:0","tags":["Rust","Iterator"],"title":"Crust of Rust: Iterators","uri":"/posts/iterators/"},{"categories":["Rust"],"content":"Generic traits vs associated types trait Iterator { type Item; fn next(\u0026mut self) -\u003e Option\u003cSelf::Item\u003e; } trait Iterator\u003cItem\u003e { fn next(\u0026mut self) -\u003e Option\u003cSelf::Item\u003e; } 为什么使用上面的 associated type 而不是下面的 generic 来实现 Iterator?因为使用 generic 来实现的话,可以对一个类型实现多个 Iterator trait 例如 Iterator\u003ci32\u003e, Iterator\u003cf64,而从语言表达上讲,我们希望一个类型只能实现一个 Iterator trait,所以使用 associated type 来实现 Iterator trait,防止二义性。 for v in vs.iter() { // borrow vs, \u0026 to v } for v in \u0026vs { // equivalent to vs.iter() } 这两条 for 语句虽然效果一样,但是后者是使用 \u003c\u0026vs\u003e into_iter 讲 \u0026vs 转为 iterator,而不是调用 iter() 方法。 ","date":"2024-02-05","objectID":"/posts/iterators/:1:1","tags":["Rust","Iterator"],"title":"Crust of Rust: Iterators","uri":"/posts/iterators/"},{"categories":["Rust"],"content":"Iterator::flatten method std::iter::Iterator::flatten Creates an iterator that flattens nested structure. This is useful when you have an iterator of iterators or an iterator of things that can be turned into iterators and you want to remove one level of indirection. flatten() 的本质是将一种 Iterator 类型转换成另一种 Iterator 类型,所以调用者和返回值 Flatten 都满足 trait Iterator,因为都是迭代器,只是将原先的 n-level 压扁为 1-level 的 Iterator 了。录影视频里只考虑 2-level 的情况。 ","date":"2024-02-05","objectID":"/posts/iterators/:1:2","tags":["Rust","Iterator"],"title":"Crust of Rust: Iterators","uri":"/posts/iterators/"},{"categories":["Rust"],"content":"DoubleEndedIterator Trait std::iter::DoubleEndedIterator It is important to note that both back and forth work on the same range, and do not cross: iteration is over when they meet in the middle. 也就是说,back 和 front 的迭代器类似于双指针,但是这两个迭代器并不会越过对方。 ","date":"2024-02-05","objectID":"/posts/iterators/:1:3","tags":["Rust","Iterator"],"title":"Crust of Rust: Iterators","uri":"/posts/iterators/"},{"categories":["Rust"],"content":"Homework 信息 实作说明: 尝试实现 Iterator 的 flat_map 方法 (Github: My Implementation) 参考资料: method std::iter::Iterator::flat_map struct std::iter::FlatMap ","date":"2024-02-05","objectID":"/posts/iterators/:2:0","tags":["Rust","Iterator"],"title":"Crust of Rust: Iterators","uri":"/posts/iterators/"},{"categories":["Rust"],"content":"Documentations 这里列举视频中一些概念相关的 documentation 学习的一手资料是官方文档,请务必自主学会阅读规格书之类的资料 ","date":"2024-02-05","objectID":"/posts/iterators/:3:0","tags":["Rust","Iterator"],"title":"Crust of Rust: Iterators","uri":"/posts/iterators/"},{"categories":["Rust"],"content":"Crate std 可以使用这里提供的搜素栏进行搜索 (BTW 不要浪费时间在 Google 搜寻上!) Trait std::iter::Iterator method std::iter::Iterator::flatten method std::iter::Iterator::rev method std::iter::Iterator::flat_map Trait std::iter::IntoIterator Struct std::iter::Flatten function std::iter::empty Struct std::iter::Empty function std::iter::once Struct std::iter::Once Trait std::iter::DoubleEndedIterator Enum std::option::Option method std::option::Option::and_then method std::option::Option::as_mut Trait std::marker::Sized ","date":"2024-02-05","objectID":"/posts/iterators/:3:1","tags":["Rust","Iterator"],"title":"Crust of Rust: Iterators","uri":"/posts/iterators/"},{"categories":["Rust"],"content":"References What is the difference between iter and into_iter? [Stack Overflow] How to run a specific unit test in Rust? [Stack Overflow] How do I implement a trait with a generic method? [Stack Overflow] 可能不是你看过最无聊的 Rust 入门喜剧 102 (1) 闭包与迭代器 [bilibili] ","date":"2024-02-05","objectID":"/posts/iterators/:4:0","tags":["Rust","Iterator"],"title":"Crust of Rust: Iterators","uri":"/posts/iterators/"},{"categories":["C","Linux Kernel Internals"],"content":" 无论是操作系统核心、C 语言函数库内部、程序开发框架,到应用程序,都不难见到 linked list 的身影,包含多种针对性能和安全议题所做的 linked list 变形,又还要考虑应用程序的泛用性 (generic programming),是很好的进阶题材。 原文地址 ","date":"2024-02-03","objectID":"/posts/c-linked-list/:0:0","tags":["Sysprog","C","Linked List"],"title":"你所不知道的 C 语言: linked list 和非连续记忆体","uri":"/posts/c-linked-list/"},{"categories":["C","Linux Kernel Internals"],"content":"Linux 核心的艺术 YouTube: The mind behind Linux | Linus Torvalds | TED 事实上 special case 和 indirect pointer 这两种写法在 clang 的最佳优化下效能并没有什么区别,我们可以不使用 indirect pointer 来写程序,但是我们需要学习 indirect pointer 这种思维方式,即 good taste。 把握程序的本质,即本质上是修改指针的值,所以可以使用指针的指针来实现,无需进行特判。 在 Unix-like 的操作系统中,类型名带有后缀 _t 表示这个类型是由 typedef 定义的,而不是语言原生的类型名,e.g. typedef struct list_entry { int value; struct list_entry *next; } list_entry_t; ","date":"2024-02-03","objectID":"/posts/c-linked-list/:1:0","tags":["Sysprog","C","Linked List"],"title":"你所不知道的 C 语言: linked list 和非连续记忆体","uri":"/posts/c-linked-list/"},{"categories":["C","Linux Kernel Internals"],"content":"linked list append \u0026 remove Source 信息 The mind behind Linux Linus on Understanding Pointers ","date":"2024-02-03","objectID":"/posts/c-linked-list/:1:1","tags":["Sysprog","C","Linked List"],"title":"你所不知道的 C 语言: linked list 和非连续记忆体","uri":"/posts/c-linked-list/"},{"categories":["C","Linux Kernel Internals"],"content":"LeetCode Source LeetCode 21. Merge Two Sorted Lists LeetCode 23. Merge k Sorted Lists Leetcode 2095. Delete the Middle Node of a Linked List LeetCode 86. Partition List 注意 原文对于 LeetCode 23. Merge k Sorted Lists 给出了 3 种解法,其时间复杂度分别为: $O(m \\cdot n)$ $O(m \\cdot n)$ $O(m \\cdot logn)$ $n$ 为 listsSize,$m$ 为 merge linked list 过程中产生的 linked list 的最大长度。 如果你对第 3 种解法的时间复杂度感到疑惑,请参考 Josh Hug 在 CS61B 的 Merge Sort 复杂度讲解。 ","date":"2024-02-03","objectID":"/posts/c-linked-list/:1:2","tags":["Sysprog","C","Linked List"],"title":"你所不知道的 C 语言: linked list 和非连续记忆体","uri":"/posts/c-linked-list/"},{"categories":["C","Linux Kernel Internals"],"content":"Circular linked list 单向 linked list 相对于双向 linked list 的优势在于,一个 cache line 可以容纳更多的 list node,而且很容易进行反向查询,这弥补了反向查询时的效能差距。例如在 64 位处理器上,地址为 64 Bit 即 8 Byte,如果 list node 的数据域存放一个 2 Byte 的整数,那么一个单向的 list node 大小为 10 Byte,双向的则为 18 Byte,又因为一般的 cache line 的大小为 64 Byte,则对于单向的 node 来说,cache line 可以存放 $64 / 10 = 6$ 个 list node,但是仅能存放 $64 / 18 = 3$ 个 list node,cache 效率明显降低。 这部分内容可以参考 jserv 的讲座 \u003c現代處理器設計: Cache 原理和實際影響\u003e ","date":"2024-02-03","objectID":"/posts/c-linked-list/:2:0","tags":["Sysprog","C","Linked List"],"title":"你所不知道的 C 语言: linked list 和非连续记忆体","uri":"/posts/c-linked-list/"},{"categories":["C","Linux Kernel Internals"],"content":"Floyd’s Cycle detection 这个“龟兔赛跑”算法保证兔子在跑两次循环圈后,一定会和刚完成一次循环圈的乌龟相遇。因为已知乌龟每次移动一步,兔子每次移动两步,可以假设在相遇点处乌龟移动的 $X$ 步,则兔子移动了 $2X$ 步,$2X$ 必为偶数,所以兔子必能在移动了 $2X$ 步后与乌龟相遇,不会出现兔子因为每次移动两步而刚好越过乌龟一步的情况。 $\\lambda$ is the length of the loop to be found, $\\mu$ is the index of the first element of the cycle. Source LeetCode 141. Linked List Cycle LeetCode 142. Linked List Cycle II LeetCode 146. LRU Cache 金刀的算法小册子 Linked List 专题 LeetCode 206. Reverse Linked List 信息 探索 Floyd Cycle Detection Algorithm ","date":"2024-02-03","objectID":"/posts/c-linked-list/:2:1","tags":["Sysprog","C","Linked List"],"title":"你所不知道的 C 语言: linked list 和非连续记忆体","uri":"/posts/c-linked-list/"},{"categories":["C","Linux Kernel Internals"],"content":"Merge Sort 实现了 recursion, non-recursion 的 merge sort Source 技巧 Merge Sort 与它的变化 不论是这里的 non-recursion 版本的 merge sort,还是后面的 non-recursion 版本的 quick sort,本质上都是通过模拟栈 (stack) 操作来实现的,关于这个模拟 stack 方法,可以参考蒋炎岩老师的录影 应用视角的操作系统 (程序的状态机模型;编译优化)。 ","date":"2024-02-03","objectID":"/posts/c-linked-list/:3:0","tags":["Sysprog","C","Linked List"],"title":"你所不知道的 C 语言: linked list 和非连续记忆体","uri":"/posts/c-linked-list/"},{"categories":["C","Linux Kernel Internals"],"content":"Linux 核心的 linked list Linux 核心使用的 linked list 是通过 Intrusive linked lists 搭配 contain_of 宏,来实现自定义的 linked list node。 sysprog21/linux-list 这个仓库将 Linux kernel 中 linked list 部分抽离出来,并改写为 user mode 的实作。本人对该仓库进行了一些改写,对 insert sort 和 quick sort 增加了 makefile 支持。 上面的仓库与 Linux kernel 的实作差异主要在于 WRITE_ONCE 宏。WRITE_ONCE 的原理简单来说是,通过 union 产生两个引用同一地址的引用 (即 __val 和 __c),然后因为对同一地址有多个引用,所以编译器进行最佳化时不会过于激进的重排序,从而达到顺序执行效果。 Source ","date":"2024-02-03","objectID":"/posts/c-linked-list/:4:0","tags":["Sysprog","C","Linked List"],"title":"你所不知道的 C 语言: linked list 和非连续记忆体","uri":"/posts/c-linked-list/"},{"categories":["C","Linux Kernel Internals"],"content":"Intrusive linked lists Intrusive linked lists 这篇文章对于 Intrusive linked list 说明的非常好,解释了其在 memory allocations 和 cache thrashing 的优势,还搭配 Linux kernel 讲解了场景应用。 ","date":"2024-02-03","objectID":"/posts/c-linked-list/:4:1","tags":["Sysprog","C","Linked List"],"title":"你所不知道的 C 语言: linked list 和非连续记忆体","uri":"/posts/c-linked-list/"},{"categories":["C","Linux Kernel Internals"],"content":"container_of Linux 核心原始程式碼巨集: container_of ","date":"2024-02-03","objectID":"/posts/c-linked-list/:4:2","tags":["Sysprog","C","Linked List"],"title":"你所不知道的 C 语言: linked list 和非连续记忆体","uri":"/posts/c-linked-list/"},{"categories":["C","Linux Kernel Internals"],"content":"Optimized QuickSort Optimized QuickSort: C Implementation (Non-Recursive) 这篇文章介绍了 non-recursion 的 quick sort 在 array 上的实作,参考该文章完成 linked list 上的 non-recursion 版本的 quick sort 实作。 非递归的快速排序中 if (L != R \u0026\u0026 \u0026begin[i]-\u003elist != head) { 其中的 \u0026begin[i]-\u003elist != head 条件判断用于空链表情况,数组版本中使用的是下标比较 L \u003c R 来判断,但是链表中使用 L != R 不足以完全表示 L \u003c R 这个条件,还需要 \u0026begin[i]-\u003elist != head 来判断链表是否为空。 ","date":"2024-02-03","objectID":"/posts/c-linked-list/:4:3","tags":["Sysprog","C","Linked List"],"title":"你所不知道的 C 语言: linked list 和非连续记忆体","uri":"/posts/c-linked-list/"},{"categories":["C","Linux Kernel Internals"],"content":"Linux 核心的 list_sort 实作 linux/list_sort.c 先将双向循环链表转换成单向链表,然后利用链表节点的 prev 来挂载 pending list (因为单向链表中 prev 没有作用,但是链表节点仍然存在 prev 字段,所以进行充分利用)。 假设 count 对应的 bits 第 k 个 bit 值为 0 且 $\u003e k$ 的 bits 都为 0,$\u003c k$ 的 bits 都为 1,则 $\u003c k $ 的这些 1 可以表示 pending list 中分别有 $2^{k-1}, 2^{k-2}, …, 2^0$ 大小的 list 各一个。 如果第 k 个 bit 值为 0 且 $\u003e k$ 的 bits 中存在值为 1 的 bit,$\u003c k$ 的 bits 均为 1,则只有 $\u003c k$ 的 bits 可以表示 pending list 中分别有 $2^{k-1}, 2^{k-2}, …, 2^0$ 大小的 list 各一个,\u003e k 的 1 表示需要进行 merge 以获得对应大小的 list。 这样也刚好能使得 merge 时是 $2: 1$ 的长度比例,因为 2 的指数之间的比例是 $2: 1$。 技巧 这部分内容在 Lab0: Linux 核心的链表排序 中有更详细的解释和讨论。 信息 List, HList, and Hash Table hash table What is the strict aliasing rule? [Stack Overflow] Unions and type-punning [Stack Overflow] Nine ways to break your systems code using volatile [Stack Overflow] WRITE_ONCE in linux kernel lists [Stack Overflow] lib/list_sort: Optimize number of calls to comparison function ","date":"2024-02-03","objectID":"/posts/c-linked-list/:4:4","tags":["Sysprog","C","Linked List"],"title":"你所不知道的 C 语言: linked list 和非连续记忆体","uri":"/posts/c-linked-list/"},{"categories":["C","Linux Kernel Internals"],"content":"Fisher–Yates shuffle Wikipedia Fisher–Yates shuffle The Fisher–Yates shuffle is an algorithm for shuffling a finite sequence. 原文所说的事件复杂度,是考虑关于构造结果链表时的复杂度,并不考虑寻找指定节点的复杂度,所以对于原始方法复杂度为 $1 + 2 + … + n = O(n^2)$,对于 modern method 复杂度为 $1 + 1 + … + 1 = O(n)$ 原文实作虽然使用了 pointer to pointer,但是使用上并没有体现 linus 所说的 good taste,重新实作如下: void shuffle(node_t **head) { srand(time(NULL)); // First, we have to know how long is the linked list int len = 0; node_t **indirect = head; while (*indirect) { len++; indirect = \u0026(*indirect)-\u003enext; } // Append shuffling result to another linked list node_t *new = NULL; node_t **new_tail = \u0026new; while (len) { int random = rand() % len; indirect = head; while (random--) indirect = \u0026(*indirect)-\u003enext; node_t *tmp = *indirect; *indirect = (*indirect)-\u003enext; tmp-\u003enext = NULL; *new_tail = tmp; new_tail = \u0026(*new_tail)-\u003enext; len--; } *head = new; } 主要是修改了新链表 new 那一部分,只需要一个 pointer to pinter new_tail 就可以避免条件判断。 ","date":"2024-02-03","objectID":"/posts/c-linked-list/:5:0","tags":["Sysprog","C","Linked List"],"title":"你所不知道的 C 语言: linked list 和非连续记忆体","uri":"/posts/c-linked-list/"},{"categories":["Rust"],"content":" In this second Crust of Rust video, we cover declarative macros, macro_rules!, by re-implementing the vec! macro from the standard library. As part of that, we cover not only how to write these, but some of the gotchas and tricks you’ll run into, and some common use-cases. 整理自 John Gjengset 的影片 ","date":"2024-01-31","objectID":"/posts/declarative-macros/:0:0","tags":["Rust","Macro","Declarative Macros"],"title":"Crust of Rust: Declarative Macros","uri":"/posts/declarative-macros/"},{"categories":["Rust"],"content":"影片注解 ","date":"2024-01-31","objectID":"/posts/declarative-macros/:1:0","tags":["Rust","Macro","Declarative Macros"],"title":"Crust of Rust: Declarative Macros","uri":"/posts/declarative-macros/"},{"categories":["Rust"],"content":"regex macro 可以使用以下 3 种分隔符来传入参数 (注意花括号 {} 的需要与 macro 名之间进行空格,末尾不需要分号,这是因为 {} 会被编译器视为一个 statement,无需使用 ; 来进行分隔): macro_rules! avec { () =\u003e {}; ... } avec!(); avec![]; avec! {} macro 定义内的 () 和 {} 也都可以使用 (), [], {} 之间的任意一种,并不影响调研 macro 的分隔符的使用(都是 3 任选 1 即可),不过推荐在 macro 定义内使用 () 和 {} 搭配。 如果需要在 macro 传入的 synatx 中使用正则表达式 (regex),则需要在外面使用 $() 进行包装: ($($elem:expr),* $(,)?) =\u003e {{ let mut v = Vec::new(); $(v.push($elem);)* v }}; 同样的,可以在 macro 体内使用 regex 对参数进行解包装,语法是相同的: $(...)[delimiter](+|*|?) 其中分隔符 (delimiter) 是可选的。它会根据内部所包含的参数 $(...) (本例中是 $(elem)) 来进行自动解包装,生成对应次数的 statement,如果有分隔符 (delimiter) 也会生成对应的符号。 ","date":"2024-01-31","objectID":"/posts/declarative-macros/:1:1","tags":["Rust","Macro","Declarative Macros"],"title":"Crust of Rust: Declarative Macros","uri":"/posts/declarative-macros/"},{"categories":["Rust"],"content":"cargo expand cargo-expand 可以将宏展开,对于宏的除错非常方便,可以以下命令来安装: $ cargo install cargo-expand 然后可以通过以下命令对 macro 进行展开: $ cargo expand 使用以下命令可以将 unit tests 与 cargo expand 结合起来,即展开的是 unit tests 之后的完整代码: $ cargo expand --lib tests ","date":"2024-01-31","objectID":"/posts/declarative-macros/:1:2","tags":["Rust","Macro","Declarative Macros"],"title":"Crust of Rust: Declarative Macros","uri":"/posts/declarative-macros/"},{"categories":["Rust"],"content":"scope 由于 Rust 中 macro 和 normal code 的作用域不一致,所以像 C 语言那种在 macro 中定义变量或在 macro 中直接修改已有变量是不可行的,操作这种 lvalue 的情况需要使用 macro 参数进行传入,否则无法通过编译。 // cannot compile macro_rules! avec { () =\u003e { let x = 1; } } // cannot compile macro_rules! avec { () =\u003e { x = 42; } } // can compile macro_rules! avec { ($x: ident) =\u003e { $x += 1; } } ","date":"2024-01-31","objectID":"/posts/declarative-macros/:1:3","tags":["Rust","Macro","Declarative Macros"],"title":"Crust of Rust: Declarative Macros","uri":"/posts/declarative-macros/"},{"categories":["Rust"],"content":"statements 在 Rust macro 中,如果需要将传入的 syntax 转换成多个 statements,需要使用 {} 进行包装: () =\u003e {{ ... }} 其中第一对 {} 是 macro 语法所要求的的,第二对 {} 则是用于包装 statements 的 {},使用 cargo expand 进行查看会更直观。 ","date":"2024-01-31","objectID":"/posts/declarative-macros/:1:4","tags":["Rust","Macro","Declarative Macros"],"title":"Crust of Rust: Declarative Macros","uri":"/posts/declarative-macros/"},{"categories":["Rust"],"content":"delimiter 注意 macro 中传入的 syntax,其使用的类似于 =\u003e 的分隔符是有限的,例如不能使用 -\u003e 作为分隔符,具体可以查阅手册。 ($arg1:ty =\u003e $arg2:ident) =\u003e { type $arg2 = $arg1; }; 技巧 当 declarative macros 变得复杂时,它的可读性会变得很差,这时候需要使用 procedural macros。但是 procedural macros 需要多花费一些编译周期 (compilition cycle),因为需要先对 procedural macros 进行编译,再编译 lib/bin 对应的源文件。 ","date":"2024-01-31","objectID":"/posts/declarative-macros/:1:5","tags":["Rust","Macro","Declarative Macros"],"title":"Crust of Rust: Declarative Macros","uri":"/posts/declarative-macros/"},{"categories":["Rust"],"content":"calculating 编写 macro 时传入的参数如果是 expression,需要先对其进行计算,然后使用 clone 方法来对该计算结果进行拷贝,这样能最大限度的避免打破 Rust 所有权制度的限制。 ($elem:expr; $count:expr) =\u003e {{ let mut v = Vec::new(); let x = $elem; for _ in 0..$count { v.push(x.clone()); } v }}; 这样传入 y.take().unwrap() 作为宏的 elem 参数就不会产生 panic。 技巧 对于会导致 compile fail 的 unit test,无法使用通常的 unit test 来测试,但是有一个技巧:可以使用 Doc-tests 的方式来构建(需要标记 compile_fail,如果不标记则默认该测试需要 compile success) /// ```compile_fail /// let v: Vec\u003cu32\u003e = vecmac::avec![42; \"foo\"]; /// ``` #[allow(dead_code)] struct CompileFailTest; ","date":"2024-01-31","objectID":"/posts/declarative-macros/:1:6","tags":["Rust","Macro","Declarative Macros"],"title":"Crust of Rust: Declarative Macros","uri":"/posts/declarative-macros/"},{"categories":["Rust"],"content":"trait Rust 中的 macro 无法限制传入参数的 Trait,例如不能限制参数必须实现 Clone 这个 Trait。 ::std::iter 带有前置双冒号 :: 的语法,是在没有显式引入 use std::iter 模块的情况下访问该模块的方式。在这种情况下,::std::iter 表示全局命名空间 (global namespace) 中的 std::iter 模块,即标准库中的 iter 模块。由于 macro 需要进行 export 建议编写 macro 时尽量使用 :: 这类语法。 技巧 计算 vector 的元素个数时使用 () 引用 [()] 进行计数是一个常见技巧,因为 () 是 zero size 的,所以并不会占用栈空间。其他的元素计数方法可以参考 The Little Book of Rust Macros 的 2.5.2 Counting 一节。 ","date":"2024-01-31","objectID":"/posts/declarative-macros/:1:7","tags":["Rust","Macro","Declarative Macros"],"title":"Crust of Rust: Declarative Macros","uri":"/posts/declarative-macros/"},{"categories":["Rust"],"content":"Homework 信息 实作说明: 尝试使用 declarative macro 来实现 HashMap 的初始化语法 (Github: My Implementation) 尝试阅读 vec macro 在 std 库的实现 Macro std::vec 参考资料: Struct std::collections::HashMap ","date":"2024-01-31","objectID":"/posts/declarative-macros/:2:0","tags":["Rust","Macro","Declarative Macros"],"title":"Crust of Rust: Declarative Macros","uri":"/posts/declarative-macros/"},{"categories":["Rust"],"content":"Documentations 这里列举视频中一些概念相关的 documentation 学习的一手资料是官方文档,请务必自主学会阅读规格书之类的资料 ","date":"2024-01-31","objectID":"/posts/declarative-macros/:3:0","tags":["Rust","Macro","Declarative Macros"],"title":"Crust of Rust: Declarative Macros","uri":"/posts/declarative-macros/"},{"categories":["Rust"],"content":"Crate std 可以使用这里提供的搜素栏进行搜索 (BTW 不要浪费时间在 Google 搜寻上!) Macro std::vec Struct std::vec::Vec Method std::vec::Vec::with_capacity method std::vec::Vec::extend method std::vec::Vec::resize Module std::iter Function std::iter::repeat method std::iter::Iterator::take method std::option::Option::take ","date":"2024-01-31","objectID":"/posts/declarative-macros/:3:1","tags":["Rust","Macro","Declarative Macros"],"title":"Crust of Rust: Declarative Macros","uri":"/posts/declarative-macros/"},{"categories":["Rust"],"content":"References 原版的 The Little Book of Rust Macros 在 Rust 更新新版本后没有持续更新,另一位大牛对这本小册子进行了相应的更新: The Little Book of Rust Macros Rust语言中文社区也翻译了该小册子: Rust 宏小册 ","date":"2024-01-31","objectID":"/posts/declarative-macros/:4:0","tags":["Rust","Macro","Declarative Macros"],"title":"Crust of Rust: Declarative Macros","uri":"/posts/declarative-macros/"},{"categories":["Rust"],"content":" We’re going to investigate a case where you need multiple explicit lifetime annotations. We explore why they are needed, and why we need more than one in this particular case. We also talk about some of the differences between the string types and introduce generics over a self-defined trait in the process. 整理自 John Gjengset 的影片 ","date":"2024-01-25","objectID":"/posts/lifetime-annotations/:0:0","tags":["Rust","Lifetime"],"title":"Crust of Rust: Lifetime Annotations","uri":"/posts/lifetime-annotations/"},{"categories":["Rust"],"content":"C 语言中的 lifetime Rust 中的 lifetime 一向是一个难点,为了更好地理解这一难点的本质,建议阅读 C 语言规格书关于 lifetime 的部分,相信你会对 Rust 的 lifetime 有不同的看法。 C11 [6.2.4] Storage durations of objects An object has a storage duration that determines its lifetime. There are four storage durations: static, thread, automatic, and allocated. ","date":"2024-01-25","objectID":"/posts/lifetime-annotations/:1:0","tags":["Rust","Lifetime"],"title":"Crust of Rust: Lifetime Annotations","uri":"/posts/lifetime-annotations/"},{"categories":["Rust"],"content":"影片注解 ","date":"2024-01-25","objectID":"/posts/lifetime-annotations/:2:0","tags":["Rust","Lifetime"],"title":"Crust of Rust: Lifetime Annotations","uri":"/posts/lifetime-annotations/"},{"categories":["Rust"],"content":"cargo check cargo check 可以给出更简洁的提示,例如相对于编译器给出的错误信息,它会整合相同的错误信息,从而提供简洁切要的提示信息。而且它是一个静态分析工具,不需要进行编译即可给出提示,所以速度会比编译快很多,在大型项目上尤为明显。 ","date":"2024-01-25","objectID":"/posts/lifetime-annotations/:2:1","tags":["Rust","Lifetime"],"title":"Crust of Rust: Lifetime Annotations","uri":"/posts/lifetime-annotations/"},{"categories":["Rust"],"content":"ref 影片大概 49 分时提到了 if let Some(ref mut remainder) = self.remainder {...} ref 的作用配合 if let 语句体的逻辑可以体会到 pointer of pointer 的美妙之处。 因为在 pattern match 中形如 \u0026mut 这类也是用于 pattern match 的,不能用于获取 reference,这也是为什么需要使用 ref mut 这类语法来获取 reference 的原因。 ","date":"2024-01-25","objectID":"/posts/lifetime-annotations/:2:2","tags":["Rust","Lifetime"],"title":"Crust of Rust: Lifetime Annotations","uri":"/posts/lifetime-annotations/"},{"categories":["Rust"],"content":"operator ? 影片大概 56 分时提到了 let remainder = self.remainder.as_mut()?; 为什么使用之前所提的 let remainder = \u0026mut self.remainder?; 这是因为使用 ? 运算符返回的是内部值的 copy,所以这种情况 remainder 里是 self.remainder? 返回的值 (是原有 self.remainder 内部值的 copy) 的 reference ","date":"2024-01-25","objectID":"/posts/lifetime-annotations/:2:3","tags":["Rust","Lifetime"],"title":"Crust of Rust: Lifetime Annotations","uri":"/posts/lifetime-annotations/"},{"categories":["Rust"],"content":"\u0026str vs String 影片大概 1:03 时提到了 str 与 String 的区别,个人觉得讲的很好: str -\u003e [char] \u0026str -\u003e \u0026[char] // fat pointer (address and size) String -\u003e Vec\u003cchar\u003e String -\u003e \u0026str (cheap -- AsRef) \u0026str -\u003e String (expensive -- memcpy) 对于 String 使用 \u0026* 可以保证将其转换成 \u0026str,因为 * 会先将 String 转换成 str。当然对于函数参数的 \u0026str,只需传入 \u0026String 即可自动转换类型。 ","date":"2024-01-25","objectID":"/posts/lifetime-annotations/:2:4","tags":["Rust","Lifetime"],"title":"Crust of Rust: Lifetime Annotations","uri":"/posts/lifetime-annotations/"},{"categories":["Rust"],"content":"lifetime 可以将结构体的 lifetime 的第一个 (一般为 'a) 视为实例的 lifetime,其它的可以表示与实例 lifetime 无关的 lifetime。由于 compiler 不够智能,所以它会将实例化时传入参数的 lifetime 中相关联的最小 lifetime 视为实例的 lifetime 约束 (即实例的 lifetime 包含于该 lifetime 内)。 当在实现结构体的方法或 Trait 时,如果在实现方法时无需使用 lifetime 的名称,则可以使用匿名 lifetime '_,或者在编译器可以推推导出 lifetime 时也可以使用匿名 lifetime '_。 only lifetime struct Apple\u003c'a\u003e { owner: \u0026'a Human, } impl Apple\u003c'_\u003e { ... } lifetime and generic struct Apple\u003c'a, T\u003e { owner: \u0026'a T, } impl\u003cT\u003e Apple\u003c'_, T\u003e { ... } compiler can know lifetime pun fn func(\u0026self) -\u003e Apple\u003c'_, T\u003e { ... } ","date":"2024-01-25","objectID":"/posts/lifetime-annotations/:2:5","tags":["Rust","Lifetime"],"title":"Crust of Rust: Lifetime Annotations","uri":"/posts/lifetime-annotations/"},{"categories":["Rust"],"content":"Documentations 这里列举视频中一些概念相关的 documentation 学习的一手资料是官方文档,请务必自主学会阅读规格书之类的资料 ","date":"2024-01-25","objectID":"/posts/lifetime-annotations/:3:0","tags":["Rust","Lifetime"],"title":"Crust of Rust: Lifetime Annotations","uri":"/posts/lifetime-annotations/"},{"categories":["Rust"],"content":"Crate std 可以使用这里提供的搜素栏进行搜索 (BTW 不要浪费时间在 Google 搜寻上!) Keywords Keyword SelfTy Keyword ref Trait std::iter::Iterator method std::iter::Iterator::eq method std::iter::Iterator::collect method std::iter::Iterator::position method std::iter::Iterator::find Enum std::option::Option method std::option::Option::take method std::option::Option::as_mut method std::option::Option::expect Primitive Type str method str::find method str::char_indices Trait std::ops::Try Macro std::try method char::len_utf8 ","date":"2024-01-25","objectID":"/posts/lifetime-annotations/:3:1","tags":["Rust","Lifetime"],"title":"Crust of Rust: Lifetime Annotations","uri":"/posts/lifetime-annotations/"},{"categories":["Toolkit"],"content":"记录一下折腾 Deepin 20.9 的物理机的过程与相关的配置。 ","date":"2024-01-24","objectID":"/posts/deepin20.9/:0:0","tags":["Linux","Deepin"],"title":"深度操作系统 Deepin 20.9 安装配置","uri":"/posts/deepin20.9/"},{"categories":["Toolkit"],"content":"安装与配置 新手教学影片: 深度操作系统deepin下载安装 (附双系统安装及分区指引) [bilibili] 安装完deepin之后该做的事情 [bilibili] ","date":"2024-01-24","objectID":"/posts/deepin20.9/:1:0","tags":["Linux","Deepin"],"title":"深度操作系统 Deepin 20.9 安装配置","uri":"/posts/deepin20.9/"},{"categories":["Toolkit"],"content":"网络代理 新手教学文档: Ubuntu 22.04LTS 相关配置 在境内可以使用 gitclone 镜像站来加快 clone 的速度。 ","date":"2024-01-24","objectID":"/posts/deepin20.9/:2:0","tags":["Linux","Deepin"],"title":"深度操作系统 Deepin 20.9 安装配置","uri":"/posts/deepin20.9/"},{"categories":["Toolkit"],"content":"编辑器: VS Code 新手教学文档: 编辑器: Visual Studio Code [HackMD] 本人的一些注解: GNU/Linux 开发工具 这里列举一下本人配置的插件: Even Better TOML CodeLLDB 用于调试 Rust Git History Native Debug 用于调试 C/C++ rust-analyzer Tokyo Night 挺好看的一个主题 Vim VSCode Great Icons 文件图标主题 问题 rust5-analyzer 插件可能会因为新版本要求 glibc 2.29 而导致启动失败,请参考这个 issue 来解决。 ","date":"2024-01-24","objectID":"/posts/deepin20.9/:3:0","tags":["Linux","Deepin"],"title":"深度操作系统 Deepin 20.9 安装配置","uri":"/posts/deepin20.9/"},{"categories":["Toolkit"],"content":"终端和 Vim 新手教学文档: 終端機和 Vim 設定 [HackMD] 本人的一些注解: GNU/Linux 开发工具 本人的终端提示符配置: \\u@\\h\\W 本人使用 Minimalist Vim Plugin Manager 来管理 Vim 插件,配置如下: \" Specify a directory for plugins (for Neovim: ~/.local/share/nvim/plugged) call plug#begin('~/.vim/plugged') Plug 'Shougo/neocomplcache' Plug 'scrooloose/nerdtree' map \u003cF5\u003e :NERDTreeToggle\u003cCR\u003e call plug#end() let g:neocomplcache_enable_at_startup = 1 let g:neocomplcache_enable_smart_case = 1 inoremap \u003cexpr\u003e\u003cTAB\u003e pumvisible()?\"\\\u003cC-n\u003e\" : \"\\\u003cTAB\u003e\" syntax on set number set cursorline colorscheme default set bg=dark set tabstop=4 set expandtab set shiftwidth=4 set ai set hlsearch set smartindent map \u003cF4\u003e : set nu!\u003cBAR\u003eset nonu?\u003cCR\u003e \" autocomplete dropdown list colorscheme hi Pmenu ctermfg=0 ctermbg=7 hi PmenuSel ctermfg=7 ctermbg=4 ","date":"2024-01-24","objectID":"/posts/deepin20.9/:4:0","tags":["Linux","Deepin"],"title":"深度操作系统 Deepin 20.9 安装配置","uri":"/posts/deepin20.9/"},{"categories":["Toolkit"],"content":"系统语言: Rust 安装教程: Installation [The book] 安装 Rust [Rust course] Channels [The rustup book] # install rust $ curl --proto '=https' --tlsv1.2 https://sh.rustup.rs -sSf | sh # install nightly toolchain $ rustup toolchain install nightly # change to nightly toolchain $ rustup default nightly # list installed toolchain $ rustup toolchain list # update installed toolchain $ rustup update 个人偏向于使用 nightly toolchain ","date":"2024-01-24","objectID":"/posts/deepin20.9/:5:0","tags":["Linux","Deepin"],"title":"深度操作系统 Deepin 20.9 安装配置","uri":"/posts/deepin20.9/"},{"categories":["Toolkit"],"content":"tldr The tldr-pages project is a collection of community-maintained help pages for command-line tools, that aims to be a simpler, more approachable complement to traditional man pages. 安装 tldr: $ sudo apt install tldr ","date":"2024-01-24","objectID":"/posts/deepin20.9/:6:0","tags":["Linux","Deepin"],"title":"深度操作系统 Deepin 20.9 安装配置","uri":"/posts/deepin20.9/"},{"categories":["Toolkit"],"content":"效果展示 Deepin Terminial Vim Deepin DDE Desktop ","date":"2024-01-24","objectID":"/posts/deepin20.9/:7:0","tags":["Linux","Deepin"],"title":"深度操作系统 Deepin 20.9 安装配置","uri":"/posts/deepin20.9/"},{"categories":["Toolkit"],"content":"FAQ 问题 重启后可能会出现,输入密码无法进入图形界面重新返回登录界面,这一循环状况。这个是 deepin 的默认 shell 是 dash 造成的,只需将默认的 shell 改为 bash 即可解决问题: $ ls -l /bin/sh lrwxrwxrwx 1 root root 9 xx月 xx xx:xx /bin/sh -\u003e /bin/dash $ sudo rm /bin/sh $ sudo ln -s /bin/bash /bin/sh 如果你已经处于无限登录界面循环这一状况,可以通过 Ctrl + Alt + \u003cF2\u003e 进入 tty2 界面进行修改: # 先查看问题日志,判断是不是 shell 导致的问题 $ cat .xsession-errors # 如果是,则重复上面的操作即可 ","date":"2024-01-24","objectID":"/posts/deepin20.9/:8:0","tags":["Linux","Deepin"],"title":"深度操作系统 Deepin 20.9 安装配置","uri":"/posts/deepin20.9/"},{"categories":["Toolkit"],"content":"在 deepin 20.9 上根据 DragonOS 构建文档 的 bootstrap.sh 的方式来构建 DragonOS 时,如果没有事先安装 Qemu 会出现 KVM 相关的依赖问题。本文记录解决这一问题的过程。 如果事先没有安装 Qemu,在使用 bootstrap.sh 时会出现如下报错: $ bash bootstrap.sh ... 下列软件包有未满足的依赖关系: qemu-kvm : 依赖: qemu-system-x86 E: 无法修正错误,因为您要求某些软件包保持现状,就是它们破坏了软件包间的依赖关系。 查询 deepin 论坛上的相关内容:qemu-kvm无法安装,可以得知是因为 qemu-kvm 在 debian 发行版上只是一个虚包,所以对于 x86 架构的机器可以直接安装 qemu-systerm-x86 Debian qemu-kvm https://packages.debian.org/search?keywords=qemu-kvm 安装 qemu-systerm-x86: $ sudo apt install qemu-systerm-x86 $ $ qemu-system-x86_64 --version QEMU emulator version 5.2.0 (Debian 1:5.2+dfsg-11+deb11u1) Copyright (c) 2003-2020 Fabrice Bellard and the QEMU Project developers 安装的 qemu 版本看起来有点低,但是先使用 bootstrap.sh 快速安装其它依赖项,然后尝试编译运行一下 DragonOS: $ bash bootstrap.sh ... |-----------Congratulations!---------------| | | | 你成功安装了DragonOS所需的依赖项! | | | | 请关闭当前终端, 并重新打开一个终端 | | 然后通过以下命令运行: | | | | make run | | | |------------------------------------------| 新开一个终端或刷新一下 ~/.bashrc: $ cd DragonOS $ make run 运行 DragonOS Ok 可以成功运行 注意 如果需要使用 RISC-V 的 Qemu 模拟器,安装 qemu-system-misc 即可: $ sudo apt install qemu-system-misc ","date":"2024-01-22","objectID":"/posts/deepin-dragonos/:0:0","tags":["Deepin","Linux","DragonOS"],"title":"Deepin 20.9 构建 DragonOS","uri":"/posts/deepin-dragonos/"},{"categories":["Toolkit","Linux Kernel Internals"],"content":"大型开源项目的规模十分庞大,例如使用 Rust 编写的 Servo 浏览器,这个项目有近十万行代码。在开发规模如此庞大的项目时,了解如何通过正确的方式进行调试非常重要,因为这样可以帮助开发者快速地找到瓶颈。 原文地址 | 教学录影 ","date":"2024-01-16","objectID":"/posts/debug-gdb/:0:0","tags":["Debug","GDB","Rust","C/C++","Sysprog"],"title":"程序调试工具 GDB","uri":"/posts/debug-gdb/"},{"categories":["Toolkit","Linux Kernel Internals"],"content":"GDB 调试 观看教学视频 拯救資工系學生的基本素養—使用 GDB 除錯基本教學 和搭配博文 ==[How to debug Rust/C/C++ via GDB][debug-gdb]==,学习 GDB 的基本操作和熟悉使用 GDB 调试 Rust/C/C++ 程序。 掌握 run/r, break/b, print/p, continue/c, step/s info/i, delete/d, backtrace/bt, frame/f, up/down, exit/q 等命令的用法。以及 GBD 的一些特性,例如 GDB 会将空白行的断点自动下移到下一代码行;使用 break 命令时可以输入源文件路径,也可以只输入源文件名称。 相关的测试文件: test.c hello_cargo/ ","date":"2024-01-16","objectID":"/posts/debug-gdb/:1:0","tags":["Debug","GDB","Rust","C/C++","Sysprog"],"title":"程序调试工具 GDB","uri":"/posts/debug-gdb/"},{"categories":["Toolkit","Linux Kernel Internals"],"content":"GDB 基本介绍 引用 “GDB, the GNU Project debugger, allows you to see what is going on ‘inside’ another program while it executes — or what another program was doing at the moment it crashed.” — from gnu.org 安装 GDB: $ sudo apt install gdb 启动 GDB 时可以加入 -q 参数 (quite),表示减少或不输出一些提示或信息。 LLDB 与 GDB 的命令类似,本文也可用于 LLDB 的入门学习。 ","date":"2024-01-16","objectID":"/posts/debug-gdb/:2:0","tags":["Debug","GDB","Rust","C/C++","Sysprog"],"title":"程序调试工具 GDB","uri":"/posts/debug-gdb/"},{"categories":["Toolkit","Linux Kernel Internals"],"content":"GDB 调试 C/C++ 要使用 GDB 来调试 C/C++,需要在编译时加上 -g 参数(必需),也可以使用 -Og 参数来对 debug 进行优化(但使用 -Og 后 compiler 可能会把一些东西移除掉,所以 debug 时可能不会符合预期),例如: $ gcc test.c -Og -g -o test $ gdb -q ./test Source ","date":"2024-01-16","objectID":"/posts/debug-gdb/:3:0","tags":["Debug","GDB","Rust","C/C++","Sysprog"],"title":"程序调试工具 GDB","uri":"/posts/debug-gdb/"},{"categories":["Toolkit","Linux Kernel Internals"],"content":"GDB 调试 Rust 在使用 build 命令构建 debug 目标文件(即位于 target/debug 目录下的目标文件,与 package 同名)后,就可以通过 gdb 来进行调试: $ cargo build $ gdb -q ./target/debug/\u003cpackage name\u003e 但是如果是使用 cargo build --release 构建的 release 目标文件(即位于 target/release 目录下的目标文件),则无法使用 GDB 进行调试,因为 release 目标未包含任何调试信息,类似于未使用 -g 参数编译 C/C++ 源代码。 Source ","date":"2024-01-16","objectID":"/posts/debug-gdb/:4:0","tags":["Debug","GDB","Rust","C/C++","Sysprog"],"title":"程序调试工具 GDB","uri":"/posts/debug-gdb/"},{"categories":["Toolkit","Linux Kernel Internals"],"content":"GDB 基本命令 ","date":"2024-01-16","objectID":"/posts/debug-gdb/:5:0","tags":["Debug","GDB","Rust","C/C++","Sysprog"],"title":"程序调试工具 GDB","uri":"/posts/debug-gdb/"},{"categories":["Toolkit","Linux Kernel Internals"],"content":"run run (r) 命令用于从程序的执行起始点开始执行,直到遇到下一个断点或者程序结束。 ","date":"2024-01-16","objectID":"/posts/debug-gdb/:5:1","tags":["Debug","GDB","Rust","C/C++","Sysprog"],"title":"程序调试工具 GDB","uri":"/posts/debug-gdb/"},{"categories":["Toolkit","Linux Kernel Internals"],"content":"continue continue (c) 命令用于从当前停止的断点位置处继续执行程序,直到遇到下一个断点或者程序结束。 注意 run 和 continue 的区别在于 run 是将程序从头开始执行。例如如果未设置任何断点,使用 run 可以反复执行程序,而如果使用 continue 则会提示 The program is not being run。 ","date":"2024-01-16","objectID":"/posts/debug-gdb/:5:2","tags":["Debug","GDB","Rust","C/C++","Sysprog"],"title":"程序调试工具 GDB","uri":"/posts/debug-gdb/"},{"categories":["Toolkit","Linux Kernel Internals"],"content":"step step (s) 命令用于 逐行 执行程序,在遇到函数调用时进入对应函数,并在函数内部的第一行暂停。step 命令以 单步方式 执行程序的每一行代码,并跟踪函数调用的进入和退出。 (gdb) step 6 bar += 3; (gdb) step 7 printf(\"bar = %d\\n\", bar); 注意 step 命令与 continue 命令相同,只能在程序处于运行态(即停留在断点处)时才能使用。 ","date":"2024-01-16","objectID":"/posts/debug-gdb/:5:3","tags":["Debug","GDB","Rust","C/C++","Sysprog"],"title":"程序调试工具 GDB","uri":"/posts/debug-gdb/"},{"categories":["Toolkit","Linux Kernel Internals"],"content":"next next (n) 命令用于执行当前行并移动到 下一行,它用于逐行执行程序,但不会进入函数调用。 ","date":"2024-01-16","objectID":"/posts/debug-gdb/:5:4","tags":["Debug","GDB","Rust","C/C++","Sysprog"],"title":"程序调试工具 GDB","uri":"/posts/debug-gdb/"},{"categories":["Toolkit","Linux Kernel Internals"],"content":"break break (b) 命令用于在可执行问卷对应的源程序中加入断点,可以在程序处于 未运行态/运行态 时加入断点(运行态是指程序停留在断点处但未执行完毕的姿态)。 可以通过指定 源文件对应的 行数/函数名 来加入断点(源文件名可以省略): (gdb) break test.c:7 (gdb) break test.c:foo 如果可执行文件由多个源文件编译链接得到,可以通过指定 源文件名字 的方式来加入断点,无需源文件路径,但如果不同路径有重名源文件,则需要指定路径来区分: (gdb) break test1.c:7 (gdb) break test2.c:main ","date":"2024-01-16","objectID":"/posts/debug-gdb/:5:5","tags":["Debug","GDB","Rust","C/C++","Sysprog"],"title":"程序调试工具 GDB","uri":"/posts/debug-gdb/"},{"categories":["Toolkit","Linux Kernel Internals"],"content":"print print (p) 命令用于在调试过程中打印 变量的值或 表达式 的结果,帮助开发者检查程序状态并查看特定变量的当前值。 # Assume x: 3, y: 4 (gdb) print x $1 = 3 (gdb) print x + y $2 = 7 使用 p 命令打印变量值时,会在左侧显示一个 $\u003cnumber\u003e,这个可以理解成临时变量,后续也可以通过这个标志来复用这些值。例如在上面的例子中: (gdb) print $1 $3 = 3 (gdb) print $1 + $3 $4 = 4 Use p/format to instead select other formats such as x for hex, t for binary, and c for char. ","date":"2024-01-16","objectID":"/posts/debug-gdb/:5:6","tags":["Debug","GDB","Rust","C/C++","Sysprog"],"title":"程序调试工具 GDB","uri":"/posts/debug-gdb/"},{"categories":["Toolkit","Linux Kernel Internals"],"content":"backtrace backtrace (bt) 命令用于打印当前调用栈的信息,也称为堆栈回溯 (backtrace)。它显示了程序在执行过程中经过的函数调用序列,以及每个函数调用的位置和参数,即可以获取以下信息: 函数调用序列:显示程序当前的函数调用序列,以及每个函数的名称和所在的源代码文件。 栈帧信息:对于每个函数调用,显示该函数的栈帧信息,包括栈帧的地址和栈帧的大小。 (gdb) backtrace (gdb) backtrace #0 foo () at test.c:7 #1 0x00005555555551d2 in main () at test.c:14 技巧 backtrace 命令对于跟踪程序的执行路径、检查函数调用的顺序以及定位错误非常有用。在实际中,一般会搭配其他GDB命令(如 up、down 和 frame)结合使用,以查看特定栈帧的更多详细信息或切换到不同的栈帧。在上面的例子中,#0 和 #1 表示栈帧的编号,可以通过 frame 配合这些编号来切换栈帧。 ","date":"2024-01-16","objectID":"/posts/debug-gdb/:5:7","tags":["Debug","GDB","Rust","C/C++","Sysprog"],"title":"程序调试工具 GDB","uri":"/posts/debug-gdb/"},{"categories":["Toolkit","Linux Kernel Internals"],"content":"where where 和 backtrace 命令都用于显示程序的调用栈信息。backtrace 提供更详细的调用栈信息,包括函数名称、文件名、行号、参数和局部变量的值。而 where 命令可以理解为 backtrace 的一个简化版本,它提供的是较为紧凑的调用栈信息,通常只包含函数名称、文件名和行号。 ","date":"2024-01-16","objectID":"/posts/debug-gdb/:5:8","tags":["Debug","GDB","Rust","C/C++","Sysprog"],"title":"程序调试工具 GDB","uri":"/posts/debug-gdb/"},{"categories":["Toolkit","Linux Kernel Internals"],"content":"frame frame (f) 命令用于选择特定的栈帧 (stack frame),从而切换到不同的函数调用上下文,每个栈帧对应于程序中的一个函数调用。 接着上一个例子,切换到 main 函数所在的栈帧: (gdb) frame 1 #1 0x00005555555551d2 in main () at test.c:14 14 int result = foo(); ","date":"2024-01-16","objectID":"/posts/debug-gdb/:5:9","tags":["Debug","GDB","Rust","C/C++","Sysprog"],"title":"程序调试工具 GDB","uri":"/posts/debug-gdb/"},{"categories":["Toolkit","Linux Kernel Internals"],"content":"up/down up 和 down 命令用于在调试过程中在不同的栈帧之间进行切换: up 用于在调用栈中向上移动到较高的栈帧,即进入调用当前函数的函数。每次执行 up 命令,GDB 将切换到上一个(更高层次)的栈帧。这可以用于查看调用当前函数的上层函数的执行上下文。 down 用于在调用栈中向下移动到较低的栈帧,即返回到当前函数调用的函数。每次执行 down 命令,GDB 将切换到下一个(较低层次)的栈帧。这可以用于返回到调用当前函数的函数的执行上下文。 这两个命令需要开发者对应函数调用堆栈的布局有一定程度的了解。 接着上一个例子: (gdb) up #1 0x00005555555551d2 in main () at test.c:14 14 int result = foo(); (gdb) down #0 foo () at test.c:7 7 printf(\"bar = %d\\n\", bar); ","date":"2024-01-16","objectID":"/posts/debug-gdb/:5:10","tags":["Debug","GDB","Rust","C/C++","Sysprog"],"title":"程序调试工具 GDB","uri":"/posts/debug-gdb/"},{"categories":["Toolkit","Linux Kernel Internals"],"content":"info info (i) 命令用于获取程序状态和调试环境的相关信息,该命令后面可以跟随不同的子命令,用于获取特定类型的信息。 一些常用的 info 子命令: info breakpoints 显示已设置的所有断点 (breakpoint) 信息,包括断点编号、断点类型、断点位置等。 info watchpoints 显示已设置的所有监视点 (watchpoint) 信息,包括监视点编号、监视点类型、监视的表达式等。 info locals 显示当前函数的局部变量的值和名称。 info args 显示当前函数的参数的值和名称。 info registers 显示当前 CPU 寄存器的值。 info threads 显示当前正在调试的所有线程 (thread) 信息,包括线程编号、线程状态等。 info frame 显示当前栈帧 (stack frame) 的信息,包括函数名称、参数、局部变量等。 info program 显示被调试程序的相关信息,例如程序入口地址、程序的加载地址等。 (gdb) info breakpoints # or simply: i b Num Type Disp Enb Address What 1 breakpoint keep y 0x000055555555518f in foo at test.c:7 2 breakpoint keep y 0x0000555555555175 in foo at test.c:4 ","date":"2024-01-16","objectID":"/posts/debug-gdb/:5:11","tags":["Debug","GDB","Rust","C/C++","Sysprog"],"title":"程序调试工具 GDB","uri":"/posts/debug-gdb/"},{"categories":["Toolkit","Linux Kernel Internals"],"content":"delete delete (d) 命令用于删除断点 (breakpoint) 或观察点 (watchpoint)。断点是在程序执行期间暂停执行的特定位置,而观察点是在特定条件满足时暂停执行的位置。 可以通过指定 断点 / 观察点 的编号或使用 delete 命令相关的参数,来删除已设置的断点 / 观察点。断点 / 观察点编号可以在使用 info breakpoints / info watchpoints 命令时获得。 ","date":"2024-01-16","objectID":"/posts/debug-gdb/:5:12","tags":["Debug","GDB","Rust","C/C++","Sysprog"],"title":"程序调试工具 GDB","uri":"/posts/debug-gdb/"},{"categories":["Toolkit","Linux Kernel Internals"],"content":"quit quit (q) 命令用于退出 GDB,返回终端页面。 (gdb) quit $ # Now, in the terminial ","date":"2024-01-16","objectID":"/posts/debug-gdb/:5:13","tags":["Debug","GDB","Rust","C/C++","Sysprog"],"title":"程序调试工具 GDB","uri":"/posts/debug-gdb/"},{"categories":["Toolkit","Linux Kernel Internals"],"content":"list list 命令用于显示当前位置的代码片段,默认情况下,它会显示当前位置的前后10行代码。 list 命令也可以显示指定范围的代码,使用 list \u003cstart\u003e,\u003cend\u003e 命令将显示从 start 行到 end 行的源代码。 ","date":"2024-01-16","objectID":"/posts/debug-gdb/:5:14","tags":["Debug","GDB","Rust","C/C++","Sysprog"],"title":"程序调试工具 GDB","uri":"/posts/debug-gdb/"},{"categories":["Toolkit","Linux Kernel Internals"],"content":"whatis whatis 命令用于获取给定标识符(如变量、函数或类型)的类型信息。 // in source code int calendar[12][31]; // in gdb (gdb) whatis calendar type = int [12][31] ","date":"2024-01-16","objectID":"/posts/debug-gdb/:5:15","tags":["Debug","GDB","Rust","C/C++","Sysprog"],"title":"程序调试工具 GDB","uri":"/posts/debug-gdb/"},{"categories":["Toolkit","Linux Kernel Internals"],"content":"x x 命令用于查看内存中的数据,使用 x 命令搭配不同的格式来显示内存中的数据,也可以搭配 / 后跟数字来指定要显示的内存单元数量。例如,x/4 \u003caddress\u003e 表示显示地址 address 开始的连续 4 个内存单元的内容。 ","date":"2024-01-16","objectID":"/posts/debug-gdb/:5:16","tags":["Debug","GDB","Rust","C/C++","Sysprog"],"title":"程序调试工具 GDB","uri":"/posts/debug-gdb/"},{"categories":["Toolkit","Linux Kernel Internals"],"content":"其他 如果被调试程序正处于运行态(即已经通过 run 命令来运行程序),此时可以通过 Ctrl+C 来中断 GDB,程序将被立即中断,并在中断时所运行到的地方暂停。这种方式被称为 手动断点,手动断点可以理解为一个临时断点,只会在该处暂停一次。 GDB 会将空白行的断点自动下移到下一非空的代码行。 set print pretty 命令可以以更易读和格式化的方式显示结构化数据,以更友好的方式输出结构体、类、数组等复杂类型的数据,更易于阅读和理解。 ","date":"2024-01-16","objectID":"/posts/debug-gdb/:5:17","tags":["Debug","GDB","Rust","C/C++","Sysprog"],"title":"程序调试工具 GDB","uri":"/posts/debug-gdb/"},{"categories":["Toolkit","Linux Kernel Internals"],"content":"References video: Linux basic anti-debug video: C Programming, Disassembly, Debugging, Linux, GDB rr (Record and Replay Framework) video: Quick demo video: Record and replay debugging with “rr” ","date":"2024-01-16","objectID":"/posts/debug-gdb/:6:0","tags":["Debug","GDB","Rust","C/C++","Sysprog"],"title":"程序调试工具 GDB","uri":"/posts/debug-gdb/"},{"categories":["C","Linux Kernel Internals"],"content":" 「指针」 扮演 「记忆体」 和 「物件」 之间的桥梁 原文地址 ","date":"2024-01-14","objectID":"/posts/c-pointer/:0:0","tags":["Sysprog","C","Pointer"],"title":"你所不知道的 C 语言: 指针篇","uri":"/posts/c-pointer/"},{"categories":["C","Linux Kernel Internals"],"content":"前言杂谈 Let’s learn programming by inventing it [CppCon 2018] ✅ 在 K\u0026R 一书中,直到 93 页才开始谈论 pointer,而全书总计 185 页,所以大概是在全书 $50.27\\%$ 的位置才开始讲 pointer。所以即使不学 pointer,你还是能够掌握 $~50\\%$ 的 C 语言的内容,但是 C 语言的核心正是 pointer,所以 Good Luck 🤣 godbolt 可以直接在网页上看到,源代码由各类 compiler 生成的 Assembly Code How to read this prototype? [Stack Overflow] ✅ Note 这个问题是关于 signal 系统调用的函数原型解读,里面的回答页给出了很多对于指针,特别是 函数指针 的说明,下面节选一些特别有意思的回答: 引用 The whole thing declares a function called signal: signal takes an int and a function pointer this function pointer takes an int and returns void signal returns a function pointer this function pointer takes an intand returns avoid` That’s where the last int comes in. You can use the spiral rule to make sense of such declarations, or the program cdecl(1). The whole thing declares a function called signal: 这里面提到了 the spiral rule 这是一个用于解析 C 语言中声明 (declaration) 的方法;另外还提到了 cdecl 这一程序,它也有类似的作用,可以使用英文进行声明或者解释。 引用 Find the leftmost identifier and work your way out, remembering that [] and () bind before *; IOW, *a[] is an array of pointers, (*a)[] is a pointer to an array, *f() is a function returning a pointer, and (*f)() is a pointer to a function. Thus, void ( *signal(int sig, void (*handler)(int)) ) (int); breaks down as signal -- signal signal( ) -- is a function signal( sig ) -- with a parameter named sig signal(int sig, ) -- of type int signal(int sig, handler ) -- and a parameter named handler signal(int sig, *handler ) -- which is a pointer signal(int sig, (*handler)( )) ) -- to a function signal(int sig, (*handler)(int)) ) -- taking an int parameter signal(int sig, void (*handler)(int)) ) -- and returning void *signal(int sig, void (*handler)(int)) ) -- returning a pointer ( *signal(int sig, void (*handler)(int)) )( ) -- to a function ( *signal(int sig, void (*handler)(int)) )(int) -- taking an int parameter void ( *signal(int sig, void (*handler)(int)) )(int); -- and returning void 这一回答强调了 * 和 []、() 优先级的关系,这在判断数组指针、函数指针时是个非常好用的技巧。 Rob Pike 于 2009/10/30 的 Golang Talk [PDF] David Brailsford 教授解说影片 Essentials: Pointer Power! - Computerphile [YouTube] ","date":"2024-01-14","objectID":"/posts/c-pointer/:1:0","tags":["Sysprog","C","Pointer"],"title":"你所不知道的 C 语言: 指针篇","uri":"/posts/c-pointer/"},{"categories":["C","Linux Kernel Internals"],"content":"阅读 C 语言规格书 一手资料的重要性毋庸置疑,对于 C 语言中的核心概念 指针,借助官方规格书清晰概念是非常重要的。 C99 [6.2.5] Types An array type of unknown size is an incomplete type. It is completed, for an identifier of that type, by specifying the size in a later declaration (with internal or external linkage). A structure or union type of unknown content is an incomplete type. It is completed, for all declarations of that type, by declaring the same structure or union tag with its defining content later in the same scope. incomplete type 和 linkage 配合可以进行 forward declaration,如果搭配 pointer 则可以进一步,在无需知道 object 内部细节即可进行程序开发。 Array, function, and pointer types are collectively called derived declarator types. A declarator type derivation from a type T is the construction of a derived declarator type from T by the application of an array-type, a function-type, or a pointer-type derivation to T. 注意 derived declarator types 表示衍生的声明类型,因为 array, function, pointer 本质都是地址,而它们的类型都是由其它类型衍生而来的,所以可以使用这些所谓的 derived declarator types 来提前声明 object,表示在某个地址会存储一个 object,这也是为什么这些类型被规格书定义为 derived declarator types。 lvalue: Locator value 危险 C 语言里只有 call by value ","date":"2024-01-14","objectID":"/posts/c-pointer/:2:0","tags":["Sysprog","C","Pointer"],"title":"你所不知道的 C 语言: 指针篇","uri":"/posts/c-pointer/"},{"categories":["C","Linux Kernel Internals"],"content":"void \u0026 void * C89 之前,函数如果没有标注返回类型,则默认返回类型 int,返回值 0。但由于这样既可以表示返回值不重要,也可以表示返回值为 0,这会造成歧义,所以引进了 void。 void * 只能表示地址,而不能对所指向的地址区域的内容进行操作。因为通过 void * 无法知道所指向区域的 size,所以无法对区域的内容进行操作,必须对 void * 进行 显示转换 才能操作指向的内容。(除此之外,针对于 gcc,对于指针本身的操作,void * 与 char * 是等价的,即对于 +/- 1 这类的操作,二者的偏移量是一致的 (这是 GNU extensions 并不是 C 语言标准);对于其它的编译器,建议将 void * 转换成 char * 再进行指针的加减运算) ","date":"2024-01-14","objectID":"/posts/c-pointer/:3:0","tags":["Sysprog","C","Pointer"],"title":"你所不知道的 C 语言: 指针篇","uri":"/posts/c-pointer/"},{"categories":["C","Linux Kernel Internals"],"content":"Alignment 这部分原文描述不是很清晰,2-byte aligned 图示如下: Alignment 如果是 2-byte aligned 且是 little-endian 的处理器,对于左边,可以直接使用 *(uint16_t *) ptr,但对于右边就无法这样(不符合 alignment): /* may receive wrong value if ptr is not 2-byte aligned */ uint16_t value = *(uint16_t *) ptr; /* portable way of reading a little-endian value */ uint16_t value = *(uint8_t *) ptr | ((*(uint8_t *) (ptr + 1)) \u003c\u003c 8); 因为内存寻址的最小粒度是 Byte,所以使用 (uint_8 *) 不需要担心 alignment 的问题。原文并没有给出 32-bit aligned 的 portable way,我们来写一下: /* may receive wrong value if ptr is not 2-byte aligned */ uint32_t value = *(uint32_t *) ptr; /* portable way of reading a little-endian value */ uint32_t value = *(uint8_t *) ptr | ((*(uint8_t *) (ptr + 1)) \u003c\u003c 8) | ((*(uint8_t *) (ptr + 2)) \u003c\u003c 16) | ((*(uint8_t *) (ptr + 3)) \u003c\u003c 24); 信息 The Lost Art of Structure Packing ","date":"2024-01-14","objectID":"/posts/c-pointer/:3:1","tags":["Sysprog","C","Pointer"],"title":"你所不知道的 C 语言: 指针篇","uri":"/posts/c-pointer/"},{"categories":["C","Linux Kernel Internals"],"content":"规格书中的 Pointer C99 [6.3.2.3] Pointers A pointer to a function of one type may be converted to a pointer to a function of another type and back again; the result shall compare equal to the original pointer. Ifaconverted pointer is used to call a function whose type is not compatible with the pointed-to type, the behavior is undefined. C11 [6.3.2.3] Pointers A pointer to a function of one type may be converted to a pointer to a function of another type and back again; the result shall compare equal to the original pointer. If a converted pointer is used to call a function whose type is not compatible with the referenced type, the behavior is undefined. C99 和 C11 都不保证 pointers (whose type is not compatible with the pointed-to / referenced type) 之间的转换是正确的。 导致这个的原因正是之前所提的 Alignment,转换后的指针类型不一定满足原有类型的 Alignment 要求,这种情况下进行 dereference 会导致异常。例如将一个 char * 指针转换成 int * 指针,然后进行 deference 有可能会产生异常。 ","date":"2024-01-14","objectID":"/posts/c-pointer/:3:2","tags":["Sysprog","C","Pointer"],"title":"你所不知道的 C 语言: 指针篇","uri":"/posts/c-pointer/"},{"categories":["C","Linux Kernel Internals"],"content":"Pointers vs. Arrays C99 6.3.2.1 Except when it is the operand of the sizeof operator or the unary \u0026 operator, or is a string literal used to initialize an array, an expression that has type ‘‘array of type’’ is converted to an expression with type ‘‘pointer to type’’ that points to the initial element of the array object and is not an lvalue. Array 只有在表示其自身为数组时才不会被 converted to Pointer,例如 // case 1: extern declaration of array extern char a[]; // case 2: defintion of array char a[10]; // case 3: size of array sizeof(a); // case 4: address of array \u0026a 在其他情况则会倍 converted to Pointer,这时 Array 可以和 Pointer 互换进行表示或操作,例如 // case 1: function parameter void func(char a[]); void func(char *a); // case 2: operation in expression char c = a[2]; char c = *(a + 2); 这也是为什么对于一个 Array a,\u0026a 和 \u0026a[0] 值虽然相同,但 \u0026a + 1 和 \u0026a[0] + 1 的结果大部分时候是大不相同的,这件事乍一看是非常惊人的,但其实不然,在了解 Array 和 Pointer 之后,也就那么一回事 🤣 Source ","date":"2024-01-14","objectID":"/posts/c-pointer/:4:0","tags":["Sysprog","C","Pointer"],"title":"你所不知道的 C 语言: 指针篇","uri":"/posts/c-pointer/"},{"categories":["C","Linux Kernel Internals"],"content":"GDB 实作 char a[10]; int main() { return 0; }; 我们以上面这个例子,通过 GDB 来对 Array 和 Pointer 进行深入研究: (gdb) print \u0026a $1 = (char (*)[10]) 0x555555558018 \u003ca\u003e (gdb) print \u0026a[0] $2 = 0x555555558018 \u003ca\u003e \"\" 符合预期,\u0026a 和 \u0026a[0] 得到的值是相同的,虽然类型看起来不同,但是现在先放到一边。 (gdb) print \u0026a + 1 $3 = (char (*)[10]) 0x555555558022 (gdb) print \u0026a[0] + 1 $4 = 0x555555558019 \u003ca+1\u003e \"\" (gdb) print a + 1 $5 = 0x555555558019 \u003ca+1\u003e \"\" Oh! 正如我们之前所说的 \u0026a + 1 与 \u0026a[0] + 1 结果并不相同(而 \u0026a[0] + 1 和 a + 1 结果相同正是我们所提到的 Array 退化为 Pointer),虽然如此,GDB 所给的信息提示我们可能是二者 Pointer 类型不相同导致的。 (gdb) whatis \u0026a type = char (*)[10] (gdb) whatis \u0026a[0] type = char * Great! 果然是 Pointer 类型不同导致的,我们可以看到 \u0026a 的类型是 char (*)[10] 一个指向 Array 的指针,\u0026a[0] 则是 char *。所以这两个 Pointer 在进行 +/- 运算时的偏移量是不同的,\u0026a[0] 的偏移量为 sizeof(a[0]) 即一个 char 的宽度 ($0x18 + 1 = 0x19$),而 \u0026a 的偏移量为 sizeof(a) 即 10 个 char 的宽度 ($0x18 + 10 = 0x22$)。 警告 在 GDB 中使用 memcpy 后直接打印可能会出现以下错误: (gdb) p memcpy(calendar, b, sizeof(b[0])) 'memcpy' has unknown return type; cast the call to its declared return type 只需加入 void * 进行类型转换即可解决该问题: (gdb) p (void *) memcpy(calendar, b, sizeof(b[0])) ... 技巧 遇到陌生的函数,可以使用 man 来快速查阅手册,例如 man strcpy, man strcat,手册可以让我们快速查询函数的一些信息,从而进入实作。 ","date":"2024-01-14","objectID":"/posts/c-pointer/:4:1","tags":["Sysprog","C","Pointer"],"title":"你所不知道的 C 语言: 指针篇","uri":"/posts/c-pointer/"},{"categories":["C","Linux Kernel Internals"],"content":"Runtime Environment 根据 Zero size arrays in C ,原文中的 char (*argv)[0] 在函数参数传递时会被转换成 char **argv。而为什么在查看地址 ((char **) argv)[0] 开始的连续 4 个 char * 内容时,会打印出 envp 中的内容,可以参考以下的进入 main 函数时的栈布局: argv 和 envp 所指的字符串区域是相连的,所以在越过 argv 字符串区域的边界后,会继续打印 envp 区域的字符串。这也是为什么打印出的字符串之间地址增长于其长度相匹配。所以从地址 (char **) argv 开始的区域只是一个 char * 数组,使用 x/4s 对这部分进行字符串格式打印显然是看不懂的。 注意 argv 和 envp 都是在 shell 进行 exec 系统调用之前进行传递(事实上是以 arguments 的形式传递给 exec) man 2 execve int execve(const char *pathname, char *const argv[], char *const envp[]); execve 实际上在内部调用了 fork,所以 argv 和 envp 的传递是在 fork 之前。(设想如果是在 fork 之后传递,可能会出现 fork 后 child process 先执行,这种情况 child process 显然无法获得这些被传递的信息) 注意到 execve 只传递了 argv 而没有传递 argc,这也很容易理解,argc 是 argv 的计数,只需 argv 即可推导出 argc。 ","date":"2024-01-14","objectID":"/posts/c-pointer/:4:2","tags":["Sysprog","C","Pointer"],"title":"你所不知道的 C 语言: 指针篇","uri":"/posts/c-pointer/"},{"categories":["C","Linux Kernel Internals"],"content":"Function Pointer 危险 与 Array 类似,Function 只有在表示自身时不会被 converted to Function Pointer (即除 sizeof 和 \u0026 运算之外),其它情况、运算时都会被 convert to Function Pointer 理解 C 语言中的 Function 以及 Function Pointer 的核心在于理解 Function Designator 这个概念,函数名字必然是 Function Designator,其它的 designator 则是根据以下两条规则进行推导得来。 C99 [ 6.3.2.1 ] A function designator is an expression that has function type. Except when it is the operand of the sizeof operator or the unary \u0026 operator, a function designator with type ‘‘function returning type’’ is converted to an expression that has type ‘‘pointer to function returning type’’. C99 [6.5.3.2-4] The unary * operator denotes indirection. If the operand points to a function, the result is a function designator. ","date":"2024-01-14","objectID":"/posts/c-pointer/:5:0","tags":["Sysprog","C","Pointer"],"title":"你所不知道的 C 语言: 指针篇","uri":"/posts/c-pointer/"},{"categories":["C","Linux Kernel Internals"],"content":"指针的修饰符 指针 p 自身不能变更,既不能改变 p 自身所存储的地址。const 在 * 之后: char * const p; 指针 p 所指向的内容不能变更,即不能通过 p 来更改所指向的内容。const 在 * 之前: const char * p; char const * p; 指针 p 自身于所指向的内容都不能变更: const char * const p; char const * const p; ","date":"2024-01-14","objectID":"/posts/c-pointer/:6:0","tags":["Sysprog","C","Pointer"],"title":"你所不知道的 C 语言: 指针篇","uri":"/posts/c-pointer/"},{"categories":["C","Linux Kernel Internals"],"content":"字符串 对于函数内部的 char *p = \"hello\"; char p[] = \"hello\"; 这两个是不一样的,因为 string literals 是必须放在 “static storage” 中,而 char p[] 则表示将资料分配在 stack 內,所以这会造成编译器隐式地生成额外代码,在执行时 (runtime) 将 string literals 从 static storage 拷贝到 stack 中,所以此时 return p 会造成 UB。而 char *p 的情形不同,此时 p 只是一个指向 static storage 的指针,进行 return p 是合法的。除此之外,无法对第一种方法的字符串进行修改操作,因为它指向的字符串存放的区域的资料是无法修改的,否则会造成 segmentationfalut 🤣 在大部分情况下,null pointer 并不是一个有效的字符串,所以在 glibc 中字符相关的大部分函数也不会对 null pointer 进行特判 (特判会增加分支,从而影响程序效能),所以在调用这些函数时需要用户自己判断是否为 null pointer,否则会造成 UB。 ","date":"2024-01-14","objectID":"/posts/c-pointer/:7:0","tags":["Sysprog","C","Pointer"],"title":"你所不知道的 C 语言: 指针篇","uri":"/posts/c-pointer/"},{"categories":["C","Linux Kernel Internals"],"content":"Linus 的“教导” Linus 親自教你 C 語言 array argument 的使用 because array arguments in C don’t actually exist. Sadly, compilers accept it for various bad historical reasons, and silently turn it into just a pointer argument. There are arguments for them, but they are from weak minds. The “array as function argument” syntax is occasionally useful (particularly for the multi-dimensional array case), so I very much understand why it exists, I just think that in the kernel we’d be better off with the rule that it’s against our coding practices. array argument 应该只用于多维数组 (multi-dimensional arrays) 的情形,这样可以保证使用下标表示时 offset 是正确的,但对于一维数组则不应该使用数组表示作为函数参数,因为这会对函数体内的 sizeof 用法误解 (以为会获得数组的 size,实际上获得的只是指针的 size)。 技巧 一个常用于计算数组中元素个数的宏: #define ARRAY_SIZE(x) (sizeof(x) / sizeof((x)[0])) 这个宏非常有用,xv6 中使用到了这个宏。 但是需要注意,使用时必须保证 x 是一个数组,而不是函数参数中由数组退化而来的指针,以及保证数组必须至少拥有一个元素的长度 (这个很容易满足,毕竟 x[0] 编译器会抛出警告)。 ","date":"2024-01-14","objectID":"/posts/c-pointer/:8:0","tags":["Sysprog","C","Pointer"],"title":"你所不知道的 C 语言: 指针篇","uri":"/posts/c-pointer/"},{"categories":["C","Linux Kernel Internals"],"content":"Lvalue \u0026 Rvalue Lvalue: locator value Rvalue: Read-only value C99 6.3.2.1 footnote The name “lvalue” comes originally from the assignment expression E1 = E2, in which the left operand E1 is required to be a (modifiable) lvalue. It is perhaps better considered as representing an object “locator value”. What is sometimes called “rvalue” is in this International Standard described as the “value of an expression”. An obvious example of an lvalue is an identifier of an object. As a further example, if E is a unary expression that is a pointer to an object, *E is an lvalue that designates the object to which E points. 即在 C 语言中 lvalue 是必须能在内存 (memory) 中可以定位 (locator) 的东西,因为可以定位 (locator) 所以才可以在表达式左边从而修改值。想像一下,在 C 语言中修改一个常数的值显然是不可能的,因为常数无法在内存 (memory) 定位 (locator) 所以常数在 C 语言中不是 lvalue。C 语言中除了 lvalue 之外的 value 都是 rvalue (这与 C++ 有些不同,C++ 的 lvalue 和 rvalue 的定义请参考 C++ 的规格书)。 ","date":"2024-01-14","objectID":"/posts/c-pointer/:9:0","tags":["Sysprog","C","Pointer"],"title":"你所不知道的 C 语言: 指针篇","uri":"/posts/c-pointer/"},{"categories":["Network"],"content":"之前学校的计网理论课学得云里雾里,对于物理层和数据链路层并没有清晰的逻辑框架,而这学期的计网课设内容为数据链路层和网络层的相关内容,写起来还是云里雾里。虽然最终艰难地把课设水过去了,但是个人认为网络对于 CSer 非常重要,特别是在互联网行业,网络知识是必不可少的。 所以决定寒假重学计网,于是在 HackMD 上冲浪寻找相关资料。然后发现了这篇笔记 110-1 計算機網路 (清大開放式課程),里面提到清大计网主要介绍 L2 ~ L4 一些著名的协议和算法,这完美符合个人的需求,而且该笔记还补充了一些额外的内容,例如 IPv6,所以当即决定搭配这篇笔记来学习清大的计算机网络概论。 ","date":"2024-01-14","objectID":"/posts/nthu-computer-network/:0:0","tags":["Network"],"title":"國立清華大學 計算機網路 重點提示","uri":"/posts/nthu-computer-network/"},{"categories":["Network"],"content":"清大计算机网络概论 本課程將介紹計算機網路的基本運作原理與標準的網路七層結構,由淺入深,可以讓我們對於計算機網路的運作有最基本的認識,本課程還會介紹全球建置最多的有線網路──IEEE 802.3 Ethernet 的基本運作原理, 還有全球建置最多的無線區域網路──IEEE 802.11 Wireless LAN 的基本運作原理, 想知道網路交換機(switches) 是如何運作的嗎 ? 想知道網際網路最重要也最關鍵的通訊協議 ── TCP/IP 是如何運作的嗎 ? 想知道網際網路最重要的路由器 (Routers) 是如何運作的嗎 ? 在本課程裡您都可以學到這些重要的基本知識。 开课学校 课程主页 课程资料 课程影片 國立清華大學 計算機網路概論 課程講義與練習題 Youtube ","date":"2024-01-14","objectID":"/posts/nthu-computer-network/:1:0","tags":["Network"],"title":"國立清華大學 計算機網路 重點提示","uri":"/posts/nthu-computer-network/"},{"categories":["Network"],"content":"Foundation Outline: Applications Network Connectivity Network Architecture Network Performance ","date":"2024-01-14","objectID":"/posts/nthu-computer-network/:2:0","tags":["Network"],"title":"國立清華大學 計算機網路 重點提示","uri":"/posts/nthu-computer-network/"},{"categories":["Network"],"content":"Applications Foundation - 5 进行 1 次 URL request 需要进行 17 次的讯息交换: 6 次讯息交换用于查询 URL 对应的 IP Address 3 次讯息交换用于建立 TCP 连接(TCP 的 3 次握手) 4 次讯息交换用于 HTTP 协议的请求和回复 4 次讯息交换用于关闭 TCP 连接(TCP 的 4 次握手) ","date":"2024-01-14","objectID":"/posts/nthu-computer-network/:2:1","tags":["Network"],"title":"國立清華大學 計算機網路 重點提示","uri":"/posts/nthu-computer-network/"},{"categories":["Network"],"content":"Network Connectivity Foundation - 8 交换机 (Switches) 可以分为很多层级,即可以有不同层级的交换机,例如 L2 层的交换机,L3 层的交换机以及 L4 层的交换机。如何判断交换机是哪个层级?很简单,只需要根据交换机所处理的讯息,L2 层交换机处理的是 MAC Address,L3 层交换机处理的是 IP Address,而 L4 层交换机处理的是 TCP 或者 UDP 相关的讯息。 交换机 (Switches) 用于网络 (Network) 内部的连接,路由 (Router) 用于连接不同的网络 (Network),从而形成 Internetwork。 地址 (Address),对于网卡来说是指 MAC Address,对于主机来说是指 IP Address。Host-to-Host connectivity 是指不同网络 (Network) 的主机,即位于 Internetwork 的不同主机之间,进行连接。 ","date":"2024-01-14","objectID":"/posts/nthu-computer-network/:2:2","tags":["Network"],"title":"國立清華大學 計算機網路 重點提示","uri":"/posts/nthu-computer-network/"},{"categories":["Network"],"content":"Network Architecture Foundation - 22 Physical Layer: 如何将原始资料在 link 上传输,例如不同介质、信息编码。(P25) Data Link Layer: 在 Physical Layer 基础上,如何将 frame 传给直接相连的主机或设备,核心是通过 Media Access Control Protocol 解决 Multiple access 产生的碰撞问题。这一层交换的数据被称为 frame。(P26) Network Layer: 在 Data Link Layer 基础上,如何将 packet 通过 Internet 送给目的地主机。核心是通过 Routing Protocols 动态转发 packet。这一层交换的数据被称为 packet。(P27) Transport Layer: 在 Network Layer 基础上,提供不同主机 processes 之间的资料传送。由于 Networkd Layer 是主机间进行资料传送,所以在 Transport Layer 不论是可靠还是不可靠的传输协议,都必须要实现最基本的机制:主机与 process 之间数据的复用和分解。这一层交换的数据被称为 message。(P28) 注意 Switch 一般处于 L2 Layer,Router 一般处于 L3 Layer。L4 Layer 及以上的 layers 通常只存在于 hosts,switches 和 routers 内部一般不具有这些 layers。(P29) Internet Architecture 的层级并不是严格的,Host 可以略过 Application Layer 而直接使用 Transport Layer、Network Layer 中的协议。(P30) Internet Architecture 的核心是 IP 协议,它作为沙漏形状的中心位置,为处于其上层的协议与处于其下层协议之间提供了一个映射关系。(P31) ","date":"2024-01-14","objectID":"/posts/nthu-computer-network/:2:3","tags":["Network"],"title":"國立清華大學 計算機網路 重點提示","uri":"/posts/nthu-computer-network/"},{"categories":["Network"],"content":"Network Performance Foundation - 36 Foundation - 37 Bandwidth: Number of bits per second (P34) Delay 可以近似理解为 Propagation time。有效利用 network 的标志是在接收对方的回应之前,发送方传送的资料充满了 pipe,即发送了 Delay $\\times$ Bandwitdh bits 的资料量。(P39) Foundation - 40 RTT 可以近似理解为 2 $\\times$ Propagation time,因为一个来回需要从 sender 到 reciever,再从 reciever 到 sender。 ","date":"2024-01-14","objectID":"/posts/nthu-computer-network/:2:4","tags":["Network"],"title":"國立清華大學 計算機網路 重點提示","uri":"/posts/nthu-computer-network/"},{"categories":["Network"],"content":"Homework Redis 作者 Salvatore Sanfilippo 的聊天室项目: smallchat,通过该项目可以入门学习网络编程 (Network Programming),请复现该项目。 Salvatore Sanfilippo 在 YouTube 上对 smallchat 的讲解: Smallchat intro smallchat client \u0026 raw line input GitHub 上也有使用 Go 和 Rust 实现该项目的仓库,如果你对 Go 或 Rust 的网络编程 (Network Programming) 感兴趣,可以参考这个仓库。 ","date":"2024-01-14","objectID":"/posts/nthu-computer-network/:2:5","tags":["Network"],"title":"國立清華大學 計算機網路 重點提示","uri":"/posts/nthu-computer-network/"},{"categories":["Network"],"content":"IEEE 802.3 Ethernet Outline: Introduction Ethernet Topologies Ethernet Frame Format Ethernet MAC Protocol – CSMA/CD 802.3 Ethernet Standards Summary: MAC Protocol – CSMA/CD Connection less, unreliable transmission Topology from Bus to Star (switches) Half-duplex transmission in Bus topology Work best under lightly loaded conditions Too much collision under heavy load Full-duplex transmission in Switch topology (point-to-point) No more collisions !! Excellent performance (wired speed) ","date":"2024-01-14","objectID":"/posts/nthu-computer-network/:3:0","tags":["Network"],"title":"國立清華大學 計算機網路 重點提示","uri":"/posts/nthu-computer-network/"},{"categories":["Network"],"content":"Introduction Ethernet - 03 Ethernet 发展过程: 传输速度从 10Mb 发展到 100Gb (P4) Ethernet 的特点: Unreliable, Connectionless, CSMA/CD (P5) ","date":"2024-01-14","objectID":"/posts/nthu-computer-network/:3:1","tags":["Network"],"title":"國立清華大學 計算機網路 重點提示","uri":"/posts/nthu-computer-network/"},{"categories":["Network"],"content":"Ethernet Topologies Ethernet - 07 Ethernet - 18 10Base5: 10Mbps, segment up to 500m (P8) 10Base2: 10Mbps, segment up to 200m (P8) 10BaseT: 10Mbps, Twisted pair, segment up to 100m (P16) Repeater, Hub 都是 physical layer 的设备,只负责 转发信号,无法防止 collision (P12, P16) Switch 则是 data-link layer 的设备,内置芯片进行 数据转发,可以防止 collision (P19) Manchester Encoding (P11): Ethernet 下层的 physical layer 使用的编码方式是 Manchester Encoding: 在一个时钟周期内,信号从低到高表示 1,从高到低表示 0 注意 Manchester Encoding 发送方在进行数据传输之前需要发送一些 bits 来进行时钟同步 (例如 P22 的 Preamble 部分),接收方完成时钟同步后,可以对一个时钟周期进行两次采样:一次前半段,一次后半段,然后可以通过两次取样电位信号的变化来获取对应的 bit (低到高表示 1,高到低表示 0)。 有些读者可能会疑惑,既然都进行时钟同步了,为什么不直接使用高电位信号表示 1,低电位信号表示 0 这样直观的编码方式?这是因为如果采取这种编码方式,那么在一个时钟周期内信号不会有变化,如果接收的是一系列的 1 或 0,信号也不会变化。这样可能会导致漏采样,或者编码出错却无法及时侦测。而采用 Manchester Encoding 接收方每个时钟周期内信号都会变化,如果接收方在一次时钟周期内的两次采样,信号没有发生变化,那么可以立即侦测到出错了 (要么是漏采样了,要么是编码出错了)。 ","date":"2024-01-14","objectID":"/posts/nthu-computer-network/:3:2","tags":["Network"],"title":"國立清華大學 計算機網路 重點提示","uri":"/posts/nthu-computer-network/"},{"categories":["Network"],"content":"Ethernet Frame Format Ethernet - 23 除开 Preamble, SFD 之外,一个 Frame 的大小为 $64 \\sim 1518$ bytes。因为 DA, SA, TYPE, FCS 占据了 $6 + 6 + 2 + 4 = 18$ bytes,所以 Data 部分的大小为 $48 ~\\sim 1500$ bytes (P43) MAC Address 是 unique 并且是与 Adaptor 相关的,所以一个主机可能没有 MAC Address (没有 Adaptor),可能有两个 MAC Address (有两个 Adaptor)。MAC Address 是由 Adaptor 的生产商来决定的。(P24) unicast address, broadcast address, multicast address (P26) ","date":"2024-01-14","objectID":"/posts/nthu-computer-network/:3:3","tags":["Network"],"title":"國立清華大學 計算機網路 重點提示","uri":"/posts/nthu-computer-network/"},{"categories":["Network"],"content":"CSMA/CD Ethernet - 46 Ethernet - 41 Ethernet - 45 Ethernet - 49 关于 CSMA/CD 的详细介绍可以查看 P34 ~ P38 关于 Ethernet Frame 的大小限制设计可以查看 P39 ~ P43 关于 CSMA/CD Collision Handling 的策略机制可以查看 P44 ~ P45, P47 ~ P48 注意 Host 在 detect collision 之后进行 backoff random delay,delay 结束后按照 1-persistent protocol (P35) 继续等待到 busy channel goes idle 后立刻进行传输。 ","date":"2024-01-14","objectID":"/posts/nthu-computer-network/:3:4","tags":["Network"],"title":"國立清華大學 計算機網路 重點提示","uri":"/posts/nthu-computer-network/"},{"categories":["Network"],"content":"IEEE 802.11 Wireless LAN 无线网络这章太难了,战术性放弃 ","date":"2024-01-14","objectID":"/posts/nthu-computer-network/:4:0","tags":["Network"],"title":"國立清華大學 計算機網路 重點提示","uri":"/posts/nthu-computer-network/"},{"categories":["Network"],"content":"IEEE 802.1D Spanning Tree Algorithm ","date":"2024-01-14","objectID":"/posts/nthu-computer-network/:5:0","tags":["Network"],"title":"國立清華大學 計算機網路 重點提示","uri":"/posts/nthu-computer-network/"},{"categories":["Network"],"content":"Referenecs 110-1 計算機網路 (清大開放式課程) 小菜学网络 NUDT 高级计算机网络实验: 基于UDP的可靠传输 可靠 UDP 的实现 (KCP over UDP) 基于 UDP 的可靠传输 [bilibili] 实现基于 UDP 的网络文件传输器,程序员的经验大礼包项目 [bilibili] ping 命令但是用来通信,学习计算机网络好项目,也可能是校园网福利 [bilibili] Implementing TCP in Rust [YouTube] Let's code a TCP/IP stack ","date":"2024-01-14","objectID":"/posts/nthu-computer-network/:6:0","tags":["Network"],"title":"國立清華大學 計算機網路 重點提示","uri":"/posts/nthu-computer-network/"},{"categories":["C","Linux Kernel Internals"],"content":"C 语言规格书阅读学习记录。 规格书草案版本为 n1256,对应 C99 标准,对应的 PDF 下载地址。 也配合 C11 标准来阅读,草案版本 n1570,对应的 PDF 下载地址。 阅读规格书需要一定的体系结构、编译原理的相关知识,但不需要很高的程度。请善用检索工具,在阅读规格书时遇到术语时,请先在规格书中进行检索,因为极大可能是规格书自己定义的术语。 ","date":"2024-01-06","objectID":"/posts/c-specification/:0:0","tags":["C","Sysprog"],"title":"C 语言规格书 重点提示","uri":"/posts/c-specification/"},{"categories":["C","Linux Kernel Internals"],"content":"6. Language ","date":"2024-01-06","objectID":"/posts/c-specification/:1:0","tags":["C","Sysprog"],"title":"C 语言规格书 重点提示","uri":"/posts/c-specification/"},{"categories":["C","Linux Kernel Internals"],"content":"6.2 Concepts ","date":"2024-01-06","objectID":"/posts/c-specification/:2:0","tags":["C","Sysprog"],"title":"C 语言规格书 重点提示","uri":"/posts/c-specification/"},{"categories":["C","Linux Kernel Internals"],"content":"6.2.2 Linkages of identifiers linkage: external internal none 一个拥有 file scope 并且关于 object 或 function 的 identifier 声明,如果使用 static 修饰,则该 identifer 有 internal linkage,e.g. // file scope static int a; static void f(); int main() {} 一个 scope 内使用 static 修饰的 identifier 声明,如果在同一 scope 内已存在该 identifier 声明,则该 identifier 的 linkage 取决于先前的 identifier 声明。如果该 identifier 不存在先前声明或者先前声明 no linkage,则该 identifier 是 external linkage,e.g. // Example 1 static int a; // a is internal linkage extern int a; // linkage is the same as prior // Example 2 extern int b; // no prior, a is external linkage extern int b; // linkage is the same as prior 如果一个 function identifier 声明没有 storage-class 修饰符,则其 linkage 等价于加上 extern 修饰的声明的 linkage,e.g. int func(int a, int b); // equal to `extern int func(int a. int b);` // and then no prior, it is external linkage 如果一个 object identifier 声明没有 storage-class 修饰符,且拥有 file scope,则其拥有 external linkage,e.g. // file scope int a; // external linkage int main() {} ","date":"2024-01-06","objectID":"/posts/c-specification/:2:1","tags":["C","Sysprog"],"title":"C 语言规格书 重点提示","uri":"/posts/c-specification/"},{"categories":["C","Linux Kernel Internals"],"content":"6.5 Expressions ","date":"2024-01-06","objectID":"/posts/c-specification/:3:0","tags":["C","Sysprog"],"title":"C 语言规格书 重点提示","uri":"/posts/c-specification/"},{"categories":["C","Linux Kernel Internals"],"content":"6.5.3 Unary operators 注意 C99 [6.2.5] Types There are three real floating types, designated as float, double, and long double. The real floating and complex types are collectively called the floating types. The integer and real floating types are collectively called real types. Integer and floating types are collectively called arithmetic types. A function type describes a function with specified return type. A function type is characterized by its return type and the number and types of its parameters. A function type is said to be derived from its return type, and if its return type is T, the function type is sometimes called ‘‘function returning T’’. The construction of a function type from a return type is called ‘‘function type derivation’’. Arithmetic types and pointer types are collectively called scalar types. C99 [6.3.2.1] Lvalues, arrays, and function designators A function designator is an expression that has function type. Except when it is the operand of the sizeof operator or the unary \u0026 operator, a function designator with type ‘‘function returning type’’ is converted to an expression that has type ‘‘pointer to function returning type’’. 6.5.3.1 Prefix increment and decrement operators Constraints 前缀自增或自减运算符的操作数,必须为实数 (real types) 类型(即不能是复数)或者是指针类型,并且其值是可变的。 Semantics ++E 等价于 (E+=1) --E 等价于 (E-=1) 6.5.3.2 Address and indirection operators Constraints \u0026 运算符的操作数必须为 function designator,[] 或 * 的运算结果,或者是一个不是 bit-field 和 register 修饰的左值。 * 运算符的操作数必须为指针类型。 Semantics \u0026*E 等价于 E,即 \u0026 和 * 被直接忽略,但是它们的 constraints 仍然起作用。所以 (\u0026*(void *)0) 并不会报错。 \u0026a[i] 等价于 a + i,即忽略了 \u0026 以及 * (由 [] 隐式指代)。 其它情况 \u0026 运算的结果为一个指向 object 或 function 的指针。 如果 * 运算符的操作数是一个指向 function 的指针,则结果为对应的 function designator。 如果 * 运算符的操作数是一个指向 object 的指针,则结果为指示该 obejct 的左值。 如果 * 运算符的操作数为非法值的指针,则对该指针进行 * 运算的行为三未定义的。 6.5.3.3 Unary arithmetic operators Constraints 单目 + 或 - 运算符的操作数必须为算数类型 (arithmetic type),~ 运算符的操作数必须为整数类型 (integer type),! 运算符的操作数必须为常数类型 (scalar type)。 Semantics 在进行单目 +、-、~ 运算之前,会对操作数进行整数提升 (integer promotions),结果的类型与操作数进行整数提升后的类型一致。 !E 等价于 (E==0),结果为 int 类型。 ","date":"2024-01-06","objectID":"/posts/c-specification/:3:1","tags":["C","Sysprog"],"title":"C 语言规格书 重点提示","uri":"/posts/c-specification/"},{"categories":["C","Linux Kernel Internals"],"content":"6.5.6 Additive operators 介绍加减法运算,其中包括了指针的运算,务必阅读这部分关于指针运算的标准说明。 ","date":"2024-01-06","objectID":"/posts/c-specification/:3:2","tags":["C","Sysprog"],"title":"C 语言规格书 重点提示","uri":"/posts/c-specification/"},{"categories":["C","Linux Kernel Internals"],"content":"6.5.7 Bitwise shift operators Constraints 位运算的操作数都必须为整数类型。 Semantics 在进行位运算之前会先对操作数进行整数提升 (integer promotion),位运算结果类型与整数提升后的左操作数一致。如果右运算数是负数,或者大于等于整数提升后的左运算数的类型的宽度,那么这个位运算行为是未定义的。 假设运算结果的类型为 T $E1 \u003c\u003c E2$ 如果 E1 是无符号,则结果为 $E1 \\times 2^{E2} \\bmod (\\max[T] + 1)$。 如果 E1 是有符号,E1 不是负数,并且 T 可以表示 $E1 \\times 2^{E2}$,则结果为 $E1 \\times 2^{E2}$。 除了以上两种行为外,其他均是未定义行为。 $E1 \u003e\u003e E2$ 如果 E1 是无符号,或者 E1 是有符号并且是非负数,则结果为 $E1 / 2^{E2}$。 如果 E1 是有符号并且是负数,则结果由具体实现决定 (implementation-defined)。 ","date":"2024-01-06","objectID":"/posts/c-specification/:3:3","tags":["C","Sysprog"],"title":"C 语言规格书 重点提示","uri":"/posts/c-specification/"},{"categories":["C","Linux Kernel Internals"],"content":"7. Library ","date":"2024-01-06","objectID":"/posts/c-specification/:4:0","tags":["C","Sysprog"],"title":"C 语言规格书 重点提示","uri":"/posts/c-specification/"},{"categories":["C","Linux Kernel Internals"],"content":"7.18 Integer types \u003cstdint.h\u003e 描述了头文件 stdint.h 必须定义和实现的整数类型,以及相应的宏。 ","date":"2024-01-06","objectID":"/posts/c-specification/:5:0","tags":["C","Sysprog"],"title":"C 语言规格书 重点提示","uri":"/posts/c-specification/"},{"categories":["C","Linux Kernel Internals"],"content":"7.18.1 Integer types 7.18.1.1 Exact-width integer types 二补数编码,固定长度 N 的整数类型: 有符号数:intN_t 无符号数:uintN_t 7.18.1.2 Minimum-width integer types 至少拥有长度 N 的整数类型: 有符号数:int_leastN_t 无符号数:uint_leastN_t 7.18.1.3 Fastest minimum-width integer types 至少拥有长度 N,且操作速度最快的整数类型: 有符号数:int_fastN_t 无符号数:uint_fastN_t 7.18.1.4 Integer types capable of holding object pointers 可以将指向 void 的有效指针转换成该整数类型,也可以将该整数类型转换回指向 void 的指针类型,并且转换结果与之前的指针值保持一致: 有符号数:intptr_t 无符号数:uintptr_t 7.18.1.5 Greatest-width integer types 可以表示任意整数类型所表示的值的整数类型,即具有最大长度的整数类型: 有符号数:intmax_t 无符号数:uintmax_t ","date":"2024-01-06","objectID":"/posts/c-specification/:5:1","tags":["C","Sysprog"],"title":"C 语言规格书 重点提示","uri":"/posts/c-specification/"},{"categories":["Toolkit"],"content":"Git 中文教学 新手入门推荐,对于 Git 的入门操作讲解十分友好。 视频地址 学习记录 ","date":"2024-01-04","objectID":"/posts/git/:1:0","tags":["Git","GitHub"],"title":"Git/GitHub 资源与问题汇总","uri":"/posts/git/"},{"categories":["Toolkit"],"content":"Git 常见问题及解决 ","date":"2024-01-04","objectID":"/posts/git/:2:0","tags":["Git","GitHub"],"title":"Git/GitHub 资源与问题汇总","uri":"/posts/git/"},{"categories":["Toolkit"],"content":"git pull/push 遇到 Port 22 connect timeout 网络问题导致 22 端口被禁止,无法正常使用 ssh。切换成 443 端口并且编写配置文件即可: $ vim ~/.ssh/config # In ~/.ssh/config Host github.com HostName ssh.github.com Port 443 ","date":"2024-01-04","objectID":"/posts/git/:2:1","tags":["Git","GitHub"],"title":"Git/GitHub 资源与问题汇总","uri":"/posts/git/"},{"categories":["Toolkit"],"content":"GitHub 支持多个账户通过 ssh 连接 Using multiple github accounts with ssh keys ","date":"2024-01-04","objectID":"/posts/git/:2:2","tags":["Git","GitHub"],"title":"Git/GitHub 资源与问题汇总","uri":"/posts/git/"},{"categories":["Toolkit"],"content":"References Git 基本原理 Learn Git Branching DIY a Git ugit 动手学习GIT - 最好学习GIT的方式是从零开始做一个 ","date":"2024-01-04","objectID":"/posts/git/:3:0","tags":["Git","GitHub"],"title":"Git/GitHub 资源与问题汇总","uri":"/posts/git/"},{"categories":["Linux Kernel Internals"],"content":" 人们对数学的加减运算可轻易在脑中辨识符号并理解其结果,但电脑做任何事都受限于实体资料储存及操作方式,换言之,电脑硬体实际只认得 0 和 1,却不知道符号 + 和 - 在数学及应用场域的意义,於是工程人员引入「补数」以便在二进位系统中,表达人们认知上的正负数。但您有没有想过,为何「二补数」(2’s complement) 被电脑广泛采用呢?背後的设计考量又是什麽?本文尝试从数学观点去解读编码背後的原理,并佐以资讯安全及程式码最佳化的考量,探讨二补数这样的编码对于程式设计有何关键影响。 原文地址:解讀計算機編碼 技巧 为了更好的理解本文的一些数学概念,例如群,以及后续其他关于数值系统、浮点数的讲座,Jserv 强烈建议我们去修读数学系的 数学导论。笔者在这里分享一下台大齐震宇老师的 2015 年的新生营讲座,这个讲座覆盖了数学导论的内容。 YouTube: 臺大 2015 數學系新生營 ","date":"2023-12-31","objectID":"/posts/binary-representation/:0:0","tags":["Sysprog","Numerics","Bitwise"],"title":"解读计算机编码","uri":"/posts/binary-representation/"},{"categories":["Linux Kernel Internals"],"content":"一补数 (Ones’ complement) ","date":"2023-12-31","objectID":"/posts/binary-representation/:1:0","tags":["Sysprog","Numerics","Bitwise"],"title":"解读计算机编码","uri":"/posts/binary-representation/"},{"categories":["Linux Kernel Internals"],"content":"9 的补数 科普短片: Not just counting, but saving lives: Curta ","date":"2023-12-31","objectID":"/posts/binary-representation/:1:1","tags":["Sysprog","Numerics","Bitwise"],"title":"解读计算机编码","uri":"/posts/binary-representation/"},{"categories":["Linux Kernel Internals"],"content":"运算原理 注意 以一补数编码形式表示的运算子,在参与运算后,运算结果符合一补数的编码: $$ [X]_{一补数} + [Y]_{一补数} = [X+Y]_{一补数} $$ 接下来进行分类讨论,以 32-bit 正数 $X$, $Y$ 为例: $X + Y = X + Y$ 显然运算子和运算结果都满足一补数编码。 $X - Y = X + (2^{32} - 1 - Y)$ 如果 $X \u003e Y$,则运算结果应为 $X - Y$ 且为正数,其一补数编码为 $X - Y$。而此时 $$ 2^{32} - 1 + X - Y $$ 显然会溢出,为了使运算结果对应一补数编码,所以此时循环进位对应 $+\\ (1 - 2_{32})$。 如果 $X \u003c Y$,则运算结果应为 $X - Y$ 且为负数,其一补数编码为 $$ 2^{32} - 1 - (Y - X) = 2_{32} - 1 - X - Y $$ 而此时 $2^{32} - 1 + X - Y$ 并不会溢出,并且满足运算结果的一补数编码,所以无需进行循环进位。 如果 $X = Y$,显然 $$ X - Y = X + 2^{32} - 1 - Y = 2^{32} - 1 $$ 为 0 成立。 $-X - Y = (2^{32} - 1 - X) + (2^{32} - 1 - Y)$,显然会导致溢出。而 $-X - Y$ 的一补数编码为 $$ 2^{32} - 1 - (X + Y) = 2^{32} - 1 - X - Y $$ 所以需要在溢出时循环进位 $+\\ (1 - 2^{32})$ 来消除运算结果中的一个 $2^{32} - 1$。 ","date":"2023-12-31","objectID":"/posts/binary-representation/:1:2","tags":["Sysprog","Numerics","Bitwise"],"title":"解读计算机编码","uri":"/posts/binary-representation/"},{"categories":["Linux Kernel Internals"],"content":"二补数 (Two’s complement) ","date":"2023-12-31","objectID":"/posts/binary-representation/:2:0","tags":["Sysprog","Numerics","Bitwise"],"title":"解读计算机编码","uri":"/posts/binary-representation/"},{"categories":["Linux Kernel Internals"],"content":"正负数编码表示 假设有 n-bit 的二补数编码 $A$,$-A$ 的推导如下: 格式一: $$ \\begin{align*} A + \\neg A \u0026= 2^n - 1 \\\\ A + \\neg A + 1 \u0026\\equiv 0 \\equiv 2^n \\ (\\bmod 2^n) \\\\ -A \u0026= \\neg A + 1 \\\\ \\end{align*} $$ 格式二: $$ \\begin{align*} A + \\neg A \u0026= 2^n - 1 \\\\ A + \\neg A - 1 \u0026= 2^n - 2 \\\\ A - 1 \u0026= 2^n - 1 - (\\neg A + 1) \\\\ \\neg (A - 1) \u0026= \\neg A + 1 \\\\ \\neg (A - 1) \u0026= -A \\\\ \\end{align*} $$ 也可以通过一补数和二补数,在时钟表上的对称轴偏差,来理解上述两种方式是等价的。 Twos’ complement 注意 在二补数编码中,将一个整数转换成其逆元,也可以依据以下的方法: 以 LSB 到 MSB 的顺序,寻找第一个值为 1 的 bit,将这个 bit 以及比其更低的 bits (包含该 bit) 都保持不变,将比该 bit 更高的 bits (不包括该 bit) 进行取反操作。下面是一些例子 (以 32-bit 为例): 0x0080 \u003c-\u003e 0xff80 0x0001 \u003c-\u003e 0xffff 0x0002 \u003c-\u003e 0xfffe ","date":"2023-12-31","objectID":"/posts/binary-representation/:2:1","tags":["Sysprog","Numerics","Bitwise"],"title":"解读计算机编码","uri":"/posts/binary-representation/"},{"categories":["Linux Kernel Internals"],"content":"加 / 减法器设计 科普短片: See How Computers Add Numbers In One Lesson ✅ 了解晶体管的原理 了解基本逻辑门元件,例如 OR, AND 逻辑门的设计 了解加法器的原理和工作流程。 ","date":"2023-12-31","objectID":"/posts/binary-representation/:2:2","tags":["Sysprog","Numerics","Bitwise"],"title":"解读计算机编码","uri":"/posts/binary-representation/"},{"categories":["Linux Kernel Internals"],"content":"阿贝尔群及对称性 技巧 群论的最大用途是关于「对称性」的研究;所有具有对称性质,群论都可派上用场。只要发生变换后仍有什么东西还维持不变,那符合对称的性质。 一个圆左右翻转后还是圆,它在这种变换下是对称的,而这刚好与群的 封闭性 (Closure) 对应。 一个时钟的时刻,从 0 时刻开始,两边的时刻相加模 12 的结果均为 0,这与群的 单位元 (Identity element) 和 逆元 (Inverse element) 对应。 上述两个例子反映了群论的性质,对于对称性研究的重要性和原理依据。 科普影片: 从五次方程到伽罗瓦理论 ","date":"2023-12-31","objectID":"/posts/binary-representation/:3:0","tags":["Sysprog","Numerics","Bitwise"],"title":"解读计算机编码","uri":"/posts/binary-representation/"},{"categories":["Linux Kernel Internals"],"content":"旁路攻击 观看科普视频: 我听得到你打了什么字 ✅ 阅读相关论文 Keyboard Acoustic Emanations 体验使用相关工具 kbd-audio 借由 Wikipedia 了解旁路攻击 (Side-channel attack) 和时序攻击 (Timing attack) 的基本概念 ✅ Black-box testing Row hammer Cold boot attack Rubber-hose cryptanalysis 延伸阅读 The password guessing bug in Tenex Side Channel Attack By Using Hidden Markov Model One\u0026Done: A Single-Decryption EM-Based Attack on OpenSSL’s Constant-Time Blinded RSA ","date":"2023-12-31","objectID":"/posts/binary-representation/:4:0","tags":["Sysprog","Numerics","Bitwise"],"title":"解读计算机编码","uri":"/posts/binary-representation/"},{"categories":["Linux Kernel Internals"],"content":"Constant-Time Functions 比较常见的常数时间实作方法是,消除分支。因为不同分支的执行时间可能会不同,这会被利用进行时序攻击。这个方法需要对 C 语言中的编码和位运算有一定的了解。 C99 7.18.1.1 Exact-width integer types C99 6.5.7.5 Bitwise shift operators Source Branchless abs 如果 n 是 signed 32-bit,则 n \u003e\u003e 31 等价于 n == 0 ? 0 : -1 方法一,原理为 $-A = \\neg (A - 1)$: #include \u003cstdint.h\u003e int32_t abs(int32_t x) { int32_t mask = (x \u003e\u003e 31); return (x + mask) ^ mask; } 方法二,原理为 $-A = \\neg A + 1$: #include \u003cstdint.h\u003e int32_t abs(int32_t x) { int32_t mask = (x \u003e\u003e 31); return (x ^ mask) - mask; } Branchless min/max Min: #include \u003cstdint.h\u003e int32_t min(int32_t a, int32_t b) { int32_t diff = (a - b); return b + (diff \u0026 (diff \u003e\u003e 31)); } 如果 diff \u003e 0 即 b 小,那么 (diff \u003e\u003e 31) == 0,则 b + (diff \u0026 (diff \u003e\u003e 31)) == b 如果 diff \u003c 0 即 a 小,那么 (diff \u003e\u003e 31) == -1,则 b + (diff \u0026 (diff \u003e\u003e 31)) == b + (a - b) == a Max: #include \u003cstdint.h\u003e int32_t max(int32_t a, int32_t b) { int32_t diff = (b - a); return b - (diff \u0026 (diff \u003e\u003e 31)); } 如果 diff \u003e 0 即 b 大, 那么 (diff \u003e\u003e 31) == 0,则 b - (diff \u0026 (diff \u003e\u003e 31)) == b 如果 diff \u003c 0 即 a 大,那么 (diff \u003e\u003e 31) == -1,则 b - (diff \u0026 (diff \u003e\u003e 31)) == b - (b - a) == a 信息 基于 C 语言标准研究与系统程序安全议题 ","date":"2023-12-31","objectID":"/posts/binary-representation/:4:1","tags":["Sysprog","Numerics","Bitwise"],"title":"解读计算机编码","uri":"/posts/binary-representation/"},{"categories":["Linux Kernel Internals","Rust"],"content":"Rust in 100 Seconds 观看短片: Rust in 100 Seconds ✅ 了解 Rust,初步了解其安全性原理 所有权 (ownership) 借用 (borrow) 警告 0:55 This is wrong, value mutability doesn’t have anything to do with the value being stored on the stack or the heap (and the example let mut hello = \"hi mom\" will be stored on the stack since it’s type is \u0026'static str), it depends on the type of the value (if it’s Sized or not). ","date":"2023-12-28","objectID":"/posts/why-rust-/:1:0","tags":["Rust","Sysprog"],"title":"Why Rust?","uri":"/posts/why-rust-/"},{"categories":["Linux Kernel Internals","Rust"],"content":"The adoption of Rust in Business (2022) 阅读报告: The adoption of Rust in Business (2022) ✅ Rust 目前蓬勃发展,预测未来是很难的,但是 Rust 已经是进行时的未来了 🤣 ","date":"2023-12-28","objectID":"/posts/why-rust-/:2:0","tags":["Rust","Sysprog"],"title":"Why Rust?","uri":"/posts/why-rust-/"},{"categories":["Linux Kernel Internals","Rust"],"content":"Visualizing memory layout of Rust’s data types YouTube: Visualizing memory layout of Rust’s data types 影片的中文翻译: 可视化 Rust 各数据结构的内存布局 [bilibili] 可搭配阅读相关的文档: [2022-05-04] 可视化 Rust 各数据类型的内存布局 ","date":"2023-12-28","objectID":"/posts/why-rust-/:3:0","tags":["Rust","Sysprog"],"title":"Why Rust?","uri":"/posts/why-rust-/"},{"categories":["Toolkit","Linux Kernel Internals"],"content":"教学影片:Git 中文教学 ","date":"2023-12-27","objectID":"/posts/git-learn/:0:0","tags":["Git","GitHub"],"title":"Git 学习记录","uri":"/posts/git-learn/"},{"categories":["Toolkit","Linux Kernel Internals"],"content":"安装与设定 注意 ✅ 观看影片 Git 教学系列 - 安装与配置,完成常用的 Git 设置。 设置 Git 的编辑器为 vim,主要用于 commit 时的编辑: $ git config --global core.editor vim 设置 Git 的合并解决冲突工具为 vimdiff: $ git config --global merge.tool vimdiff 启用 Git 命令行界面的颜色显示: $ git config --global color.ui true 设置常用命令的别名: $ git config --global alias.st status $ git config --global alias.ch checkout $ git config --global alias.rst reset HEAD 效果为:命令 git st 等价于 git status,其余的类似。 设置 Windows 和 Mac/Linux 的换行符同步: # In Windows $ git config --global core.autocrlf true # In Mac/Linux $ git config --global core.autocrlf input 效果为:在 Windows 提交时自动将 CRLF 转为 LF,检出代码时将 LF 转换成 CRLF。在 Mac/Linux 提交时将 CRLF转为 LF,检出代码时不转换。这是因为 Windows 的换行符为 \\r\\n,而 Mac/Linux 的换行符仅为 \\n。 ","date":"2023-12-27","objectID":"/posts/git-learn/:1:0","tags":["Git","GitHub"],"title":"Git 学习记录","uri":"/posts/git-learn/"},{"categories":["Toolkit","Linux Kernel Internals"],"content":"Add 和 Commit ","date":"2023-12-27","objectID":"/posts/git-learn/:2:0","tags":["Git","GitHub"],"title":"Git 学习记录","uri":"/posts/git-learn/"},{"categories":["Toolkit","Linux Kernel Internals"],"content":"指定 Commit 注意 ✅ 观看影片 Git 教学系列 - 指定 Commit,掌握 git log、git show、git diff 的常用方法。理解 Hash Value 和 commit 对于 Git 版本控制的核心作用。 只要 commit 了,资料基本不可能丢失,即使误操作了也是可以补救回来的(除非把 .git/ 文件夹也删除了)。 ","date":"2023-12-27","objectID":"/posts/git-learn/:3:0","tags":["Git","GitHub"],"title":"Git 学习记录","uri":"/posts/git-learn/"},{"categories":["Toolkit","Linux Kernel Internals"],"content":"Hash Value Every commit has a unique hash value. Calculate by SHA1 Hash value can indicate a commit absolutely. ","date":"2023-12-27","objectID":"/posts/git-learn/:3:1","tags":["Git","GitHub"],"title":"Git 学习记录","uri":"/posts/git-learn/"},{"categories":["Toolkit","Linux Kernel Internals"],"content":"Indicate Commit git manage references to commit HEAD Branch Tag Remote Also, We can indicate commit by ^, ~ 通俗地将,不论是 HEAD、Branch、Tag、Remote,其本质都是使用 Hash Value 进行索引的 commit,所以 ~ 和 ^ 也可以作用于它们。 可以通过 git log 来查看 commit 以及对应的 Hash 值。事实上,这个命令十分灵活,举个例子: git log 4a6ebc -n1 这个命令的效果是从 Hash 值为 4a6bc 的 commit 开始打印 1 条 commit 记录(没错,对应的是 -n1),因为 Git 十分聪明,所以 commit 对应的 Hash 值只需前 6 位即可(因为这样已经几乎不会发生 Hash 冲突)。 Examples 打印 master 分支的最新一个 commit: git log master -n1 打印 master 分支的最新一个 commit(仅使用一行打印 commit 信息): git log master -n1 --oneline 打印 HEAD 所指向的 commit: git log HEAD -n1 --oneline 打印 HEAD 所指向的 commit 的前一个 commit: git log HEAD^ -n1 --oneline ^ 可以持续使用,比如 HEAD^^ 表示 HEAD 所指向的 commit 的前两个 commit。当 ^ 数量过多时,可以使用 ~ 搭配数字来达到相同效果。例如: git log HEAD^^^^^ -n1 --oneline git log HEAD~5 -n1 --oneline 一般来说,使用 ^ 就已经足够了,几乎不会遇到使用 ~ 的场景,因为这种场景一般会去找图形化界面吧。🤣 打印与文件 README.md 相关的 commits(仅使用一行显示): git log --oneline README.md 打印与文件 README.md 相关的 commits(显示详细信息,包括文件内容的增减统计): git log --stat README.md 打印与文件 README.md 相关的 commits(显示详细信息,包括文件内容的增减细节): git log --patch README.md 在打印的 commit 信息中抓取与 README 符合的信息(可以与 --stat 或 --patch 配合使用): git log -S README ","date":"2023-12-27","objectID":"/posts/git-learn/:3:2","tags":["Git","GitHub"],"title":"Git 学习记录","uri":"/posts/git-learn/"},{"categories":["Toolkit","Linux Kernel Internals"],"content":"View History git log \u003cpath\u003e|\u003ccommit\u003e -n: limit number --oneline: view hash and commit summary --stat: view files change --patch: view lines change -S or --grep: find modification ","date":"2023-12-27","objectID":"/posts/git-learn/:3:3","tags":["Git","GitHub"],"title":"Git 学习记录","uri":"/posts/git-learn/"},{"categories":["Toolkit","Linux Kernel Internals"],"content":"View Commit git show \u003ccommit\u003e Equal to log -n1 ","date":"2023-12-27","objectID":"/posts/git-learn/:3:4","tags":["Git","GitHub"],"title":"Git 学习记录","uri":"/posts/git-learn/"},{"categories":["Toolkit","Linux Kernel Internals"],"content":"See Difference 查看当前的修改,可以查看已经修改但没有 staged 文件的变化: git diff 查看当前的修改,可以查看已经修改且 staged 文件的变化: git diff --staged 查看当前与指定的 commit 的差异: git diff \u003ccommit\u003e # e.g. git diff master^ 查两个指定的 commit 之间的差异: git diff \u003ccommit\u003e \u003ccommit\u003e # e.g. git diff master^ master^^ ","date":"2023-12-27","objectID":"/posts/git-learn/:3:5","tags":["Git","GitHub"],"title":"Git 学习记录","uri":"/posts/git-learn/"},{"categories":["Toolkit","Linux Kernel Internals"],"content":"Path Add and Amend 注意 ✅ 观看影片 Git 教学系列 - Patch Add and Amend,掌握 git add -p、git checkout -p、git add ---amend 的用法,使用 add 和 checkout 时强烈建议使用 -p,掌握修改 commit 的两种方法。 ","date":"2023-12-27","objectID":"/posts/git-learn/:4:0","tags":["Git","GitHub"],"title":"Git 学习记录","uri":"/posts/git-learn/"},{"categories":["Toolkit","Linux Kernel Internals"],"content":"Only Add Related git add -p 推荐尽量使用这个 git add -p 而不是单纯的 git add。 使用 git add -p 后,Git 会帮我们把涉及的修改分成 section,然后我们就可以对每一个 section 涉及的修改进行 review,选择 y(yes) 表示采纳该 sction 对应的修改,选择 n(no) 表示不采纳。 如果觉得 section 切割的粒度太大了,可以选择 s(split) 来进行更细粒度的划分。如果这样仍然觉得粒度不够,可以选择 e(edit) 对 section 涉及的修改,进行以行为粒度的 review,具体操作可以查阅此时给出的提示。 还有一些其它的选项,比如 j、J、k、K,这些是类似 vim,用于切换进行 review 的 section,不太常用。q(quit) 表示退出。 由于可以针对一个文件的不同 section 进行 review,所以在进行 git add -p 之后,使用 git status 可以发现同一个文件会同时处于两种状态。 ","date":"2023-12-27","objectID":"/posts/git-learn/:4:1","tags":["Git","GitHub"],"title":"Git 学习记录","uri":"/posts/git-learn/"},{"categories":["Toolkit","Linux Kernel Internals"],"content":"Checkout Also git checkout -p 这个操作比较危险,因为这个操作的效果与 git add -p 相反,如果选择 y 的话,文件涉及的修改就会消失,如果涉及的修改没有 commit 的话,那么涉及的修改是无法救回的。但是怎么说,这个操作还是比直接使用 git checkout 稍微保险一点,因为会先进入 review 界面,而不是直接撤销修改。所以,请一定要使用 git checkout -p! ","date":"2023-12-27","objectID":"/posts/git-learn/:4:2","tags":["Git","GitHub"],"title":"Git 学习记录","uri":"/posts/git-learn/"},{"categories":["Toolkit","Linux Kernel Internals"],"content":"Modify Commit 有两种方式来修改最新的 commit: # 1. Use git commit --amend git commit --amend # 2. Use reset HEAD^ then re-commit git reset HEAD^ git add -p git commit git commit --amend 并不是直接替换原有的 commit,而是创建了一个新的 commit 并重新设置了 HEAD 的指向。所以,新旧两个 commit 的 Hash Value 并不相同,事实上,如果你拥有旧 commit 的 Hash Value,是可以通过 git checkout \u003ccommit\u003e 切换到那个 commit 的。其原理如下图: 但是注意,git reset HEAD^ 是会撤销原先的 commit(仅限于本地 Git 存储库)。 ","date":"2023-12-27","objectID":"/posts/git-learn/:4:3","tags":["Git","GitHub"],"title":"Git 学习记录","uri":"/posts/git-learn/"},{"categories":["Toolkit","Linux Kernel Internals"],"content":"Branch and Merge 注意 ✅ 观看影片 Git 教学系列 - Branch and Merge,掌握创建、删除、切换分支的用法,掌握合并分支、解决冲突的方法。 git checkout \u003ccommit\u003e git branch \u003cname\u003e git branch \u003cname\u003e \u003ccommit\u003e git branch [-d|-D] \u003cname\u003e git merge \u003cname\u003e --no-ff ","date":"2023-12-27","objectID":"/posts/git-learn/:5:0","tags":["Git","GitHub"],"title":"Git 学习记录","uri":"/posts/git-learn/"},{"categories":["Toolkit","Linux Kernel Internals"],"content":"Move and Create Branch Checkout: move HEAD git checkout \u003ccommit\u003e: Move HEAD to commit git checkout \u003cpath\u003e: WARNING: discard change 可以将路径上的文件复原到之前 commit 的状态。 Branch: git branch: List branch git branch \u003cname\u003e: Create branch Or just: git checkout -b Examples 修改一个文件并恢复: # modify file load.cpp git status git checkout load.cpp git status 删除一个文件并恢复: rm load.cpp git status git checkout load.cpp git status 正如上一节所说的,git checkout 尽量带上 -p 参数,因为如果一不小心输入了 git checkout .,那就前功尽弃了。 显示分支: # only show name git branch # show more infomation git branch -v 切换分支: # switch to branch 'main' git checkout main 创建分支: # 1. using `git branch` git branch cload # 2. using `git checkout -b` git checkout -b asmload # 3. create a new branch in \u003ccommit\u003e git branch cload \u003ccommit\u003e 切换到任一 commit: git checkout \u003ccommit\u003e 直接 checkout 到任一 commit 会有警告,这是因为,当你以该 commit 为基点进行一系列的 commit,这些新的 commit 会在你切换分支后消失,因为没有 branch 来引用它们。之前可以被引用是因为 HEAD 引用,切换分支后 HEAD 不再引用这些 commit,所以就会消失。在这种情况,Git 会在发出警告的同时建议我们使用 git branch 来创建分支进行引用。 ","date":"2023-12-27","objectID":"/posts/git-learn/:5:1","tags":["Git","GitHub"],"title":"Git 学习记录","uri":"/posts/git-learn/"},{"categories":["Toolkit","Linux Kernel Internals"],"content":"View Branch 列出仓库的所有分支: git branch 也可以通过 log 来查看分支: git log --decorate: 在 log 的首行显示所有的 references(可能需要通过 git config log.decorate auto 来开启) --graph: 以图形化的方式显示 branch 的关系(主要是 commit 的引用) ","date":"2023-12-27","objectID":"/posts/git-learn/:5:2","tags":["Git","GitHub"],"title":"Git 学习记录","uri":"/posts/git-learn/"},{"categories":["Toolkit","Linux Kernel Internals"],"content":"Delete Branch 删除分支: git branch -d \u003cname\u003e 对于有没有 merge 的 commit 的分支,Git 会警告,需要使用 -D 来强制删除: git branch -D \u003cname\u003e for no-merge commit WARNING: Discard Commit Git 会发出警告的原因同样是 no-merge commit 在删除分支后就无法被引用,所以会发出警告。 ","date":"2023-12-27","objectID":"/posts/git-learn/:5:3","tags":["Git","GitHub"],"title":"Git 学习记录","uri":"/posts/git-learn/"},{"categories":["Toolkit","Linux Kernel Internals"],"content":"Merge 合并分支。默认使用 fast-forward,即如果没有冲突,直接将要合并的分支提前到被合并分支的 commit 处,而不会另外生成一个 merge commit。但这样会使得被合并的分支在合并后,没有历史痕迹。可以通过 --no-ff (no fast forward) 来强制生成 merge commit。推荐使用 merge 时加上 --no-ff 这个参数。 git merge \u003cbranch\u003e 通常是 main/master 这类主分支合并其它分支: git checkout main/master git merge \u003cbranch\u003e ","date":"2023-12-27","objectID":"/posts/git-learn/:5:4","tags":["Git","GitHub"],"title":"Git 学习记录","uri":"/posts/git-learn/"},{"categories":["Toolkit","Linux Kernel Internals"],"content":"Resolve Conflict Manually resolve: Check every codes between \u003c\u003c\u003c\u003c\u003c\u003c\u003c, \u003e\u003e\u003e\u003e\u003e\u003e\u003e Edit code to what it should be Use mergetool like vimdiff: It shows: local, base, remote, file to be edited Edit “file ro be edited” to what is should be Add and Commit # 1. 合并分支 git merge \u003cbranch\u003e # 2. 检查状态,查看 unmerged 的文件 git status # 3. 编辑 unmerged 文件,编辑冲突区域代码即可 vim \u003cfile\u003e # 4. 添加解决完冲突的文件 git add \u003cfile\u003e # 5. 进行 merge commit git commit 冲突区域就是 \u003c\u003c\u003c\u003c\u003c\u003c\u003c 和 \u003e\u003e\u003e\u003e\u003e\u003e\u003e 内的区域,在 merge 操作后,Git 已经帮我们把 unmerged 文件修改为待解决冲突的状态,直接编辑文件即可。在编辑完成后,需要手动进行 add 和 commit,此次 commit 的信息 Git 已经帮我们写好了,一般不需要修改。 如果使用的是 mergetool,以 vimdiff 为例,只需将第 3 步的 vim \u003cfile\u003e 改为 git mergetool 即可。vimdiff 会提供 4 个视窗:底部视窗是我们的编辑区,顶部左边是当前合并分支的状态,顶部中间是 base (合并分支和被合并的共同父节点) 的状态,顶部右边是 remote 的状态,按需要选择、编辑。 vimdiff 在编辑完后会保留 *.orig 的文件,这个文件是待解决冲突的文件副本。 ","date":"2023-12-27","objectID":"/posts/git-learn/:5:5","tags":["Git","GitHub"],"title":"Git 学习记录","uri":"/posts/git-learn/"},{"categories":["Toolkit","Linux Kernel Internals"],"content":"Merge Conflict Prevent very long development branch. Split source code clearly. ","date":"2023-12-27","objectID":"/posts/git-learn/:5:6","tags":["Git","GitHub"],"title":"Git 学习记录","uri":"/posts/git-learn/"},{"categories":["Toolkit","Linux Kernel Internals"],"content":"Rebase 注意 ✅ 观看影片 Git 教学系列 - Branch and Merge,掌握 TODO 的方法。git rebase 是 Git 的精华,可以让我们实现更细粒度的操作,可以说学会了 rebase 才算真正入门了 Git。 这个视频讲得比较乱,所以推荐配合视频给出的参考文章 Git-rebase 小笔记 来学习。 ","date":"2023-12-27","objectID":"/posts/git-learn/:6:0","tags":["Git","GitHub"],"title":"Git 学习记录","uri":"/posts/git-learn/"},{"categories":["Toolkit"],"content":"网络代理 根据项目 clash-for-linux-backup 来配置 Ubuntu 的网络代理。 $ git clone https://github.com/Elegybackup/clash-for-linux-backup.git clash-for-linux 过程当中可能需要安装 curl 和 net-tools,根据提示进行安装即可: sudo apt install curl sudo apt install net-tools 安装并启动完成后,可以通过 localhost:9090/ui 来访问 Dashboard。 启动代理: $ cd clash-for-linux $ sudo bash start.sh $ source /etc/profile.d/clash.sh $ proxy_on 关闭代理: $ cd clash-for-linux $ sudo bash shutdown.sh $ proxy_off ","date":"2023-12-27","objectID":"/posts/ubuntu22.04lts/:1:0","tags":["Linux","Ubuntu"],"title":"Ubuntu 22.04LTS 相关配置","uri":"/posts/ubuntu22.04lts/"},{"categories":["Toolkit"],"content":"搜狗输入法 根据 搜狗输入法 Linux 安装指导 来安装搜狗输入法。 安装时无需卸载系统 ibus 输入法框架 (与上面的安装指导不一致) 通过 Ctrl + space 唤醒搜狗输入法 通过 Ctrl + Shift + Z 呼出特殊符号表 ","date":"2023-12-27","objectID":"/posts/ubuntu22.04lts/:2:0","tags":["Linux","Ubuntu"],"title":"Ubuntu 22.04LTS 相关配置","uri":"/posts/ubuntu22.04lts/"},{"categories":["Toolkit"],"content":"快捷键 新建终端: Ctrl + Alt + T 锁屏: Super + L:锁定屏幕并熄屏。 显示桌面: Super + d 或者 Ctrl + Alt + d 最小化所有运行的窗口并显示桌面,再次键入则重新打开之前的窗口。 显示所有的应用程序: Super + a 可以通过 ESC 来退出该显示。 显示当前运行的所有应用程序: Super 移动窗口位置: Super + 左箭头:当前窗口移动到屏幕左半边区域 Super + 右箭头:当前窗口移动到屏幕右半边区域 Super + 上箭头:当前窗口最大化 Super + 下箭头:当前窗口恢复正常 隐藏当前窗口到任务栏: Super + h 切换当前的应用程序: Super + Tab:以应用程序为粒度显示切换选项 Alt + Tab:以窗口为粒度显示切换选项 切换虚拟桌面/工作区: Ctrl + Alt + 左/右方向键 自定义键盘快捷键: Settings -\u003e Keyboard -\u003e Keyboard Shortcus | View and Customize Shortcuts -\u003e Custom Shortcuts ","date":"2023-12-27","objectID":"/posts/ubuntu22.04lts/:3:0","tags":["Linux","Ubuntu"],"title":"Ubuntu 22.04LTS 相关配置","uri":"/posts/ubuntu22.04lts/"},{"categories":["Toolkit","Linux Kernel Internals"],"content":" 摘要 GNU/Linux 开发工具,几乎从硬件到软件,Linux 平台能够自下而上提供各类触及“灵魂”的学习案例,让所有课程从纸上谈兵转变成沙场实战,会极大地提升工程实践的效率和技能。 原文地址 ","date":"2023-12-25","objectID":"/posts/gnu-linux-dev/:0:0","tags":["Sysprog","Linux"],"title":"GNU/Linux 开发工具","uri":"/posts/gnu-linux-dev/"},{"categories":["Toolkit","Linux Kernel Internals"],"content":"安装 Windows / Ubuntu 双系统 因为有些操作必须在物理硬件上才能执行。 ","date":"2023-12-25","objectID":"/posts/gnu-linux-dev/:1:0","tags":["Sysprog","Linux"],"title":"GNU/Linux 开发工具","uri":"/posts/gnu-linux-dev/"},{"categories":["Toolkit","Linux Kernel Internals"],"content":"Markdown 与 LaTeX 速览 LaTeX 语法示例一节,作为工具书册,在需要使用时知道如何查询。 速览 Markdown 语法示例一节,作为工具书册,在需要使用时知道如何查询。 注意 编写 Markdown 文本以及 LaTeX 语法表示的数学式可以通过: Hugo + FixIt ✅ VS Code + Markdown Preview Enhanced ","date":"2023-12-25","objectID":"/posts/gnu-linux-dev/:2:0","tags":["Sysprog","Linux"],"title":"GNU/Linux 开发工具","uri":"/posts/gnu-linux-dev/"},{"categories":["Toolkit","Linux Kernel Internals"],"content":"Git 和 Github 阅读 SSH key 产生方法一节,配置好 Git 和 Github 的 SSH key。同时也可作为工具书册,在需要使用时知道如何查询。 推荐通过 LearnGitBranching 来熟悉 Git 命令!!! 以下资源作为自学资源,用于补充自己不熟悉的操作,或者作为以上资料的补充工具手册。 Git 中文教学 - YouTube (学习记录) 30 天精通 Git 版本控制 - GitHub 警告 原文档中的将公钥复制到 clipboard 中使用了 clip 命令,但是这个命令在 Ubuntu 中并没有对应的命令。可以使用 xclip + alias 达到近似效果。 $ sudo apt install xclip # using alias to implement clip, you can add this to bashrc $ alias clip='xclip -sel c' $ clip \u003c ~/.ssh/id_rsa.pub ","date":"2023-12-25","objectID":"/posts/gnu-linux-dev/:3:0","tags":["Sysprog","Linux"],"title":"GNU/Linux 开发工具","uri":"/posts/gnu-linux-dev/"},{"categories":["Toolkit","Linux Kernel Internals"],"content":"编辑器: Visual Studio Code 认真阅读,跟随教学文档进行安装、设置。重点阅读 设定、除错(调试) 这两部分。更新 VS Code 部分作为手册,在需要时进行参考。 以下资源作为自学资源,用于补充自己不熟悉的操作,或者作为以上资料的补充工具手册。 开开心心学 Vistual Studio Code 完成 SSH key 的生成。 完成 VS Code 的设置。 安装 Git History 插件。 安装 Native Debug 插件,并进行 Debug (test-stopwatch.c) 操作。 安装 VSCode Great Icons 文件图标主题,另外推荐两款颜色主题:One Dark Pro, Learn with Sumit。 VS Code 控制台使用说明: 可以在面板的输出,点击 GIT 选项显示 VS Code 背后执行的 git 命令。 可以使用 ctrl + shift + P 呼出命令区,然后通过输入 Git branch 和 Git checkout 等并选择对应选项,来达到创建分支、切换分支等功能。 技巧 在 VS Code 设置中,需要在设置中打开 Open Default Settings 选项才能在左侧面板观察到预设值。键位绑定同理。 要想进行调试,需要在使用 gcc 生成目标文件时,加入 -g 参数来生产调试信息。 原文档中的 GDB 教学链接-除错程式-gdb 已失效,这是目前的有效链接。也可通过该影片 拯救资工系学生的基本素养-使用 GDB 除错基本教学 来补充学习 GDB 的操作。 ","date":"2023-12-25","objectID":"/posts/gnu-linux-dev/:4:0","tags":["Sysprog","Linux"],"title":"GNU/Linux 开发工具","uri":"/posts/gnu-linux-dev/"},{"categories":["Toolkit","Linux Kernel Internals"],"content":"终端和 Vim 认真阅读,跟随教学影片 快快乐乐学 Vim 和教学文档配置好 终端提示符、Vim。 完成命令行提示符配置 完成 Vim 的设定 安装并使用 Minial Vim Plugin Manager 来管理 Vim 插件 (neocomplcache, nerdtree) 安装并使用 byobu 来管理多个终端视图。 技巧 在 .vimrc 中增加插件后,打开 vim,执行 :PlugInstall 来安装插件,完成后在 vim 执行 :source ~/.vimrc。(可以通过 :PlugStatus 来查看插件安装状态) 使用 F4 键来[显示/不显示][行数/相对行数]。 使用 F5 键来呼入/呼出文件树(nerdtree),在文件树恻通过 ENTER 键来访问目录/文件。 使用 Ctrl-w-h/Ctrl-w-l 切换到 文件树/编辑区。 自动补全时使用 ENTER 键来选中,使用方向键或 Ctrl-N/Ctrl-U/Ctrl-P 来上下选择。 在 Vim 中可以通过 :set paste,并在 insert 模式下,将粘贴板的内容通过 Ctrl-Shift-V 进行粘贴。 byobu 使用说明: 在终端输入 byobu F2 新增 Terminial 分页。F3, F4 在 Terminial 分页中切换。Ctrl +F6 删除当前 Terminial 分页。 Shift + F2 水平切割 Terminial。Ctrl +F2 垂直切割 Terminial。Shift + 方向键 切换。 在 byobu 中暂时无法使用之前设置的 F4 或 F5 快捷键,但是可以直接通过命令 :set norelative 来关闭相对行数。 推荐观看影片 How to Do 90% of What Plugins Do (With Just Vim) 来扩展 Vim 插件的使用姿势。 以下资源为 Cheat Sheet,需要使用时回来参考即可。 Vim Cheat Sheet Bash terminal Cheat Sheet ","date":"2023-12-25","objectID":"/posts/gnu-linux-dev/:5:0","tags":["Sysprog","Linux"],"title":"GNU/Linux 开发工具","uri":"/posts/gnu-linux-dev/"},{"categories":["Toolkit","Linux Kernel Internals"],"content":"Makefile 速览教学文档,作为工具书册,在需要使用时知道如何查询。 gcc 的 -MMD 和 -MF 参数对我们编写 Makefile 是一个巨大利器。理解 Makefile 的各种变量定义的原理。 对之前的 test-stopwatch.c 编写了一个 Makefile 来自动化管理。 ","date":"2023-12-25","objectID":"/posts/gnu-linux-dev/:6:0","tags":["Sysprog","Linux"],"title":"GNU/Linux 开发工具","uri":"/posts/gnu-linux-dev/"},{"categories":["Toolkit","Linux Kernel Internals"],"content":"Linux 性能分析工具: Perf 认真阅读,复现教学文档中的所有例子,初步体验 perf 在性能分析上的强大。 安装 perf 并将 kernel.perf_event_paranoid 设置为 1。 动手使用 perf_top_example.c,体验 perf 的作用。 搭配影片: Branch Prediction 对照阅读: Fast and slow if-statements: branch prediction in modern processors 编译器提供的辅助机制: Branch Patterns, Using GCC 动手使用 perf_top_while.c,体验 perf top 的作用。 动手使用 perf_stat_cache_miss.c,体验 perf stat 的作用。(原文的结果有些不直观,务必亲自动手验证) 动手使用 perf_record_example.c,体验 perf record 的作用。(原文的操作不是很详细,可以参考下面的 Success) Source 成功 $ perf record -e branch-misses:u,branch-instructions:u ./perf_record_example [ perf record: Woken up 1 times to write data ] [ perf record: Captured and wrote 0.009 MB perf.data (94 samples) ] 输出第一行表示 perf 工具在收集性能数据时被唤醒了 1 次,以将数据写入输出文件。 输出第二行表示 perf 工具已经取样并写入了一个名为 perf.data 的二进制文件,文件大小为 0.009 MB,其中包含了 94 个采样。(可以通过 ls 命令来检查 perf.data 文件是否存在) 接下来通过 perf report 对之前输出的二进制文件 perf.data 进行分析。可以通过方向键选择,并通过 ENTER 进入下一层查看分析结果。 $ perf report Available samples 5 branch-misses:u 89 branch-instructions:u 技巧 perf 需要在 root 下进行性能分析。 perf top 是对于哪个程序是性能瓶颈没有头绪时使用,可以查看哪个程序(以及程序的哪个部分)是热度点。 在 perf top 时可以通过 h 键呼出帮助列表。 可以通过方向键选择需要进一步分析的部分,并通过 a 键来查看指令级别粒度的热点。 perf stat 是对某一个要优化的程序进行性能分析,对该程序涉及的一系列 events 进行取样检查。 perf record 的精度比 perf stat 更高,可以对取样的 events 进行函数粒度的分析。 ","date":"2023-12-25","objectID":"/posts/gnu-linux-dev/:7:0","tags":["Sysprog","Linux"],"title":"GNU/Linux 开发工具","uri":"/posts/gnu-linux-dev/"},{"categories":["Toolkit","Linux Kernel Internals"],"content":"Linux 绘图工具: gnuplot 阅读教程,搭配教学影片 轻轻松松学 gnuplot,使用 gnuplot 完成所给例子相应图像的绘制。 使用 runtime.gp 完成 runtime.png 的绘制生成。 使用 statistic.gp 完成降雨量折线图 statistic.png 的绘制生成。 注意 原文所给的 statistic.gp 是使用 Times_New_Roman 来显示中文的,但笔者的 Ubuntu 中并没有这个字体,所以会显示乱码。可以通过 fc-list :lang=zh 命令来查询当前系统中的已安装的中文字体。 Source 安装 gnuplot: $ sudo apt-get install gnuplot gnuplot script 的使用流程: # 创建并编写一个后缀名为 .gp 的文件 $ vim script.gp # 根据 script 内的指令进行绘图 $ gnuplot script.gp # 根据 script 指定的图片保存路径打开图片 $ eog [name of picture] 下面以一个 script 进行常用指令的说明: reset set ylabel 'time(sec)' set style fill solid set title 'performance comparison' set term png enhanced font 'Verdana,10' set output 'runtime.png' plot [:][:0.100]'output.txt' using 2:xtic(1) with histogram title 'original', \\ '' using ($0-0.06):($2+0.001):2 with labels title ' ', \\ '' using 3:xtic(1) with histogram title 'optimized' , \\ '' using 4:xtic(1) with histogram title 'hash' , \\ '' using ($0+0.3):($3+0.0015):3 with labels title ' ', \\ '' using ($0+0.4):($4+0.0015):4 with labels title ' ' reset 指令的作用为,将之前 set 指令设置过的内容全部重置。 set style fill solid 将绘制出的柱形或区域使用实心方式填充。 set term png enhanced font 'Verdana,10' term png 生成的图像以 png 格式进行保存。(term 是 terminial 的缩写) enhanced 启用增强文本模式,允许在标签和注释中使用特殊的文本格式,如上下标、斜体、下划线等。 font 'Verdana,10' 指定所使用的字体为 Verdana,字号为10。可进行自定义设置。 其它指令查询原文或手册即可。 $0 在 gnuplot 中表示伪列,可以简单理解为行号,以下为相应图示: 原始数据集: append() 0.048240 0.040298 0.057908 findName() 0.006495 0.002938 0.000001 (人为)增加了 伪列 表示的数据集(最左边 0, 1 即为伪列): 0 append() 0.048240 0.040298 0.057908 1 findName() 0.006495 0.002938 0.000001 技巧 gnuplot 在绘制生成图像时是安装指令的顺序进行的,并且和一般的画图软件类似,在最上层进行绘制。所以在编写 script 的指令时需要注意顺序,否则生成图像的部分可能并不像预期一样位于最上层。(思考上面 script 的 3, 4 列的 label 的绘制顺序) gnuplot script 中的字符串可以使用 '' 或者 \"\" 来包裹,同样类似于 Python。 直接在终端输入 gnuplot 会进入交互式的命令界面,也可以使用 gnulpot 指令来绘图(类似与 Python)。在这种交互式界面环境中,如果需要在输入完指令后立即显示图像到新窗口,而不是保存图像再打开,只需输入进行指令: set term wxt ehanced persist raise term wxt 将图形终端类型设置为WXT,这会在新窗口中显示绘图。 ersist 该选项使绘图窗口保持打开状态,即使脚本执行完毕也不会自动关闭。 raise 该选项将绘图窗口置于其他窗口的前面,以确保它在屏幕上的可见性。 一些额外的教程: Youtube - gnuplot Tutorlal 这个教程有五部影片,到发布者的主页搜寻即可。 ","date":"2023-12-25","objectID":"/posts/gnu-linux-dev/:8:0","tags":["Sysprog","Linux"],"title":"GNU/Linux 开发工具","uri":"/posts/gnu-linux-dev/"},{"categories":["Toolkit","Linux Kernel Internals"],"content":"Linux 绘图工具: Graphviz 官方网站 一小时实践入门 Graphviz 安装: $ sudo apt install graphviz 查看安装版本: $ dot -V dot - graphviz version 2.43.0 (0) 通过脚本生成图像: $ dot -Tpng example.dot -o example.png Graphviz 在每周测验题的原理解释分析时会大量使用到,请务必掌握以实作出 Readable 的报告。 ","date":"2023-12-25","objectID":"/posts/gnu-linux-dev/:9:0","tags":["Sysprog","Linux"],"title":"GNU/Linux 开发工具","uri":"/posts/gnu-linux-dev/"},{"categories":["Toolkit","Linux Kernel Internals"],"content":"其它工具 ","date":"2023-12-25","objectID":"/posts/gnu-linux-dev/:10:0","tags":["Sysprog","Linux"],"title":"GNU/Linux 开发工具","uri":"/posts/gnu-linux-dev/"},{"categories":["Toolkit","Linux Kernel Internals"],"content":"man $ man man The table below shows the section numbers of the manual followed by the types of pages they contain. 1 Executable programs or shell commands 2 System calls (functions provided by the kernel) 3 Library calls (functions within program libraries) 4 Special files (usually found in /dev) 5 File formats and conventions, e.g. /etc/passwd 6 Games 7 Miscellaneous (including macro packages and conventions), e.g. man(7), groff(7), man-pages(7) 8 System administration commands (usually only for root) 9 Kernel routines [Non standard] ","date":"2023-12-25","objectID":"/posts/gnu-linux-dev/:10:1","tags":["Sysprog","Linux"],"title":"GNU/Linux 开发工具","uri":"/posts/gnu-linux-dev/"},{"categories":["Toolkit","Linux Kernel Internals"],"content":"cloc man 1 cloc cloc - Count, or compute differences of, lines of source code and comments. ","date":"2023-12-25","objectID":"/posts/gnu-linux-dev/:10:2","tags":["Sysprog","Linux"],"title":"GNU/Linux 开发工具","uri":"/posts/gnu-linux-dev/"},{"categories":["Toolkit","Linux Kernel Internals"],"content":"top man 1 top top - display Linux processes ","date":"2023-12-25","objectID":"/posts/gnu-linux-dev/:10:3","tags":["Sysprog","Linux"],"title":"GNU/Linux 开发工具","uri":"/posts/gnu-linux-dev/"},{"categories":["Toolkit","Linux Kernel Internals"],"content":"htop man 1 htop htop - interactive process viewer ","date":"2023-12-25","objectID":"/posts/gnu-linux-dev/:10:4","tags":["Sysprog","Linux"],"title":"GNU/Linux 开发工具","uri":"/posts/gnu-linux-dev/"},{"categories":["Mathematics"],"content":"这里记录一些收集到的数学开放式课程学校资料。 中大數學系開放式課程 國立台灣大學 齊震宇 數學導論:相關講義 | 教學錄影 數學潛水艇:綫性代數、拓撲 微積分 分析 國立台灣大學 謝銘倫 綫性代數 ","date":"2023-12-23","objectID":"/posts/math/:0:0","tags":["Mathematics"],"title":"数学开放式课程学习指引","uri":"/posts/math/"},{"categories":["Mathematics"],"content":"逻辑、集合论 ","date":"2023-12-23","objectID":"/posts/math/:1:0","tags":["Mathematics"],"title":"数学开放式课程学习指引","uri":"/posts/math/"},{"categories":["Mathematics"],"content":"簡易邏輯 在联结词构成的复合命题中,命题的语义和真值需要分开考虑,特别是联结词 $\\implies$。以 $p \\implies q$ 为例 (注意 $p$ 和 $q$ 都是抽象命题,可指代任意命题,类似于未知数 $x$),如果从 $p$ 和 $q$ 的语义考虑,很容易就陷入语义的 “如果 $p$ 则 $q$” 这类语义混淆中,导致强加因果,但因为在逻辑上 $p$ 和 $q$ 可以没有任何关系,所以此时忽略它们的语义,而只根据它们的真值和相关定义上推断该复合命题的真值 ($p \\implies q$ 等价于 $(\\neg p) \\lor q$)。简单来说,逻辑上的蕴涵式包括语义上的因果关系,即因果关系是蕴涵式的真子集。 第 16 页的趣味问题可以通过以下 “标准” 方式 (这里的 “标准” 指的是一种通用方法思路,并非应试教育中的得分点) 来解决: 令悟空、八戒、悟净和龙马分别为 $a, b, c, d$,令命题「$X$ 第 $Y$」为 $XY$,例如 “悟空第一” 则表示为命题 $a1$,则有以下命题成立: $$ \\begin{split} \u0026 (c1 \\land (\\neg b2)) \\lor ((\\neg c1) \\land b2) \\\\ \\land\\ \u0026 (c2 \\land (\\neg d3)) \\lor ((\\neg c2) \\land d3) \\\\ \\land\\ \u0026 (d4 \\land (\\neg a2)) \\lor ((\\neg d4) \\land a2) \\\\ \\end{split} $$ 然后化简该表达式即可得到结果 (因为这个问题是精心设计过的,所以会有一个唯一解)。 性质也是命题,即其真假值可以谈论 (但未必能确定)。但正如第 22 页的注一所说,一般不讨论性质 $A(x)$ 的真假值,只有将具体成代入时才有可能谈论其真假值 (这很好理解,抽象的不定元 $x$ 可以代表无限多的东西,代入性质 $A(x)$ 的真假值可能并不相同)。但是需要注意后面集合论中虽然也使用了性质来定义群体,但是此时的性质表示 $A(x)$ 这个命题为真,即 $x$ 满足 $A$ 这个性质。所以需要细心看待逻辑学和集合论的性质一次,它们有共同点也有不同点。 处理更多不定元的性质时,按照第 24 ~ 27 页的注二的方法,将其转换成简单形式的单一不定元的性质进行处理。 ","date":"2023-12-23","objectID":"/posts/math/:1:1","tags":["Mathematics"],"title":"数学开放式课程学习指引","uri":"/posts/math/"},{"categories":["Mathematics"],"content":"集合概念簡介 第 42 页上运用了类似的化简不定元技巧,通过括号将同一时间处理不定元数量减少为 1.除此之外,这里还有一个不等式和区间/射线符号的技巧: 不带等号的不等式和区间/射线符号搭配使用时,需要反转区间/射线符号,这个技巧可以从区间/射线符号的定义推导而来。以该页最后的例子为例: $$ (\\bigcup_{m \\in (0,1)}(1-\\frac{1}{k}, 9-m]) \\land (8 \u003c 9-m \u003c 9) \\\\ \\begin{split} (1-\\frac{1}{k}, 9-m] \u0026= \\{x \\in \\mathbb{R} | 1-\\frac{1}{k} \u003c x \u003c= 9-m \u003c 9 \\} \\\\ \u0026= \\{x \\in \\mathbb{R} | 1-\\frac{1}{k} \u003c x \u003c 9 \\} \\\\ \u0026= (1-\\frac{1}{k}, 9) \\\\ \\end{split} $$ 倒数第二个例子也类似: $$ (\\bigcap_{j \\in \\mathbb{R},\\ j\u003e0}(-j, 9)) \\land (-j\u003c0) \\\\ \\begin{split} (-j, 9) \u0026= \\{x \\in \\mathbb{R} | -j \u003c 0 \u003c= x \u003c 9 \\} \\\\ \u0026= \\{x \\in \\mathbb{R} | 0 \u003c= x \u003c 9 \\} \\\\ \u0026= [0, 9) \\\\ \\end{split} $$ 第 45 ~ 46 页分别展示了例行公事式的证明和受过教育似的证明这两种方法,需要注意的是第一钟方法使用的是 逻辑上等价 进行推导 (因为它一次推导即可证明两个集合等价),而第二种方法使用的是 蕴涵 进行推导 (因为它是通过两个方向分别证明包含关系,不需要等价性推导)。例行公事式的证明的要点在于,事先说明后续证明涉及的元素 $x$ 的性质,然后在后续证明过程中某一步将这个性质加入,进而构造出另一个集合的形式。 练习一: 例行公事式的证明 练习二: 例行公事式的证明 ","date":"2023-12-23","objectID":"/posts/math/:1:2","tags":["Mathematics"],"title":"数学开放式课程学习指引","uri":"/posts/math/"},{"categories":["Mathematics"],"content":"初等整數論 第 13 页 (反转了的) 辗转相除法的阅读顺序是:先阅读左边,在阅读右边,右边的推导是将上面的式子代入到下面的式子得来。 ","date":"2023-12-23","objectID":"/posts/math/:2:0","tags":["Mathematics"],"title":"数学开放式课程学习指引","uri":"/posts/math/"},{"categories":["Mathematics"],"content":"群、群作用與 Burnside 引理 第 16 页的 $Perm(X)$ 表示 $X$ 的元素进行置换对应的所有映射构成的集合,这个集合的基数为 $8!$。表示这个集合的某个元素 (也就是置换对应的映射),可以用投影片上的形如 $(1\\ 4\\ 2)(3\\ 7)(5)(6)$ 来表示,比较直观的体现这个映射的效果。 第 18 页子群定义的结合律一般不需要特别考虑,因为子群的任意元素属于群,而群的元素都满足结合律,所以子群的任意元素都满足结合律。 第 18 页的证明提示「消去律」,是指在证明子群性质时利用群的 可逆 和 单位元 性质进行证明。因为依据定义,群的单位元可作用的范围比子群的单位元作用范围广。 ","date":"2023-12-23","objectID":"/posts/math/:3:0","tags":["Mathematics"],"title":"数学开放式课程学习指引","uri":"/posts/math/"},{"categories":["Mathematics"],"content":"綫性代數","date":"2023-12-23","objectID":"/posts/math/:4:0","tags":["Mathematics"],"title":"数学开放式课程学习指引","uri":"/posts/math/"},{"categories":["draft"],"content":"博客(英语:Blog)是一种在线日记型式的个人网站,借由张帖子章、图片或视频来记录生活、抒发情感或分享信息。博客上的文章通常根据张贴时间,以倒序方式由新到旧排列。 ","date":"2023-12-23","objectID":"/posts/hello_world/:0:0","tags":["draft"],"title":"Hello, World","uri":"/posts/hello_world/"},{"categories":["draft"],"content":"数学公式 行内公式:$N(b,d)=(b-1)M$ 公式块: $$ \\int_{a}^{b}x(t)dt = \\dfrac{b - a}{N} \\\\ =\\sum_{k=1}^{N}x(t_k)\\cdot\\dfrac{b-a}{N} $$ $$ \\begin{aligned} \\int_{a}^{b}x(t)dt \u0026= \\dfrac{b - a}{N} \\\\ \u0026=\\sum_{k=1}^{N}x(t_k)\\cdot\\dfrac{b-a}{N} \\\\ \\end{aligned} $$ $$ \\mathrm{Integrals\\ are\\ numerically\\ approximated\\ as\\ finite\\ series}:\\\\ \\begin{split} \\int_{a}^{b}x(t)dt \u0026= \\dfrac{b - a}{N} \\\\ \u0026=\\sum_{k=1}^{N}x(t_k)\\cdot\\dfrac{b-a}{N} \\end{split} \\\\ where\\ t_k = a + (b-a)\\cdot k/N $$ $$ \\begin{align*} p(x) = 3x^6 + 14x^5y \u0026+ 590x^4y^2 + 19x^3y^3 \\\\ \u0026- 12x^2y^4 - 12xy^5 + 2y^6 - a^3b^3 - a^2b - ab + c^5d^3 + c^4d^3 - cd \\end{align*} $$ $$ \\begin{split} \u0026(X \\in B) = X^{-1}(B) = {s \\in S: X(s) \\in B} \\subset S \\\\ \u0026\\Rightarrow P(x \\in B) = P({s \\in S: X(s) \\in B}) \\end{split} $$ ","date":"2023-12-23","objectID":"/posts/hello_world/:1:0","tags":["draft"],"title":"Hello, World","uri":"/posts/hello_world/"},{"categories":["draft"],"content":"代码块 let i: i32 = 13; let v = vec![1, 2, 3, 4, 5, 65]; for x in v.iter() { println!(\"{}\", x); } typedef struct Block_t { int head; int data; } Block_t; ","date":"2023-12-23","objectID":"/posts/hello_world/:2:0","tags":["draft"],"title":"Hello, World","uri":"/posts/hello_world/"},{"categories":["draft"],"content":"Admonition 注意 111年度資工所心得 摘要 Georgia Tech - Advanced Operating Systems: Part 1 / Part 2 / Part 3 / Part 4 信息 Reddit: Best book to learn in-depth knowledge about the Linux Kernel? Project: Linux From Scratch Book: Linux Kernel Development Video: Steven Rostedt - Learning the Linux Kernel with tracing 技巧 WIkipedia: Xenix / Multics / Plan9 / FreeBSD 成功 一个 成功 横幅 问题 一个 问题 横幅 警告 一个 警告 横幅 失败 一个 失败 横幅 危险 一个 危险 横幅 Bug 一个 Bug 横幅 示例 一个 示例 横幅 引用 一个 引用 横幅 ","date":"2023-12-23","objectID":"/posts/hello_world/:3:0","tags":["draft"],"title":"Hello, World","uri":"/posts/hello_world/"},{"categories":["draft"],"content":"References FixIt 快速上手 使用 Hugo + Github 搭建个人博客 Markdown 基本语法 Emoji 支持 扩展 Shortcodes 概述 图表支持 URL management ","date":"2023-12-23","objectID":"/posts/hello_world/:4:0","tags":["draft"],"title":"Hello, World","uri":"/posts/hello_world/"}] \ No newline at end of file +[{"categories":["draft"],"content":"This post is used to record the process of my English learning. ","date":"2024-03-30","objectID":"/posts/english/:0:0","tags":["draft"],"title":"English Everyday","uri":"/posts/english/"},{"categories":["draft"],"content":"Preface 工欲善其事,必先利其器 If I had eight hours to chop down a tree, I’d spend six hours sharpening my axe. 单词书: Merriam-Webster’s Vocabulary Builder 写作书: The Elements of Style 语法书: https://grammar.codeyu.com/ 发音教学: 一些 YouTube channels: https://www.youtube.com/@LearnEnglishWithTVSeries https://www.youtube.com/@letstalk https://www.youtube.com/@bbclearningenglish https://www.youtube.com/@coachshanesesl 一些 B 站 UP 主: 妈妈不用担心我的英语 英语兔 一些 GitHub 仓库: https://github.com/byoungd/English-level-up-tips https://github.com/xiaolai/everyone-can-use-english https://github.com/IammyselfBOOKS/New_concept_English https://github.com/protogenesis/NewConceptEnglish 仓库中关于新概念英语的网址,录音是正确的,但是有一些正文不太准确,可以下载书籍进行对比 ","date":"2024-03-30","objectID":"/posts/english/:1:0","tags":["draft"],"title":"English Everyday","uri":"/posts/english/"},{"categories":["draft"],"content":"New Concept English ","date":"2024-03-30","objectID":"/posts/english/:2:0","tags":["draft"],"title":"English Everyday","uri":"/posts/english/"},{"categories":["draft"],"content":"NCE 1 001: Excuse me! 003: Sorry, sir! 005: Nice to meet you! 007: Are you a teacher? 009: How are you today? 011: Is this your shirt? 013: A new dress 015: Your passports, please 017: How do you do 019: Tired and thirsty 021: Which book? 023: Which glasses? 025: Mrs. Smith’s kitchen 027: Mrs. Smith’s living room 029: Come in, Amy 031: Where’s Sally 033: A fine day 035: Our village 037: Making a bookcase 039: Don’t drop it! 041: Penny’s bag 043: Hurry up! 045: The boss’s letter 047: A cup of coffee 049: At the butcher’s 051: A pleasant climate 053: An interesting climate 055: The Sawyer family 057: An unusual day 059: Is that all? 063: Thank you, doctor. 065: Not a baby. 067: The weekend 069: The car race 071: He’s awful 073: The way to King Street 075: Uncomfortable shoes 077: Terrible toothache 079: Peggy’s shopping list 081: Roast beef and potato 083: Going on a holiday Source: Cambridge Dictionary handbag n. a small bag used by a woman to carry everyday personal items. umbrella n. a device consisting of a circular canopy of cloth on a folding metal frame supported by a central rod, used as protection against rain. nationality n. the official right to belong to a particular country. engineer n. a person whose job is to design or build machines, engines, or electrical equipment, or things such as roads, railways, or bridges, using scientific principles. perhaps adv. used to show that something is possible or that you are not certain about something. refrigerator n. a piece of kitchen equipment that uses electricity to preserve food at a cold temperature. armchair n. a comfortable chair with sides that support your arms. stereo n. wardrobe n. a tall cupboard in which you hang your clothes. dust v. to use a cloth to remove dust from the surface of something. sweep v. to clean something, especially a floor by using a brush to collect the dirt into one place from which it can be removed. aeroplane n. vase n. tobacco n. kettle n. cupboard n. beef n. lamb n. steak n. mince n. mild pad n. chalk n. greengrocer n. phrase n. jam n. grocer n. bear n. wine n. cinema n. ","date":"2024-03-30","objectID":"/posts/english/:2:1","tags":["draft"],"title":"English Everyday","uri":"/posts/english/"},{"categories":["Linux Kernel Internals"],"content":" 在「Linux 核心设计/实作」Spring 2023 课程进度页面的原始档案的基础上,稍作修改以记录我的学习进度 原始页面 | PDF 成功 如果你学习时感到挫折,感到进度推进很慢,这很正常,因为 Jserv 的一个讲座,需要我们花费一个星期去消化 🤣 并且 Jserv 也提到前 6 周课程的密度是比较大的 所以没必要为此焦虑,如果你觉得某个内容不太理解,可以尝试先去看其他讲座,将原先不懂的知识交给大脑隐式消化,过段时间再回来看,你的理解会大有不同。 Instructor: Jim Huang (黃敬群) \u003cjserv.tw@gmail.com\u003e 往年課程進度 Linux 核心設計 (線上講座) 注意: 下方課程進度表標註有 * 的項目,表示內附錄影的教材 注意: 新開的「Linux 核心實作」課程內容幾乎與「Linux 核心設計」一致,採線上為主的進行方式 ","date":"2024-02-28","objectID":"/posts/linux2023/:0:0","tags":["Sysprog","Linux"],"title":"Linux 核心设计/实作 (Linux Kernel Internals)","uri":"/posts/linux2023/"},{"categories":["Linux Kernel Internals"],"content":"Linux 核心設計/實作 (Spring 2023) 課程進度表暨線上資源 ","date":"2024-02-28","objectID":"/posts/linux2023/:1:0","tags":["Sysprog","Linux"],"title":"Linux 核心设计/实作 (Linux Kernel Internals)","uri":"/posts/linux2023/"},{"categories":["Linux Kernel Internals"],"content":"第 1 週: 誠實面對自己 (Feb 13, 14, 16) 教材解說* (僅止於概況,請詳閱下方教材及個別的對應解說錄影) 課程簡介和注意須知 / 課程簡介解說錄影* 每週均安排隨堂測驗,採計其中最高分的 9 次 學期評分方式: 隨堂測驗 (20%) + 個人作業+報告及專題 (30%) + 自我評分 (50%) 歷屆修課學生心得: 向景亘, 張家榮, 蕭奕凱, 方鈺學 分組報告示範: ARM-Linux, Xvisor GNU/Linux 開發工具共筆*: 務必 自主 學習 Linux 操作, Git, HackMD, LaTeX 語法 (特別是數學式), GNU make, perf, gnuplot 確認 Ubuntu Linux 22.04-LTS (或更新的版本) 已順利安裝到你的電腦中 透過 Computer Systems: A Programmer’s Perspective 學習系統軟體*: 本課程指定的教科書 (請及早購買: 天瓏書店) 軟體缺失導致的危害 1970 年代推出的首款廣體民航客機波音 747 軟體由大約 40 萬行程式碼構成,而 2011 年引進的波音 787 的軟體規模則是波音 747 的 16 倍,約 650 萬行程式碼。換言之,你我的性命緊繫於一系列極為複雜的軟體系統之中,能不花點時間了解嗎? 軟體開發的安全性設計和測試驗證應獲得更高的重視 The adoption of Rust in Business (2022) 搭配觀看短片: Rust in 100 Seconds 解讀計算機編碼 人們對數學的加減運算可輕易在腦中辨識符號並理解其結果,但電腦做任何事都受限於實體資料儲存及操作方式,換言之,電腦硬體實際只認得 0 和 1,卻不知道符號 + 和 - 在數學及應用場域的意義,於是工程人員引入「補數」以表達人們認知上的正負數 您有沒有想過,為何「二補數」(2’s complement) 被電腦廣泛採用呢?背後的設計考量是什麼?本文嘗試從數學觀點去解讀編碼背後的原理 你所不知道的 C 語言:指標篇* linked list 和非連續記憶體操作* 安排 linked list 作為第一份作業及隨堂測驗的考量點: 檢驗學員對於 C 語言指標操作的熟悉程度 (附帶思考:對於 Java 程式語言來說,該如何實作 linked list 呢?) linked list 本質上就是對非連續記憶體的操作,乍看僅是一種單純的資料結構,但對應的演算法變化多端,像是「如何偵測 linked list 是否存在環狀結構?」和「如何對 linked list 排序並確保空間複雜度為 O(1) 呢?」 linked list 的操作,例如走訪 (traverse) 所有節點,反映出 Locality of reference (cache 用語) 的表現和記憶體階層架構 (memory hierarchy) 高度相關,學員很容易從實驗得知系統的行為,從而思考其衝擊和效能改進方案 無論是作業系統核心、C 語言函式庫內部、應用程式框架,到應用程式,都不難見到 linked list 的身影,包含多種針對效能和安全議題所做的 linked list 變形,又還要考慮到應用程式的泛用性 (generic programming),是很好的進階題材 題目 1 + 分析* 題目2 / 參考題解1, 參考題解2 題目3 / 參考題解 題目4 / 參考題解 題目5 / 參考題解 佳句偶得:「大部分的人一輩子洞察力不彰,原因之一是怕講錯被笑。想了一點點就不敢繼續也沒記錄或分享,時間都花在讀書查資料看別人怎麼想。看完就真的沒有自己的洞察了」(出處) 作業: 截止繳交日: Feb 28, 2023 lab0* quiz1 第 1 週隨堂測驗: 題目 (內含作答表單) 課堂問答簡記 ","date":"2024-02-28","objectID":"/posts/linux2023/:1:1","tags":["Sysprog","Linux"],"title":"Linux 核心设计/实作 (Linux Kernel Internals)","uri":"/posts/linux2023/"},{"categories":["Linux Kernel Internals"],"content":"第 2 週: C 語言程式設計 (Feb 20, 21, 23) 教材解說* (僅止於概況,請詳閱下方教材及個別的對應解說錄影) Linux v6.2 發布: 接下來會是讓學員眼花撩亂的主版號/次版號的飛快跳躍 / kernel.org Linux: 作業系統術語及概念* 系統軟體開發思維 C 語言: 數值系統* 儘管數值系統並非 C 語言所特有,但在 Linux 核心大量存在 u8/u16/u32/u64 這樣透過 typedef 所定義的型態,伴隨著各式 alignment 存取,若學員對數值系統的認知不夠充分,可能立即就被阻擋在探索 Linux 核心之外 —— 畢竟你完全搞不清楚,為何在 Linux 核心存取特定資料需要繞一大圈。 C 語言: Bitwise 操作* Linux 核心原始程式碼存在大量 bit(-wise) operations (簡稱 bitops),頗多乍看像是魔法的 C 程式碼就是 bitops 的組合 類神經網路的 ReLU 及其常數時間複雜度實作 從 √2 的存在談開平方根的快速運算 Linux 核心的 hash table 實作 為什麼要深入學習 C 語言?* C 語言發明者 Dennis M. Ritchie 說:「C 很彆扭又缺陷重重,卻異常成功。固然有歷史的巧合推波助瀾,可也的確是因為它能滿足於系統軟體實作的程式語言期待:既有相當的效率來取代組合語言,又可充分達到抽象且流暢,能用於描述在多樣環境的演算法。」 Linux 核心作為世界上最成功的開放原始碼計畫,也是 C 語言在工程領域的瑰寶,裡頭充斥各式「藝術」,往往會嚇到初次接觸的人們,但總是能夠用 C 語言標準和開發工具提供的擴展 (主要來自 gcc 的 GNU extensions) 來解釋。 基於 C 語言標準研究與系統程式安全議題 藉由研讀漏洞程式碼及 C 語言標準,討論系統程式的安全議題 透過除錯器追蹤程式碼實際運行的狀況,了解其運作原理; 取材自 dangling pointer, CWE-416 Use After Free, CVE-2017-16943 以及 integer overflow 的議題; C 語言:記憶體管理、對齊及硬體特性* 搭配閱讀: The Lost Art of Structure Packing 從虛擬記憶體談起,歸納出現代銀行和虛擬記憶體兩者高度相似: malloc 給出 valid pointer 不要太高興,等你要開始用的時候搞不好作業系統給個 OOM ——簡單來說就是一張支票,能不能拿來開等到兌現才知道。 探討 heap (動態配置產生,系統會存放在另外一塊空間)、data alignment,和 malloc 實作機制等議題。這些都是理解 Linux 核心運作的關鍵概念。 C 語言: bit-field bit field 是 C 語言一個很被忽略的特徵,但在 Linux 和 gcc 這類系統軟體很常出現,不僅是精準規範每個 bit 的作用,甚至用來「擴充」C 語言 參考題目 / 參考題目* / 參考題解 1, 參考題解 2, 參考題解 3 作業: 截止繳交日 Mar 7 quiz2 第 2 週隨堂測驗: 題目 (內含作答表單) 課堂問答簡記 ","date":"2024-02-28","objectID":"/posts/linux2023/:1:2","tags":["Sysprog","Linux"],"title":"Linux 核心设计/实作 (Linux Kernel Internals)","uri":"/posts/linux2023/"},{"categories":["Linux Kernel Internals"],"content":"第 3 週: 並行和 C 語言程式設計 (Feb 27, 28, Mar 2) 教材解說* (僅止於概況,請詳閱下方教材及個別的對應解說錄影) 公告 2 月 28 日沒有實體課程,但安排線上測驗 (「Linux 核心設計」課程的學員務必參加),在 15:20-23:59 之間依據 Google Calendar 進行作答 第二次作業已指派,可在 2 月 28 日晚間起開始繳交,截止繳交日 Mar 7 3 月 1 日晚間安排第一次作業的檢討直播 (事後有錄影),請參見 Google Calendar Linux: 發展動態回顧* 從 Revolution OS 看作業系統生態變化* 並行和多執行緒程式設計*: 應涵蓋 Part 1 到 Part 4 Part 1: 概念、执行顺序 Part 2 Part 3 Part 4 C 語言: 函式呼叫* 著重在計算機架構對應的支援和行為分析 C 語言: 遞迴呼叫* 或許跟你想像中不同,Linux 核心的原始程式碼裡頭也用到遞迴函式呼叫,特別在較複雜的實作,例如檔案系統,善用遞迴可大幅縮減程式碼,但這也導致追蹤程式運作的難度大增 C 語言: 前置處理器應用* C 語言之所以不需要時常發佈新的語言特徵又可以保持活力,前置處理器 (preprocessor) 是很重要的因素,有心者可逕行「擴充」C 語言 C 語言: goto 和流程控制* goto 在 C 語言被某些人看做是妖魔般的存在,不過實在不用這樣看待,至少在 Linux 核心原始程式碼中,goto 是大量存在 (跟你想像中不同吧)。有時不用 goto 會寫出更可怕的程式碼 C 語言程式設計技巧* 作業: 截止繳交日: Mar 21 fibdrv* quiz3 review* Week3 隨堂測驗: 題目 (內含作答表單) ","date":"2024-02-28","objectID":"/posts/linux2023/:1:3","tags":["Sysprog","Linux"],"title":"Linux 核心设计/实作 (Linux Kernel Internals)","uri":"/posts/linux2023/"},{"categories":["Linux Kernel Internals"],"content":"第 4 週: 數值系統 + 編譯器 (Mar 6, 7, 9) 教材解說* (僅止於概況,請詳閱下方教材及個別的對應解說錄影) 公告: 請填寫 Google 表單,以利後續追蹤 《Demystifying the Linux CPU Scheduler》的書稿已寄送給成功大學的選課學生,旁聽的學員預計在 3 月 13 日取得 (第 5 週進度) 貢獻程式碼到 Linux 核心 第一次給 Linux Kernel 發 patch 提交第一份 Patch 到 Linux Kernel 第一次發 patch 到 LKML 追求神乎其技的程式設計之道 「可以看出抄襲風氣在台灣並不只是小時候在學校抄抄作業而已;媒體工作者在報導中任意抄襲及轉載是種不尊重自己專業的表現,不但隱含著一種應付了事的心態,更代表著這些人對於自己的工作沒有熱情,更沒有著一點堅持。如果要說我在美國看到這邊和台灣有什麼最大的不同,我想關鍵的差異就在對自己的工作有沒有熱情和堅持而已了。」 「程式藝術家也不過是在『簡潔』、『彈性』、『效率』這三大目標上進行一連串的取捨 (trade-off) 和最佳化。」 Linux 核心的紅黑樹 CS:APP 第 2 章重點提示和練習* 核心開發者當然要熟悉編譯器行為 Linus Torvalds 教你分析 gcc 行為 Pointers are more abstract than you might expect in C / HackerNews 討論 C 編譯器原理和案例分析* C 語言: 未定義行為*: C 語言最初為了開發 UNIX 和系統軟體而生,本質是低階的程式語言,在語言規範層級存在 undefined behavior,可允許編譯器引入更多最佳化 C 語言: 編譯器和最佳化原理* 《Demystifying the Linux CPU Scheduler》第 1 章 作業: 截止繳交日: Mar 30 quiz4 Week4 隨堂測驗: 題目 (內含作答表單) 課堂問答簡記 第 5 週 (Mar 13, 14, 16): Linux CPU scheduler 教材解說* (僅止於概況,請詳閱下方教材及個別的對應解說錄影) 公告: 本週導入客製化作業,讓學員選擇改進前四週的作業或自訂題目 (例如貢獻程式碼到 Linux 核心),隨後安排授課教師和學員的線上一對一討論 浮點數運算*: 工程領域往往是一系列的取捨結果,浮點數更是如此,在軟體發開發有太多失誤案例源自工程人員對浮點數運算的掌握不足,本議程希望藉由探討真實世界的血淋淋案例,帶著學員思考 IEEE 754 規格和相關軟硬體考量點,最後也會探討在深度學習領域為了改善資料處理效率,而引入的 BFloat16 這樣的新標準 float16 vs. bfloat16 記憶體配置器涉及 bitwise 操作及浮點數運算。傳統的即時系統和該領域的作業系統 (即 RTOS) 為了讓系統行為更可預測,往往捨棄動態記憶體配置的能力,但這顯然讓系統的擴充能力大幅受限。後來研究人員提出 TLSF (Two-Level Segregated Fit) 嘗試讓即時系統也能享用動態記憶體管理,其關鍵訴求是 “O(1) cost for malloc, free, realloc, aligned_alloc” Benchmarking Malloc with Doom 3 tlsf-bsd TLSF: Part 1: Background, Part 2: The floating point Linux 核心模組運作原理 Linux: 不只挑選任務的排程器*: 排程器 (scheduler) 是任何一個多工作業系統核心都具備的機制,但彼此落差極大,考量點不僅是演算法,還有當應用規模提昇時 (所謂的 scalability) 和涉及即時處理之際,會招致不可預知的狀況 (non-determinism),不僅即時系統在意,任何建構在 Linux 核心之上的大型服務都會深受衝擊。是此,Linux 核心的排程器經歷多次變革,需要留意的是,排程的難度不在於挑選下一個可執行的行程 (process),而是讓執行完的行程得以安插到合適的位置,使得 runqueue 依然依據符合預期的順序。 C 語言: 動態連結器* C 語言: 連結器和執行檔資訊* C 語言: 執行階段程式庫 (CRT)* 作業: 截止繳交 Apr 10 assessment Week5 隨堂測驗: 題目 (內含作答表單) 課堂問答簡記 第 6 週 (Mar 20, 21, 23): System call + CPU Scheduler 教材解說* (僅止於概況,請詳閱下方教材及個別的對應解說錄影) 公告 自 3 月 22 日起,開放讓學員 (選課的學生 + 完成前二次作業過半要求的旁聽者) 跟授課教師預約一對一線上討論,請參照課程行事曆裡頭標注 “Office hour” 的時段,發訊息到 Facebook 粉絲專頁,簡述你的學習狀況並選定偏好的時段 (建議是 30 分鐘)。留意課程發送的公告信件 選修課程的學員在本學期至少要安排一次一對一討論,否則授課教師難以評估學習狀況,從而會影響評分,請重視自己的權益。 coroutine Linux: 賦予應用程式生命的系統呼叫 vDSO: 快速的 Linux 系統呼叫機制 UNIX 作業系統 fork/exec 系統呼叫的前世今生 《Demystifying the Linux CPU Scheduler》 1.2.1 System calls 1.2.2 A different kind of software 1.2.3 User and kernel stacks 1.3 Process management 2.1 Introduction 2.2 Prior to CFS 2.3 Completely Fair Scheduler (CFS) 3.1 Structs and their role 作業: 截止繳交 Apr 17 quiz5, quiz6 Week6 隨堂測驗: 題目 (內含作答表單) 課堂問答簡記 第 7 週 (Mar 27, 28, 30): Process, 並行和多執行緒 教材解說-1*, 教材解說-2* (僅止於概況,請詳閱下方教材及個別的對應解說錄影) 公告: 第 5 次作業 和 第 6 次作業 作業已指派 本週測驗順延到 4 月 4 日和 4 月 6 日,3 月 30 日晚間安排課程講解 4 月 3 日晚間依舊講課 (事後有錄影)、4 月 4 日下午到晚間安排在家測驗,4 月 6 日晚間安排測驗 Linux: 不僅是個執行單元的 Process*: Linux 核心對於 UNIX Process 的實作相當複雜,不僅蘊含歷史意義 (幾乎每個欄位都值得講古),更是反映出資訊科技產業的變遷,核心程式碼的 task_struct 結構體更是一絕,廣泛涵蓋 process 狀態、處理器、檔案系統、signal 處理、底層追蹤機制等等資訊,更甚者,還很曖昧地保存著 thread 的必要欄位,好似這兩者天生就脫不了干係 探討 Linux 核心設計的特有思維,像是如何透過 LWP 和 NPTL 實作執行緒,又如何透過行程建立記憶體管理的一種抽象層,再者回顧行程間的 context switch 及排程機制,搭配 signal 處理 測試 Linux 核心的虛擬化環境 建構 User-Mode Linux 的實驗環境* 〈Concurrency Primer〉導讀 The C11 and C++11 Concurrency Model Time to move to C11 atomics? C11 atomic variables and the kernel C11 atomics part 2: “consume” semantics An introduction to lockless algorithms 並行和多執行緒程式設計* CS:APP 第 12 章 Concurrency / 錄影* Synchronization: Basic / 錄影* Synchronization: Advanced / 錄影* Thread-Level Parallelism / 錄影* 課堂問答簡記 第 8 週 (Apr 3, 4, 6): 並行程式設計, lock-free, Linux 同步機制 教材解說* (僅止於概況,請詳閱下方教材及個別的對應解說錄影) 公告: 4 月 4 日下午到晚間安排在家測驗,請在當日 15:00 刷新課程進度表/行事曆,以得知測驗方式 4 月 6 日晚間安排測驗 並行和多執行緒程式設計,涵蓋 Atomics 操作 POSIX Threads (請對照 CS:APP 第 12 章自行學習) Lock-free 程式設計 案例: Hazard pointer 案例: Ring buffer 案例: Thread Pool Linux: 淺談同步機制* 利用 lkm 來變更特定 Linux 行程的內部狀態 Week8 隨堂測驗: 題目 (內含作答表單) 第 9 週 (Apr 10, 11, 13): futex, RCU, 伺服器開發與 Linux 核心對應的系統呼叫 教材解說* (僅止於概況,請詳閱下方教材及個別的對應解說錄影) 第二次作業檢討 公告: 請於 4 月 14 日 10:00PM 刷新本頁面,以得知新指派的作業 4 月 13 日晚間安排課程測驗和","date":"2024-02-28","objectID":"/posts/linux2023/:1:4","tags":["Sysprog","Linux"],"title":"Linux 核心设计/实作 (Linux Kernel Internals)","uri":"/posts/linux2023/"},{"categories":null,"content":"ccrysisa's friends","date":"2024-04-29","objectID":"/friends/","tags":null,"title":"友情链接","uri":"/friends/"},{"categories":null,"content":" Reminder ","date":"2024-04-29","objectID":"/friends/:0:0","tags":null,"title":"友情链接","uri":"/friends/"},{"categories":null,"content":"Base info - nickname: Lruihao avatar: https://lruihao.cn/images/avatar.jpg url: https://lruihao.cn description: Lruihao's Note ","date":"2024-04-29","objectID":"/friends/:1:0","tags":null,"title":"友情链接","uri":"/friends/"},{"categories":null,"content":"Friendly Reminder Notice If you want to exchange link, please leave a comment in the above format. (personal non-commercial blogs / websites only) Website failure, stop maintenance and improper content may be unlinked! Those websites that do not respect other people’s labor achievements, reprint without source, or malicious acts, please do not come to exchange. ","date":"2024-04-29","objectID":"/friends/:2:0","tags":null,"title":"友情链接","uri":"/friends/"},{"categories":["Rust"],"content":" Rust is a statically compiled, fast language with great tooling and a rapidly growing ecosystem. That makes it a great fit for writing command line applications: They should be small, portable, and quick to run. Command line applications are also a great way to get started with learning Rust; or to introduce Rust to your team! 整理自 Command line apps in Rust ","date":"2024-04-29","objectID":"/posts/rust-cli/:0:0","tags":["Rust","CLI"],"title":"Command Line Applications in Rust","uri":"/posts/rust-cli/"},{"categories":["Rust"],"content":"重点提示 ","date":"2024-04-29","objectID":"/posts/rust-cli/:1:0","tags":["Rust","CLI"],"title":"Command Line Applications in Rust","uri":"/posts/rust-cli/"},{"categories":["Rust"],"content":"Arguments C 语言的 CLI 程序处理参数的逻辑是过程式的,即每次执行都会通过 argv 来获取本次执行的参数并进行相应的处理 (Rust 的 std::env::args() 处理 CLI 程序的参数方式也类似,都是对每次执行实例进行过程式的处理),而 Clap 不同,它类似于面向对象的思想,通过定义一个结构体 (object),每次运行时通过 clap::Parser::parse 获取并处理本次运行的参数 (即实例化 object),这样开发的 CLI 程序扩展性会更好。 ","date":"2024-04-29","objectID":"/posts/rust-cli/:1:1","tags":["Rust","CLI"],"title":"Command Line Applications in Rust","uri":"/posts/rust-cli/"},{"categories":["Rust"],"content":"BufReader Struct std::io::BufReader 中关于系统调用 (syscall) 的开销,以及如何使用 buffer 这一机制减少 syscall 调用以此提高效能,进行了比较直观的描述: It can be excessively inefficient to work directly with a Read instance. For example, every call to read on TcpStream results in a system call. A BufReader performs large, infrequent reads on the underlying Read and maintains an in-memory buffer of the results. BufReader can improve the speed of programs that make small and repeated read calls to the same file or network socket. It does not help when reading very large amounts at once, or reading just one or a few times. It also provides no advantage when reading from a source that is already in memory, like a Vec. When the BufReader is dropped, the contents of its buffer will be discarded. Creating multiple instances of a BufReader on the same stream can cause data loss. Reading from the underlying reader after unwrapping the BufReader with BufReader::into_inner can also cause data loss. ","date":"2024-04-29","objectID":"/posts/rust-cli/:1:2","tags":["Rust","CLI"],"title":"Command Line Applications in Rust","uri":"/posts/rust-cli/"},{"categories":["Rust"],"content":"Documentations 这里列举视频中一些概念相关的 documentation 学习的一手资料是官方文档,请务必自主学会阅读规格书之类的资料 ","date":"2024-04-29","objectID":"/posts/rust-cli/:2:0","tags":["Rust","CLI"],"title":"Command Line Applications in Rust","uri":"/posts/rust-cli/"},{"categories":["Rust"],"content":"Crate std 可以使用这里提供的搜素栏进行搜索 (BTW 不要浪费时间在 Google 搜寻上!) Function std::fs::read_to_string Function std::env::args Struct std::path::PathBuf Struct std::io::BufReader method std::iter::Iterator::nth Primitive Type str: method str::lines method str::contains expect: method std::option::Option::expect method std::result::Result::expect ","date":"2024-04-29","objectID":"/posts/rust-cli/:2:1","tags":["Rust","CLI"],"title":"Command Line Applications in Rust","uri":"/posts/rust-cli/"},{"categories":["Rust"],"content":"Crate clap method clap::Parser::parse ","date":"2024-04-29","objectID":"/posts/rust-cli/:2:2","tags":["Rust","CLI"],"title":"Command Line Applications in Rust","uri":"/posts/rust-cli/"},{"categories":["Rust"],"content":"References","date":"2024-04-29","objectID":"/posts/rust-cli/:3:0","tags":["Rust","CLI"],"title":"Command Line Applications in Rust","uri":"/posts/rust-cli/"},{"categories":["C","Linux Kernel Internals"],"content":" AMaCC 是由成功大學師生開發的 self-compiling 的 C 語言編譯器,可產生 Arm 架構的執行檔 (ELF 格式,運作在 GNU/Linux)、也支援 just-in-time (JIT) 編譯和執行,原始程式碼僅 1500 行,在這次講座中,我們就來揭開 AMaCC 背後的原理和實作議題。 預期會接觸到 IR (Intermediate representation), dynamic linking, relocation, symbol table, parsing tree, language frontend, Arm 指令編碼和 ABI 等等。 原文地址 ","date":"2024-04-23","objectID":"/posts/c-compiler-construction/:0:0","tags":["Sysprog","C","Compiler"],"title":"你所不知道的 C 语言: 编译器原理和案例分析","uri":"/posts/c-compiler-construction/"},{"categories":["C","Linux Kernel Internals"],"content":"如何打造一个具体而微的 C 语言编译器 用十分鐘 向 jserv 學習作業系統設計 用1500 行建構可自我編譯的 C 編譯器 / 投影片 AMaCC 是由成功大學師生開發的 self-compiling 的 C 語言編譯器,可產生 Arm 架構的執行檔 (ELF 格式,運作在 GNU/Linux)、也支援 just-in-time (JIT) 編譯和執行,原始程式碼僅 1500 行,在這次講座中,我們就來揭開 AMaCC 背後的原理和實作議題。 預期會接觸到 IR (Intermediate representation), dynamic linking, relocation, symbol table, parsing tree, language frontend, Arm 指令編碼和 ABI 等等。 Wikipedia: Executable and Linkable Format ","date":"2024-04-23","objectID":"/posts/c-compiler-construction/:1:0","tags":["Sysprog","C","Compiler"],"title":"你所不知道的 C 语言: 编译器原理和案例分析","uri":"/posts/c-compiler-construction/"},{"categories":["C","Linux Kernel Internals"],"content":"编译器和软件工业强度息息相关 形式化驗證 (Formal Verification) ","date":"2024-04-23","objectID":"/posts/c-compiler-construction/:2:0","tags":["Sysprog","C","Compiler"],"title":"你所不知道的 C 语言: 编译器原理和案例分析","uri":"/posts/c-compiler-construction/"},{"categories":["C","Linux Kernel Internals"],"content":"背景知识 你所不知道的 C 语言: 编译器和最佳化原理篇 你所不知道的 C 语言: 函数呼叫篇 你所不知道的 C 语言: 动态连链接器和执行时期篇 虚拟机器设计与实作 ","date":"2024-04-23","objectID":"/posts/c-compiler-construction/:3:0","tags":["Sysprog","C","Compiler"],"title":"你所不知道的 C 语言: 编译器原理和案例分析","uri":"/posts/c-compiler-construction/"},{"categories":["C","Linux Kernel Internals"],"content":"C 程序的解析和语意 手把手教你构建 C 语言编译器 descent 點評幾本編譯器設計書籍 desent 教你逐步開發編譯器 c4 是很好的切入點,原作者 Robert Swierczek 還又另一個 更完整的 C 編譯器實作,这个实作支持 preprocessor AMaCC 在 Robert Swierczek 的基礎上,額外實作 C 語言的 struct, switch-case, for, C-style comment 支援,並且重寫了 IR 執行程式碼,得以輸出合法 GNU/Linux ELF 執行檔 (支援 armhf ABI) 和 JIT 編譯 徒手写一个 RISC-V 编译器!初学者友好的实战课程 注意 上面的第一个链接是关于 c4 的教程,非常值得一看和一做 (Make your hands dirty!),同时它也是 AMaCC 的基础 (AMaCC 在这个基础上进行了重写和扩展)。 最后一个关于虚拟机器的讲座也不错,对于各类虚拟机器都进行了介绍和说明,并搭配了相关实作 RTMux 进行了讲解。 ","date":"2024-04-23","objectID":"/posts/c-compiler-construction/:4:0","tags":["Sysprog","C","Compiler"],"title":"你所不知道的 C 语言: 编译器原理和案例分析","uri":"/posts/c-compiler-construction/"},{"categories":["C","Linux Kernel Internals"],"content":"手把手教你构建 C 语言编译器 原文地址 编译原理课程教导的是如何完成一个「编译器的编译器」 即 Compiler-compiler,这个难度比较大,因为需要考虑通用性,但是实作一个简单的编译器并没有这么难。 Wikipedia: Compiler-compiler ","date":"2024-04-23","objectID":"/posts/c-compiler-construction/:5:0","tags":["Sysprog","C","Compiler"],"title":"你所不知道的 C 语言: 编译器原理和案例分析","uri":"/posts/c-compiler-construction/"},{"categories":["C","Linux Kernel Internals"],"content":"设计 一般而言,编译器的编写分为 3 个步骤: 词法分析器,用于将字符串转化成内部的表示结构。 语法分析器,将词法分析得到的标记流(token)生成一棵语法树。 目标代码的生成,将语法树转化成目标代码。 ","date":"2024-04-23","objectID":"/posts/c-compiler-construction/:5:1","tags":["Sysprog","C","Compiler"],"title":"你所不知道的 C 语言: 编译器原理和案例分析","uri":"/posts/c-compiler-construction/"},{"categories":["C","Linux Kernel Internals"],"content":"虚拟机 在该项目的虚拟机设计中,函数调用时 callee 的参数位于 caller 的栈帧 (frame) 内,并且函数调用需要使用到 4 条指令: CALL 指令将 callee 的返回地址压入栈,然后跳转到 callee 的入口处 ENT 指令保存 ebp 寄存器的值并为 callee 的栈帧设置 esp 和 ebp 寄存器,在栈中给 callee 的局部变量分配空间 ADJ 指令在 callee 逻辑执行完毕后,释放之前分配给局部变量的栈空间 LEV 指令将 ebp 和 esp 寄存器恢复为对应 caller 栈帧,并跳转到之前保存的 callee 的返回地址处 除此之外,在 callee 执行期间,可能需要通过 LEA 指令来访问函数参数和局部变量。 sub_function(arg1, arg2, arg3); | .... | high address +---------------+ | arg: 1 | new_bp + 4 +---------------+ | arg: 2 | new_bp + 3 +---------------+ | arg: 3 | new_bp + 2 +---------------+ |return address | new_bp + 1 +---------------+ | old BP | \u003c- new BP +---------------+ | local var 1 | new_bp - 1 +---------------+ | local var 2 | new_bp - 2 +---------------+ | .... | low address Wikipedia: Stack machine Wikipedia: x86 calling conventions 问题 PRTF means? else if (op == PRTF) { tmp = sp + pc[1]; ax = printf((char *)tmp[-1], tmp[-2], tmp[-3], tmp[-4], tmp[-5], tmp[-6]); } 这里 c4 对于 PRTF 指令的处理暂时没看明白… 问题 -m32 error gcc 通过 -m32 参数编译本节代码时可能会遇到以下报错: fatal error: bits/wordsize.h: No such file or directory 这是因为当前安装的 gcc 只有 64 位的库而没有 32 位的库,通过以下命令安装 32 位库解决问题: $ sudo apt install gcc-multilib Stack Overflow: “fatal error: bits/libc-header-start.h: No such file or directory” while compiling HTK ","date":"2024-04-23","objectID":"/posts/c-compiler-construction/:5:2","tags":["Sysprog","C","Compiler"],"title":"你所不知道的 C 语言: 编译器原理和案例分析","uri":"/posts/c-compiler-construction/"},{"categories":["C","Linux Kernel Internals"],"content":"IR (Intermediate representation) Interpreter, Compiler, JIT from scratch How to JIT - an introduction How to write a very simple JIT compiler How to write a UNIX shell, with a lot of background 注意 JIT (Just in time) 表示“即时”,形象描述就是“及时雨” 🤣 原理是将解释执行的“热点“编译成位于一个内存区域的 machine code,从而减轻内存的压力。因为解释执行时会在内存中跳来跳去,而一个区域的 machine code 是连续执行,内存压力没这么大并且可以充分利用 cache 从而提高效能。另一个因素可以参考 你所不知道的 C 語言: goto 和流程控制篇,从 VM 的 swith-case 和 computed goto 在效能差异的主要因素「分支预测」进行思考。 最后两个链接对于提高系统编程 (System programming) 能力非常有益,Just do it! ","date":"2024-04-23","objectID":"/posts/c-compiler-construction/:6:0","tags":["Sysprog","C","Compiler"],"title":"你所不知道的 C 语言: 编译器原理和案例分析","uri":"/posts/c-compiler-construction/"},{"categories":["C","Linux Kernel Internals"],"content":"How to write a UNIX shell 系统编程 (System Programming) 的入门项目,阅读过程需要查询搭配 man 手册,以熟悉库函数和系统调用的原型和作用。 Linux manual page: fflush / elf / exec / perror / getline / strchr / waitpid / fprintf / pipe / dup / close ","date":"2024-04-23","objectID":"/posts/c-compiler-construction/:6:1","tags":["Sysprog","C","Compiler"],"title":"你所不知道的 C 语言: 编译器原理和案例分析","uri":"/posts/c-compiler-construction/"},{"categories":["C","Linux Kernel Internals"],"content":"延伸阅读 Linux manual page: bsearch ","date":"2024-04-23","objectID":"/posts/c-compiler-construction/:6:2","tags":["Sysprog","C","Compiler"],"title":"你所不知道的 C 语言: 编译器原理和案例分析","uri":"/posts/c-compiler-construction/"},{"categories":["C","Linux Kernel Internals"],"content":"程序语言设计和编译器考量 YouTube: Brian Kernighan on successful language design ","date":"2024-04-23","objectID":"/posts/c-compiler-construction/:7:0","tags":["Sysprog","C","Compiler"],"title":"你所不知道的 C 语言: 编译器原理和案例分析","uri":"/posts/c-compiler-construction/"},{"categories":["Toolkit"],"content":"今日闲来无事,刷了会 B 站,突发奇想将发灰了一年多的 WSL2 找回来折腾一下 (之前为了在 VMware 中开启嵌套虚拟化,不得以将 WSL2 打入冷宫 🤣)。由于这一年内功功力大涨,很快就完成了 WLS2 的再召集启用,下面列出配置的主要步骤和相关材料。 操作系统: Windows 10 ","date":"2024-04-20","objectID":"/posts/wsl2/:0:0","tags":["Windows","WSL","Ubuntu"],"title":"Windows 10 WSL2 Ubuntu 22.04 配置指南","uri":"/posts/wsl2/"},{"categories":["Toolkit"],"content":"安装 WSL2 和 Linux 发行版 启用或关闭 Windows 功能 适用于 Linux 的 Windows 子系统 虚拟机平台 以管理员身份运行 PowerShell \u003e bcdedit /v ... hypervisorlaunchtype Auto # 保证上面这个虚拟化选项是 Auto,如果是 Off 则使用下面命令设置 \u003e bcdedit /set hypervisorlaunchtype auto 在 PowerShell 中安装 wsl2 \u003e wsl --update \u003e wsl --set-default-version 2 \u003e wsl -l -o NAME FRIENDLY NAME ... Ubuntu-18.04 Ubuntu 18.04 LTS Ubuntu-20.04 Ubuntu 20.04 LTS Ubuntu-22.04 Ubuntu 22.04 LTS ... \u003e wsl --install Ubuntu-22.04 # 后面需要创建用户和密码,自行设置 上面以 Ubuntu2 22.04 发行版为例,你也可以安装其它的发行版。安装过程中可能会出现无法访问源 / 仓库的问题,这个是网络问题,请自行通过魔法/科学方法解决 ","date":"2024-04-20","objectID":"/posts/wsl2/:1:0","tags":["Windows","WSL","Ubuntu"],"title":"Windows 10 WSL2 Ubuntu 22.04 配置指南","uri":"/posts/wsl2/"},{"categories":["Toolkit"],"content":"迁移至非系统盘 以管理员身份运行 PowerShell # 查看已安装的 Linux 发行版 \u003e wsl -l- v # 停止正在运行的发行版 \u003e wsl --shutdown # 导出发行版的镜像 (以 Ubuntu 22.04 为例) \u003e wsl --export Ubuntu-22.04 D:/ubuntu.tar # 导出镜像后,卸载原有发行版以释放 C 盘空间 \u003e wsl --unregister Ubuntu-22.04 # 重新导入发行版镜像。并指定该子系统储存的目录 (即进行迁移) \u003e wsl --import Ubuntu-22.04 D:\\Ubuntu\\ D:\\ubuntu.tar --version 2 # 上面命令完成后,在目录 D:\\Ubuntu 下会出现名为 ext4.vhdx 的文件,这个就是子系统的虚拟磁盘 # 设置启用子系统时的默认用户 (建议使用迁移前创建的用户),否则启动子系统时进入的是 root 用户 \u003e ubuntu-22.04.exe config --default-user \u003cusername\u003e ","date":"2024-04-20","objectID":"/posts/wsl2/:2:0","tags":["Windows","WSL","Ubuntu"],"title":"Windows 10 WSL2 Ubuntu 22.04 配置指南","uri":"/posts/wsl2/"},{"categories":["Toolkit"],"content":"Windows Terminal 美化 ","date":"2024-04-20","objectID":"/posts/wsl2/:3:0","tags":["Windows","WSL","Ubuntu"],"title":"Windows 10 WSL2 Ubuntu 22.04 配置指南","uri":"/posts/wsl2/"},{"categories":["Toolkit"],"content":"其它 目前 Windows 10 上的 WSL2 应该都支持 WSLg (如果你一直更新的话),可以使用 gedit 来测试一下 WLSg 的功能,可以参考微软的官方文档: https://github.com/microsoft/wslg ","date":"2024-04-20","objectID":"/posts/wsl2/:4:0","tags":["Windows","WSL","Ubuntu"],"title":"Windows 10 WSL2 Ubuntu 22.04 配置指南","uri":"/posts/wsl2/"},{"categories":["Toolkit"],"content":"效果展示 ","date":"2024-04-20","objectID":"/posts/wsl2/:5:0","tags":["Windows","WSL","Ubuntu"],"title":"Windows 10 WSL2 Ubuntu 22.04 配置指南","uri":"/posts/wsl2/"},{"categories":["CSAPP","Linux Kernel Internals"],"content":" 千万不要小看数值系统,史上不少有名的 软体缺失案例 就因为开发者未能充分掌握相关议题,而导致莫大的伤害与损失。 原文地址 搭配 CMU: 15-213: Intro to Computer Systems: Schedule for Fall 2015 可以在 这里 找到相关的投影片和录影 B 站上有一个汉化版本的 录影 ","date":"2024-04-19","objectID":"/posts/csapp-ch2/:0:0","tags":["Sysprog","CSAPP"],"title":"CS:APP 第 2 章重点提示和练习","uri":"/posts/csapp-ch2/"},{"categories":["CSAPP","Linux Kernel Internals"],"content":"数值系统 ","date":"2024-04-19","objectID":"/posts/csapp-ch2/:1:0","tags":["Sysprog","CSAPP"],"title":"CS:APP 第 2 章重点提示和练习","uri":"/posts/csapp-ch2/"},{"categories":["CSAPP","Linux Kernel Internals"],"content":"导读 YouTube: 十进制,十二进制,六十进制从何而来? YouTube: 老鼠和毒药问题怎么解?二进制和易经八卦有啥关系? YouTube: 小精靈遊戲中的幽靈是怎麼追蹤人的? 鮮為人知的 bug 解读计算机编码 你所不知道的 C 语言: 未定义/未指定行为篇 你所不知道的 C 语言: 数值系统篇 基于 C 语言标准研究与系统程式安全议题 熟悉浮点数每个位的表示可以获得更大的最佳化空间 Faster arithmetic by flipping signs Faster floating point arithmetic with Exclusive OR 看了上面的第 3 个影片后,对 pac-man 256 莫名感兴趣 🤣 ","date":"2024-04-19","objectID":"/posts/csapp-ch2/:1:1","tags":["Sysprog","CSAPP"],"title":"CS:APP 第 2 章重点提示和练习","uri":"/posts/csapp-ch2/"},{"categories":["CSAPP","Linux Kernel Internals"],"content":"Bits, Bytes \u0026 Integers 信息 第一部分录影 ✅ / 投影片 ✅ / 阅读章节: 2.1 ✅ 信息 第二部分录影 ✅ / 投影片 ✅ / 阅读章节: 2.2-2.3 计算乘法至多需要多少位可以从无符号数和二补数的编码方式来思考。无符号数乘法最大值为 $2^{2w}-2^{2+1}+1$ 不超过 $2^{2w}$,依据无符号数编码方式至多需要 $2w$ bits 表示;二补数乘法最小值为 $-2^{2w-2}+2^{w-1}$,依据而二补数编码 MSB 表示值 $-2^{2w-2}$,所以 MSB 为第 $2w-2$ 位,至多需要 $2w-1$ bits 表示二补数乘法的最小值;二补数乘法最大值为 $2^{2w-2}$,因为 MSB 为符号位,所以 MSB 的右一位表示值 $2^{2w-2}$,即第 $2w-2$ 位,所以至多需要 $2w$ 位来表示该值 (因为还需要考虑一个符号位)。 ","date":"2024-04-19","objectID":"/posts/csapp-ch2/:1:2","tags":["Sysprog","CSAPP"],"title":"CS:APP 第 2 章重点提示和练习","uri":"/posts/csapp-ch2/"},{"categories":["Linux","Linux Kernel Internals"],"content":" Linux 核心原始程式碼中,許多地方出現紅黑樹的蹤影,例如:hr_timer 使用紅黑樹來記錄計時器 (timer) 端發出的要求、ext3 檔案系統使用紅黑樹來追蹤目錄內容變更,以及 CFS (Completely Fair Scheduler) 這個 Linux 預設 CPU 排程器,由於需要頻繁地插入跟移除節點 (任務),因此開發者選擇用紅黑樹 (搭配一些效能調整)。VMA(Virtual Memory Area)也用紅黑樹來紀錄追蹤頁面 (page) 變更,因為後者不免存在頻繁的讀取 VMA 結構,如 page fault 和 mmap 等操作,且當大量的已映射 (mapped) 區域時存在時,若要尋找某個特定的虛擬記憶體地址,鏈結串列 (linked list) 的走訪成本過高,因此需要一種資料結構以提供更有效率的尋找,於是紅黑樹就可勝任。 原文地址 ","date":"2024-04-12","objectID":"/posts/linux-rbtree/:0:0","tags":["Sysprog","Linux","Red Black Tree"],"title":"Linux 核心的红黑树","uri":"/posts/linux-rbtree/"},{"categories":["Linux","Linux Kernel Internals"],"content":"简述红黑树 Left-Leaning Red-Black Trees: 论文 by Robert Sedgewick 投影片 by Robert Sedgewick 解说录影: Left Leaning Red Black Trees (Part 1) Left Leaning Red Black Trees (Part 2) 2-3-4 tree: Problem: Doesn’t work if parent is a 4-node 为解决该问题,投影片主要说明的是 Split 4-nodes on the way down 方法,这个方法的逻辑是:在插入节点前向下走访的过程中,如果发现某个节点是 4-nodes 则对该节点进行 split 操作,具体例子可以参考投影片的 P24 ~ P25。 LLRB tree: Problem: Doesn’t work if parent is a 4-node 为解决该问题,投影片主要说明的也是 Split 4-nodes on the way down 方法,其逻辑和之前相同,除此之外,在插入节点后向上返回的过程中,进行 rotate 操作,保证了 LLRB 节点的结构满足要求 (即红边的位置)。 技巧 在拆分 4-node 时 3-node 和 4-node 的孩子节点的大小关系不太直观,这时可以参考解说录影的老师的方法,使用数字或字母标识节点,进而可以直观看出 4-node 转换前后的等价性。 如果我们将 4-node 节点的拆分放在插入节点后向上返回的过程进行处理,则会将原本的树结构转换成 2-3 tree,因为这样插入节点后,不会保留有 4-node (插入产生的 4-node 立刻被拆分)。 注意 红黑树的 perfect-balance 的特性在于:它随着节点的增多而逐渐将 4-nodes (因为新增节点都是 red 边,所以叶节点很大概率在插入结束后会是 4-node) 从根节点方向移动 (on the way down 时 split 4-nodes 的效果),当 4-node 移动到根节点时,进行颜色反转并不会破坏树的平衡,只是树高加 1 (这很好理解,因为根节点是唯一的,只要保证 4-node 的根节点拆分操作保持平衡即可,显然成立)。 ","date":"2024-04-12","objectID":"/posts/linux-rbtree/:1:0","tags":["Sysprog","Linux","Red Black Tree"],"title":"Linux 核心的红黑树","uri":"/posts/linux-rbtree/"},{"categories":["Linux","Linux Kernel Internals"],"content":"Maple tree 解说录影: The Linux Maple Tree - Matthew Wilcox, Oracle ","date":"2024-04-12","objectID":"/posts/linux-rbtree/:2:0","tags":["Sysprog","Linux","Red Black Tree"],"title":"Linux 核心的红黑树","uri":"/posts/linux-rbtree/"},{"categories":["Concurrency","Linux Kernel Internals"],"content":" 原文地址 ","date":"2024-04-10","objectID":"/posts/posix-threads/:0:0","tags":["Sysprog","Linux","Concurrency"],"title":"并行程序设计: POSIX Threads","uri":"/posts/posix-threads/"},{"categories":["Concurrency","Linux Kernel Internals"],"content":"Process vs. Thread vs. Coroutines With threads, the operating system switches running tasks preemptively according to its scheduling algorithm. With coroutines, the programmer chooses, meaning tasks are cooperatively multitasked by pausing and resuming functions at set points. coroutine switches are cooperative, meaning the programmer controls when a switch will happen. The kernel is not involved in coroutine switches. 一图胜千语: 具体一点,从函数执行流程来看: $\\rightarrow$ 在使用 coroutinues 后执行流程变成 $\\rightarrow$ ","date":"2024-04-10","objectID":"/posts/posix-threads/:1:0","tags":["Sysprog","Linux","Concurrency"],"title":"并行程序设计: POSIX Threads","uri":"/posts/posix-threads/"},{"categories":["Concurrency","Linux Kernel Internals"],"content":"Thread \u0026 Process Wikipedia: Light-weight process On Linux, user threads are implemented by allowing certain processes to share resources, which sometimes leads to these processes to be called “light weight processes”. Wikipedia: Thread-local storage On a modern machine, where multiple threads may be modifying the errno variable, a call of a system function on one thread may overwrite the value previously set by a call of a system function on a different thread, possibly before following code on that different thread could check for the error condition. The solution is to have errno be a variable that looks as if it is global, but is physically stored in a per-thread memory pool, the thread-local storage. ","date":"2024-04-10","objectID":"/posts/posix-threads/:1:1","tags":["Sysprog","Linux","Concurrency"],"title":"并行程序设计: POSIX Threads","uri":"/posts/posix-threads/"},{"categories":["Concurrency","Linux Kernel Internals"],"content":"PThread (POSIX threads) POSIX 的全称是 Portable Operating System Interfaces,结合上图,所以你明白 pthread 的 P 代表的意义了吗? Answer 从 CPU 厂商群魔乱舞中诞生的标准,自然是要保证可移植 Portable 的啦 🤣 下面的这个由 Lawrence Livermore National Laboratory 撰写的教程文档写的非常棒,值得一读 (他们还有关于 HPC 高性能计算的相关教程文档): POSIX Threads Programming ","date":"2024-04-10","objectID":"/posts/posix-threads/:1:2","tags":["Sysprog","Linux","Concurrency"],"title":"并行程序设计: POSIX Threads","uri":"/posts/posix-threads/"},{"categories":["Concurrency","Linux Kernel Internals"],"content":"Synchronizing Threads 3 basic synchronization primitives (为什么是这 3 个?请从 synchronized-with 关系进行思考) mutex locks condition variables semaphores 取材自 Ching-Kuang Shene 教授的讲义: Part IV Other Systems: IIIPthreads: A Brief Review Conditions in Pthreads are usually used with a mutex to enforce mutual exclusion. mutex locks pthread_mutex_t mutex = PTHREAD_MUTEX_INITIALIZER; int pthread_mutex_init(pthread_mutex_t *mutex, pthread_mutexattr_t *attr); int pthread_mutex_destroy(pthread_mutex_t *mutex); int pthread_mutex_lock(pthread_mutex_t *mutex); int pthread_mutex_unlock(pthread_mutex_t *mutex); int pthread_mutex_trylock(pthread_mutex_t *mutex); Only the owner can unlock a mutex. Since mutexes cannot be copied, use pointers. If pthread_mutex_trylock() returns EBUSY, the lock is already locked. Otherwise, the calling thread becomes the owner of this lock. With pthread_mutexattr_settype(), the type of a mutex can be set to allow recursive locking or report deadlock if the owner locks again 注意 单纯的 Mutex 无法应对复杂情形的「生产者-消费者」问题,例如单生产者单消费者、多生产者单消费者、单生产者多消费者,甚至是多生产者多消费者 😵 需要配合 condition variables 我有用 Rust 写过一个「多生产者单消费者」的程序,相关的博客解说在 这里 condition variables int pthread_cond_init(pthread_cond_t *cond, const pthread_condattr_t *attr); int pthread_cond_destroy(pthread_cond_t *cond); int pthread_cond_wait(pthread_cond_t *cond, pthread_mutex_t *mutex); int pthread_cond_signal(pthread_cond_t *cond); int pthread_cond_broadcast(pthread_cond_t *cond); // all threads waiting on a condition need to be woken up Condition variables allow a thread to block until a specific condition becomes true blocked thread goes to wait queue for condition When the condition becomes true, some other thread signals the blocked thread(s) Conditions in Pthreads are usually used with a mutex to enforce mutual exclusion. the wait call should occur under the protection of a mutex 使用 condition variables 改写之前 mutex 部分的 producer 实作 (实作是单生产者单消费者模型,且缓冲区有 MAX_SIZE 个元素): void producer(char *buf) { for (;;) { pthread_mutex_lock(lock); while (count == MAX_SIZE) pthread_cond_wait(notFull, lock); buf[count] = getChar(); count++; pthread_cond_signal(notEmpty); pthread_mutex_unlock(lock); } } semaphores semaphores 是站在「资源的数量」的角度来看待问题,这与 condition variables 是不同的 sem_t semaphore; int sem_init(sem_t *sem, int pshared, unsigned int value); int sem_wait(sem_t *sem); int sem_post(sem_t *sem); Can do increments and decrements of semaphore value Semaphore can be initialized to any value Thread blocks if semaphore value is less than or equal to zero when a decrement is attempted As soon as semaphore value is greater than zero, one of the blocked threads wakes up and continues no guarantees as to which thread this might be ","date":"2024-04-10","objectID":"/posts/posix-threads/:1:3","tags":["Sysprog","Linux","Concurrency"],"title":"并行程序设计: POSIX Threads","uri":"/posts/posix-threads/"},{"categories":["Concurrency","Linux Kernel Internals"],"content":"POSIX Threads","date":"2024-04-10","objectID":"/posts/posix-threads/:2:0","tags":["Sysprog","Linux","Concurrency"],"title":"并行程序设计: POSIX Threads","uri":"/posts/posix-threads/"},{"categories":["C","Linux Kernel Internals"],"content":" 本次講座將選定幾個案例,藉此解說 C 語言程式設計的技巧,像是對矩陣操作進行包裝、初始化特定結構的成員、追蹤物件配置的記憶體、Smart Pointer 等等。 原文地址 ","date":"2024-04-10","objectID":"/posts/c-trick/:0:0","tags":["Sysprog","C"],"title":"你所不知道的 C 语言: 技巧篇","uri":"/posts/c-trick/"},{"categories":["C","Linux Kernel Internals"],"content":"从矩阵操作谈起 C 语言也可作实现 Object-oriented programming (需要搭配前置处理器扩充语法) GNU Manual 6.29 Designated Initializers Stack Overflow: Why does C++11 not support designated initializer lists as C99? 从 C99 (含) 以后,C 和 C++ 就分道扬镳了。相关差异可以参考: Incompatibilities Between ISO C and ISO C++ 结构体的成员函数实作时使用 static,并搭配 API gateway 可以获得一部分 namespace 的功能 Fun with C99 Syntax ","date":"2024-04-10","objectID":"/posts/c-trick/:1:0","tags":["Sysprog","C"],"title":"你所不知道的 C 语言: 技巧篇","uri":"/posts/c-trick/"},{"categories":["C","Linux Kernel Internals"],"content":"明确初始化特定结构的成员 静态空间初始化配置: 动态空间初始化配置: Initializing a heap-allocated structure in C ","date":"2024-04-10","objectID":"/posts/c-trick/:2:0","tags":["Sysprog","C"],"title":"你所不知道的 C 语言: 技巧篇","uri":"/posts/c-trick/"},{"categories":["C","Linux Kernel Internals"],"content":"追踪物件配置的记忆体 ","date":"2024-04-10","objectID":"/posts/c-trick/:3:0","tags":["Sysprog","C"],"title":"你所不知道的 C 语言: 技巧篇","uri":"/posts/c-trick/"},{"categories":["C","Linux Kernel Internals"],"content":"Smart Pointer ","date":"2024-04-10","objectID":"/posts/c-trick/:4:0","tags":["Sysprog","C"],"title":"你所不知道的 C 语言: 技巧篇","uri":"/posts/c-trick/"},{"categories":["C","Linux Kernel Internals"],"content":"C99 Variable Length Arrays (VLA) ","date":"2024-04-10","objectID":"/posts/c-trick/:5:0","tags":["Sysprog","C"],"title":"你所不知道的 C 语言: 技巧篇","uri":"/posts/c-trick/"},{"categories":["C","Linux Kernel Internals"],"content":"字符串和数值转换 Integer to string conversion ","date":"2024-04-10","objectID":"/posts/c-trick/:6:0","tags":["Sysprog","C"],"title":"你所不知道的 C 语言: 技巧篇","uri":"/posts/c-trick/"},{"categories":["C","Linux Kernel Internals"],"content":"GCC 支援 Plan 9 C Extension GCC 6.65 Unnamed Structure and Union Fields ","date":"2024-04-10","objectID":"/posts/c-trick/:7:0","tags":["Sysprog","C"],"title":"你所不知道的 C 语言: 技巧篇","uri":"/posts/c-trick/"},{"categories":["C","Linux Kernel Internals"],"content":"GCC transparent union GCC 6.35.1 Common Type Attributes ","date":"2024-04-10","objectID":"/posts/c-trick/:8:0","tags":["Sysprog","C"],"title":"你所不知道的 C 语言: 技巧篇","uri":"/posts/c-trick/"},{"categories":["C","Linux Kernel Internals"],"content":"高阶的 C 语言的「开发框架」 cello 是上面提到的技巧的集大成者,在 C 语言基础上,提供以下进阶特征: ","date":"2024-04-10","objectID":"/posts/c-trick/:9:0","tags":["Sysprog","C"],"title":"你所不知道的 C 语言: 技巧篇","uri":"/posts/c-trick/"},{"categories":["C","Linux Kernel Internals"],"content":"善用 GNU extension 的 typeof GCC 6.7 Referring to a Type with typeof typeof 在 C23 中已由 GNU extenison 转正为 C 语言标准 ","date":"2024-04-10","objectID":"/posts/c-trick/:10:0","tags":["Sysprog","C"],"title":"你所不知道的 C 语言: 技巧篇","uri":"/posts/c-trick/"},{"categories":["Rust"],"content":" 从基础到进阶讲解探讨 Rust 生命周期,不仅仅是 lifetime kata,还有更多的 lifetime 资料,都来讲解和探讨,从「入门 Rust」到「进阶 Rust」 整理自 B 站 UP 主 @这周你想干啥 的 教学影片合集 注意 学习 John Gjengset 的教学影片 Subtying and Variance 时发现自己对 Rust 生命周期 (lifetime) 还是不太理解,于是便前来补课 🤣 同时完成 LifetimeKata 的练习。 ","date":"2024-04-05","objectID":"/posts/rust-lifetime/:0:0","tags":["Rust","Lifetime"],"title":"Rust Lifetime: 由浅入深理解生命周期","uri":"/posts/rust-lifetime/"},{"categories":["Rust"],"content":"引用 \u0026 生命周期 ","date":"2024-04-05","objectID":"/posts/rust-lifetime/:1:0","tags":["Rust","Lifetime"],"title":"Rust Lifetime: 由浅入深理解生命周期","uri":"/posts/rust-lifetime/"},{"categories":["Rust"],"content":"生命周期标注 在 单线程 的程序中,通过函数参数传入的引用,无论其生命周期标注,它的生命周期在该函数的函数体范围内都是有效的。因为从 状态机 模型来考虑,该函数没有传入的引用的所有权 (因为是通过参数传入的),所以该函数不可能在其函数体范围内某一节点,就结束传入的引用的生命周期。但是在 多线程 的程序,上述规则就不一定成立了。 fn split\u003c'a, 'b\u003e(text: \u0026'a str, delimiter: \u0026'b str) {...} 单线程情况下,参数 text 和 delimiter 在函数 split 范围内都是有效的。 也因为这个状态机模型,Rust 的生命周期对于参数在函数体内的使用的影响并不大,主要影响的是涉及生命周期的参数或返回值在 函数调用后的生命周期使用约束,下面给出一些技巧: 技巧 最大共同生命周期: 从引用的当前共同使用开始,直到任一引用对应的 object 消亡,即其生命周期结束。 当以相同的生命周期标注,来对函数参数的生命周期进行标注时,其表示参数的最大共同生命周期,例如: fn f\u003c'a\u003e(x: \u0026'a i32, y: \u0026'a i32, z: \u0026'a i32) {...} 生命周期 'a 表示参数 x, y, z 的最大共同生命周期,即 x, y, z 可以共同存活的最大生命周期。 当以相同的生命周期标注,来对函数参数和返回值的生命周期进行标注时,其表示返回值的生命周期必须在参数的生命周期内,例如: fn f\u003c'a\u003e(x: \u0026'a i32) -\u003e \u0026'a i32 {...} 生命周期 'a 表示返回值的生命周期必须在参数 x 的生命周期内,即返回值的生命周期是参数和返回值的最大共同生命周期。所以在返回值可以使用的地方,参数都必须存活,这也是常出现问题的地方。 最后看一下将这两个技巧结合起来的一个例子: fn f\u003c'a\u003e(x: \u0026'a i32, y: \u0026'a i32) -\u003e \u0026'a i32 {...} 参数取最大生命周期在容器情况下会被另一个技巧取代,即容器和容器内部元素都被标注为相同的生命周期,这种情况会让容器的生命周期和容器内的元素的生命周期保持一致。这是因为隐式规则: 容器的生命周期 $\\leq$ 容器内元素的生命周期,这显然已经满足了最大生命周期的要求,而此时标注一样的生命周期,会被编译器推导为二者的生命周期相同,即 容器的生命周期和容器内的元素的生命周期一致: fn strtok(x: \u0026'a mut \u0026'a str, y: char) {...} ","date":"2024-04-05","objectID":"/posts/rust-lifetime/:2:0","tags":["Rust","Lifetime"],"title":"Rust Lifetime: 由浅入深理解生命周期","uri":"/posts/rust-lifetime/"},{"categories":["Rust"],"content":"生命周期在函数上的省略规则 The Rust Reference: Lifetime elision Each elided lifetime in the parameters becomes a distinct lifetime parameter. If there is exactly one lifetime used in the parameters (elided or not), that lifetime is assigned to all elided output lifetimes. If the receiver has type \u0026Self or \u0026mut Self, then the lifetime of that reference to Self is assigned to all elided output lifetime parameters. 正如投影片所说,虽然有生命周期的省略规则,但有时并不符合我们的预期,这时候需要我们手动标注。这种情况常出现于可变引用 \u0026mut self 的使用: struct S {} impl S { fn as_slice_mut\u003c'a\u003e(\u0026'a mut self) -\u003e \u0026'a [u8] {...} } let s: S = S{}; let x: \u0026[u8] = s.as_slice_mut(); //--- ... // | ... // | scope ... // | // end of s's lifetime //--- 在上面例子中,由于将方法 as_slice_mut 的可变引用参数和返回值的生命周期都标注为相同 'a,所以在范围 scope 中,在编译器看来 s 的可变引用仍然有效 (回想一下之前的参数和返回值的生命周期推导技巧),所以在这个范围内无法使用 s 的引用 (不管是可变还是不可变,回想一下 Rust 的引用规则),这是一个很常见的可变引用引起非预期的生命周期的例子,下一节会进一步介绍。 ","date":"2024-04-05","objectID":"/posts/rust-lifetime/:3:0","tags":["Rust","Lifetime"],"title":"Rust Lifetime: 由浅入深理解生命周期","uri":"/posts/rust-lifetime/"},{"categories":["Rust"],"content":"关注可变引用 fn insert_value\u003c'a\u003e(my_vec: \u0026'a mut Vec\u003c\u0026'a i32\u003e, value: \u0026'a i32) {...} 这个例子和之前的例子很类似,同样的,使用我们的参数生命周期推导技巧,调用函数 insert_value 后,当参数 vec 和 value 的最大共同生命周期的范围很广时,这时就需要注意,在这个范围内,我们无法使用 my_vec 对应的 object 的任何其它引用 (因为编译器会认为此时还存在可变引用 my_vec),从而编译错误。这就是容器和引用使用相同生命周期标注,而导致的强约束。为避免这种非预期的生命周期,应当将函数原型改写如下: fn insert_value\u003c'a, 'b\u003e(my_vec: \u0026'a mut Vec\u003c\u0026'b i32\u003e, value: \u0026'b i32) {...} 这样改写会包含一个隐式的生命周期规则: 'a $\\leq$ 'b,这很好理解,容器的生命周期应该比所引用的 object 短,这个隐式规则在下一节的 struct/enum 的生命周期标注非常重要。 ","date":"2024-04-05","objectID":"/posts/rust-lifetime/:4:0","tags":["Rust","Lifetime"],"title":"Rust Lifetime: 由浅入深理解生命周期","uri":"/posts/rust-lifetime/"},{"categories":["Rust"],"content":"struct / enum 生命周期标注 struct / enum 的生命周期标注也可以通过之前所提的 状态机 模型来进行理解,因为 struct / enum 本身不具备引用对应的 object 的所有权,在进行方法 (method) 调用时并不能截断引用对应的 object 的生命周期。 struct / enum 生命周期标注主要需要特别注意一点,就是 struct / enum 本身的可变引用的生命周期标注,最好不要和为引用的成员的生命周期的标注,标注为相同,因为这极大可能会导致 生命周期强约束,例如: fn strtok(x: \u0026mut 'a Vec\u003c'a i32\u003e, y: \u0026'a i32) {...} 如果参数 Vec\u003c'a i32\u003e 的 vector 里的 i32 引用的生命周期是 static 的话,依据我们之前所提的技巧,会将可变引用 \u0026'a mut 的生命周期也推导为 static,这就导致再也无法借用 x 对应的 object。 ","date":"2024-04-05","objectID":"/posts/rust-lifetime/:5:0","tags":["Rust","Lifetime"],"title":"Rust Lifetime: 由浅入深理解生命周期","uri":"/posts/rust-lifetime/"},{"categories":["Rust"],"content":"'static 和 '_ fn foo(_input: \u0026'a str) -\u003e 'static str { \"abc\" } 如果不进行 static 生命周期标注,依据省略规则,编译器会把返回值的生命周期推导为 'a,即和输入参数一样,这就不符合我们预期使用了。 如果使用 static 标注 struct / enum 里的成员,则无需标注 struct / enum 的生命周期,因为 static 表示在整个程序运行起见都有效,没必要对容器进行生命周期标注。 struct UniqueWords { sentence: \u0026'static str, unique_words: Vec\u003c\u0026'static str\u003e, } impl UniqueWords {...} 在没有使用到 struct 的生命周期标注时,impl 可以不显式指明生命周期,而是通过 '_ 让编译器自行推导: struct Counter\u003c'a\u003e { inner: \u0026'a mut i32, } impl Counter\u003c'_\u003e { fn increment(\u0026mut self) {...} } // is the same as impl\u003c'a\u003e Counter\u003c'a\u003e { fn increment(\u0026mut self) {...} } 函数返回值不是引用,但是返回值类型里有成员是引用,依据省略规则,编译器无法自行推导该成员的生命周期,此时可以通过 '_ 来提示编译器依据省略规则,对返回值的成员的生命周期进行推导: struct StrWrap\u003c'a\u003e(\u0026'a str); fn make_wrapper(string: \u0026str) -\u003e StrWrap\u003c'_\u003e {...} // is the same as fn make_wrapper\u003c'a\u003e(string: \u0026'a str) -\u003e StrWrap\u003c'a\u003e {...} ","date":"2024-04-05","objectID":"/posts/rust-lifetime/:6:0","tags":["Rust","Lifetime"],"title":"Rust Lifetime: 由浅入深理解生命周期","uri":"/posts/rust-lifetime/"},{"categories":["Rust"],"content":"生命周期型变和绑定 因为 Rust 是没有继承的概念,所以下面以 scala 来对类型的型变举例子进行讲解 (但是不需要你对 Scala 有任何了解): class A class B extends A // B is subclass of A private def f1(a: A): Unit = { println(\"f1 works!\") } def main(args: Array[STring]): Unit = { val a = new A val b = new B f1(a) // succeed f1(b) // succeed } 这个例子很好理解,因为 B 是 A 的子类,所以作为参数传入函数 f1 显然是可以接受的。但是当泛型 (generic) 和子类 (subclass) 结合起来时,就变得复杂了: class A class B extends A // B is subclass of A class Foo[T] // generic private def f1(a: Foo[A]): Unit = { println(\"f1 works!\") } def main(args: Array[STring]): Unit = { val foo_a = new Foo[A] val foo_b = new Foo[B] f1(a) // succeed f1(b) // error } 在编译器看来,虽然 B 是 A 的子类,但是编译器认为 Foo[A] 和 Foo[B] 是两个独立的类型,这个被称为 不变 (invariant)。而我们的直觉是,这种情况 Foo[B] 应该是 Foo[A] 的子类,这就引出了 协变 (covariant)。将上面例子的第 3 行的 Foo 定义修改如下: class Foo[+T] // covariant 就可以让编译器推导出 Foo[B] 是 Foo[A] 的子类,进而让第 14 行编译通过。 除此之外,还有 逆变 (contra-variant),它会将子类关系反转。将上面例子的第 3 行的 Foo 定义修改如下: class Foo[-T] // contra-variant 编译器就会推导出关系: Foo[A] 是 Foo[B] 的子类,这个关系刚好是 A 和 B 的反转。 在 Scala 中,函数之间的关系也体现了协变和逆变,即 参数是逆变的,返回值是协变的: class A class B extends A // B is subclass of A class C extends B // C is subclass of B /* * `A =\u003e C` is subclass of `B =\u003e B` */ 为什么 A =\u003e C 是 B =\u003e B 的子类呢?其实也很好理解,B =\u003e B 的返回值是 B,这个返回值可以用 C 来代替,但不能用 A 来代替,这显然满足协变。B =\u003e B 的参数是 B,这个参数可以用 A 来代替而不能用 C 来代替 (因为有一部分 B 不一定是 C,而 B 则一定是 A),这满足逆变。 T 可以表示所有情况: ownership, immutable reference, mutable reference,例如 T 可以表示 i32, \u0026i32, \u0026mut i32 (如果你使用过 into_iterator 的话应该不陌生) T: 'a 是说:如果 T 里面含有引用,那么这个引用的生命周期必须是 'a 的子类,即比 'a 长或和 'a 相等。T: 'static 也类似,表示 T 里面的引用 (如果有的话),要么比 'static 长或和 'static 相等,因为不可能有比 'static 更长的生命周期,所以这个标注表示 要么 T 里面的引用和 'static 一样长,要么 T 里面没有引用只有所有权 (owneship)。 The Rust Reference: Subtyping and Variance The Rustonomicon: Subtyping and Variance 基本和我们之前所说的一致,这里需要注意一点: 凡是涉及可变引用的 T,都是不变 (invariant)。这也很好理解,因为可变引用需要保证所引用的类型 T 是一致并且是唯一的,否则会扰乱 Rust 的引用模型。因为可变引用不仅可以改变所指向的 object 的内容,还可以改变自身,即改变指向的 object,如果此时 T 不是不变 (invariant) 的,那么可以将这个可变引用指向 T 的子类,这会导致该可变引用指向的 object 被可变借用一次后无法归还,从而导致后续再也无法引用该 object。此外还会导致原本没有生命周期约束的两个独立类型,被生命周期约束,具体见后面的例子。 struct Foo\u003c'a\u003e { _phantom: PhantomData\u003c\u0026'a i32\u003e, } fn foo\u003c'short, 'long: 'short\u003e( // long is subclass of short mut short_foo: Foo\u003c'short\u003e, mut long_foo: Foo\u003c'long\u003e, ) { short_foo = long_foo; // succeed long_foo = short_foo; // error } 下面是一个可变引用的例子。参数 short_foo 和 long_foo 没有关系,是两个独立的类型,所以无法相互赋值,这保证了可变引用的模型约束。除此之外,如果可变引用的型变规则不是不变 (inariant) 则会导致 short_foo 和 long_foo 在函数 foo 调用后的生命周期约束为: short_foo $\\leq$ long_foo (协变) long_foo $\\leq$ short_foo (逆变) 而它们本身可能并没有这种约束,生命周期是互相独立的。 struct Foo\u003c'a\u003e { _phantom: PhantomData\u003c\u0026'a i32\u003e, } fn foo\u003c'short, 'long: 'short\u003e( // long is subclass of short mut short_foo: \u0026mut Foo\u003c'short\u003e, mut long_foo: \u0026mut Foo\u003c'long\u003e, ) { short_foo = long_foo; // error long_foo = short_foo; // error } ","date":"2024-04-05","objectID":"/posts/rust-lifetime/:7:0","tags":["Rust","Lifetime"],"title":"Rust Lifetime: 由浅入深理解生命周期","uri":"/posts/rust-lifetime/"},{"categories":["Rust"],"content":"Documentations 这里列举视频中一些概念相关的 documentation 学习的一手资料是官方文档,请务必自主学会阅读规格书之类的资料 ","date":"2024-04-05","objectID":"/posts/rust-lifetime/:8:0","tags":["Rust","Lifetime"],"title":"Rust Lifetime: 由浅入深理解生命周期","uri":"/posts/rust-lifetime/"},{"categories":["Rust"],"content":"Crate std 可以使用这里提供的搜素栏进行搜索 (BTW 不要浪费时间在 Google 搜寻上!) ","date":"2024-04-05","objectID":"/posts/rust-lifetime/:8:1","tags":["Rust","Lifetime"],"title":"Rust Lifetime: 由浅入深理解生命周期","uri":"/posts/rust-lifetime/"},{"categories":["Rust"],"content":"References LifetimeKata The Rust Reference 泛型中的型变 (协变,逆变,不可变) Variant Types and Polymorphism ","date":"2024-04-05","objectID":"/posts/rust-lifetime/:9:0","tags":["Rust","Lifetime"],"title":"Rust Lifetime: 由浅入深理解生命周期","uri":"/posts/rust-lifetime/"},{"categories":["C","Linux Kernel Internals"],"content":" goto 在 C 語言被某些人看做是妖魔般的存在,不過實在不用這樣看待,至少在 Linux 核心原始程式碼中,goto 是大量存在 (跟你想像中不同吧)。有時不用 goto 會寫出更可怕的程式碼。 原文地址 Stack Overflow: GOTO still considered harmful? ","date":"2024-04-05","objectID":"/posts/c-control-flow/:0:0","tags":["Sysprog","C","Control flow"],"title":"你所不知道的 C 语言: goto 和流程控制篇","uri":"/posts/c-control-flow/"},{"categories":["C","Linux Kernel Internals"],"content":"MISRA C MISRA-C:2004 Guidelines for the use of the C language in critical systems MISRA C 禁用 goto 和 continue,但可用 break: Rule 14.4 (required): The goto statement shall not be used. Rule 14.5 (required): The continue statement shall not be used. Rule 14.6 (required): For any iteration statement there shall be at most one break statement used for loop termination. These rules are in the interests of good structured programming. One break statement is allowed in a loop since this allows, for example, for dual outcome loops or for optimal coding. Stack Overflow 上的相关讨论: Why “continue” is considered as a C violation in MISRA C:2004? 使用 goto 可能会混淆静态分析的工具 (当然使用 goto 会极大可能写出 ugly 的程式码): Case in point: MISRA C forbids goto statements primarily because it can mess up static analysis. Yet this rule is gratuitously followed even when no static analysis tools are used, thus yielding none of the gains that you trade off for occasionally writing ugly code. ","date":"2024-04-05","objectID":"/posts/c-control-flow/:1:0","tags":["Sysprog","C","Control flow"],"title":"你所不知道的 C 语言: goto 和流程控制篇","uri":"/posts/c-control-flow/"},{"categories":["C","Linux Kernel Internals"],"content":"GOTO 没有想象中那么可怕 虽然 MISRA C 这类规范都明确禁止了使用 goto,但 goto 并没有想像中的那么可怕,在一些领域还是极具活力的。 在 C 语言中 goto 语句是实作错误处理的极佳选择 (如果你看过 xv6 应该不陌生),有时不用 goto 可能会写出更可怕的程式码: Using goto for error handling in C Wikipedia: RAII C requires significant administrative code since it doesn’t support exceptions, try-finally blocks, or RAII at all. A typical approach is to separate releasing of resources at the end of the function and jump there with gotos in the case of error. This way the cleanup code need not be duplicated. 相关实作: goto 在 Linux 核心广泛应用 OpenBSD’s httpd Linux kernel 里 NFS inode 验证的函数: fs/nfs/inode.c 以 goto 为关键字进行检索 Wikipedia: Common usage patterns of Goto To make the code more readable and easier to follow Error handling (in absence of exceptions), particularly cleanup code such as resource deallocation. ","date":"2024-04-05","objectID":"/posts/c-control-flow/:2:0","tags":["Sysprog","C","Control flow"],"title":"你所不知道的 C 语言: goto 和流程控制篇","uri":"/posts/c-control-flow/"},{"categories":["C","Linux Kernel Internals"],"content":"switch \u0026 goto switch 通过 jump table 的内部实作可以比大量的 if-else 效率更高。 GCC: 6.3 Labels as Values You can get the address of a label defined in the current function (or a containing function) with the unary operator ‘\u0026\u0026’. The value has type void *. To use these values, you need to be able to jump to one. This is done with the computed goto statement6, goto *exp;. 下面这篇文章以 VM 为例子对 computed goto 和 switch 的效能进行了对比 (之前学的 RISC-V 模拟器派上用场了hhh): Computed goto for efficient dispatch tables the condition serves as an offset into a lookup table that says where to jump next. To anyone with a bit of experience with assembly language programming, the computed goto immediately makes sense because it just exposes a common instruction that most modern CPU architectures have - jump through a register (aka. indirect jump). computed goto 比 switch 效能更高的原因: The switch does a bit more per iteration because of bounds checking. The effects of hardware branch prediction. C99: If no converted case constant expression matches and there is no default label, no part of the switch body is executed. 因为标准的这个要求,所以编译器对于 switch 会生成额外的 safe 检查代码,以符合上面情形的 “no part of the switch body is executed” 的要求。 Since the switch statement has a single “master jump” that dispatches all opcodes, predicting its destination is quite difficult. On the other hand, the computed goto statement is compiled into a separate jump per opcode, so given that instructions often come in pairs it’s much easier for the branch predictor to “home in” on the various jumps correctly. 作者提到,ta 个人认为分支预测是导致效能差异的主要因素: I can’t say for sure which one of the two factors weighs more in the speed difference between the switch and the computed goto, but if I had to guess I’d say it’s the branch prediction. 除此之外,有这篇文章的 disassembly 部分可以得知,switch 的底层是通过所谓的 jump table 来实作的: 引用 How did I figure out which part of the code handles which opcode? Note that the “table jump” is done with: jmpq *0x400b20(,%rdx,8) bounds checking 是在 switch 中執行的一個環節,每次迴圈中檢查是否有 default case 的狀況,即使程式中的 switch 沒有用到 default case,編譯期間仍會產生強制檢查的程式,所以 switch 會較 computed goto 多花一個 bounds checking 的步驟 branch prediction 的部分,switch 需要預測接下來跳到哪個分支 case,而 computed goto 則是在每個 instruction 預測下一個 instruction,這之中比較直覺的想法是 computed goto 的prediction可以根據上個指令來預測,但是 switch 的prediction每次預測沒辦法根據上個指令,因此在 branch prediction accuracy 上 computed goto 會比較高。 所以在实际中大部分也是采取 computed goto 来实作 VM: Ruby 1.9 (YARV): also uses computed goto. Dalvik (the Android Java VM): computed goto Lua 5.2: uses a switch ","date":"2024-04-05","objectID":"/posts/c-control-flow/:3:0","tags":["Sysprog","C","Control flow"],"title":"你所不知道的 C 语言: goto 和流程控制篇","uri":"/posts/c-control-flow/"},{"categories":["C","Linux Kernel Internals"],"content":"do {…} while (0) 宏 用于避免 dangling else,即 if 和 else 未符合预期的配对 (常见于未使用 {} 包裹) Stack Overflow: C multi-line macro: do/while(0) vs scope block 我写了 相关笔记 记录在前置处理器篇。 ","date":"2024-04-05","objectID":"/posts/c-control-flow/:4:0","tags":["Sysprog","C","Control flow"],"title":"你所不知道的 C 语言: goto 和流程控制篇","uri":"/posts/c-control-flow/"},{"categories":["C","Linux Kernel Internals"],"content":"用 goto 实作 RAII 开发风格 RAII in C If you have functions or control flows that allocate resources and a failure occurs, then goto turns out to be one of the nicest ways to unwind all resources allocated before the point of failure. Linux 核心中的实作: shmem.c 以 goto 为关键字进行检索 ","date":"2024-04-05","objectID":"/posts/c-control-flow/:5:0","tags":["Sysprog","C","Control flow"],"title":"你所不知道的 C 语言: goto 和流程控制篇","uri":"/posts/c-control-flow/"},{"categories":["C","Linux Kernel Internals"],"content":"检阅 C 语言规格书 ISO/IEC 9899:201x Committee Draft 6.8.6 Jump statements A jump statement causes an unconditional jump to another place. The identifier in a goto statement shall name a label located somewhere in the enclosing function. A goto statement shall not jump from outside the scope of an identifier having a variably modified type to inside the scope of that identifier. 规格书后面的例子也值得一看 (特别是当你看不懂规格书严格的英语语法想表达什么的时候 🤣) 从规格书中也可以得知,goto 虽然是无条件跳转 (对应汇编语言的 jmp 这类无条件跳转指令),但它的跳转范围是有限制的 (jump to another place),而不是可以跳转到任意程式码 (这也是为什么 setjmp/longjmp 被称为「长跳转」的原因,与 goto 这类「短跳转」相对应)。 相关实作: CPython 的 Modules/_asynciomodule.c 以 goto 为关键字进行检索 Modern C 作者也总结了 3 项和 goto 相关的规范 (大可不必视 goto 为洪水猛兽,毕竟我们有规范作为指导是不是): Rule 2.15.0.1: Labels for goto are visible in the whole function that contains them. Rule 2.15.0.2: goto can only jump to a label inside the same function. Rule 2.15.0.3: goto should not jump over variable initializations. ","date":"2024-04-05","objectID":"/posts/c-control-flow/:6:0","tags":["Sysprog","C","Control flow"],"title":"你所不知道的 C 语言: goto 和流程控制篇","uri":"/posts/c-control-flow/"},{"categories":["C","Linux Kernel Internals"],"content":"和想象中不同的 switch-case switch-case 语句中的 case 部分本质上是 label,所以使用其它语句 (例如 if) 将其包裹起来并不影响 switch 语句的跳转。 Something You May Not Know About the Switch Statement in C/C++ How to Get Fired Using Switch Statements \u0026 Statement Expressions ","date":"2024-04-05","objectID":"/posts/c-control-flow/:7:0","tags":["Sysprog","C","Control flow"],"title":"你所不知道的 C 语言: goto 和流程控制篇","uri":"/posts/c-control-flow/"},{"categories":["C","Linux Kernel Internals"],"content":"Duff’s Device 这个技巧常用于内存数据的复制,类似于 memcpy Wikipedia: Duff’s Device Duff’s Device 的详细解释 Tom Duff 本人的解释 ","date":"2024-04-05","objectID":"/posts/c-control-flow/:8:0","tags":["Sysprog","C","Control flow"],"title":"你所不知道的 C 语言: goto 和流程控制篇","uri":"/posts/c-control-flow/"},{"categories":["C","Linux Kernel Internals"],"content":"co-routine 应用 Wikipedia: Coroutine 不借助操作系统也可以实作出多工交执行的 illusion (通过 switch-case 黑魔法来实现 🤣) ","date":"2024-04-05","objectID":"/posts/c-control-flow/:9:0","tags":["Sysprog","C","Control flow"],"title":"你所不知道的 C 语言: goto 和流程控制篇","uri":"/posts/c-control-flow/"},{"categories":["RISC-V"],"content":" pretask 作为社区入门探索,目的是帮助实习生一起搭建工作环境,熟悉 oerv 的工作流程和合作方式。pretask 分为三个步骤: 任务一:通过 QEMU 仿真 RISC-V 环境并启动 openEuler RISC-V 系统,设法输出 neofetch 结果并截图提交 任务二:在 openEuler RISC-V 系统上通过 obs 命令行工具 osc,从源代码构建 RISC-V 版本的 rpm 包,比如 pcre2。提示首先需要在 openEuler 的 OBS 上注册账号 任务三:尝试使用 qemu user \u0026 nspawn 或者 docker 加速完成任务二 ","date":"2024-03-28","objectID":"/posts/oerv-pretask/:0:0","tags":["RISC-V","openEuler","QEMU","Neofetch","Container","chroot","nspawn"],"title":"OERV 之 Pretask","uri":"/posts/oerv-pretask/"},{"categories":["RISC-V"],"content":"Pretask 1: Neofetch 任务一:通过 QEMU 仿真 RISC-V 环境并启动 openEuler RISC-V 系统,设法输出 neofetch 结果并截图提交 由于工作内容是对软件包进行: 编译 -\u003e 失败 -\u003e 定位问题 -\u003e 修复 -\u003e 重新编译,所以我们倾向于直接从源码编译,根据 neofetch wiki 从 git 拉取最新数据进行构建: # enter into euler openEuler RISC-V QEMU $ git clone https://github.com/dylanaraps/neofetch $ cd neofetch $ make install $ neofetch ","date":"2024-03-28","objectID":"/posts/oerv-pretask/:1:0","tags":["RISC-V","openEuler","QEMU","Neofetch","Container","chroot","nspawn"],"title":"OERV 之 Pretask","uri":"/posts/oerv-pretask/"},{"categories":["RISC-V"],"content":"Pretask 2: Open Build Service (OBS) 任务二:在 openEuler RISC-V 系统上通过 obs 命令行工具 osc,从源代码构建 RISC-V 版本的 rpm 包,比如 pcre2。提示首先需要在 openEuler 的 OBS 上注册账号 观看教学影片: openEuler构建之OBS使用指导 - bilibili 并对比阅读 Beginnerʼs Guide openSUSE:Build Service 新手入门 如何通过OpenSUSE Open Build Service(OBS)构建Debian包 for RISCV-64 了解掌握 OBS 的基本概念、OBS 网页 以及 OSC 命令行工具 的使用方法。 这部分内容很重要,和后续工作内容息息相关,在这里不要图快,打牢基础比较好。 OBS 的 Package 中 _service 配置文件,revision 字段是对应与 Git 仓库的 commit id (如果你使用的 Source Code Management (SCM) 方式是 Git 托管的话) 参考仓库: https://gitee.com/zxs-un/doc-port2riscv64-openEuler 内的相关文档 osc命令工具的安装与~/.oscrc配置文件 在 openEuler 上安装 osc build 本地构建工具 使用 osc build 在本地构建 openEuler OBS 服务端的内容 在 openEuler RISC-V QEMU 虚拟机内完成 OBS、OSC 相关基础设施的安装: # install osc and build $ sudo yum install osc build # configure osc in ~/.oscrc [general] apiurl = https://build.openeuler.openatom.cn no_verify = 1 # 未配置证书情况下不验证 [https://build.openeuler.openatom.cn] user=username # 用户名 pass=password # 明文密码 trusted_prj=openEuler:selfbuild:function # 此项目为openEuler:Mailine:RISC-V项目的依赖库 在 openEuler RISC-V QEMU 虚拟机内完成 pcre2 的本地编译构建: # 选定 pcre2 包 $ osc co openEuler:Mainline:RISC-V/pcre2 $ cd openEuler\\:Mainline\\:RISC-V/pcre2/ # 更新并下载相关文件到本地 $ osc up -S # 重命名刚刚下载的文件 $ rm -f _service;for file in `ls | grep -v .osc`;do new_file=${file##*:};mv $file $new_file;done # 查看一下仓库信息,方便后续构建 $ osc repos standard_riscv64 riscv64 mainline_gcc riscv64 # 指定仓库和架构并进行本地构建 $ osc build standard_riscv64 riscv64 总计用时 1301s ","date":"2024-03-28","objectID":"/posts/oerv-pretask/:2:0","tags":["RISC-V","openEuler","QEMU","Neofetch","Container","chroot","nspawn"],"title":"OERV 之 Pretask","uri":"/posts/oerv-pretask/"},{"categories":["RISC-V"],"content":"Pretask 3: 容器加速构建 任务三:尝试使用 qemu user \u0026 nspawn 或者 docker 加速完成任务二 参考 文档 由于 deepin 20.9 的 Python3 版本仅为 3.7,构建 osc 和 qemu 显然不太够,所以我通过 KVM 构建了一个 openEuler 22.03 LTS SP3 的虚拟机,在上面进行这项任务。 Deepin 20.9 KVM 安装和管理 openEuler 22.03 LTS SP3 编译 QEMU 时常见错误修正: ERROR: Python package 'sphinx' was not found nor installed. $ sudo yum install python3-sphinx ERROR: cannot find ninja $ sudo yum install ninja-build openEuler 22.03 LTS SP3 没有预先安装好 nspawn,所以需要手动安装: $ sudo yum install systemd-container systemd-nspawn 其余同任务二。 尝试使用 nspawn 来构建 pcre2: $ osc build standard_riscv64 riscv64 --vm-type=nspawn 会遇到以下报错 (且经过相当多时间排错,仍无法解决该问题,个人猜测是平台问题): can't locate file/copy.pm: /usr/lib64/perl5/vendor_perl/file/copy.pm: permission denied at /usr/bin/autoreconf line 49. 所以退而求其次,使用 chroot 来构建: $ osc build standard_riscv64 riscv64 --vm-type=chroot 总计用时 749s,比 qemu-system-riscv64 快了将近 2 倍,效能提升相当可观。 ","date":"2024-03-28","objectID":"/posts/oerv-pretask/:3:0","tags":["RISC-V","openEuler","QEMU","Neofetch","Container","chroot","nspawn"],"title":"OERV 之 Pretask","uri":"/posts/oerv-pretask/"},{"categories":["RISC-V"],"content":"References https://gitee.com/zxs-un/doc-port2riscv64-openEuler/blob/master/doc/build-osc-config-oscrc.md https://gitee.com/zxs-un/doc-port2riscv64-openEuler/blob/master/doc/build-osc-build-tools.md https://gitee.com/zxs-un/doc-port2riscv64-openEuler/blob/master/doc/build-osc-obs-service.md https://gitee.com/openeuler/RISC-V/blob/master/doc/tutorials/qemu-user-mode.md https://stackoverflow.com/questions/5308816/how-can-i-merge-multiple-commits-onto-another-branch-as-a-single-squashed-commit ","date":"2024-03-28","objectID":"/posts/oerv-pretask/:4:0","tags":["RISC-V","openEuler","QEMU","Neofetch","Container","chroot","nspawn"],"title":"OERV 之 Pretask","uri":"/posts/oerv-pretask/"},{"categories":["Toolkit"],"content":"本篇主要介绍在 deepin20.9 操作系统平台下,使用 KVM 虚拟化技术来创建和安装 Linux 发行版,并以创建安装 openEuler 22.03 LTS SP3 的 KVM 虚拟机作为示范,让学员领略 KVM 虚拟化技术的强大魅力。 ","date":"2024-03-28","objectID":"/posts/deepin-kvm/:0:0","tags":["Linux","Deepin","KVM","QEMU","openEuler"],"title":"Deepin 20.9 KVM 安装和管理","uri":"/posts/deepin-kvm/"},{"categories":["Toolkit"],"content":"什么是虚拟化? 什么是虚拟化技术?KVM 虚拟化和 Virtual Box、VMware 这类虚拟机软件的区别是什么?请阅读下面的这篇文章。 KVM 与 VMware 的区别盘点 ","date":"2024-03-28","objectID":"/posts/deepin-kvm/:1:0","tags":["Linux","Deepin","KVM","QEMU","openEuler"],"title":"Deepin 20.9 KVM 安装和管理","uri":"/posts/deepin-kvm/"},{"categories":["Toolkit"],"content":"配置虚拟化环境 首先需要检查 CPU 是否支持虚拟化 (以 Intel 处理器为例): # intel vmx,amd svm $ egrep '(vmx|svm)' /proc/cpuinfo ...vmx... $ lscpu | grep Virtualization Virtualization: VT-x 检查 KVM 模块是否已加载: $ lsmod | grep -i kvm kvm_intel 278528 11 kvm 901120 1 kvm_intel 确保 CPU 支持虚拟化并且 KVM 模块已被加载,接下来是安装 QEMU 和 virt-manager (虚拟系统管理器)。直接通过 apt 安装的 QEMU 版本过低,而通过 GitHub 下载最新的 QEMU 源码编译安装需要Python3.9,而 deepin 20.9 的 Python 3 版本是 3.7 (保险起见不要随便升级),所以折中一下,编译安装 QEMU 7.2.0 🤣 安装 QEMU: $ wget https://download.qemu.org/qemu-7.2.0.tar.xz $ tar xvJf qemu-7.2.0.tar.xz $ mv qemu-7.2.0 qemu $./configure $ sudo make -j$(nproc) # in ~/.bashrc export PATH=$PATH:/path/to/qemu/build 安装 virt-manager: $ sudo apt install virt-manager ","date":"2024-03-28","objectID":"/posts/deepin-kvm/:2:0","tags":["Linux","Deepin","KVM","QEMU","openEuler"],"title":"Deepin 20.9 KVM 安装和管理","uri":"/posts/deepin-kvm/"},{"categories":["Toolkit"],"content":"安装 openEuler KVM 虚拟机 可以在启动器看到一个虚拟机管理应用图标,如下: 点击打开 (需要输入密码认证,以下图片中的 “本地” 可能会显示为 “QEMU/KVM”): 接下来创建虚拟机: 下图的操作系统选择对应的类型 (可以在 这里 下载 openEuler 22.03 LTS SP3 镜像,对于 openEuler 这类未被收录的类型,选择 Generic): 这里选择 iso 镜像后可能会显示路径搜索问题,选择 “是” 将该路径加入存储池即可 接下来是处理器和内存配置,建议配置 8 核 8G 内存,根据自己物理机配置选择即可: 接下来是虚拟磁盘的大小设置和存放位置,建议选择自定义存储路径,并搭配 更改 KVM 虚拟机默认存储路径,特别是如果你的根目录空间不太够的情况: 在对应的存储卷中创建虚拟磁盘 (注意: 如果你更改了默认存储路径,请选择对应的存储池而不是 default): 创建虚拟磁盘 (名称可以自定义,分配默认初始为 0,它会随着虚拟机使用而增大,当然也可以直接将分配等于最大容量,这样就会直接分配相应的磁盘空间,玩过虚拟机的学员应该很熟悉): 接下来自定义虚拟机名称并生成虚拟机即可: 最后就是熟悉的安装界面: 参考 这里 安装 openEuler 即可。 ","date":"2024-03-28","objectID":"/posts/deepin-kvm/:3:0","tags":["Linux","Deepin","KVM","QEMU","openEuler"],"title":"Deepin 20.9 KVM 安装和管理","uri":"/posts/deepin-kvm/"},{"categories":["Toolkit"],"content":"透过 SSH 连接 KVM 虚拟机 首先先检查 Guest OS 上 ssh 服务是否开启 (一般是开启的): $ sudo systemctl status sshd sshd.service - OpenSSH server daemon Loaded: loaded (/usr/lib/systemd/system/sshd.service; enabled; vendor preset: enabled) Active: active (running) since Thu 2024-03-28 14:40:15 CST; 20min ago ... 然后在 Guest OS 上获取其 IP 地址 (ens3 的 inet 后的数字即是,openEuler 启动时也会输出一下 IP 地址): $ ip addr 在 Host OS 上通过 ssh 连接登录 GuestOS: $ ssh user@ip # user: user name in the guest os # ip ip addr of guest os ","date":"2024-03-28","objectID":"/posts/deepin-kvm/:4:0","tags":["Linux","Deepin","KVM","QEMU","openEuler"],"title":"Deepin 20.9 KVM 安装和管理","uri":"/posts/deepin-kvm/"},{"categories":["Toolkit"],"content":"Development Tools 由于是最小安装,很多趁手的工具都没有,俗话说“工欲善其事,必先利其器”,所以先安装必要的开发工具。幸好 openEuler 提供了整合包 Development Tools,直接安装即可: $ sudo yum group install -y \"Development Tools\" ","date":"2024-03-28","objectID":"/posts/deepin-kvm/:5:0","tags":["Linux","Deepin","KVM","QEMU","openEuler"],"title":"Deepin 20.9 KVM 安装和管理","uri":"/posts/deepin-kvm/"},{"categories":["Toolkit"],"content":"Neofetch 安装 neofetch 来酷炫地输出一下系统信息: $ git clone https://github.com/dylanaraps/neofetch $ cd neofetch $ make install $ neofetch ","date":"2024-03-28","objectID":"/posts/deepin-kvm/:6:0","tags":["Linux","Deepin","KVM","QEMU","openEuler"],"title":"Deepin 20.9 KVM 安装和管理","uri":"/posts/deepin-kvm/"},{"categories":["Toolkit"],"content":"References 使用 KVM 安装和管理 deepin Linux 下使用 KVM 虚拟机安装 OpenEuler 系统 KVM 更改虚拟机默认存储路径 实践 KVM on Deepin ","date":"2024-03-28","objectID":"/posts/deepin-kvm/:7:0","tags":["Linux","Deepin","KVM","QEMU","openEuler"],"title":"Deepin 20.9 KVM 安装和管理","uri":"/posts/deepin-kvm/"},{"categories":["C","Linux Kernel Internals"],"content":" C 语言之所以不需要时常发布新的语言特性又可以保持活力,前置处理器 (preprocessor) 是很重要的因素,有心者可进行「扩充」C 语言。 原文地址 ","date":"2024-03-25","objectID":"/posts/c-preprocessor/:0:0","tags":["Sysprog","C","Preprocessor"],"title":"你所不知道的 C 语言: 前置处理器应用篇","uri":"/posts/c-preprocessor/"},{"categories":["C","Linux Kernel Internals"],"content":"不要小看 preprocessor man gcc -D name Predefine name as a macro, with definition 1. -D name=definition The contents of definition are tokenized and processed as if they appeared during translation phase three in a #define directive. In particular, the definition is truncated by embedded newline characters. 在 Makefile 中往 CFLAGS 加入 -D’;’=’;’ 这类搞怪信息,会导致编译时出现一些不明所以的编译错误 (恶搞专用 🤣) 早期的 C++ 是和 C 语言兼容的,那时候的 C++ 相当于 C 语言的一种 preprocessor,将 C++ 代码预编译为对应的 C 语言代码,具体可以参考 C with Classes。事实上现在的 C++ 和 C 语言早已分道扬镳,形同陌路,虽然语法上有相似的地方,但请把这两个语言当成不同的语言看待 🤣 体验一下 C++ 模版 (template) 的威力 ❌ 丑陋 ✔️ : C 语言: 大道至简 ✅ ","date":"2024-03-25","objectID":"/posts/c-preprocessor/:1:0","tags":["Sysprog","C","Preprocessor"],"title":"你所不知道的 C 语言: 前置处理器应用篇","uri":"/posts/c-preprocessor/"},{"categories":["C","Linux Kernel Internals"],"content":"Object Oriented Programming 面向对象编程时,善用前置处理器可大幅简化和开发 #: Stringizing convert a macro argument into a string constant ##: Concatenation merge two tokens into one while expanding macros. 宏的实际作用: generate (产生/生成) 程式码 Rust 的过程宏 (procedural macros) 进一步强化了这一目的,可以自定义语法树进行代码生成。 可以 gcc -E -P 来观察预处理后的输出: man gcc -E Stop after the preprocessing stage; do not run the compiler proper. The output is in the form of preprocessed source code, which is sent to the standard output. Input files that don't require preprocessing are ignored. -P Inhibit generation of linemarkers in the output from the preprocessor. This might be useful when running the preprocessor on something that is not C code, and will be sent to a program which might be confused by the linemarkers. 可以依据不同时期的标准来对 C 源程序编译生成目标文件: Feature Test Macros The exact set of features available when you compile a source file is controlled by which feature test macros you define. 使用 gcc -E -P 观察 objects.h 预处理后的输出,透过 make 和 make check 玩一下这个最简单光线追踪引擎 GitHub: raytracing object oriented programming 不等于 class based programming, 只需要满足 Object-oriented programming (OOP) is a computer programming model that organizes software design around data, or objects, rather than functions and logic. 这个概念的就是 OOP。 Source ","date":"2024-03-25","objectID":"/posts/c-preprocessor/:2:0","tags":["Sysprog","C","Preprocessor"],"title":"你所不知道的 C 语言: 前置处理器应用篇","uri":"/posts/c-preprocessor/"},{"categories":["C","Linux Kernel Internals"],"content":"C11: _Generic 阅读 C11 规格书 6.5.1.1 Generic selection The controlling expression of a generic selection is not evaluated. If a generic selection has a generic association with a type name that is compatible with the type of the controlling expression, then the result expression of the generic selection is the expression in that generic association. Otherwise, the result expression of the generic selection is the expression in the default generic association. None of the expressions from any other generic association of the generic selection is evaluated. #define cbrt(X) \\ _Generic((X), \\ long double: cbrtl, \\ default: cbrt, \\ const float: cbrtf, \\ float: cbrtf \\ )(X) 经过 func.c/func.cpp 的输出对比,C++ 模版在字符类型的的判定比较准确,C11 的 _Generic 会先将 char 转换成 int 导致结果稍有瑕疵,这是因为在 C 语言中字符常量 (例如 ‘a’) 的类型是 int 而不是 char。 Stack Overflow: What to do to make ‘_Generic(‘a’, char : 1, int : 2) == 1’ true ","date":"2024-03-25","objectID":"/posts/c-preprocessor/:3:0","tags":["Sysprog","C","Preprocessor"],"title":"你所不知道的 C 语言: 前置处理器应用篇","uri":"/posts/c-preprocessor/"},{"categories":["C","Linux Kernel Internals"],"content":"Block Wikipedia: Blocks (C language extension) Blocks are a non-standard extension added by Apple Inc. to Clang’s implementations of the C, C++, and Objective-C programming languages that uses a lambda expression-like syntax to create closures within these languages. Like function definitions, blocks can take arguments, and declare their own variables internally. Unlike ordinary C function definitions, their value can capture state from their surrounding context. A block definition produces an opaque value which contains both a reference to the code within the block and a snapshot of the current state of local stack variables at the time of its definition. The block may be later invoked in the same manner as a function pointer. The block may be assigned to variables, passed to functions, and otherwise treated like a normal function pointer, although the application programmer (or the API) must mark the block with a special operator (Block_copy) if it’s to be used outside the scope in which it was defined. 使用 BLock 可以减少宏展开时的重复计算次数。目前 clang 是支持 Block 这个扩展的,但是在编译时需要加上参数 -fblocks: $ clang -fblocks blocks-test.c -lBlocksRuntime 同时还需要 BlocksRuntime 这个库,按照仓库 README 安装即可: # clone repo $ git clone https://github.com/mackyle/blocksruntime.git $ cd blocksruntime/ # building $ ./buildlib # testing $ ./checktests # installing $ sudo ./installlib 除了 Block 之外,常见的避免 double evaluation 的方法还有利用 typeof 提前计算: #define DOUBLE(a) ((a) + (a)) #define DOUBLE(a) ({ \\ __typeof__(a) _x_in_DOUBLE = (a); \\ _x_in_DOUBLE + _x_in_DOUBLE; \\ }) ","date":"2024-03-25","objectID":"/posts/c-preprocessor/:4:0","tags":["Sysprog","C","Preprocessor"],"title":"你所不知道的 C 语言: 前置处理器应用篇","uri":"/posts/c-preprocessor/"},{"categories":["C","Linux Kernel Internals"],"content":"ARRAY_SIZE 宏 // get the number of elements in array #define ARRAY_SIZE(arr) (sizeof(arr) / sizeof((arr)[0])) 这样实作的 ARRAY_SIZE 宏有很大的隐患,例如它无法对传入的 arr 进行类型检查,如果碰上不合格的 C 程序员,在数组隐式转换成指针后使用 ARRAY_SIZE 宏会得到非预期的结果,我们需要在编译器就提醒程序员不要错用这个宏。 注意 阅读以下博客以理解 Linux 核心的 ARRAY_SIZE 原理机制和实作手法: Linux Kernel: ARRAY_SIZE() Linux 核心的 ARRAY_SIZE 宏在上面那个简陋版的宏的基础上,加上了类型检查,保证传入的是数组而不是指针: #define ARRAY_SIZE(arr) (sizeof(arr) / sizeof((arr)[0]) + __must_be_array(arr)) /* \u0026a[0] degrades to a pointer: a different type from an array */ #define __must_be_array(a) BUILD_BUG_ON_ZERO(__same_type((a), \u0026(a)[0])) /* Are two types/vars the same type (ignoring qualifiers)? */ #ifndef __same_type # define __same_type(a, b) __builtin_types_compatible_p(typeof(a), typeof(b)) #endif 6.54 Other built-in functions provided by GCC You can use the built-in function __builtin_types_compatible_p to determine whether two types are the same. This built-in function returns 1 if the unqualified versions of the types type1 and type2 (which are types, not expressions) are compatible, 0 otherwise. The result of this built-in function can be used in integer constant expressions. 6.6 Referring to a Type with typeof Another way to refer to the type of an expression is with typeof. The syntax of using of this keyword looks like sizeof, but the construct acts semantically like a type name defined with typedef. 所以 Linux 核心的 ARRAY_SIZE 宏额外加上了 __must_be_array 宏,但是这个宏在编译成功时会返回 0,编译失败自然就不需要考虑返回值了 🤣 所以它起到的作用是之前提到的类型检查,透过 BUILD_BUG_ON_ZERO 宏和 __same_type 宏。 从 Linux 核心 「提炼」 出的 array_size _countof Macro ","date":"2024-03-25","objectID":"/posts/c-preprocessor/:5:0","tags":["Sysprog","C","Preprocessor"],"title":"你所不知道的 C 语言: 前置处理器应用篇","uri":"/posts/c-preprocessor/"},{"categories":["C","Linux Kernel Internals"],"content":"do { … } while (0) 宏 用于避免 dangling else,即 if 和 else 未符合预期的配对 (常见于未使用 {} 包裹) 考虑以下情形: #define handler(cond) if (cond) foo() if (\u003ccondition1\u003e) handler(\u003cconditional2\u003e) else bar() 这个写法乍一看没什么问题,但是我们把它展开来看一下: if (\u003ccondition1\u003e) if (\u003cconditional2\u003e) foo() else bar() 显然此时由于未使用 {} 区块进行包裹,导致 else 部分与 handler 宏的 if 逻辑进行配对了。do {...} while (0) 宏的作用就是提供类似于 {} 区块的隔离性 (因为它的循环体只能执行一遍 🤣) 注意 下面的讨论是关于为什么要使用 do {...} while(0) 而不是 {},非常值得一读: Stack Overflow: C multi-line macro: do/while(0) vs scope block The more elegant solution is to make sure that macro expand into a regular statement, not into a compound one. 主要是考虑到对包含 {} 的宏,像一般的 statement 一样加上 ; 会导致之前的 if 语句结束,从而导致后面的 else 语句无法配对进而编译失败,而使用 do {...} while (0) 后面加上 ; 并不会导致这个问题。 ","date":"2024-03-25","objectID":"/posts/c-preprocessor/:6:0","tags":["Sysprog","C","Preprocessor"],"title":"你所不知道的 C 语言: 前置处理器应用篇","uri":"/posts/c-preprocessor/"},{"categories":["C","Linux Kernel Internals"],"content":"应用: String switch in C 这篇博文展示了如何在 C 语言中对 string 使用 switch case: String switch in C #define STRING_SWITCH_L(s) switch (*((int32_t *)(s)) | 0x20202020) #define MULTICHAR_CONSTANT(a,b,c,d) ((int32_t)((a) | (b) \u003c\u003c 8 | (c) \u003c\u003c 16 | (d) \u003c\u003c 24)) Note that STRING_SWITCH_L performs a bitwise OR with the 32-bit integral value – this is a fast means of lowering the case of four characters at once. 这里有一个 | 0x20202020 的位运算操作,这个运算的作用是将对应的字符转换成对应小写字符,具体可以参考本人于数值系统篇的 记录 (提示: 字符 ' ' 对应的 ASCII 编码为 0x20)。 然后 MULTICHAR_CONSTANT 则是将参数按小端字节序计算出对应的数值。 这篇博文说明了在 C 语言中对 string 使用 switch case 提升效能的原理 (除此之外还讲解了内存对齐相关的效能问题): More on string switch in C ","date":"2024-03-25","objectID":"/posts/c-preprocessor/:7:0","tags":["Sysprog","C","Preprocessor"],"title":"你所不知道的 C 语言: 前置处理器应用篇","uri":"/posts/c-preprocessor/"},{"categories":["C","Linux Kernel Internals"],"content":"应用: Linked List 的各式变种 宏和函数调用的效能对比: Simple code for checking the speed difference between function call and macro 在進行函式呼叫時,我們除了需要把參數推入特定的暫存器或是堆疊,還要儲存目前暫存器的值到堆疊。在函式呼叫數量少的狀況,影響不顯著,但隨著數量增長,就會導致程式運作比用 macro 實作時慢。 这也是为什么 Linux 核心对于 linked list 的功能大量采用宏来实现。 静态的 linked list 初始化需要使用到 compound literal: C99 6.5.2.5 Compound literals The type name shall specify an object type or an array of unknown size, but not a variable length array type. A postfix expression that consists of a parenthesized type name followed by a braceenclosed list of initializers is a compound literal. It provides an unnamed object whose value is given by the initializer list. If the type name specifies an array of unknown size, the size is determined by the initializer list as specified in 6.7.8, and the type of the compound literal is that of the completed array type. Otherwise (when the type name specifies an object type), the type of the compound literal is that specified by the type name. In either case, the result is an lvalue. C99 6.7.8 Initialization Each brace-enclosed initializer list has an associated current object. When no designations are present, subobjects of the current object are initialized in order according to the type of the current object: array elements in increasing subscript order, structure members in declaration order, and the first named member of a union. In contrast, a designation causes the following initializer to begin initialization of the subobject described by the designator. Initialization then continues forward in order, beginning with the next subobject after that described by the designator. ","date":"2024-03-25","objectID":"/posts/c-preprocessor/:8:0","tags":["Sysprog","C","Preprocessor"],"title":"你所不知道的 C 语言: 前置处理器应用篇","uri":"/posts/c-preprocessor/"},{"categories":["C","Linux Kernel Internals"],"content":"其它应用 ","date":"2024-03-25","objectID":"/posts/c-preprocessor/:9:0","tags":["Sysprog","C","Preprocessor"],"title":"你所不知道的 C 语言: 前置处理器应用篇","uri":"/posts/c-preprocessor/"},{"categories":["C","Linux Kernel Internals"],"content":"Unit Test 测试框架本质是提供一个框架模版,让程序员将精力放在测试逻辑的编写上。使用 C 语言的宏配合前置处理器,可以很方便地实现这个功能。 unity/unity_fixture.h Google Test ","date":"2024-03-25","objectID":"/posts/c-preprocessor/:9:1","tags":["Sysprog","C","Preprocessor"],"title":"你所不知道的 C 语言: 前置处理器应用篇","uri":"/posts/c-preprocessor/"},{"categories":["C","Linux Kernel Internals"],"content":"Object Model 同样的,使用 C 语言的宏和前置处理器,可以让 C 语言拥有 OOP 的表达能力: ObjectC: use as a superset of the C language adding a lot of modern concepts missing in C ","date":"2024-03-25","objectID":"/posts/c-preprocessor/:9:2","tags":["Sysprog","C","Preprocessor"],"title":"你所不知道的 C 语言: 前置处理器应用篇","uri":"/posts/c-preprocessor/"},{"categories":["C","Linux Kernel Internals"],"content":"Exception Handling 通过宏和 setjmp/longjmp 可以很轻松地实作出 C 语言的异常机制: ExtendedC library extends the C programming language through complex macros and other tricks that increase productivity, safety, and code reuse without needing to use a higher-level language such as C++, Objective-C, or D. include/exception.h ","date":"2024-03-25","objectID":"/posts/c-preprocessor/:9:3","tags":["Sysprog","C","Preprocessor"],"title":"你所不知道的 C 语言: 前置处理器应用篇","uri":"/posts/c-preprocessor/"},{"categories":["C","Linux Kernel Internals"],"content":"ADT 与之前所提的 Linux 核心的 linked list 类似,使用宏取代函数调用可以降低 抽象数据类型 (ADT) 的相关操作的效能损失: pearldb: A Lightweight Durable HTTP Key-Value Pair Database in C klib/ksort.h 通过宏展开实作的排序算法 成功 Linux 核心原始程式码也善用宏来扩充 ","date":"2024-03-25","objectID":"/posts/c-preprocessor/:9:4","tags":["Sysprog","C","Preprocessor"],"title":"你所不知道的 C 语言: 前置处理器应用篇","uri":"/posts/c-preprocessor/"},{"categories":["C","Linux Kernel Internals"],"content":"Linux 核心宏: BUILD_BUG_ON_ZERO 原文地址 简单来说就是编译时期就进行检查的 assert,我写了 相关笔记 来说明它的原理。 ","date":"2024-03-25","objectID":"/posts/c-preprocessor/:10:0","tags":["Sysprog","C","Preprocessor"],"title":"你所不知道的 C 语言: 前置处理器应用篇","uri":"/posts/c-preprocessor/"},{"categories":["C","Linux Kernel Internals"],"content":"Linux 核心原始程式码宏: max, min 原文地址 ","date":"2024-03-25","objectID":"/posts/c-preprocessor/:11:0","tags":["Sysprog","C","Preprocessor"],"title":"你所不知道的 C 语言: 前置处理器应用篇","uri":"/posts/c-preprocessor/"},{"categories":["C","Linux Kernel Internals"],"content":"Linux 核心原始程式码宏: contain_of 原文地址","date":"2024-03-25","objectID":"/posts/c-preprocessor/:12:0","tags":["Sysprog","C","Preprocessor"],"title":"你所不知道的 C 语言: 前置处理器应用篇","uri":"/posts/c-preprocessor/"},{"categories":["Systems"],"content":"操作系统使用正确的抽象使构造庞大的计算机软件/硬件生态从不可能变为可能。这门课围绕操作系统是 如何设计 (应用程序视角)、怎样实现 (硬件视角) 两个角度展开,分为两个主要部分: 原理课 (并发/虚拟化/持久化):以教科书内容为主,介绍操作系统的原理性内容。课程同时注重讲解操作系统相关的代码实现和编程技巧,包括操作系统中常用的命令行/代码工具、教学操作系统 xv6 的代码讲解等 理解操作系统最重要的实验部分: Mini labs (应用程序视角;设计):通过实现一系列有趣的 (黑科技) 代码理解操作系统中对象存在的意义和操作系统 API 的使用方法、设计理念 OS labs (计算机硬件视角;实现):基于一个简化的硬件抽象层实现多处理器操作系统内核,向应用程序提供一些基础操作系统 API 时隔一年,在跟随 B 站 up 主 @踌躇月光 从零编写一个基于 x86 架构的内核 Txics 后,终于可以跟得上 @绿导师 的课程了 🤣 这次以 2022 年的 OS 课程 作为主线学习,辅以 2023 年课程 和 2024 年课程 的内容加以补充、扩展,并搭配南大的 ICS 课程进行作业,后期可能会加入清华大学的 rCore 实验 (待定)。 tux 问题 JYY 2022 年的 OSDI 课程讲义和阅读材料是分开的,2023 年和 2024 年进行了改进,讲义和阅读材料合并成类似于共笔的材料,所以下面有一些 lectures 是没有阅读材料链接的。 ","date":"2024-03-24","objectID":"/posts/nju-osdi/:0:0","tags":["Linux","QEMU","RISC-V"],"title":"南京大学 操作系统: 设计与实现 重点提示","uri":"/posts/nju-osdi/"},{"categories":["Systems"],"content":"第 1 周: 绪论 ","date":"2024-03-24","objectID":"/posts/nju-osdi/:1:0","tags":["Linux","QEMU","RISC-V"],"title":"南京大学 操作系统: 设计与实现 重点提示","uri":"/posts/nju-osdi/"},{"categories":["Systems"],"content":"操作系统概述 (为什么要学操作系统) 信息 直播录影 / 讲义页面 一个 Talk 的经典三段式结构: Why? What? How? (这个真是汇报的大杀器 🤣) 1950s 的计算机 I/O 设备的速度已经严重低于处理器的速度,中断机制出现 (1953) 希望使用计算机的人越来越多;希望调用 API 而不是直接访问设备 批处理系统 = 程序的自动切换 (换卡) + 库函数 API 操作系统中开始出现 设备、文件、任务 等对象和 API 1960s 的计算机 可以同时载入多个程序而不用 “换卡” 了 能载入多个程序到内存且灵活调度它们的管理程序,包括程序可以调用的 API 既然操作系统已经可以在程序之间切换,为什么不让它们定时切换呢? 操作系统机制出现和发展的原因,不需要死记硬背,这些机制都是应需求而诞生、发展的,非常的自然。 什么是操作系统? 程序视角:对象 + API 硬件视角:一个 C 程序 实验环境: deepin 20.9 $ uname -a Linux cai-PC 5.15.77-amd64-desktop #2 SMP Thu Jun 15 16:06:18 CST 2023 x86_64 GNU/Linux 安装 tldr: $ sudo apt install tldr 有些系统可能没有预装 man 手册: $ sudo apt install manpages manpages-de manpages-de-dev manpages-dev manpages-posix manpages-posix-dev glibc-doc ","date":"2024-03-24","objectID":"/posts/nju-osdi/:1:1","tags":["Linux","QEMU","RISC-V"],"title":"南京大学 操作系统: 设计与实现 重点提示","uri":"/posts/nju-osdi/"},{"categories":["Systems"],"content":"操作系统上的程序 (什么是程序和编译器) 信息 直播录影 / 讲义页面 / 阅读材料 UNIX 哲学: Make each program do one thing well Expect the output of every program to become the input to another 什么是程序 计算机是构建在状态机 (数字电路) 之上的,所以运行在计算机之上的程序 (不管是操作系统还是应用,无论是源代码还是二进制) 都是状态机。C程序的状态机模型中,状态是由堆栈确定的,所以函数调用是状态迁移,因为它改变了堆栈,即改变了状态机的状态。明确这一点之后,我们可以通过模拟堆栈的方式,来将任意的递归程序改写为非递归程序,例如经典的汉诺塔程序。 程序 = 状态机 源代码 $S$ (状态机): 状态迁移 = 执行语句 二进制代码 $C$ (状态机): 状态迁移 = 执行指令 注意 jyy 所给的非递归汉诺塔程序也是通过模拟堆栈状态转移实现的,但是比较晦涩的一点是,对于每一个堆栈状态,都有可能需要执行最多 4 条语句 (对应 for 循环和 pc),这一点比较难懂。 只使用纯\"计算\"的指令 (无论是 deterministic 还是 non-deterministic) 无法使程序停下来,因为将程序本质是状态机,而状态机通过“计算”的指令只能从一个状态迁移到另一个状态,无法实现销毁状态机的操作 (对应退出/停下程序),要么死循环,要么 undefined behavior。这时需要程序对应的状态机之外的另一个东西来控制、管理该状态机,以实现程序的停下/退出操作,这就是 OS 的 syscall 存在的意义,它可以游离在程序对应的状态机之外,并修改状态机的内容 (因为程序呼叫 syscall 时已经全权授予 OS 对其状态内容进行修改)。 空的 _start 函数可以成功编译并链接,但是由于函数是空的,它会编译生成 retq 指令,这会导致 pc 跳转到不合法的区域,而正确的做法应该是使用 syscall exit 来结束该程序 (熟悉 C 语言函数调用的同学应该能看懂这段描述)。 // start.c int _start() {} // start.o 0000000000000000 \u003c_start\u003e: 0: 55 push %rbp 1: 48 89 e5 mov %rsp,%rbp 4: 90 nop 5: 5d pop %rbp 6: c3 retq 通过 syscall 实现了和 mininal.S 功能一致的最小 C 语言 hello, world 程序 mininal.c: #include \u003csys/syscall.h\u003e #include \u003cunistd.h\u003e int main() { char buf[] = \"\\033[01;31mHello, OS World\\033[0m\\n\"; syscall(SYS_write, 1, buf, sizeof(buf)); syscall(SYS_exit, 42); } System Calls Manual 如何在程序的两个视角之间切换? 从“状态机”的角度可以帮我们解决一个重要的基本问题: 什么是编译器??? 编译器: 源代码 S (状态机) $\\rightarrow$ 二进制代码 C (状态机) $$C=compile(S)$$ 即编译器的功能是将源代码对应的状态机 $S$ 转换成二进制代码对应的状态机 $C$。但是这里需要注意,这两个状态机不需要完全等价,只需要满足 $S$ 与 $C$ 的可观测行为严格一致 即可,这也是编译优化的理论基础:在保证观测一致性 (sound) 的前提下改写代码 (rewriting)。 Jserv 的讲座 並行程式設計: 執行順序 对这个有更清晰的讲解 可以通过以下指令来观察编译器的优化情况,以理解什么是观测一致性: $ gcc -On -c a.c # n couldbe 0, 1, 2, 3 $ objdump -d a.o 操作系统中的一般程序 对于操作系统之上的程序,它们看待操作系统的视角是 API (syscall),所以这门课中有一个很重要的工具:strace (system call trace 追踪程序运行时使用的系统调用,可以查看程序和操作系统的交互): $ sudo apt install strace $ strace ./hello-goodbye Linux manual page: strace 技巧 可以通过 apt-file 来检索文件名可能在那些 package 里,例如: $ sudo apt install apt-file $ sudo apt-file update $ sudo apt-file search \u003cfilename\u003e ","date":"2024-03-24","objectID":"/posts/nju-osdi/:1:2","tags":["Linux","QEMU","RISC-V"],"title":"南京大学 操作系统: 设计与实现 重点提示","uri":"/posts/nju-osdi/"},{"categories":["Systems"],"content":"第 2 周: 并发","date":"2024-03-24","objectID":"/posts/nju-osdi/:2:0","tags":["Linux","QEMU","RISC-V"],"title":"南京大学 操作系统: 设计与实现 重点提示","uri":"/posts/nju-osdi/"},{"categories":["Rust"],"content":" In this episode of Crust of Rust, we go over subtyping and variance — a niche part of Rust that most people don’t have to think about, but which is deeply ingrained in some of Rust’s borrow ergonomics, and occasionally manifests in confusing ways. In particular, we explore how trying to implement the relatively straightforward strtok function from C/C++ in Rust quickly lands us in a place where the function is more or less impossible to call due to variance! 整理自 John Gjengset 的影片 ","date":"2024-03-17","objectID":"/posts/subtying-and-variance/:0:0","tags":["Rust","Subtying","Variance"],"title":"Crust of Rust: Subtying and Variance","uri":"/posts/subtying-and-variance/"},{"categories":["Rust"],"content":"影片注解 ","date":"2024-03-17","objectID":"/posts/subtying-and-variance/:1:0","tags":["Rust","Subtying","Variance"],"title":"Crust of Rust: Subtying and Variance","uri":"/posts/subtying-and-variance/"},{"categories":["Rust"],"content":"strtok A sequence of calls to this function split str into tokens, which are sequences of contiguous characters separated by any of the characters that are part of delimiters. cplusplus: strtok cppreference: strtok ","date":"2024-03-17","objectID":"/posts/subtying-and-variance/:1:1","tags":["Rust","Subtying","Variance"],"title":"Crust of Rust: Subtying and Variance","uri":"/posts/subtying-and-variance/"},{"categories":["Rust"],"content":"shortening lifetimes 影片大概 19 分时给出了为何 cargo test 失败的推导,个人觉得非常巧妙 pub fn strtok\u003c'a\u003e(s: \u0026'a mut \u0026'a str, delimiter: char) { ... } let mut x = \"hello world\"; strtok(\u0026mut x, ' '); 为了更直观地表示和函数 strtok 的返回值 lifetime 无关,这里将返回值先去掉了。在调用 strtok 时,编译器对于参数 s 的 lifetime 推导如下: \u0026'a mut \u0026'a str \u0026 mut x \u0026'a mut \u0026'a str \u0026 mut \u0026'static str \u0026'a mut \u0026'static str \u0026 mut \u0026'static str \u0026'static mut \u0026'static str \u0026 mut \u0026'static str \u0026'static mut \u0026'static str \u0026'static mut \u0026'static str 所以 strtok 在接收参数 s 后 (通过传入 \u0026mut x),会推导其 lifetime 为 static,这就会导致后面使用 x 的不可变引用 (\u0026x) 时发生冲突。 ","date":"2024-03-17","objectID":"/posts/subtying-and-variance/:1:2","tags":["Rust","Subtying","Variance"],"title":"Crust of Rust: Subtying and Variance","uri":"/posts/subtying-and-variance/"},{"categories":["Rust"],"content":"Documentations 这里列举视频中一些概念相关的 documentation 学习的一手资料是官方文档,请务必自主学会阅读规格书之类的资料 ","date":"2024-03-17","objectID":"/posts/subtying-and-variance/:2:0","tags":["Rust","Subtying","Variance"],"title":"Crust of Rust: Subtying and Variance","uri":"/posts/subtying-and-variance/"},{"categories":["Rust"],"content":"Crate std 可以使用这里提供的搜素栏进行搜索 (BTW 不要浪费时间在 Google 搜寻上!) method str::find ","date":"2024-03-17","objectID":"/posts/subtying-and-variance/:2:1","tags":["Rust","Subtying","Variance"],"title":"Crust of Rust: Subtying and Variance","uri":"/posts/subtying-and-variance/"},{"categories":["Rust"],"content":"References The Rust Reference: Subtyping and Variance The Rustonomicon: Subtyping and Variance ","date":"2024-03-17","objectID":"/posts/subtying-and-variance/:3:0","tags":["Rust","Subtying","Variance"],"title":"Crust of Rust: Subtying and Variance","uri":"/posts/subtying-and-variance/"},{"categories":["C","Linux Kernel Internals"],"content":" 在许多应用程序中,递归 (recursion) 可以简单又优雅地解决貌似繁琐的问题,也就是不断地拆解原有问题为相似的子问题,直到无法拆解为止,并且定义最简化状况的处理机制,一如数学思维。递归对 C 语言程序开发者来说,绝对不会陌生,但能掌握者却少,很多人甚至难以讲出汉诺塔之外的使用案例。 究竟递归是如何优雅地解决真实世界的问题,又如何兼顾执行效率呢》我们从运作原理开始探讨,搭配若干 C 程序解说,并且我们将以简化过的 UNIX 工具为例,分析透过递归来大幅缩减程式码。 或许跟你想象中不同,Linux 核心的原始程式码里头也用到递归函数呼叫,特别在较复杂的实作,例如文件系统,善用递归可大幅缩减程式码,但这也导致追踪程序运作的难度大增。 原文地址 ","date":"2024-03-16","objectID":"/posts/c-recursion/:0:0","tags":["Sysprog","C","Recursion"],"title":"你所不知道的 C 语言: 递归调用篇","uri":"/posts/c-recursion/"},{"categories":["C","Linux Kernel Internals"],"content":"Recursion To Iterate is Human, to Recurse, Divine. http://coder.aqualuna.me/2011/07/to-iterate-is-human-to-recurse-divine.html 注意 笔者的递归 (Recursion) 是通过 UC Berkeley 的 CS61A: Structure and Interpretation of Computer Programs CS70: Discrete Mathematics and Probability Theory 学习的,这个搭配式的学习模式使得我在实作——递归 (cs61a) 和理论——归纳法 (cs70) 上相互配合理解,从而对递归在实作和理论上都有了充分认知。 ","date":"2024-03-16","objectID":"/posts/c-recursion/:1:0","tags":["Sysprog","C","Recursion"],"title":"你所不知道的 C 语言: 递归调用篇","uri":"/posts/c-recursion/"},{"categories":["C","Linux Kernel Internals"],"content":"递归并没有想象的那么慢 以最大公因数 (Greatest Common Divisor, GCD) 为例,分别以循环和递归进行实作: unsigned gcd_rec(unsigned a, unsigned b) { if (!b) return a; return gcd_rec(b, a % b); } unsigned gcd_itr(unsigned a, unsigned b) { while (b) { unsigned t = b; b = a % b; a = t; } return a; } 这两个函数在 clang/llvm 优化后的编译输出 (clang -S -O2 gcd.c) 的汇编是一样的: .LBB0_2: movl %edx, %ecx xorl %edx, %edx divl %ecx movl %ecx, %eax testl %edx, %edx jne .LBB1_2 技巧 遞迴 (Recursion) Tail recursion 可以被编译器进行k空间利用最优化,从而达到和循环一样节省空间,但这需要编译器支持,有些编译器并不支持 tail recursion 优化 🤣 虽然如此,将一般的递归改写为 tail recursion 还是可以获得极大的效能提升。 Source ","date":"2024-03-16","objectID":"/posts/c-recursion/:2:0","tags":["Sysprog","C","Recursion"],"title":"你所不知道的 C 语言: 递归调用篇","uri":"/posts/c-recursion/"},{"categories":["C","Linux Kernel Internals"],"content":"案例分析: 等效电阻 r ----------###------------- A -------- A | | | # # # R(r, n - 1) # r # ==\u003e # R(r, n) # # # | | | --------------------------- B -------- B $$ R(r,n)= \\begin{cases} r \u0026 \\text{if n = 1}\\\\ 1 / (\\frac1r + \\frac1{R(r, n - 1) + r}) \u0026 \\text{if n \u003e 1} \\end{cases} $$ def circuit(n, r): if n == 1: return r else: return 1 / (1 / r + 1 / (circuit(n - 1, r) + r)) ","date":"2024-03-16","objectID":"/posts/c-recursion/:3:0","tags":["Sysprog","C","Recursion"],"title":"你所不知道的 C 语言: 递归调用篇","uri":"/posts/c-recursion/"},{"categories":["C","Linux Kernel Internals"],"content":"案例分析: 数列输出 man 3 printf RETURN VALUE Upon successful return, these functions return the number of characters printed (excluding the null byte used to end output to strings). 可以通过 ulimit -s 来改 stack size,预设为 8MB ulimit User limits - limit the use of system-wide resources. -s The maximum stack size. 现代编译器的最优化可能会造成递归实作的非预期改变,因为编译器可能会对递归实作在编译时期进行一些优化,从而提高效能和降低内存使用。 ","date":"2024-03-16","objectID":"/posts/c-recursion/:4:0","tags":["Sysprog","C","Recursion"],"title":"你所不知道的 C 语言: 递归调用篇","uri":"/posts/c-recursion/"},{"categories":["C","Linux Kernel Internals"],"content":"递归程序设计 Recursive Programming ","date":"2024-03-16","objectID":"/posts/c-recursion/:5:0","tags":["Sysprog","C","Recursion"],"title":"你所不知道的 C 语言: 递归调用篇","uri":"/posts/c-recursion/"},{"categories":["C","Linux Kernel Internals"],"content":"Fibonacci sequence 使用矩阵配合快速幂算法,可以将时间复杂度从 $O(n)$ 降低到 $O(\\log n)$ 方法 时间复杂度 空间复杂度 Rcursive $O(2^n)$ $O(n)$ Iterative $O(n)$ $O(1)$ Tail recursion $O(n)$ $O(1)$ Q-Matrix $O(\\log n)$ $O(n)$ Fast doubling $O(\\log n)$ $O(1)$ 原文的 Q-Matrix 实作挺多漏洞的,下面为修正后的实作 (注意矩阵乘法的 memset 是必须的,否则会使用到栈上超出生命周期的 obeject): void matrix_multiply(int a[2][2], int b[2][2], int t[2][2]) { memset(t, 0, sizeof(int) * 2 * 2); for (int i = 0; i \u003c 2; i++) for (int j = 0; j \u003c 2; j++) for (int k = 0; k \u003c 2; k++) t[i][j] += a[i][k] * b[k][j]; } void matrix_pow(int a[2][2], int n, int t[2][2]) { if (n == 1) { t[0][0] = a[0][0]; t[0][1] = a[0][1]; t[1][0] = a[1][0]; t[1][1] = a[1][1]; return; } if (n % 2 == 0) { int t1[2][2]; matrix_pow(a, n \u003e\u003e 1, t1); matrix_multiply(t1, t1, t); return; } else { int t1[2][2], t2[2][2]; matrix_pow(a, n \u003e\u003e 1, t1); matrix_pow(a, (n \u003e\u003e 1) + 1, t2); matrix_multiply(t1, t2, t); return; } } int fib(int n) { if (n \u003c= 0) return 0; int A1[2][2] = {{1, 1}, {1, 0}}; int result[2][2]; matrix_pow(A1, n, result); return result[0][1]; } Fast doubling 公式: $$ \\begin{split} F(2k) \u0026= F(k)[2F(k+1) - F(k)] \\\\ F(2k+1) \u0026= F(k+1)^2+F(k)^2 \\end{split} $$ 具体推导: $$ \\begin{split} \\begin{bmatrix} F(2n+1) \\\\ F(2n) \\end{bmatrix} \u0026= \\begin{bmatrix} 1 \u0026 1 \\\\ 1 \u0026 0 \\end{bmatrix}^{2n} \\begin{bmatrix} F(1) \\\\ F(0) \\end{bmatrix}\\\\ \\\\ \u0026= \\begin{bmatrix} 1 \u0026 1 \\\\ 1 \u0026 0 \\end{bmatrix}^n \\begin{bmatrix} 1 \u0026 1 \\\\ 1 \u0026 0 \\end{bmatrix}^n \\begin{bmatrix} F(1) \\\\ F(0) \\end{bmatrix}\\\\ \\\\ \u0026= \\begin{bmatrix} F(n+1) \u0026 F(n) \\\\ F(n) \u0026 F(n-1) \\end{bmatrix} \\begin{bmatrix} F(n+1) \u0026 F(n) \\\\ F(n) \u0026 F(n-1) \\end{bmatrix} \\begin{bmatrix} 1 \\\\ 0 \\end{bmatrix}\\\\ \\\\ \u0026= \\begin{bmatrix} F(n+1)^2 + F(n)^2\\\\ F(n)F(n+1) + F(n-1)F(n) \\end{bmatrix} \\end{split} $$ 然后根据 $F(k + 1) = F(k) + F(k - 1)$ 可得 $F(2k)$ 情况的公式。 原文中非递增情形比较晦涩,但其本质是通过累加来逼近目标值: else { t0 = t3; // F(n-2); t3 = t4; // F(n-1); t4 = t0 + t4; // F(n) i++; } ","date":"2024-03-16","objectID":"/posts/c-recursion/:6:0","tags":["Sysprog","C","Recursion"],"title":"你所不知道的 C 语言: 递归调用篇","uri":"/posts/c-recursion/"},{"categories":["C","Linux Kernel Internals"],"content":"案例分析: 字符串反转 原文对于时间复杂度的分析貌似有些问题,下面给出本人的见解。第一种方法的时间复杂度为: $$ T(n) = 2T(n-1) + T(n-2) $$ 所以第一种方法的时间复杂度为 $O(2^n)$。 第二种方法只是列出了程式码,而没有说明递归函数的作用,在本人看来,递归函数一定要明确说明其目的,才能比较好理解递归的作用,所以下面给出递归函数 rev_core 的功能说明: // 返回字符串 head 的最大下标 (下标相对于 idx 偏移),并且将字符串 head 相对于 // 整条字符串的中间对称点进行反转 int rev_core(char *head, int idx) { if (head[idx] != '\\0') { int end = rev_core(head, idx + 1); if (idx \u003e end / 2) swap(head + idx, head + end - idx); return end; } return idx - 1; } char *reverse(char *s) { rev_core(s, 0); return s; } 时间复杂度显然为 $O(n)$ ","date":"2024-03-16","objectID":"/posts/c-recursion/:7:0","tags":["Sysprog","C","Recursion"],"title":"你所不知道的 C 语言: 递归调用篇","uri":"/posts/c-recursion/"},{"categories":["C","Linux Kernel Internals"],"content":"案例分析: 建立目录 mkdir [Linux manual page (2)] DESCRIPTION Create the DIRECTORY(ies), if they do not already exist. 补充一下递归函数 mkdir_r 的功能描述: // 从路径 `path` 的第 `level` 层开始创建目录 int mkdir_r(const char *path, int level); ","date":"2024-03-16","objectID":"/posts/c-recursion/:8:0","tags":["Sysprog","C","Recursion"],"title":"你所不知道的 C 语言: 递归调用篇","uri":"/posts/c-recursion/"},{"categories":["C","Linux Kernel Internals"],"content":"案例分析: 类似 find 的程序 opendir [Linux manual page (3)] RETURN VALUE The opendir() and fdopendir() functions return a pointer to the directory stream. On error, NULL is returned, and errno is set to indicate the error. readdir [Linux manual page (3)] RETURN VALUE On success, readdir() returns a pointer to a dirent structure. (This structure may be statically allocated; do not attempt to free(3) it.) If the end of the directory stream is reached, NULL is returned and errno is not changed. If an error occurs, NULL is returned and errno is set to indicate the error. To distinguish end of stream from an error, set errno to zero before calling readdir() and then check the value of errno if NULL is returned. 练习: 连同文件一起输出 练习: 将输出的 . 和 .. 过滤掉 ","date":"2024-03-16","objectID":"/posts/c-recursion/:9:0","tags":["Sysprog","C","Recursion"],"title":"你所不知道的 C 语言: 递归调用篇","uri":"/posts/c-recursion/"},{"categories":["C","Linux Kernel Internals"],"content":"案例分析: Merge Sort Program for Merge Sort in C MapReduce with POSIX Thread ","date":"2024-03-16","objectID":"/posts/c-recursion/:10:0","tags":["Sysprog","C","Recursion"],"title":"你所不知道的 C 语言: 递归调用篇","uri":"/posts/c-recursion/"},{"categories":["C","Linux Kernel Internals"],"content":"函数式程序开发 Toward Concurrency Functional programming in C Functional Programming 风格的 C 语言实作 ","date":"2024-03-16","objectID":"/posts/c-recursion/:11:0","tags":["Sysprog","C","Recursion"],"title":"你所不知道的 C 语言: 递归调用篇","uri":"/posts/c-recursion/"},{"categories":["C","Linux Kernel Internals"],"content":"递归背后的理论 YouTube: Lambda Calculus - Computerphile YouTube: Essentials: Functional Programming’s Y Combinator - Computerphile 第一个影片相对还蛮好懂,第二个影片对于非 PL 背景的人来说完全是看不懂,所以暂时先放弃了 第一个影片主要介绍函数式编程的核心概念: 函数可以像其它 object 一样被传递使用,没有额外的限制,并且 object 是可以由函数来定义、构建的,例如我们可以定义 true 和 false: TRUE: $\\lambda x.\\ \\lambda y.\\ x$ FALSE: $\\lambda x.\\ \\lambda y.\\ y$ 因为 true 和 false 就是用来控制流程的,为 true 时我们 do somthing,为 false 我们 do other,所以上面这种定义是有意义的,当然你也可以定义为其它,毕竟函数式编程让我们可以定义任意我们想定义的东西 🤣 接下来我们就可以通过先前定义的 TRUE 和 FALSE 来实现 NOT, AND, OR 这类操作了: NOT: $\\lambda b.\\ b.$ FALSE TRUE AND: $\\lambda x.\\ \\lambda y.\\ x.\\ y.$ FALSE OR: $\\lambda x.\\ \\lambda y.\\ x$ TRUE $y.$ 乍一看这个挺抽象的,其实上面的实现正体现了函数式编程的威力,我们以 NOT TRUE 的推导带大家体会一下: NOT TRUE $\\ \\ \\ \\ $ $b.$ FALSE TRUE $\\ \\ \\ \\ $ TRUE FALSE TRUE $\\ \\ \\ \\ $ TRUE (FALSE TRUE) $\\ \\ \\ \\ $ FALSE 其余推导同理 ","date":"2024-03-16","objectID":"/posts/c-recursion/:12:0","tags":["Sysprog","C","Recursion"],"title":"你所不知道的 C 语言: 递归调用篇","uri":"/posts/c-recursion/"},{"categories":["Linux","Linux Kernel Internals"],"content":" Linux 核心如同其它复杂的资讯系统,也提供 hash table 的实作,但其原始程式码中却藏有间接指针 (可参见 你所不知道的 C 语言: linked list 和非连续内存) 的巧妙和数学奥秘。 原文地址 ","date":"2024-03-16","objectID":"/posts/linux-hashtable/:0:0","tags":["Sysprog","Linux","Hash table"],"title":"Linux 核心的 hash table 实作","uri":"/posts/linux-hashtable/"},{"categories":["Linux","Linux Kernel Internals"],"content":"间接指针 Linux 核心的 hashtable 结构示意图: 不难看出,pprev 是指向上一个节点 next 的指针,即是指向 hlist_node * 的指针,而不是指向上一个节点 (hlist_node) 的指针,因为 hashtable 的数组中存放的是 hlist_node *,所以这样也简化了表示方法,将拉链和数组元素相互联系了起来。需要使用间接指针来实现 doubly linked 本质上是因为:拉链节点和数组节点在表示和操作上的不等价。 当然也可以将数组元素和拉链元素都统一为带有两个指针 prev 和 next 的 doubly linked list node,这样解决了之前所提的不等价,可以消除特判,但这样会导致存取数组元素时内存开销增大,进而降低 cache 的利用率。 信息 List, HList, and Hash Table 内核基础设施——hlist_head/hlist_node 结构解析 hlist数据结构图示说明 ","date":"2024-03-16","objectID":"/posts/linux-hashtable/:1:0","tags":["Sysprog","Linux","Hash table"],"title":"Linux 核心的 hash table 实作","uri":"/posts/linux-hashtable/"},{"categories":["Linux","Linux Kernel Internals"],"content":"hash 函数 Wikipedia: Hash function A hash function is any function that can be used to map data of arbitrary size to fixed-size values, though there are some hash functions that support variable length output. ","date":"2024-03-16","objectID":"/posts/linux-hashtable/:2:0","tags":["Sysprog","Linux","Hash table"],"title":"Linux 核心的 hash table 实作","uri":"/posts/linux-hashtable/"},{"categories":["Linux","Linux Kernel Internals"],"content":"常见 hash 策略 Division method $$ h(k) = k % N $$ Mid-square $$ h(k) = bits_{i,i+r-1}(k^2) $$ Folding addition $$ key = 3823749374 \\\\ 382\\ |\\ 374\\ |\\ 937\\ |\\ 4 \\\\ index = 382 + 374 + 937 + 4 = 1697 \\\\ $$ 先将 key 切成片段后再相加,也可以对相加后的结果做其他运算 Multiplication Method ","date":"2024-03-16","objectID":"/posts/linux-hashtable/:2:1","tags":["Sysprog","Linux","Hash table"],"title":"Linux 核心的 hash table 实作","uri":"/posts/linux-hashtable/"},{"categories":["Linux","Linux Kernel Internals"],"content":"Linux 核心的 hash 函数 Linux 核心的 hash.h 使用的是 Multiplication Method 策略,但是是通过整数和位运算实现的,没有使用到浮点数。 $$ \\begin{split} h(K) \u0026= \\lfloor m \\cdot (KA - \\lfloor KA \\rfloor) \\rfloor \\\\ h(K) \u0026= K \\cdot 2^w \\cdot A \u003e\u003e (w - p) \\end{split} $$ 上面两条式子的等价关键在于,使用 二进制编码 表示的整数和小数配合进行推导,进而只使用整数来实现,具体推导见原文。 $(\\sqrt{5} - 1 ) / 2 = 0.618033989$ $2654435761 / 4294967296 = 0.618033987$ $2^{32} = 4294967296$ 因此 val * GOLDEN_RATIO_32 \u003e\u003e (32 - bits) $\\equiv K \\times A \\times 2^w \u003e\u003e (w - p)$,其中 GOLDEN_RATIO_32 等于 $2654435761$ Linux 核心的 64 bit 的 hash 函数: #ifndef HAVE_ARCH_HASH_64 #define hash_64 hash_64_generic #endif static __always_inline u32 hash_64_generic(u64 val, unsigned int bits) { #if BITS_PER_LONG == 64 /* 64x64-bit multiply is efficient on all 64-bit processors */ return val * GOLDEN_RATIO_64 \u003e\u003e (64 - bits); #else /* Hash 64 bits using only 32x32-bit multiply. */ return hash_32((u32)val ^ __hash_32(val \u003e\u003e 32), bits); #endif } Linux 核心采用 golden ratio 作为 $A$,这是因为这样碰撞较少,且分布均匀: ","date":"2024-03-16","objectID":"/posts/linux-hashtable/:3:0","tags":["Sysprog","Linux","Hash table"],"title":"Linux 核心的 hash table 实作","uri":"/posts/linux-hashtable/"},{"categories":["C","Linux Kernel Internals"],"content":" 本讲座将带着学员重新探索函数呼叫背后的原理,从程序语言和计算机结构的发展简史谈起,让学员自电脑软硬件演化过程去掌握 calling convention 的考量,伴随着 stack 和 heap 的操作,再探讨 C 程序如何处理函数呼叫、跨越函数间的跳跃 (如 setjmp 和 longjmp),再来思索资讯安全和执行效率的议题。着重在计算机架构对应的支援和行为分析。 原文地址 ","date":"2024-03-15","objectID":"/posts/c-function/:0:0","tags":["Sysprog","C","Function"],"title":"你所不知道的 C 语言: 函数调用篇","uri":"/posts/c-function/"},{"categories":["C","Linux Kernel Internals"],"content":"function prototype Very early C compilers and language 一个小故事,可以解释 C 语言的一些设计理念,例如 switch-case 中每个 case 都需要 break The Development of the C Language Dennis M. Ritchie 讲述 C 语言漫长的发展史,并搭配程式码来说明当初为何如此设计、取舍考量。了解这些历史背景可以让我们成为更专业的 C 语言 Programmer Rationale for International Standard – Programming Languages – C 讲述 C 语言标准的变更,并搭配程式码解释变更的原理和考量 在早期的 C 语言中,并不需要 function prototype,因为当编译器发现一个函数名出现在表达式并且后面跟着左括号 (,例如 a = func(...),就会将该函数解读为:返回值类型预设为 int,参数类型和个数由调用者提供来决定,按照这样规则编写程式码,可以在无需事先定义函数即可先写调用函数的逻辑。但是这样设计也会造成潜在问题:程序员在调用函数时需要谨慎处理,需要自己检查调用时的参数类型和个数符合函数定义 (因为当时的编译器无法正确判断调用函数时的参数是否符合预期的类型和个数,当时编译器的能力与先前提到的规则是一体两面),并且返回值类型预设为 int (当时还没有 void 类型),所以对于函数返回值,也需要谨慎处理。 显然 function prototype 的缺失导致程式码编写极其容易出错,所以从 C99 开始就规范了 function prototype,这个规范除了可以降低 programmer 心智负担之外,还可以提高程序效能。编译器的最佳化阶段 (optimizer) 可以通过 function prototype 来得知内存空间的使用情形,从而允许编译器在函数调用表达式的上下文进行激进的最佳化策略,例如 const 的使用可以让编译器知道只会读取内存数据而不会修改内存数据,从而没有 side effect,可以进行激进的最优化。 int compare(const char *string1, const char *string2); void func2(int x) { char *str1, *str2; // ... x = compare(str1, str2); // ... } Rust 的不可变引用也是编译器可以进行更激进的最优化处理的一个例子 注意 为什么早期的 C 语言没有 function prototype 呢?因为早期的 C 语言,不管有多少个源程序文件,都是先通过 cat 合并成一个单元文件,在进行编译链接生成目标文件。这样就导致了就算写了 function prototye,使用 cat 合并时,这些 prototype 不一定会出现在我们期望的程序开始处,即无法利用 prototype 对于函数调用进行检查,所以干脆不写 prototype。 在 preprocessor 出现后,通过 #include 这类语法并搭配 preprocessor 可以保证对于每个源文件,都可以通过 function prototype 对函数调用进行参数个数、类型检查,因为 #include 语句位于源文件起始处,并且此时 C 语言程序的编译过程改变了: 对单一源文件进行预处理、编译,然后再对得到的目标文件进行链接。所以此时透过 preprocessor 可以保证 function prototype 位于函数调用之前,可以进行严格地检查。 ","date":"2024-03-15","objectID":"/posts/c-function/:1:0","tags":["Sysprog","C","Function"],"title":"你所不知道的 C 语言: 函数调用篇","uri":"/posts/c-function/"},{"categories":["C","Linux Kernel Internals"],"content":"编程语言的 function C 语言不允许 nested function 以简化编译器的设计 (当然现在的 gcc 提供 nested funtion 的扩展),即 C 语言的 function 是一等公民,位于语法最顶层 (top-level),因为支持 nested function 需要 staic link 机制来确认外层函数。 编程语言中的函数,与数学的函数不完全一致,编程语言的函数隐含了状态机的转换过程 (即有 side effect),只有拥有 Referential Transparency 特性的函数,才能和数学上的函数等价。 ","date":"2024-03-15","objectID":"/posts/c-function/:2:0","tags":["Sysprog","C","Function"],"title":"你所不知道的 C 语言: 函数调用篇","uri":"/posts/c-function/"},{"categories":["C","Linux Kernel Internals"],"content":"Process 与 C 程序 程序存放在磁盘时叫 Program,加载到内存后叫 “Process” Wikipedia: Application binary interface In computer software, an application binary interface (ABI) is an interface between two binary program modules. Often, one of these modules is a library or operating system facility, and the other is a program that is being run by a user. 在 Intel x86 架构中,当返回值可以放在寄存器时就放在寄存器中返回,以提高效能,如果放不下,则将返回值的起始地址放在寄存器中返回。 ","date":"2024-03-15","objectID":"/posts/c-function/:3:0","tags":["Sysprog","C","Function"],"title":"你所不知道的 C 语言: 函数调用篇","uri":"/posts/c-function/"},{"categories":["C","Linux Kernel Internals"],"content":"stack ","date":"2024-03-15","objectID":"/posts/c-function/:4:0","tags":["Sysprog","C","Function"],"title":"你所不知道的 C 语言: 函数调用篇","uri":"/posts/c-function/"},{"categories":["C","Linux Kernel Internals"],"content":"Layout System V Application Binary Interface AMD64 Architecture Processor Supplement [PDF] ","date":"2024-03-15","objectID":"/posts/c-function/:4:1","tags":["Sysprog","C","Function"],"title":"你所不知道的 C 语言: 函数调用篇","uri":"/posts/c-function/"},{"categories":["C","Linux Kernel Internals"],"content":"PEDA 实验需要使用到 GDB 的 PEDA 扩展: Enhance the display of gdb: colorize and display disassembly codes, registers, memory information during debugging. $ git clone https://github.com/longld/peda.git ~/peda $ echo \"source ~/peda/peda.py\" \u003e\u003e ~/.gdbinit 技巧 动态追踪 Stack 实验的 call funcA 可以通过 GDB 指令 stepi 或 si 来实现 ","date":"2024-03-15","objectID":"/posts/c-function/:4:2","tags":["Sysprog","C","Function"],"title":"你所不知道的 C 语言: 函数调用篇","uri":"/posts/c-function/"},{"categories":["C","Linux Kernel Internals"],"content":"从递归观察函数调用 infinite.c int func(int x) { static int count = 0; int y = x; // local var return ++count \u0026\u0026 func(x++); } int main() { return func(0); } func 函数在调用时,一个栈帧的内容包括: x (parameter), y (local variable), return address。这些数据的类型都是 int,即占据空间相同,这也是为什么计时器 count 的变化大致呈现 $x : \\frac{x}{2} : \\frac{x}{3}$ 的比例。 ","date":"2024-03-15","objectID":"/posts/c-function/:5:0","tags":["Sysprog","C","Function"],"title":"你所不知道的 C 语言: 函数调用篇","uri":"/posts/c-function/"},{"categories":["C","Linux Kernel Internals"],"content":"stack-based buffer overflow ","date":"2024-03-15","objectID":"/posts/c-function/:5:1","tags":["Sysprog","C","Function"],"title":"你所不知道的 C 语言: 函数调用篇","uri":"/posts/c-function/"},{"categories":["C","Linux Kernel Internals"],"content":"ROP ","date":"2024-03-15","objectID":"/posts/c-function/:6:0","tags":["Sysprog","C","Function"],"title":"你所不知道的 C 语言: 函数调用篇","uri":"/posts/c-function/"},{"categories":["C","Linux Kernel Internals"],"content":"heap 使用 malloc 时操作系统可能会 overcommit,而正因为这个 overcommit 的特性,malloc 返回有效地址也不见得是安全的。除此之外,因为 overcommit,使用 malloc 后立即搭配使用 memset 代价也很高 (因为操作系统 overcommit 可能会先分配一个小空间而不是一下子分配全部,因为它优先重复使用之前已使用过的小块空间),并且如果是设置为 0,则有可能会对原本为 0 的空间进行重复设置,降低效能。此时可以应该善用 calloc,虽然也会 overcommit,但是会保证分配空间的前面都是 0 (因为优先分配的是需要操作系统参与的大块空间),无需使用 memset 这类操作而降低效能。 ","date":"2024-03-15","objectID":"/posts/c-function/:7:0","tags":["Sysprog","C","Function"],"title":"你所不知道的 C 语言: 函数调用篇","uri":"/posts/c-function/"},{"categories":["C","Linux Kernel Internals"],"content":"malloc / free ","date":"2024-03-15","objectID":"/posts/c-function/:7:1","tags":["Sysprog","C","Function"],"title":"你所不知道的 C 语言: 函数调用篇","uri":"/posts/c-function/"},{"categories":["C","Linux Kernel Internals"],"content":"RAII ","date":"2024-03-15","objectID":"/posts/c-function/:8:0","tags":["Sysprog","C","Function"],"title":"你所不知道的 C 语言: 函数调用篇","uri":"/posts/c-function/"},{"categories":["C","Linux Kernel Internals"],"content":"setjmp \u0026 longjmp setjmp(3) — Linux manual page The functions described on this page are used for performing “nonlocal gotos”: transferring execution from one function to a predetermined location in another function. The setjmp() function dynamically establishes the target to which control will later be transferred, and longjmp() performs the transfer of execution. 具体解说可以阅读 lab0-c 的「自動測試程式」部分 ","date":"2024-03-15","objectID":"/posts/c-function/:9:0","tags":["Sysprog","C","Function"],"title":"你所不知道的 C 语言: 函数调用篇","uri":"/posts/c-function/"},{"categories":["Concurrency","Linux Kernel Internals"],"content":" 多執行緒環境下,程式會出問題,往往在於執行順序的不確定性。一旦顧及分散式系統 (distributed systems),執行順序和衍生的時序 (timing) 問題更加複雜。 我們將從如何定義程式執行的順序開始說起,為了簡單起見,我們先從單執行緒的觀點來看執行順序這件事,其中最關鍵知識就是 Sequenced-before,你將會發現就連單執行緒的程式,也可能會產生不確定的執行順序。 原文地址 ","date":"2024-03-11","objectID":"/posts/concurrency-ordering/:0:0","tags":["Sysprog","Linux","Concurrency"],"title":"并行程序设计: 执行顺序","uri":"/posts/concurrency-ordering/"},{"categories":["Concurrency","Linux Kernel Internals"],"content":"Evaluation 所謂求值 (Evaluation),其實指二件事情,一是 value computations,對一串運算式計算的結果;另一是 side effect,亦即修改物件狀態,像是修改記憶體內變數的值、呼叫函式庫的 I/O 處理函式之類的操作。 ","date":"2024-03-11","objectID":"/posts/concurrency-ordering/:1:0","tags":["Sysprog","Linux","Concurrency"],"title":"并行程序设计: 执行顺序","uri":"/posts/concurrency-ordering/"},{"categories":["Concurrency","Linux Kernel Internals"],"content":"Sequenced-before sequenced-before 是種對 同一個執行緒 下,求值順序關係的描述。 若 A is sequenced-before B,代表 A 的求值會先完成,才進行對 B 的求值 若 A is not sequenced before B 而且 B is sequenced before A,代表 B 的求值會先完成,才開始對 A 的求值。 若 A is not sequenced before B 而且 B is not sequenced before A,代表兩種可能,一種是順序不定,甚至這兩者的求值過程可能會重疊(因為 CPU 重排指令交錯的關係)或不重疊。 而程式語言的工作,就是定義一連串關於 sequenced-before 的規範,舉例來說: 以下提到的先於、先進行之類的用詞,全部的意思都是 sequenced-before,也就是「先完成之後才開始進行」 i++ 這類的後置運算子,value computation 會先於 side effect 對於 assignment operator 而言 (=, +=, -= 一類),會先進行運算元的 value computation,之後才是 assignment 的 side effect,最後是整個 assignment expression 的 value computation。 虽然规格书定义了关于 sequenced-before 的规范,但不可能面面俱到,还是存在有些执行顺序是未定义的,例如 f1() + f2() + f3(),规格书只规定了 + 操作是在对 f1(), f2(), f3() 求值之后进行的,但是对于求值时的 f1() 这类函数呼叫,并没有规定哪个函数先进行调用求值,所以在求值时第一个调用的可能是 f1() 或 f2() 或 f3()。 sequenced-before 的规范缺失导致了 partial order 场景的出现,二这可能会导致未定义行为,例如经典的头脑体操 i = i++: 出现这个未定义行为的原因是,i++ 的 side effect 与 = 之间不存在 sequenced-bofore 关系 (因为 partial order),而这会导致该语句的执行结果是不确定的 (没想到吧,单线程的程序你也有可能不确定执行顺序 🤣) 警告 注意: 在 C++17 後,上方敘述不是未定義行為 假設 i 初始值為 0,由於 = 在 C++17 後為 sequenced,因此 i++ 的計算與 side effect 都會先完成,所以 i++ 得到 0,隨後 side-effect 導致 i 遞增 1,因此此時 i 為 1;之後執行 i = 這邊,所以利用右側表達式的值來指定數值,亦即剛才的 0,因此 i 最後結果為 0。 所以 i 值轉變的順序為 $0 \\rightarrow 1 \\rightarrow 0$,第一個箭頭為 side effect 造成的結果,第二個則是 = 造成的結果。 C++ sequenced-before graphs Order of evaluation from cppreference What are sequence points, and how do they relate to undefined behavior? ","date":"2024-03-11","objectID":"/posts/concurrency-ordering/:2:0","tags":["Sysprog","Linux","Concurrency"],"title":"并行程序设计: 执行顺序","uri":"/posts/concurrency-ordering/"},{"categories":["Concurrency","Linux Kernel Internals"],"content":"Happens-before 短片: Happened Before Relationship Happened Before Relation (cont) 从上图可以看出 happens-before 其实就是在 sequenced-before 基础上增加了多执行绪 communication 情形,可以理解为 happens-before 将 sequenced-before 扩大到涵盖多执行绪的情形了,即执行顺序有先后次序 (执行顺序其实不太准确,执行结果显现 的先后次序更加准确 🤣) 图中的 concurrent events 其实就是多执行绪下没有先后次序的情形 Java 规格书 17.4.5. Happens-before Order 也能佐证我们的观点: If one action happens-before another, then the first is visible to and ordered before the second. 引用 換言之,若望文生義說 “Happens-Before” 是「先行發生」,那就南轅北轍。Happens-Before 並非說前一操作發生在後續操作的前面,它真正要表達的是:「前面一個操作的效果對後續操作是 可見」。 這裡的關鍵是,Happens-before 強調 visible,而非實際執行的順序。 實際程式在執行時,只需要「看起來有這樣的效果」即可,編譯器有很大的空間可調整程式執行順序,亦即 compile-time memory ordering。 因此我們得知一個關鍵概念: A happens-before B 不代表實際上 A happening before B (注意時態,後者強調進行中,前者則是從結果來看),亦即只要 A 的效果在 B 執行之前,對於 B 是 visible 即可,實際的執行順序不用細究。 C11 正式将并行和 memory order 相关的规范引入到语言的标准: 5.1.2.4 Multi-threaded executions and data races All modifications to a particular atomic object M occur in some particular total order, called the modification order of M. If A and B are modifications of an atomic object M, and A happens before B, then A shall precede B in the modification order of M, which is defined below. cppreference std::memory_order Regardless of threads, evaluation A happens-before evaluation B if any of the following is true: A is sequenced-before B A inter-thread happens before B 引用 通常程式開發者逐行撰寫程式,期望前一行的效果會影響到後一行的程式碼。稍早已解釋何謂 Sequenced-before,現在可注意到,Sequenced-before 實際就是同一個執行緒內的 happens-before 關係。 在多执行绪情况下,如果没法确保 happens-before 关系,程序往往会产生意料之外的结果,例如: int counter = 0; 如果现在有两个执行绪在同时执行,执行绪 A 执行 counter++,执行绪 B 将 counter 的值打印出来。因为 A 和 B 两个执行绪不具备 happens-before 关系,没有保证 counter++ 后的效果对打印 counter 是可见的,导致打印出来的可能是 1 也可能是 0,这个也就是图中的 concurrent events 关系。 引用 因此,程式語言必須提供適當的手段,讓程式開發者得以建立跨越執行緒間的 happens-before 的關係,如此一來才能確保程式執行的結果正確。 ","date":"2024-03-11","objectID":"/posts/concurrency-ordering/:3:0","tags":["Sysprog","Linux","Concurrency"],"title":"并行程序设计: 执行顺序","uri":"/posts/concurrency-ordering/"},{"categories":["Concurrency","Linux Kernel Internals"],"content":"The Happens-Before Relation The Happens-Before Relation Let A and B represent operations performed by a multithreaded process. If A happens-before B, then the memory effects of A effectively become visible to the thread performing B before B is performed. No matter which programming language you use, they all have one thing in common: If operations A and B are performed by the same thread, and A’s statement comes before B’s statement in program order, then A happens-before B. Happens-Before Does Not Imply Happening Before In this case, though, the store to A doesn’t actually influence the store to B. (2) still behaves the same as it would have even if the effects of (1) had been visible, which is effectively the same as (1)’s effects being visible. Therefore, this doesn’t count as a violation of the happens-before rule. Happening Before Does Not Imply Happens-Before The happens-before relationship only exists where the language standards say it exists. And since these are plain loads and stores, the C++11 standard has no rule which introduces a happens-before relation between (2) and (3), even when (3) reads the value written by (2). 这里说的 happens-before 关系必须要在语言标准中有规定的才算,单执行绪的情况自然在标准内,多执行绪的情况,标准一般会制定相关的同步原语之间的 happens-before 关系,例如对 mutex 的连续两个操作必然是 happens-before 关系,更多的例子见后面的 synchronized-with 部分。 ","date":"2024-03-11","objectID":"/posts/concurrency-ordering/:3:1","tags":["Sysprog","Linux","Concurrency"],"title":"并行程序设计: 执行顺序","uri":"/posts/concurrency-ordering/"},{"categories":["Concurrency","Linux Kernel Internals"],"content":"Synchronized-with 引用 synchronized-with 是個發生在二個不同執行緒間的同步行為,當 A synchronized-with B 時,代表 A 對記憶體操作的效果,對於 B 是可見的。而 A 和 B 是二個不同的執行緒的某個操作。 不難發現,其實 synchronized-with 就是跨越多個執行緒版本的 happens-before。 ","date":"2024-03-11","objectID":"/posts/concurrency-ordering/:4:0","tags":["Sysprog","Linux","Concurrency"],"title":"并行程序设计: 执行顺序","uri":"/posts/concurrency-ordering/"},{"categories":["Concurrency","Linux Kernel Internals"],"content":"从 Java 切入 synchronized 关键字 引用 Mutual Exclusive 對同一個物件而言,不可能有二個前綴 synchronized 的方法同時交錯執行,當一個執行緒正在執行前綴 synchronized 的方法時,其他想執行 synchronized 方法的執行緒會被阻擋 (block)。 確立 Happens-before 關係 對同一個物件而言,當一個執行緒離開 synchronized 方法時,會自動對接下來呼叫 synchronized 方法的執行緒建立一個 Happens-before 關係,前一個 synchronized 的方法對該物件所做的修改,保證對接下來進入 synchronized 方法的執行緒可見。 volatile 关键字 引用 A write to a volatile field happens-before every subsequent read of that same volatile thread create/join ","date":"2024-03-11","objectID":"/posts/concurrency-ordering/:4:1","tags":["Sysprog","Linux","Concurrency"],"title":"并行程序设计: 执行顺序","uri":"/posts/concurrency-ordering/"},{"categories":["Concurrency","Linux Kernel Internals"],"content":"C++ 的观点 The library defines a number of atomic operations and operations on mutexes that are specially identified as synchronization operations. These operations play a special role in making assignments in one thread visible to another. 又是 visible 说明强调的还是 happens-before 这一关系 🤣 #include \u003ciostream\u003e // std::cout #include \u003cthread\u003e // std::thread #include \u003cmutex\u003e // std::mutex std::mutex mtx; // mutex for critical section int count = 0; void print_thread_id (int id) { // critical section (exclusive access to std::cout signaled by locking mtx): mtx.lock(); std::cout \u003c\u003c \"thread #\" \u003c\u003c id \u003c\u003c \" count:\" \u003c\u003c count \u003c\u003c '\\n'; count++; mtx.unlock(); } int main () { std::thread threads[10]; // spawn 10 threads: for (int i=0; i\u003c10; ++i) threads[i] = std::thread(print_thread_id,i+1); for (auto\u0026 th : threads) th.join(); return 0; } 这段程序里每个执行的 thread 之间都是 happens-before / synchronized-with 关系,因为它们的执行体都被 mutex 包裹了,而对 mutex 的操作是 happens-before 关系的。如果没有使用 mutex,那么 thread 之间不存在 happens-before 关系,打印出来的内容也是乱七八糟的。 cppreference std::mutex cplusplus std::mutex::lock ","date":"2024-03-11","objectID":"/posts/concurrency-ordering/:4:2","tags":["Sysprog","Linux","Concurrency"],"title":"并行程序设计: 执行顺序","uri":"/posts/concurrency-ordering/"},{"categories":["Concurrency","Linux Kernel Internals"],"content":"深入 Synchronizes-with The Synchronizes-With Relation ","date":"2024-03-11","objectID":"/posts/concurrency-ordering/:4:3","tags":["Sysprog","Linux","Concurrency"],"title":"并行程序设计: 执行顺序","uri":"/posts/concurrency-ordering/"},{"categories":["Concurrency","Linux Kernel Internals"],"content":"Memory Consistency Models 技巧 相关论文 / 技术报告 (可以用来参考理解): Shared Memory Consistency Models: A Tutorial 1995 Sarita V. Adve, Kourosh Gharachorloo ","date":"2024-03-11","objectID":"/posts/concurrency-ordering/:5:0","tags":["Sysprog","Linux","Concurrency"],"title":"并行程序设计: 执行顺序","uri":"/posts/concurrency-ordering/"},{"categories":["Concurrency","Linux Kernel Internals"],"content":" 透过建立 Concurrency 和 Parallelism、Mutex 与 Semaphore 的基本概念,本讲座将透过 POSIX Tread 探讨 thread pool, Lock-Free Programming, lock-free 使用的 atomic 操作, memory ordering, M:N threading model 等进阶议题。 原文地址 ","date":"2024-03-08","objectID":"/posts/concurrency-concepts/:0:0","tags":["Sysprog","Linux","Concurrency"],"title":"并行程序设计: 概念","uri":"/posts/concurrency-concepts/"},{"categories":["Concurrency","Linux Kernel Internals"],"content":"Mutex 与 Semaphore Mutex 和 Semaphore 在实作上可能是没有差异的 (例如早期的 Linux),但是 Mutex 与 Semaphore 在使用上是有显著差异的: process 使用 Mutex 就像使用一把锁,谁先跑得快就能先获得锁,释放锁 “解铃还须系铃人”,并且释放锁后不一定能立即调度到等待锁的 process (如果想立即调度到等待锁的 process 需要进行显式调度) process 使用 Semaphore 就如同它的名字类似 “信号枪”,process 要么是等待信号的选手,要么是发出信号的裁判,并且裁判在发出信号后,选手可以立即收到信号并调度 (无需显式调度)。并不是你跑得快就可以先获得,如果你是选手,跑得快你也得停下来等裁判到场发出信号 🤣 注意 关于 Mutex 与 Semphore 在使用手法上的差异,可以参考我使用 Rust 实现的 Channel,里面的 Share\u003cT\u003e 结构体包含了 Mutex 和 Semphore,查看相关方法 (send 和 recv) 来研究它们在使用手法的差异。 除此之外,Semaphore 的选手和裁判的数量比例不一定是 $1:1$,可以是 $m:n$ ","date":"2024-03-08","objectID":"/posts/concurrency-concepts/:1:0","tags":["Sysprog","Linux","Concurrency"],"title":"并行程序设计: 概念","uri":"/posts/concurrency-concepts/"},{"categories":["Concurrency","Linux Kernel Internals"],"content":"CTSS Fernando J. Corbato: 1963 Timesharing: A Solution to Computer Bottlenecks ","date":"2024-03-08","objectID":"/posts/concurrency-concepts/:2:0","tags":["Sysprog","Linux","Concurrency"],"title":"并行程序设计: 概念","uri":"/posts/concurrency-concepts/"},{"categories":["Concurrency","Linux Kernel Internals"],"content":"可重入性 (Reentrancy) 一個可再進入 (reentrancy) 的函式是可被多個工作同時呼叫,而不會有資料不一致的問題。簡單來說,一個可再進入的函式,會避免在函式中使用任何共享記憶區 (global memory),所有的變數與資料均存在呼叫者的資料區或函式本身的堆疊區 (stack memory)。 ","date":"2024-03-08","objectID":"/posts/concurrency-concepts/:3:0","tags":["Sysprog","Linux","Concurrency"],"title":"并行程序设计: 概念","uri":"/posts/concurrency-concepts/"},{"categories":["Concurrency","Linux Kernel Internals"],"content":"经典的 Fork-join 模型 $\\rightarrow$ $\\rightarrow$ $\\rightarrow$ Fork-join Parallelism Fork/join model 从图也可以看出,设定一个 join 点是非常必要的 (通常是由主执行绪对 join 点进行设置),因为 fork 之后新增的执行绪有可能立刻就执行完毕了,然后当主执行绪到达 join 点时,即可 join 操作进行下一步,也有可能 fork 之后新增的执行绪是惰性的,它们只有当主执行绪到达 join 点时,才会开始执行直到完毕,即主执行绪先抵达 join 点等待其它执行绪完成执行,从而完成 join 操作接着进行下一步。 因为 fork 操作时分叉处的执行绪的执行流程,对于主执行绪是无法预测的 (立刻执行、惰性执行、…),所以设定一个 join 点可以保证在这个 join 点时主执行绪和其它分叉的执行绪的执行预期行为一致,即在这个 join 点,不管是主执行绪还是分叉执行绪都完成了相应的执行流程。 ","date":"2024-03-08","objectID":"/posts/concurrency-concepts/:4:0","tags":["Sysprog","Linux","Concurrency"],"title":"并行程序设计: 概念","uri":"/posts/concurrency-concepts/"},{"categories":["Concurrency","Linux Kernel Internals"],"content":"Concurrency 和 Parallelism Rob Pike: Concurrency Is Not Parallelism / slides Stack Overflow 上的相关讨论 Concurrency 是指程式架構,將程式拆開成多個可獨立運作的工作。案例: 裝置驅動程式,可獨立運作,但不需要平行化。 Parallelism 是指程式執行,同時執行多個程式。Concurrency 可能會用到 parallelism,但不一定要用 parallelism 才能實現 concurrency。案例: 向量內積計算 Concurrent, non-parallel execution Concurrent, parallel execution Tim Mattson (Intel): Introduction to OpenMP [YouTube] ","date":"2024-03-08","objectID":"/posts/concurrency-concepts/:5:0","tags":["Sysprog","Linux","Concurrency"],"title":"并行程序设计: 概念","uri":"/posts/concurrency-concepts/"},{"categories":["RISC-V"],"content":" 本文对通过 QEMU 仿真 RISC-V 环境并启动 OpenEuler RISC-V 系统的流程进行详细介绍,以及介绍如何通过 mugen 测试框架来对 RISC-V 版本的 openEuler 进行系统、软件等方面测试,并根据测试日志对错误原因进行分析。 ","date":"2024-03-07","objectID":"/posts/openeuler-riscv-qemu/:0:0","tags":["RISC-V","openEuler","QEMU","Mugen"],"title":"openEuler RISC-V 系统: QEMU 仿真","uri":"/posts/openeuler-riscv-qemu/"},{"categories":["RISC-V"],"content":"实验环境 操作系统: deepin 20.9 ","date":"2024-03-07","objectID":"/posts/openeuler-riscv-qemu/:1:0","tags":["RISC-V","openEuler","QEMU","Mugen"],"title":"openEuler RISC-V 系统: QEMU 仿真","uri":"/posts/openeuler-riscv-qemu/"},{"categories":["RISC-V"],"content":"安装支持 RISC-V 架构的 QEMU 模拟器 $ sudo apt install qemu-system-misc $ qemu-system-riscv64 --version QEMU emulator version 5.2.0 (Debian 1:5.2+dfsg-11+deb11u1) Copyright (c) 2003-2020 Fabrice Bellard and the QEMU Project developers 虽然 deepin 仓库提供的 QEMU 软件包版本比较低 (5.2.0),但是根据「引用文档」的说明,不低于 5.0 即可 通过上面安装的 QEMU 版本过低,无法支持 VGA 这类虚拟外设 (virtio),需要手动编译安装: 如果之前通过 apt 安装了 QEMU 的可以先进行卸载: $ sudo apt remove qemu-system-risc $ sudo apt autoremove 安装必要的构建工具: $ sudo apt install build-essential git libglib2.0-dev libfdt-dev libpixman-1-dev zlib1g-dev ninja-build libslirp-dev 下载 QEMU 源码包 (此处以 7.2 版本为例): $ wget https://download.qemu.org/qemu-7.2.0.tar.xz 解压源码包、修改名称: $ tar xvJf qemu-7.2.0.tar.xz $ mv qemu-7.2.0 qemu 进入 qemu 对应目录并配置编译选项: $./configure 编译安装: $ sudo make -j$(nproc) 在 ~/.bashrc 中添加环境变量: export PATH=$PATH:/path/to/qemu/build 刷新一下 ~/.bashrc (或新开一个终端) 查看一下 QEMU 是否安装成功: $ source ~/.bashrc $ qemu-system-riscv64 --version QEMU emulator version 7.2.0 Copyright (c) 2003-2022 Fabrice Bellard and the QEMU Project developers ","date":"2024-03-07","objectID":"/posts/openeuler-riscv-qemu/:2:0","tags":["RISC-V","openEuler","QEMU","Mugen"],"title":"openEuler RISC-V 系统: QEMU 仿真","uri":"/posts/openeuler-riscv-qemu/"},{"categories":["RISC-V"],"content":"下载 openEuler RISC-V 系统镜像 实验指定的测试镜像 (当然如果不是实验指定的话,你也可以使用其他的镜像): https://repo.tarsier-infra.com/openEuler-RISC-V/preview/openEuler-23.09-V1-riscv64/QEMU/ 由于我是直接使用 ssh 连接 openEuler RISC-V 的 QEMU 虚拟机,所以只下载了: fw_payload_oe_uboot_2304.bin 启动用内核 openEuler-23.09-V1-base-qemu-preview.qcow2.zst 不带有桌面镜像的根文件系统 start_vm.sh 启动不带有桌面镜像的根文件系统用脚本 $ wget https://repo.tarsier-infra.com/openEuler-RISC-V/preview/openEuler-23.09-V1-riscv64/QEMU/fw_payload_oe_uboot_2304.bin $ wget https://repo.tarsier-infra.com/openEuler-RISC-V/preview/openEuler-23.09-V1-riscv64/QEMU/openEuler-23.09-V1-base-qemu-preview.qcow2.zst $ wget https://repo.tarsier-infra.com/openEuler-RISC-V/preview/openEuler-23.09-V1-riscv64/QEMU/start_vm.sh 解压缩根文件系统的磁盘映像: # install unzip tool zstd for zst $ sudo apt install zstd $ unzstd openEuler-23.09-V1-base-qemu-preview.qcow2.zst ","date":"2024-03-07","objectID":"/posts/openeuler-riscv-qemu/:3:0","tags":["RISC-V","openEuler","QEMU","Mugen"],"title":"openEuler RISC-V 系统: QEMU 仿真","uri":"/posts/openeuler-riscv-qemu/"},{"categories":["RISC-V"],"content":"启动 openEuler RISC-V 系统并连接 确认当前在刚刚下载了内核、根文件系统、启动脚本的目录,然后在一个终端上执行启动脚本: $ bash start_vm.sh 安心等待输出完毕出现提示登录界面 (时间可能会有点长),然后输入账号和密码进行登录即可 或者启动 QEMU 虚拟机后,新开一个终端通过 ssh 进行登录: $ ssh -p 12055 root@localhost $ ssh -p 12055 openeuler@localhost 通过 exit 命令可以退出当前登录账号,通过快捷键 Ctrl + A, X 可以关闭 QEMU 虚拟机 (本质上是信号 signal 处理 🤣) 建议登录后修改账号的密码 (相关命令: passwd) ","date":"2024-03-07","objectID":"/posts/openeuler-riscv-qemu/:4:0","tags":["RISC-V","openEuler","QEMU","Mugen"],"title":"openEuler RISC-V 系统: QEMU 仿真","uri":"/posts/openeuler-riscv-qemu/"},{"categories":["RISC-V"],"content":"Mugen 测试框架 根据 「mugen」 README 的使用教程,在指定测试镜像上完成 「安装依赖软件」「配置测试套环境变量」「用例执行」这三个部分,并给出实验总结 ","date":"2024-03-07","objectID":"/posts/openeuler-riscv-qemu/:5:0","tags":["RISC-V","openEuler","QEMU","Mugen"],"title":"openEuler RISC-V 系统: QEMU 仿真","uri":"/posts/openeuler-riscv-qemu/"},{"categories":["RISC-V"],"content":"安装 git \u0026 克隆 mugen 仓库 登录普通用户 openeuler 然后发现此时没有安装 git 无法克隆 mugen 仓库,先安装 git: $ sudo dnf install git $ git clone https://gitee.com/openeuler/mugen.git 原始设定的 vim 配置不太优雅,我根据我的 vim 配置进行了设置,具体见 「Vim 配置」 ","date":"2024-03-07","objectID":"/posts/openeuler-riscv-qemu/:5:1","tags":["RISC-V","openEuler","QEMU","Mugen"],"title":"openEuler RISC-V 系统: QEMU 仿真","uri":"/posts/openeuler-riscv-qemu/"},{"categories":["RISC-V"],"content":"安装依赖软件 进入 mugen 目录执行安装依赖软件脚本 (因为我使用的是普通用户,需要使用 sudo 提高权级): $ sudo bash dep_install.sh ... Complete! ","date":"2024-03-07","objectID":"/posts/openeuler-riscv-qemu/:5:2","tags":["RISC-V","openEuler","QEMU","Mugen"],"title":"openEuler RISC-V 系统: QEMU 仿真","uri":"/posts/openeuler-riscv-qemu/"},{"categories":["RISC-V"],"content":"配置测试套环境变量 $ sudo bash mugen.sh -c --ip $ip --password $passwd --user $user --port $port 这部分仓库的文档对于本机测试没有很清楚地说明,参考文章 「基于openEuler虚拟机本地执行mugen测试脚本」完成配置 执行完成后会多出一个环境变量文件 ./conf/env.json ","date":"2024-03-07","objectID":"/posts/openeuler-riscv-qemu/:5:3","tags":["RISC-V","openEuler","QEMU","Mugen"],"title":"openEuler RISC-V 系统: QEMU 仿真","uri":"/posts/openeuler-riscv-qemu/"},{"categories":["RISC-V"],"content":"用例执行 \u0026 结果分析 我对于 openEuler RISC-V 是否支持了 binutils 比较感兴趣,便进行了测试: $ sudo bash mugen.sh -f binutils -x ... INFO - A total of 8 use cases were executed, with 8 successes 0 failures and 0 skips. 执行结果显示已正确支持 binutils 接下来对程序静态分析工具 cppcheck 的支持进行测试: $ sudo bash mugen.sh -f cppcheck -x ... INFO - A total of 2 use cases were executed, with 1 successes 1 failures and 0 skips. 根据文档 suite2cases 中 json文件的写法 的解释,分析刚刚执行的测试套 suit2cases/cppcheck.json,是测试用例 oe_test_cppcheck 失败了。 观察该用例对应的脚本 testcases/cli-test/cppcheck/oe_test_cppcheck/oe_test_cppcheck.sh,并打开对应的日志 logs/cppcheck/oe_test_cppcheck/$(date).log 在里面检索 LOG_ERROR,找到两处相关错误: + LOG_ERROR 'oe_test_cppcheck.sh line 70' + LOG_ERROR 'oe_test_cppcheck.sh line 95' 比照用例脚本,对应的测试逻辑是: cppcheck --std=c99 --std=posix test.cpp 70--\u003e CHECK_RESULT $? if [ $VERSION_ID != \"22.03\" ]; then cppcheck -DA --force file.c | grep \"A=1\" 95--\u003e CHECK_RESULT $? 1 else cppcheck -DA --force file.c | grep \"A=1\" CHECK_RESULT $? fi Cppcheck manual P11 The flag -D tells Cppcheck that a name is defined. There will be no Cppcheck analysis without this define. The flag –force and –max-configs is used to control how many combinations are checked. When -D is used, Cppcheck will only check 1 configuration unless these are used. 这里面 CHECK_RESULT 是一个自定义的 shell 函数,扫一下 mugen 的库目录 libs,在 locallibs/common_lib.sh 里找到该函数的定义,它的逻辑比较好懂 (类似于 assert),但是函数开头的变量定义让我有些迷糊,于是求教于 GPT: actual_result=$1 expect_result=${2-0} mode=${3-0} error_log=$4 GPT: actual_result 变量被赋值为第一个参数的值。 expect_result 变量被赋值为第二个参数的值,如果第二个参数不存在,则默认为 0。 mode 变量被赋值为第三个参数的值,如果第三个参数不存在,则默认为 0。 error_log 变量被赋值为第四个参数的值。 所以,涉及错误的两个测试逻辑都很好理解了: CHECK_RESULT $? 表示上一条命令返回值的预期是 0 CHECK_RESULT $? 1 表示上一条命令返回值的预期是 1 接下来我们就实际测试一下这两个用例: 安装 cppcheck: $ sudo dnf install cppcheck 执行测试脚本 70 行对应的上一条命令: $ cppcheck --std=c99 --std=posix test.cpp cppcheck: error: unknown --std value 'posix' $ echo $? 1 测试失败原因是 cppcheck risc-v 版本不支持指定 C/C++ 标准为 posix (同时查询了下 「cppcheck manual」目前 cppcheck 支持的标准里并未包括 posix) 执行测试脚本 95 行对应的上一条命令: $ cppcheck -DA --force file.c | grep \"A=1\" Checking file.c: A=1... file.c:5:6: error: Array 'a[10]' accessed at index 10, which is out of bounds. [arrayIndexOutOfBounds] a[10] = 0; ^ $ echo $? 0 测试失败原因是 grep 在之前的 cppcheck 的输出里匹配到 A=1,所以导致返回值为 0。这部分测试的逻辑是: 仅对于 22.03 版本 openEuler 上的 cppcheck 在以参数 -DA 执行时才会输出包含 A=1 的信息,但是个人猜测是在比 22.03 及更高版本的 openEuler 上使用 cppcheck 搭配 -DA 都可以输出包含 A=1 的信息 ","date":"2024-03-07","objectID":"/posts/openeuler-riscv-qemu/:5:4","tags":["RISC-V","openEuler","QEMU","Mugen"],"title":"openEuler RISC-V 系统: QEMU 仿真","uri":"/posts/openeuler-riscv-qemu/"},{"categories":["RISC-V"],"content":"实验总结和讨论 初步体验了使用 QEMU 构建 openEuler RISC-V 系统虚拟机的流程,以及使用 ssh 连接 QEMU 虚拟机的技巧。实验过程中最大感触是 mugen 的文档,相对于 cppcheck 这类产品的文档,不够详细,很多内容需要阅读源码来理解 (好处是精进了我对 shell 脚本编程的理解 🤣)。 我个人比较期待 RISC-V 配合 nommu 在嵌入式这类低功耗领域的发展,同时也对 RISC-V Hypervisor Extension 在虚拟化方面的发展感兴趣。 ","date":"2024-03-07","objectID":"/posts/openeuler-riscv-qemu/:5:5","tags":["RISC-V","openEuler","QEMU","Mugen"],"title":"openEuler RISC-V 系统: QEMU 仿真","uri":"/posts/openeuler-riscv-qemu/"},{"categories":["RISC-V"],"content":"References openEuler RISC-V: 通过 QEMU 仿真 RISC-V 环境并启动 OpenEuler RISC-V 系统 openEuler RISC-V: 使用 QEMU 安装 openEuler RISC-V 23.03 Ariel Heleneto: 通过 QEMU 仿真 RISC-V 环境并启动 OpenEuler RISC-V 系统 openEuler: mugen openEuler Docs: 使用 DNF 管理软件包 基于 openEuler 虚拟机本地执行 mugen 测试脚本 Video: Mugen 框架的使用 https://openbuildservice.org/help/manuals/obs-user-guide/ https://gitee.com/openEuler/RISC-V#/openeuler/RISC-V/ https://gitee.com/zxs-un/doc-port2riscv64-openEuler ","date":"2024-03-07","objectID":"/posts/openeuler-riscv-qemu/:6:0","tags":["RISC-V","openEuler","QEMU","Mugen"],"title":"openEuler RISC-V 系统: QEMU 仿真","uri":"/posts/openeuler-riscv-qemu/"},{"categories":["C","Linux Kernel Internals"],"content":" 借由阅读 C 语言标准理解规范是研究系统安全最基础的步骤,但很多人都忽略阅读规范这点,而正因对于规范的不了解、撰写程序的不严谨,导致漏洞的产生的案例比比皆是,例如 2014 年的 OpenSSL Heartbleed Attack1 便是便是因为使用 memcpy 之际缺乏对应内存范围检查,造成相当大的危害。本文重新梳理 C 语言程序设计的细节,并借由调试器帮助理解程序的运作。 原文地址 ","date":"2024-03-05","objectID":"/posts/c-std-security/:0:0","tags":["Sysprog","C","Security"],"title":"基于 C 语言标准研究与系统程序安全议题","uri":"/posts/c-std-security/"},{"categories":["C","Linux Kernel Internals"],"content":"目标 借由研读漏洞程序及 C 语言标准,讨论系统程序的安全议题 通过调试器追踪程序实际运行的状况,了解其运作原理 取材自 dangling pointer, CWE-416 Use After Free, CVE-2017-16943 以及 integer overflow 的议题 ","date":"2024-03-05","objectID":"/posts/c-std-security/:1:0","tags":["Sysprog","C","Security"],"title":"基于 C 语言标准研究与系统程序安全议题","uri":"/posts/c-std-security/"},{"categories":["C","Linux Kernel Internals"],"content":"实验环境 编译器版本: gcc 11 调试器: GDB 操作系统: Ubuntu Linux 22.04 ","date":"2024-03-05","objectID":"/posts/c-std-security/:2:0","tags":["Sysprog","C","Security"],"title":"基于 C 语言标准研究与系统程序安全议题","uri":"/posts/c-std-security/"},{"categories":["C","Linux Kernel Internals"],"content":"主题 (一): Integer type 资料处理 ","date":"2024-03-05","objectID":"/posts/c-std-security/:3:0","tags":["Sysprog","C","Security"],"title":"基于 C 语言标准研究与系统程序安全议题","uri":"/posts/c-std-security/"},{"categories":["C","Linux Kernel Internals"],"content":"I. Integer Conversion \u0026 Integer Promotion #include \u003cstdint.h\u003e #include \u003cstdio.h\u003e unsigned int ui = 0; unsigned short us = 0; signed int si = -1; int main() { int64_t r1 = ui + si; int64_t r2 = us + si; printf(\"%lld %lld\\n\", r1, r2); } 上述程式码执行结果为: r1 输出为十进制的 4294967295,r2 输出为十进制的 -1。这个结果和 C11 规格书中提到的 Integer 的两个特性有关: Integer Conversion 和 Integer Promotion。 (1) Integer Conversion C11 6.3.1.1 Boolean, characters, and integers Every integer type has an integer conversion rank defined as follows: No two signed integer types shall have the same rank, even if they hav e the same representation. The rank of a signed integer type shall be greater than the rank of any signed integer type with less precision. The rank of long long int shall be greater than the rank of long int, which shall be greater than the rank of int, which shall be greater than the rank of short int, which shall be greater than the rank of signed char. The rank of any unsigned integer type shall equal the rank of the corresponding signed integer type, if any. The rank of any standard integer type shall be greater than the rank of any extended integer type with the same width. The rank of any extended signed integer type relative to another extended signed integer type with the same precision is implementation-defined, but still subject to the other rules for determining the integer conversion rank. 依据上述标准可排出 integer 的 rank: long long int \u003e long int \u003e int \u003e short int \u003e signed char unsigned int == signed int, if they are both in same precision and same size (2) Integer Promotion 当 integer 进行通常的算数运算 (Usual arithmetic) 时,会先进行 integer promotions 转换成 int 或 unsigned int 或者保持不变 (转换后的运算子被称为 promoted operands),然后 promoted operands 再根据自身类型以及对应的 rank 进行 arithmetic conversions,最终得到结果的类型。 C11 6.3.1.1 Boolean, characters, and integers If an int can represent all values of the original type (as restricted by the width, for a bit-field), the value is converted to an int; otherwise, it is converted to an unsigned int. These are called the integer promotions. All other types are unchanged by the integer promotions. C11 6.3.1.8 Usual arithmetic conversions Otherwise, the integer promotions are performed on both operands. Then the following rules are applied to the promoted operands: If both operands have the same type, then no further conversion is needed. Otherwise, if both operands have signed integer types or both have unsigned integer types, the operand with the type of lesser integer conversion rank is converted to the type of the operand with greater rank. Otherwise, if the operand that has unsigned integer type has rank greater or equal to the rank of the type of the other operand, then the operand with signed integer type is converted to the type of the operand with unsigned integer type. Otherwise, if the type of the operand with signed integer type can represent all of the values of the type of the operand with unsigned integer type, then the operand with unsigned integer type is converted to the type of the operand with signed integer type. Otherwise, both operands are converted to the unsigned integer type corresponding to the type of the operand with signed integer type. /* In the case that the rank is smaller than int */ char c1, c2; // Both of them are char c1 = c1 + c2; // Both are promoted to int, thus result of c1 becomes to integer /* In the case that the rank is same as int */ signed int si = -1; /* si \u0026 ui are at the same rank both are unchanged by the integer promotions */ unsigned int ui = 0; int result = si + ui; // si is converted to unsigned int, result is unsigned ","date":"2024-03-05","objectID":"/posts/c-std-security/:3:1","tags":["Sysprog","C","Security"],"title":"基于 C 语言标准研究与系统程序安全议题","uri":"/posts/c-std-security/"},{"categories":["C","Linux Kernel Internals"],"content":"II. 衍生的安全议题: Integer Overflow Stack Overflow: What is an integer overflow error? ","date":"2024-03-05","objectID":"/posts/c-std-security/:3:2","tags":["Sysprog","C","Security"],"title":"基于 C 语言标准研究与系统程序安全议题","uri":"/posts/c-std-security/"},{"categories":["C","Linux Kernel Internals"],"content":"主题 (二): Object 的生命周期 ","date":"2024-03-05","objectID":"/posts/c-std-security/:4:0","tags":["Sysprog","C","Security"],"title":"基于 C 语言标准研究与系统程序安全议题","uri":"/posts/c-std-security/"},{"categories":["C","Linux Kernel Internals"],"content":"I. Dangling Pointer C11 6.2.4 Storage durations of objects (2) The lifetime of an object is the portion of program execution during which storage is guaranteed to be reserved for it. An object exists, has a constant address, and retains its last-stored value throughout its lifetime. If an object is referred to outside of its lifetime, the behavior is undefined. The value of a pointer becomes indeterminate when the object it points to (or just past) reaches the end of its lifetime. Stack Overflow: What is a dangling pointer? When a pointer is pointing at the memory address of a variable but after some time that variable is deleted from that memory location while the pointer is still pointing to it, then such a pointer is known as a dangling pointer and this problem is known as the dangling pointer problem. 所以在 object 的生命周期结束后,应将指向 object 原本处于的内存空间的指针置为 NULL,避免 dangling pointer。 ","date":"2024-03-05","objectID":"/posts/c-std-security/:4:1","tags":["Sysprog","C","Security"],"title":"基于 C 语言标准研究与系统程序安全议题","uri":"/posts/c-std-security/"},{"categories":["C","Linux Kernel Internals"],"content":"II. CWE-416 Use After Free OWASP: Using freed memory Referencing memory after it has been freed can cause a program to crash. The use of heap allocated memory after it has been freed or deleted leads to undefined system behavior and, in many cases, to a write-what-where condition. ","date":"2024-03-05","objectID":"/posts/c-std-security/:4:2","tags":["Sysprog","C","Security"],"title":"基于 C 语言标准研究与系统程序安全议题","uri":"/posts/c-std-security/"},{"categories":["C","Linux Kernel Internals"],"content":"III. 案例探讨: CVE-2017-16943 Abusing UAF leads to Exim RCE Road to Exim RCE - Abusing Unsafe Memory Allocator in the Most Popular MTA ","date":"2024-03-05","objectID":"/posts/c-std-security/:4:3","tags":["Sysprog","C","Security"],"title":"基于 C 语言标准研究与系统程序安全议题","uri":"/posts/c-std-security/"},{"categories":["C","Linux Kernel Internals"],"content":"实验结果与验证 Source ","date":"2024-03-05","objectID":"/posts/c-std-security/:5:0","tags":["Sysprog","C","Security"],"title":"基于 C 语言标准研究与系统程序安全议题","uri":"/posts/c-std-security/"},{"categories":["C","Linux Kernel Internals"],"content":"(ㄧ) Integer Promotion 验证 测试程式码: #include \u003cstdint.h\u003e #include \u003cstdio.h\u003e unsigned int ui = 0; unsigned short us = 0; signed int si = -1; int main() { int64_t r1 = ui + si; int64_t r2 = us + si; printf(\"%lld %lld\\n\", r1, r2); } 验证结果: $ gcc -g -o integer-promotion.o integer-promotion.c $ ./integer-promotion.o 4294967295 -1 ","date":"2024-03-05","objectID":"/posts/c-std-security/:5:1","tags":["Sysprog","C","Security"],"title":"基于 C 语言标准研究与系统程序安全议题","uri":"/posts/c-std-security/"},{"categories":["C","Linux Kernel Internals"],"content":"(二) Object 生命周期 测试程式码: #include \u003cinttypes.h\u003e #include \u003cstdint.h\u003e #include \u003cstdio.h\u003e #include \u003cstdlib.h\u003e int main(int argc, char *argv[]) { char *p, *q; uintptr_t pv, qv; { char a = 3; p = \u0026a; pv = (uintptr_t) p; } { char b = 4; q = \u0026b; qv = (uintptr_t) q; } if (p != q) { printf(\"%p is different from %p\\n\", (void *) p, (void *) q); printf(\"%\" PRIxPTR \" is not the same as %\" PRIxPTR \"\\n\", pv, qv); } else { printf(\"Surprise!\\n\"); } return 0; } 验证结果: $ gcc -g -o uaf.o uaf.c $ ./uaf.o Surprise! $ gcc -g -o uaf.o uaf.c -fsanitize-address-use-after-scope $ ./uaf.o 0x7ffca405c596 is different from 0x7ffca405c597 7ffca405c596 is not the same as 7ffca405c597 $ clang -g -o uaf.o uaf.c $ ./uaf.o 0x7fff86b298ff is different from 0x7fff86b298fe 7fff86b298ff is not the same as 7fff86b298fe gcc 可以通过显式指定参数 -fsanitize-address-use-after-scope 来避免 Use-After-Scope 的问题,否则在 scope 结束后,接下来的其他 scope 会使用之前已结束的 scope 的内存空间,从而造成 Use-After-Scope 问题 (使用 GDB 在上面两种不同的情况下,查看变量 a, b 所在的地址),而 clang 则是默认开启相关保护。 “OpenSSL Heartbleed”, Synopsys ↩︎ ","date":"2024-03-05","objectID":"/posts/c-std-security/:5:2","tags":["Sysprog","C","Security"],"title":"基于 C 语言标准研究与系统程序安全议题","uri":"/posts/c-std-security/"},{"categories":["Rust"],"content":" In this Crust of Rust episode, we implement some common sorting algorithms in Rust. This episode doesn't aim to explain any single concept, but rather showcase what writing “normal” Rust code is like, and explaining various “odd bits” we come across along the way. The thinking here is that sorting algorithms are both familiar and easy to compare across languages, so this might serve as a good bridge into Rust if you are familiar with other languages. 整理自 John Gjengset 的影片 问题 You may note that the url of this posy is “orst”. Why was it given this name? Since “sort” when sorted becomes “orst”. 🤣 ","date":"2024-03-04","objectID":"/posts/orst/:0:0","tags":["Rust","Sort","Algorithm"],"title":"Crust of Rust: Sorting Algorithms","uri":"/posts/orst/"},{"categories":["Rust"],"content":"影片注解 ","date":"2024-03-04","objectID":"/posts/orst/:1:0","tags":["Rust","Sort","Algorithm"],"title":"Crust of Rust: Sorting Algorithms","uri":"/posts/orst/"},{"categories":["Rust"],"content":"Total order vs Partial order Wikipedia: Total order Wikipedia: Partial order Stack Overflow: What does it mean by “partial ordering” and “total ordering” in the discussion of Lamport's synchronization Algorithm? This definition says that in a total order any two things are comparable. Wheras in a partial order a thing needs neither to be “smaller” than an other nor the other way around, in a total order each thing is either “smaller” than an other or the other way around. 简单来说,在 total order 中任意两个元素都可以进行比较,而在 partial order 中则不一定满足。例如对于集合 $$ S = \\{a,\\ b,\\ c\\} $$ 在 total order 中,$a, b, c$ 任意两个元素之间都必须能进行比较,而在 partial order 中没有怎么严格的要求,可能只有 $a \u003c b, b \u003c c$ 这两条比较规则。 在 Rust 中,浮点数 (f32, f64) 只实现了 PartialOrd 这个 Trait 而没有实现 Ord,因为根据 IEEE 754,浮点数中存在一些特殊值,例如 NaN,它们是没法进行比较的。出于相同原因,浮点数也只实现了 PartialEq 而没有实现 Eq trait。 ","date":"2024-03-04","objectID":"/posts/orst/:1:1","tags":["Rust","Sort","Algorithm"],"title":"Crust of Rust: Sorting Algorithms","uri":"/posts/orst/"},{"categories":["Rust"],"content":"Trait \u0026 Generic pub fn sort\u003cT, S\u003e(slice: \u0026mut [T]) where T: Ord, S: Sorter\u003cT\u003e, { S::sort(slice); } sort::\u003c_, StdSorter\u003e(\u0026mut things); 这段代码巧妙地利用泛型 (generic) 来传递了\"参数\",当然这种技巧只限于可以通过类型来调用方法的情况 (上面代码段的 S::sort(...) 以及 sort::\u003c_, StdSorter\u003e(...) 片段)。 思考以下代码表示的意义: pub trait Sorter\u003cT\u003e { fn sort(slice: \u0026mut [T]) where T: Ord; } pub trait Sorter { fn sort\u003cT\u003e(slice: \u0026mut [T]) where T: Ord; } 第一个表示的是有多个 tait,例如 Sorter\u003ci32\u003e, Sorter\u003ci64\u003e 等,第二个表示只有一个 trait Sorter,但是实现这个 trait 需要实现多个方法,例如 sort\u003ci32\u003e, sort\u003ci64\u003e 等,所以第一种写法更加普适和使用 (因为未必能完全实现第二种 trait 要求的所有方法)。 ","date":"2024-03-04","objectID":"/posts/orst/:1:2","tags":["Rust","Sort","Algorithm"],"title":"Crust of Rust: Sorting Algorithms","uri":"/posts/orst/"},{"categories":["Rust"],"content":"Bubble sort Wikipedia: Bubble sort n := length(A) repeat swapped := false for i := 1 to n-1 inclusive do { if this pair is out of order } if A[i-1] \u003e A[i] then { swap them and remember something changed } swap(A[i-1], A[i]) swapped := true end if end for until not swapped ","date":"2024-03-04","objectID":"/posts/orst/:1:3","tags":["Rust","Sort","Algorithm"],"title":"Crust of Rust: Sorting Algorithms","uri":"/posts/orst/"},{"categories":["Rust"],"content":"Insertion sort Wikipedia: Insertion sort i ← 1 while i \u003c length(A) j ← i while j \u003e 0 and A[j-1] \u003e A[j] swap A[j] and A[j-1] j ← j - 1 end while i ← i + 1 end while 使用 Binary search algorithm 可以将 insertion sort 的 comparsion 次数降到 $O(nlogn)$,但是 swap 次数仍然是 $O(n^2)$ 🤣 // use binary search to find index // then use .insert to splice in i let i = match slice[..unsorted].binary_search(\u0026slice[unsorted]) { // [ a, c, e].binary_search(c) =\u003e Ok(1) Ok(i) =\u003e i, // [ a, c, e].binary_search(b) =\u003e Err(1) Err(i) =\u003e i, }; slice[i..=unsorted].rotate_right(1); match 的内部逻辑也可以改写为 OK(i) | Err(i) =\u003e i ","date":"2024-03-04","objectID":"/posts/orst/:1:4","tags":["Rust","Sort","Algorithm"],"title":"Crust of Rust: Sorting Algorithms","uri":"/posts/orst/"},{"categories":["Rust"],"content":"Selection sort Wikipedia: Selection sort 引用 There are many different ways to sort the cards. Here’s a simple one, called selection sort, possibly similar to how you sorted the cards above: Find the smallest card. Swap it with the first card. Find the second-smallest card. Swap it with the second card. Find the third-smallest card. Swap it with the third card. Repeat finding the next-smallest card, and swapping it into the correct position until the array is sorted. source 使用函数式编程可以写成相当 readable 的程式码,以下为获取 slice 最小值对应的 index: let smallest_in_rest = slice[unsorted..] .iter() .enumerate() .min_by_key(|\u0026(_, v)| v) .map(|(i, _)| unsorted + i) .expect(\"slice is not empty\"); ","date":"2024-03-04","objectID":"/posts/orst/:1:5","tags":["Rust","Sort","Algorithm"],"title":"Crust of Rust: Sorting Algorithms","uri":"/posts/orst/"},{"categories":["Rust"],"content":"Quicksort Wikipedia: Quicksort 可以通过 extra allocation 和 in-place 两种方式来实现 quicksort,其中 extra allocation 比较好理解,in-place 方式的 pseudocode 如下: Quicksort(A,p,r) { if (p \u003c r) { q \u003c- Partition(A,p,r) Quicksort(A,p,q) Quicksort(A,q+1,r) } } Partition(A,p,r) x \u003c- A[p] i \u003c- p-1 j \u003c- r+1 while (True) { repeat { j \u003c- j-1 } until (A[j] \u003c= x) repeat { i \u003c- i+1 } until (A[i] \u003e= x) if (i \u003c j) swap(A[i], A[j]) else return(j) } } source method slice::split_at_mut 实现 Quick sort 时使用了 split_at_mut 来绕开引用检查,因为如果你此时拥有一个指向 pivot 的不可变引用,就无法对 slice 剩余的部分使用可变引用,而 split_at_mut 则使得原本的 slice 被分为两个可变引用,从而绕开了之前的单一引用检查。 后面发现可以使用更符合语义的 split_first_mut,当然思路还是一样的 注意 我个人认为实现 Quick sort 的关键在于把握以下两个 invariants: left: current checking index for element which is equal or less than the pivot right: current checking index for element which is greater than the pivot 即这两个下标对应的元素只是当前准备检查的,不一定符合元素的排列规范,如下图所示: [ \u003c= pivot ] [ ] [ ... ] [ ] [ \u003e pivot ] ^ ^ | | left right 所以当 left == right 时两边都没有对所指向的元素进行检查,分情况讨论 (该元素是 $\u003c= pivot$ 或 $\u003e pivot$) 可以得出: 当 left \u003e right 时,right 指向的是 $\u003c= pivot$ 的元素,将其与 pivot 进行 swap 即可实现 partition 操作。(其实此时 left 指向的是 $\u003e pivot$ 部分的第一个元素,right 指向的是 $\u003c= pivot$ 部分的最后一个元素,但是需要注意 rest 与 slice 之间的下标转换) ","date":"2024-03-04","objectID":"/posts/orst/:1:6","tags":["Rust","Sort","Algorithm"],"title":"Crust of Rust: Sorting Algorithms","uri":"/posts/orst/"},{"categories":["Rust"],"content":"Benchmark 通过封装类型 SortEvaluator 及实现 trait PartialEq, Eq, PartialOrd, Ord 来统计排序过程中的比较操作 (eq, partial_cmp, cmp) 的次数。 Stack Overflow: Why can't the Ord trait provide default implementations for the required methods from the inherited traits using the cmp function? ","date":"2024-03-04","objectID":"/posts/orst/:1:7","tags":["Rust","Sort","Algorithm"],"title":"Crust of Rust: Sorting Algorithms","uri":"/posts/orst/"},{"categories":["Rust"],"content":"R and ggplot2 # install R $ sudo apt install r-base # install ggplot2 by R $ R \u003e install.packages(\"ggplot2\") Are there Unix-like binaries for R? https://ggplot2.tidyverse.org/ 问题 deepin 软件源下载的 R 语言包可能版本过低 (3.5),可以通过添加库源的方式来下载高版本的 R 语言包: 1.添加 Debian buster (oldstable) 库源到 /etc/apt/sourcelist 里: # https://mirrors.tuna.tsinghua.edu.cn/CRAN/ deb http://cloud.r-project.org/bin/linux/debian buster-cran40/ 2.更新软件,可能会遇到没有公钥的问题 (即出现下方的 NO_PUBKEY): $ sudo apt update ... NO_PUBKEY XXXXXX ... 此时可以 NO_PUBKEY 后的 XXXXXX 就是公钥,我们只需要将其添加一下即可: $ sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys XXXXXX 添加完公钥后再重新更新一次软件源 3.通过指定库源的方式来安装 R (如果未指定库源则还是从默认源进行下载 3.5 版本): $ sudo apt install buster-cran40 r-base $ R --version R version 4.3.3 (2024-02-29) 大功告成,按照上面安装 ggplot2 即可 ","date":"2024-03-04","objectID":"/posts/orst/:1:8","tags":["Rust","Sort","Algorithm"],"title":"Crust of Rust: Sorting Algorithms","uri":"/posts/orst/"},{"categories":["Rust"],"content":"Homework 信息 实作说明: 添加标准库的 sort_unstable 进入基准测试 将交换操作 (swap) 纳入基准测试 尝试实现 Merge sort 尝试实现 Heapsort 参考资料: Wikipedia: Merge sort Wikipedia: Heapsort ","date":"2024-03-04","objectID":"/posts/orst/:2:0","tags":["Rust","Sort","Algorithm"],"title":"Crust of Rust: Sorting Algorithms","uri":"/posts/orst/"},{"categories":["Rust"],"content":"Documentations 这里列举视频中一些概念相关的 documentation 学习的一手资料是官方文档,请务必自主学会阅读规格书之类的资料 ","date":"2024-03-04","objectID":"/posts/orst/:3:0","tags":["Rust","Sort","Algorithm"],"title":"Crust of Rust: Sorting Algorithms","uri":"/posts/orst/"},{"categories":["Rust"],"content":"Crate std 可以使用这里提供的搜素栏进行搜索 (BTW 不要浪费时间在 Google 搜寻上!) Module std::cmp Trait std::cmp::Ord Trait std::cmp::PartialOrd Trait std::cmp::Eq Trait std::cmp::PartialEq Primitive Type slice method slice::sort method slice::sort_unstable method slice::sort_by method slice::sort_by_key method slice::swap method slice::binary_search method slice::rotate_right method slice::split_at_mut method slice::split_first_mut method slice::to_vec Trait std::iter::Iterator method std::iter::Iterator::min method std::iter::Iterator::min_by_key method std::iter::Iterator::enumerate Enum std::option::Option method std::option::Option::expect method std::option::Option::map Enum std::result::Result method std::result::Result::expect method std::result::Result::map Module std::time method std::time::Instant::now method std::time::Instant::elapsed method std::time::Duration::as_secs_f64 ","date":"2024-03-04","objectID":"/posts/orst/:3:1","tags":["Rust","Sort","Algorithm"],"title":"Crust of Rust: Sorting Algorithms","uri":"/posts/orst/"},{"categories":["Rust"],"content":"Crate rand Function rand::thread_rng method rand::seq::SliceRandom::shuffle ","date":"2024-03-04","objectID":"/posts/orst/:3:2","tags":["Rust","Sort","Algorithm"],"title":"Crust of Rust: Sorting Algorithms","uri":"/posts/orst/"},{"categories":["Rust"],"content":"References orst [Github] Sorting algorithm [Wikipedia] Timsort [Wikipedia] Difference between Benchmarking and Profiling [Stack Overflow] ","date":"2024-03-04","objectID":"/posts/orst/:4:0","tags":["Rust","Sort","Algorithm"],"title":"Crust of Rust: Sorting Algorithms","uri":"/posts/orst/"},{"categories":["Linux","Linux Kernel Internals"],"content":" 本講座將以 Thorsten Leemhuis 在 FOSDEM 2020 開場演說 “Linux kernel – Solving big problems in small steps for more than 20 years” (slides) 為主軸,嘗試歸納自 21 世紀第一年開始的 Linux 核心 2.4 版到如今的 5.x 版,中間核心開發者如何克服 SMP (Symmetric multiprocessing), scalability, 及各式硬體架構和周邊裝置支援等難題,過程中提出全面移除 BKL (Big kernel lock)、實作虛擬化技術 (如 Xen 和 KVM)、提出 namespace 和 cgroups 從而確立容器化 (container) 的能力,再來是核心發展的明星技術 eBPF 會在既有的基礎之上,帶來 XDP 和哪些令人驚豔的機制呢?又,Linux 核心終於正式納入發展十餘年的 PREEMPT_RT,使得 Linux 核心得以成為硬即時的作業系統,對內部設計有哪些衝擊?AIO 後繼的 io_uring 讓 Linux 有更優雅且高效的非同步 I/O 存取,我們該如何看待? 原文地址 ","date":"2024-03-03","objectID":"/posts/linux-dev-review/:0:0","tags":["Sysprog","Linux"],"title":"Linux 核心设计: 发展动态回顾","uri":"/posts/linux-dev-review/"},{"categories":["Linux","Linux Kernel Internals"],"content":"开篇点题 前置知识: Linux 核心设计: 操作系统术语及概念 FOSDEM 2020, T. Leemhuis: YouTube: Linux kernel – Solving big problems in small steps for more than 20 years slides (这个投影片共有 248 页,所以加载时可能会比较慢 🤣) 以上面的讲座为主轴,回顾 Linux 的发展动态,由此展望 Linux 未来的发展方向。 SMP (Symmetric multiprocessing) scalability BKL (Big kernel lock) Xen, KVM namespace, cgroups, container - 云服务 eBPF, XDP - 网络封包的高效过滤 (在内核即可处理封包的过滤,无需在用户态制定规则) PREEMPT_RT - 硬即时操作系统 (hard real time os) io_uring - 高效的非同步 I/O (Linux 大部分系统调用都是非同步的) nommu - 用于嵌入式降低功耗 Linux 相关人物 (可在 YouTube 上找到他们的一些演讲): Jonathan Corbet ","date":"2024-03-03","objectID":"/posts/linux-dev-review/:1:0","tags":["Sysprog","Linux"],"title":"Linux 核心设计: 发展动态回顾","uri":"/posts/linux-dev-review/"},{"categories":["Linux","Linux Kernel Internals"],"content":"Linux 2.4 Version 2.4 of the LINUX KERNEL–Why Should a System Administrator Upgrade? 自 2004 年開始,釋出過程發生變化,新核心每隔 2-3 個月定期釋出,編號為 2.6.0, 2.6.1,直到 2.6.39 这件事对于操作系统的开发有很大的影响,是一个巨大的变革。透过这种发行机制,CPU 厂商可以直接在最新的 Linux kernel 上适配正在开发的 CPU 及相关硬体,而无需拿到真正的 CPU 硬体再进行相应的开发,这使得 Linux 获得了更多厂商的支持和投入,进而进入了飞速发展期。 LInux 核心的道路: 只提供机制不提供策略。例如 khttp (in-kernel httpd) 的弃用,通过提供更高效的系统调用来提高网页服务器的效能,而不是像 Windows NT 一样用户态性能不够就把程式搬进 kernel 🤣 ","date":"2024-03-03","objectID":"/posts/linux-dev-review/:2:0","tags":["Sysprog","Linux"],"title":"Linux 核心设计: 发展动态回顾","uri":"/posts/linux-dev-review/"},{"categories":["Linux","Linux Kernel Internals"],"content":"SMP 支援 相关故事: Digital Domain and TITANIC (泰坦尼克号) Red Hat Sinks Titanic Linux Helps Bring Titanic to Life Digital Domain: TITANIC Industrial Light and Magic MaterialX Joins the Academy Software Foundation as a Hosted Project 制作《泰坦尼克号》的特效时,使用了安装 Linux 操作系统的 Alpha 处理器,而 Alpha 是多核处理器,所以当年将 Linux 安装到 Alpha 上需要支援 SMP,由此延伸出了 BLK (Big kernel lock)。 Linux 2.4 在 SMP 的效率问题也正是 BLK 所引起的: BLK 用于锁定整个 Linux kernel,而整个 Linux kernel 只有一个 BLK 实作机制: 在执行 schedule 时当前持有 BLK 的 process 需要释放 BLK 以让其他 process 可以获得 BLK,当轮到该 process 执行时,可以重新获得 BLK 从上面的实作机制可以看出,这样的机制效率是很低的,虽然有多核 (core),但是当一个 process 获得 BLK 时,只有该 process 所在的 core 可以执行,其他 core 只能等待 BLK 已于 v.6.39 版本中被彻底去除 Linux 5.5’s Scheduler Sees A Load Balancing Rework For Better Perf But Risks Regressions ✅ When testing on a dual quad-core ARM64 system they found the performance ranged from less than 1% to upwards of 10% for the Hackbench scheduler test. With a 224-core ARM64 server, the performance ranged from less than 1% improvements to 12% better performance with Hackbench and up to 33% better performance with Dbench. More numbers and details via the v4 patch revision. ","date":"2024-03-03","objectID":"/posts/linux-dev-review/:3:0","tags":["Sysprog","Linux"],"title":"Linux 核心设计: 发展动态回顾","uri":"/posts/linux-dev-review/"},{"categories":["Linux","Linux Kernel Internals"],"content":"虚拟化 Cloud Hypervisor Xen and the Art of Virtualization ","date":"2024-03-03","objectID":"/posts/linux-dev-review/:4:0","tags":["Sysprog","Linux"],"title":"Linux 核心设计: 发展动态回顾","uri":"/posts/linux-dev-review/"},{"categories":["Linux","Linux Kernel Internals"],"content":"DPDK (Data Plane Development Kit) 一言以蔽之: Kernel-bypass networking,即略过 kernel 直接让 User programs 处理网络封包,以提升效能。一般实作于高频交易的场景。 YouTube: Kernel-bypass networking for fun and profit Stack Overflow“zero copy networking” vs “kernel bypass”? ","date":"2024-03-03","objectID":"/posts/linux-dev-review/:5:0","tags":["Sysprog","Linux"],"title":"Linux 核心设计: 发展动态回顾","uri":"/posts/linux-dev-review/"},{"categories":["Linux","Linux Kernel Internals"],"content":"XDP: eXpress Data Path 常和 eBPF 配合实现在 kernel 进行定制化的封包过滤,从而减少 cop to/from kernel/user 这类操作的效能损失。 LPC2018 - Path to DPDK speeds for AF XDP / slides ","date":"2024-03-03","objectID":"/posts/linux-dev-review/:6:0","tags":["Sysprog","Linux"],"title":"Linux 核心设计: 发展动态回顾","uri":"/posts/linux-dev-review/"},{"categories":["Linux","Linux Kernel Internals"],"content":"AIO Synchronous / Asynchronous I/O:在從/向核心空間讀取/寫入資料 (i.e. 實際進行 I/O 操作) 的過程,使用者層級的行程是否會被 blocked。 AIO 在某些情景下处理不当,性能甚至低于 blocked 的 I/O 方法,这也引导出了 io_uring 技巧 UNIX 哲学: Everything is a file. Linux 不成文规范: Everything is a file descriptor. Kernel Recipes 2019 - Faster IO through io_uring / slides io_uring ","date":"2024-03-03","objectID":"/posts/linux-dev-review/:7:0","tags":["Sysprog","Linux"],"title":"Linux 核心设计: 发展动态回顾","uri":"/posts/linux-dev-review/"},{"categories":["Linux","Linux Kernel Internals"],"content":"Container Container 构建在 Linux 核心的基础建设上: namespace, cgroups, capabilities, seccomp +----------------------+ | +------------------+ | | | cgroup | | | | namespace | | | | union-capable fs | | | | | | | | Container | | | +------------------+ | | | | +------------------+ | | | Container | | | +------------------+ | | | | +------------------+ | | | Container | | | +------------------+ | | | | Linux kernel (host) | +----------------------+ YouTube: Containers: cgroups, Linux kernel namespaces, ufs, Docker, and intro to Kubernetes pods Stack Overflow: difference between cgroups and namespaces cgroup: Control Groups provide a mechanism for aggregating/partitioning sets of tasks, and all their future children, into hierarchical groups with specialized behaviour. namespace: wraps a global system resource in an abstraction that makes it appear to the processes within the namespace that they have their own isolated instance of the global resource. Wikipedia: UnionFS Wikipedia: Microservices ","date":"2024-03-03","objectID":"/posts/linux-dev-review/:8:0","tags":["Sysprog","Linux"],"title":"Linux 核心设计: 发展动态回顾","uri":"/posts/linux-dev-review/"},{"categories":["Linux","Linux Kernel Internals"],"content":"BPF/cBPF/eBPF 技巧 run small programs in kernel mode 20 years ago, this idea would likely have been shot down immediately Netflix talks about Extended BPF - A new software type / slides ","date":"2024-03-03","objectID":"/posts/linux-dev-review/:9:0","tags":["Sysprog","Linux"],"title":"Linux 核心设计: 发展动态回顾","uri":"/posts/linux-dev-review/"},{"categories":["Linux","Linux Kernel Internals"],"content":"Real-Time Linux 核心设计: PREEMPT_RT 作为迈向硬即时操作系统的机制 ","date":"2024-03-03","objectID":"/posts/linux-dev-review/:10:0","tags":["Sysprog","Linux"],"title":"Linux 核心设计: 发展动态回顾","uri":"/posts/linux-dev-review/"},{"categories":["Linux","Linux Kernel Internals"],"content":"printk Why printk() is so complicated (and how to fix it) ","date":"2024-03-03","objectID":"/posts/linux-dev-review/:11:0","tags":["Sysprog","Linux"],"title":"Linux 核心设计: 发展动态回顾","uri":"/posts/linux-dev-review/"},{"categories":["Linux","Linux Kernel Internals"],"content":"ZFS, BtrFS, RAID ZFS versus RAID: Eight Ironwolf disks, two filesystems, one winner ","date":"2024-03-03","objectID":"/posts/linux-dev-review/:12:0","tags":["Sysprog","Linux"],"title":"Linux 核心设计: 发展动态回顾","uri":"/posts/linux-dev-review/"},{"categories":["Linux","Linux Kernel Internals"],"content":"Rust Linux 核心采纳 Rust 的状况 ","date":"2024-03-03","objectID":"/posts/linux-dev-review/:13:0","tags":["Sysprog","Linux"],"title":"Linux 核心设计: 发展动态回顾","uri":"/posts/linux-dev-review/"},{"categories":["RISC-V"],"content":" The intention is to give specific actionable optimization recommendations for software developers writing code for RISC-V application processors. 近日 RISE 基金会发布了一版 《RISC-V Optimization Guide》,其目的是为给 RISC-V 应用处理器编写代码的软件开发人员提供具体可行的优化建议。本次活动的主要内容是解读和讨论该文档内容。 原文地址 原文 PDF 解说录影 ","date":"2024-02-29","objectID":"/posts/riscv-optimization-guide/:0:0","tags":["RISC-V","Optimization","Architecture"],"title":"RISC-V Optimization Guide 重点提示","uri":"/posts/riscv-optimization-guide/"},{"categories":["RISC-V"],"content":"相关知识 RISC-V ISA 规格书: https://riscv.org/technical/specifications/ 推荐参考 体系结构如何作用于编译器后端-邱吉 [bilibili] 这个讲座是关于微架构、指令集是怎样和编译器、软件相互协作、相互影响的 Overview 这个讲座介绍的是通用 CPU 并不仅限于 RISC-V 上 ","date":"2024-02-29","objectID":"/posts/riscv-optimization-guide/:1:0","tags":["RISC-V","Optimization","Architecture"],"title":"RISC-V Optimization Guide 重点提示","uri":"/posts/riscv-optimization-guide/"},{"categories":["RISC-V"],"content":"Detecting RISC-V Extensions on Linux 参考以下文章构建 Linux RISC-V 然后进行原文的 riscv_hwprobe 系统调用实验: How To Set Up The Environment for RISCV-64 Linux Kernel Development In Ubuntu 20.04 Running 64- and 32-bit RISC-V Linux on QEMU ","date":"2024-02-29","objectID":"/posts/riscv-optimization-guide/:2:0","tags":["RISC-V","Optimization","Architecture"],"title":"RISC-V Optimization Guide 重点提示","uri":"/posts/riscv-optimization-guide/"},{"categories":["RISC-V"],"content":"Multi-versioning 最新进展: https://reviews.llvm.org/D151730 相关介绍: https://maskray.me/blog/2023-02-05-function-multi-versioning ","date":"2024-02-29","objectID":"/posts/riscv-optimization-guide/:2:1","tags":["RISC-V","Optimization","Architecture"],"title":"RISC-V Optimization Guide 重点提示","uri":"/posts/riscv-optimization-guide/"},{"categories":["RISC-V"],"content":"Optimizing Scalar Integer ","date":"2024-02-29","objectID":"/posts/riscv-optimization-guide/:3:0","tags":["RISC-V","Optimization","Architecture"],"title":"RISC-V Optimization Guide 重点提示","uri":"/posts/riscv-optimization-guide/"},{"categories":["RISC-V"],"content":"Materializing Constants RV64I 5.2 Integer Computational Instructions Additional instruction variants are provided to manipulate 32-bit values in RV64I, indicated by a ‘W’ suffix to the opcode. These “*W” instructions ignore the upper 32 bits of their inputs and always produce 32-bit signed values, i.e. bits XLEN-1 through 31 are equal. ADDIW is an RV64I instruction that adds the sign-extended 12-bit immediate to register rs1 and produces the proper sign-extension of a 32-bit result in rd. 原文 Prefer idiomatic LUI/ADDI sequence for 32 bit constants 部分使用 lui 和 addiw 构建 0x1fffff 的说明比较晦涩难懂 (说实话我没看懂原文的 addiw 为什么需要减去 4096 😇) 注意 根据下面的参考文章,如果 addiw 的立即数的 MSB 被置为 1 时,只需在 lui 时多加一个 1 即可构建我们想要的 32-bit 数值。而原文中除了对 lui 加 1 外,还对 addiw 进行减去 4096 的操作: addiw a0, a0, (0xfff - 4096) ; addiw a0, a0, -1 这乍一看不知道为何需要减去 4096,其实本质很简单,根据上面的 ISA manual addiw 的立即数是 12-bit 的 signed number,即应该传入的是数值。但是直接使用 0xfff 表示传入的仅仅是 0xfff 这个编码对应的数值 (可以表示 12-bit signed 下的数值 -1,也可以表示 unsigned 编码下 0xfff 对应的数值 4095,在 12-bit signed 下 integer overflow),为了保证 addiw 的立即数的数值符合我们的预期 (即 0xfff 在 12-bit signed 下数值是 -1) 以及避免 integer overflow,所以需要将 0xfff - 4096 得到 12-bit signed 数值 -1 (虽然这个编码和 0xfff 是一样的…)。 addiw a0, a0, -1 ; right addiw a0, a0, 4095 ; integer overflow 解读计算机编码 C 语言: 数值系统篇 RV32G 下 lui/auipc 和 addi 结合加载立即数时的补值问题 [zhihu] RISC-V build 32-bit constants with LUI and ADDI [Stack Overflow] 原文 Fold immediates into consuming instructions where possible 部分,相关的 RISC-V 的 imm 优化: Craig Topper: 2022 LLVM Dev Mtg: RISC-V Sign Extension Optimizations 改进RISC-V的代码生成-廖春玉 [bilibili] ","date":"2024-02-29","objectID":"/posts/riscv-optimization-guide/:3:1","tags":["RISC-V","Optimization","Architecture"],"title":"RISC-V Optimization Guide 重点提示","uri":"/posts/riscv-optimization-guide/"},{"categories":["RISC-V"],"content":"Avoid branches using conditional moves Zicond extension 提供了我们在 RISC-V 上实作常数时间函数 (contant-time function) 的能力,用于避免分支预测,从而减少因分支预测失败带来的高昂代价。 $$ a0 = \\begin{cases} constant1 \u0026 \\text{if } x \\neq 0 \\newline constant2 \u0026 \\text{if } x = 0 \\end{cases} $$ 原文使用了 CZERO.NEZ,下面我们使用 CZERO.EQZ 来实作原文的例子: li t2, constant2 li t3, (constant1 - constant2) CZERO.EQZ t3, t3, a0 add a0, t3, t2 原文也介绍了如何使用 seqz 来实作 constant-time function,下面使用 snez 来实作原文的例子: li t2, constant1 li t3, constant2 snez t0, a0 addi t0, t0, -1 xor t1, t2, t3 and t1, t1, t0 xor a0, t1, t2 如果有 \\‘M\\’ 扩展可以通过 mul 指令进行简化 (通过 snez 来实作原文例子): li t2, constant1 li t3, constant2 xor t1, t2, t3 snez t0, a0 mul t1, t1, t0 xor a0, t1, t3 ","date":"2024-02-29","objectID":"/posts/riscv-optimization-guide/:3:2","tags":["RISC-V","Optimization","Architecture"],"title":"RISC-V Optimization Guide 重点提示","uri":"/posts/riscv-optimization-guide/"},{"categories":["RISC-V"],"content":"Padding Use canonical NOPs, NOP ( ADDI X0, X0, 0 ) and C.NOP ( C.ADDI X0, 0 ), to add padding within a function. Use the canonical illegal instruction ( either 2 or 4 bytes of zeros depending on whether the C extension is supported ) to add padding between functions. 因为在函数内部的执行频率高,使用合法的 NOPs 进行对齐 padding,防止在乱序执行时,流水线在遇见非法指令后就不再执行后续指令,造成效能损失 如果控制流被传递到两个函数之间,那么加大可能是程序执行出错了,使用非法的指令进行对齐 padding 可以帮助我们更好更快地 debug ","date":"2024-02-29","objectID":"/posts/riscv-optimization-guide/:3:3","tags":["RISC-V","Optimization","Architecture"],"title":"RISC-V Optimization Guide 重点提示","uri":"/posts/riscv-optimization-guide/"},{"categories":["RISC-V"],"content":"Align char array to greater alignment Why use wider load/store usage for memory copy? C 语言: 内存管理、对齐及硬体特性 ","date":"2024-02-29","objectID":"/posts/riscv-optimization-guide/:3:4","tags":["RISC-V","Optimization","Architecture"],"title":"RISC-V Optimization Guide 重点提示","uri":"/posts/riscv-optimization-guide/"},{"categories":["RISC-V"],"content":"Use shifts to clear leading/trailing bits 实作 64-bit 版本的原文例子 (retain the highest 12 bits): slli x6, x5, 52 slri x7, x5, 52 RV64I 5.2 Integer Computational Instructions LUI (load upper immediate) uses the same opcode as RV32I. LUI places the 20-bit U-immediate into bits 31–12 of register rd and places zero in the lowest 12 bits. The 32-bit result is sign-extended to 64 bits. ","date":"2024-02-29","objectID":"/posts/riscv-optimization-guide/:3:5","tags":["RISC-V","Optimization","Architecture"],"title":"RISC-V Optimization Guide 重点提示","uri":"/posts/riscv-optimization-guide/"},{"categories":["RISC-V"],"content":"Optimizing Scalar Floating Point ","date":"2024-02-29","objectID":"/posts/riscv-optimization-guide/:4:0","tags":["RISC-V","Optimization","Architecture"],"title":"RISC-V Optimization Guide 重点提示","uri":"/posts/riscv-optimization-guide/"},{"categories":["RISC-V"],"content":"Optimizing Vector What about vector instructions? YouTube: Introduction to SIMD Introduction to the RISC-V Vector Extension [PDF] 2020 RISC-V Summit: Tutorial: RISC-V Vector Extension Demystified ","date":"2024-02-29","objectID":"/posts/riscv-optimization-guide/:5:0","tags":["RISC-V","Optimization","Architecture"],"title":"RISC-V Optimization Guide 重点提示","uri":"/posts/riscv-optimization-guide/"},{"categories":["Rust"],"content":" In this (fifth) Crust of Rust video, we cover multi-produce/single-consumer (mpsc) channels, by re-implementing some of the std::sync::mpsc types from the standard library. As part of that, we cover what channels are used for, how they work at a high level, different common channel variants, and common channel implementations. In the process, we go over some common Rust concurrency primitives like Mutex and Condvar. 整理自 John Gjengset 的影片 ","date":"2024-02-29","objectID":"/posts/channels/:0:0","tags":["Rust","Channel"],"title":"Crust of Rust: Channels","uri":"/posts/channels/"},{"categories":["Rust"],"content":"Channel Wikipedia: Channel 引用 In computing, a channel is a model for interprocess communication and synchronization via message passing. A message may be sent over a channel, and another process or thread is able to receive messages sent over a channel it has a reference to, as a stream. YouTube: Channels in Rust Source ","date":"2024-02-29","objectID":"/posts/channels/:1:0","tags":["Rust","Channel"],"title":"Crust of Rust: Channels","uri":"/posts/channels/"},{"categories":["Rust"],"content":"Concurrency vs Parallelism What is the difference between concurrency and parallelism? Concurrency vs. Parallelism — A brief view ","date":"2024-02-29","objectID":"/posts/channels/:1:1","tags":["Rust","Channel"],"title":"Crust of Rust: Channels","uri":"/posts/channels/"},{"categories":["Rust"],"content":"影片注解 ","date":"2024-02-29","objectID":"/posts/channels/:2:0","tags":["Rust","Channel"],"title":"Crust of Rust: Channels","uri":"/posts/channels/"},{"categories":["Rust"],"content":"Sender \u0026 Receiver multi-produce/single-consumer (mpsc) Why does the recevier type need to have an arc protected by mutex if the channel may only have a single consumer thread? Because a send and a recevie might happen at the same time, and they need to be mutually exclusive to each other as well. Why not use a boolean semaphore over the implementation in mutex? A boolean semaphore is basically a boolean flag that you check and atomically update. The problem there is if the flag is currently set (someone else is in the critical section), with a boolean semaphore, you have to spin, you have to repeatedly check it. Whereas with a mutex, the operating system can put the thread to sleep and wake it back up when the mutex is available, which is generally more efficient although adds a little bit of latency. ","date":"2024-02-29","objectID":"/posts/channels/:2:1","tags":["Rust","Channel"],"title":"Crust of Rust: Channels","uri":"/posts/channels/"},{"categories":["Rust"],"content":"Condition Variable method std::sync::Condvar::wait This function will atomically unlock the mutex specified (represented by guard) and block the current thread. This means that any calls to notify_one or notify_all which happen logically after the mutex is unlocked are candidates to wake this thread up. When this function call returns, the lock specified will have been re-acquired. method std::sync::Condvar::notify_one If there is a blocked thread on this condition variable, then it will be woken up from its call to wait or wait_timeout. Calls to notify_one are not buffered in any way. wait \u0026 notify ","date":"2024-02-29","objectID":"/posts/channels/:2:2","tags":["Rust","Channel"],"title":"Crust of Rust: Channels","uri":"/posts/channels/"},{"categories":["Rust"],"content":"Clone 对 struct Sender\u003cT\u003e 标注属性宏 #[derive(clone)] 会实现以下的 triat: impl\u003cT: Clone\u003e Clone for Sender\u003cT\u003e { ... } 但是对于 Sender\u003cT\u003e 的成员 Arc\u003cInner\u003cT\u003e\u003e 来说,Arc 可以 clone 无论内部类型 T 是否实现了 Clone 这个 trait,所以我们需要手动实现 Clone 这个 trait。这也是 #[derive(clone)] 和手动实现 impl Clone 的一个细小差别。 impl\u003cT\u003e Clone for Sender\u003cT\u003e { ... } 为了防止调用 clone 产生的二义性 (因为编译器会自动解引用),建议使用 explict 方式来调用 Arc::clone(),这样编译器就会知道调用的是 Arc 的 clone 方法,而不是 Arc 内部 object 的 clone 方法。 let inner = Arc\u003cInner\u003cT\u003e\u003e; inner.clone(); // Inner\u003cT\u003e's clone method? or Arc::clone method? Arc::clone(\u0026inner); // explict Arc::clone ! ","date":"2024-02-29","objectID":"/posts/channels/:2:3","tags":["Rust","Channel"],"title":"Crust of Rust: Channels","uri":"/posts/channels/"},{"categories":["Rust"],"content":"dbg Macro std::dbg 引用 Prints and returns the value of a given expression for quick and dirty debugging. let a = 2; let b = dbg!(a * 2) + 1; // ^-- prints: [src/main.rs:2] a * 2 = 4 assert_eq!(b, 5); The macro works by using the Debug implementation of the type of the given expression to print the value to stderr along with the source location of the macro invocation as well as the source code of the expression. 调试的大杀器,作用类似于 kernel 中的 debugk 宏 🤣 常用于检测程序运行时是否执行了某些语句,以及这些语句的值如何。 ","date":"2024-02-29","objectID":"/posts/channels/:2:4","tags":["Rust","Channel"],"title":"Crust of Rust: Channels","uri":"/posts/channels/"},{"categories":["Rust"],"content":"Performance optimization Every operation takes the lock and that's fine if you have a channel that is not very high performance, but if you wanted like super high performance, like you have a lot of sends that compete with each other, then you might not want the sends to contend with one another. Image that you have 10 threads are trying to send at the same time, you could perhaps write an implementation that allows them to do that. The only thing that really needs to be synchronized is the senders with the receivers, as opposed to the senders with one another, whereas we're actually locking all of them. 使用 VecDeque 作为缓冲区,会导致 send 时的效能问题。因为 send 是使用 push_back 方法来将 object 加入到 VecDeque 中,这个过程 VecDeque 可能会发生 resize 操作,这会花费较长时间并且在这个过程时 sender 仍然持有 Mutex,所以导致其他 sender 和 recevier 并不能使用 VecDeque,所以在实作中并不使用 VecDeque 以避免相应的效能损失。 因为只有一个 receiver,所以可以通过缓冲区来提高效能,一次性接受大批数据并进行缓存,而不是每次只接收一个数据就放弃 Mutex (Batch recv optimization)。当然这个如果使用 VecDeque 依然会在 recv 时出现上面的 resize 效能问题。 ","date":"2024-02-29","objectID":"/posts/channels/:2:5","tags":["Rust","Channel"],"title":"Crust of Rust: Channels","uri":"/posts/channels/"},{"categories":["Rust"],"content":"Synchronous channels Module std::sync::mpsc These channels come in two flavors: An asynchronous, infinitely buffered channel. The channel function will return a (Sender, Receiver) tuple where all sends will be asynchronous (they never block). The channel conceptually has an infinite buffer. A synchronous, bounded channel. The sync_channel function will return a (SyncSender, Receiver) tuple where the storage for pending messages is a pre-allocated buffer of a fixed size. All sends will be synchronous by blocking until there is buffer space available. Note that a bound of 0 is allowed, causing the channel to become a “rendezvous” channel where each sender atomically hands off a message to a receiver. ","date":"2024-02-29","objectID":"/posts/channels/:2:6","tags":["Rust","Channel"],"title":"Crust of Rust: Channels","uri":"/posts/channels/"},{"categories":["Rust"],"content":"Channel flavors Synchronous channels: Channel where send() can block. Limited capacity. Mutex + Condvar + VecDeque Atomic VecDeque (atomic queue) + thread::park + thread::Thread::notify Asynchronous channels: Channel where send() cannot block. Unbounded. Mutex + Condvar + VecDeque Mutex + Condvar + LinkedList Atomic linked list, linked list of T Atomic block linked list, linked list of atomic VecDeque Rendezvous channels: Synchronous with capacity = 0. Used for thread synchronization. Oneshot channels: Any capacity. In practice, only one call to send(). ","date":"2024-02-29","objectID":"/posts/channels/:2:7","tags":["Rust","Channel"],"title":"Crust of Rust: Channels","uri":"/posts/channels/"},{"categories":["Rust"],"content":"async/await Module std::future Keyword async Keyword await ","date":"2024-02-29","objectID":"/posts/channels/:2:8","tags":["Rust","Channel"],"title":"Crust of Rust: Channels","uri":"/posts/channels/"},{"categories":["Rust"],"content":"Homework 信息 实作说明: 尝试实现 Synchronous channels 使用 Atomic 存储 senders 以提高效能 使用两个 ConVar 来指示 sender 和 receiver 进行 block 和 wake up receiver 被 drop 时需要通知所有 senders 以释放资源 使用 linked list 来取代 VecDeque 以避免 resize 的效能损失 尝试阅读 std 中 mpsc 的实现 Module std::sync::mpsc 对比阅读其他库关于 channel 的实现: crossbeam, flume 参考资料: Module std::sync::atomic Module std::sync::mpsc Crate crossbeam Crate flume ","date":"2024-02-29","objectID":"/posts/channels/:3:0","tags":["Rust","Channel"],"title":"Crust of Rust: Channels","uri":"/posts/channels/"},{"categories":["Rust"],"content":"Documentations 这里列举视频中一些概念相关的 documentation 学习的一手资料是官方文档,请务必自主学会阅读规格书之类的资料 ","date":"2024-02-29","objectID":"/posts/channels/:4:0","tags":["Rust","Channel"],"title":"Crust of Rust: Channels","uri":"/posts/channels/"},{"categories":["Rust"],"content":"Crate std 可以使用这里提供的搜素栏进行搜索 (BTW 不要浪费时间在 Google 搜寻上!) Module std::sync::mpsc Function std::sync::mpsc::channel Struct std::sync::mpsc::Sender Struct std::sync::mpsc::Receiver Module std::sync Struct std::sync::Arc Struct std::sync::Mutex Struct std::sync::Condvar method std::sync::Condvar::wait method std::sync::Condvar::notify_one Module std::sync::atomic Trait std::marker::Send Struct std::collections::VecDeque Function std::mem::take Function std::mem::swap Macro std::dbg ","date":"2024-02-29","objectID":"/posts/channels/:4:1","tags":["Rust","Channel"],"title":"Crust of Rust: Channels","uri":"/posts/channels/"},{"categories":["Rust"],"content":"References Go 语言也有 channel: 解说 Go channel 底层原理 [bilibili] 可能不是你看过最无聊的 Rust 入门喜剧 102 (3) 多线程并发 [bilibili] ","date":"2024-02-29","objectID":"/posts/channels/:5:0","tags":["Rust","Channel"],"title":"Crust of Rust: Channels","uri":"/posts/channels/"},{"categories":["C","Linux Kernel Internals"],"content":" Linux 核心作为世界上最成功的开放原始码计划,也是 C 语言在工程领域的瑰宝,里头充斥则各种“艺术”,往往会吓到初次接触的人们,但总是能够使用 C 语言标准和开发工具提供的扩展 (主要是来自 gcc 的 GNU extensions) 来解释。 工欲善其事,必先利其器 原文地址 If I had eight hours to chop down a tree, I’d spend six hours sharpening my axe. —— Abraham Lincoln 语言规格: C89/C90 -\u003e C99 -\u003e C11 -\u003e C17/C18 -\u003e C2x ","date":"2024-02-28","objectID":"/posts/c-standards/:0:0","tags":["Sysprog","C","Standard"],"title":"你所不知道的 C 语言: 开发工具和规格标准","uri":"/posts/c-standards/"},{"categories":["C","Linux Kernel Internals"],"content":"C vs C++ C is quirky, flawed, and an enormous success. Although accidents of history surely helped, it evidently satisfied a need for a system implementation language efficient enough to displace assembly language, yet sufficiently abstract and fluent to describe algorithms and interactions in a wide variety of environments. —— Dennis M. Ritchie David Brailsford: Why C is so Influential - Computerphile Linus Torvalds: c++ in linux kernel And I really do dislike C++. It’s a really bad language, in my opinion. It tries to solve all the wrong problems, and does not tackle the right ones. The things C++ “solves” are trivial things, almost purely syntactic extensions to C rather than fixing some true deep problem. Bjarne Stroustrup: Learning Standard C++ as a New Language [PDF] C++ 标准更新飞快: C++11, C++14, C++17, … 从 C99, C++98 开始,C 语言和 C++ 分道扬镳 in C, everything is a representation (unsigned char [sizeof(TYPE)]). —— Rich Rogers 第一個 C 語言編譯器是怎樣編寫的? 介绍了自举 (sel-hosting/compiling) 以及 C0, C1, C2, C3, … 等的演化过程 ","date":"2024-02-28","objectID":"/posts/c-standards/:1:0","tags":["Sysprog","C","Standard"],"title":"你所不知道的 C 语言: 开发工具和规格标准","uri":"/posts/c-standards/"},{"categories":["C","Linux Kernel Internals"],"content":"C 语言规格书 ","date":"2024-02-28","objectID":"/posts/c-standards/:2:0","tags":["Sysprog","C","Standard"],"title":"你所不知道的 C 语言: 开发工具和规格标准","uri":"/posts/c-standards/"},{"categories":["C","Linux Kernel Internals"],"content":"main 阅读 C 语言规格书可以让你洞察本质,不在没意义的事情上浪费时间,例如在某乎大肆讨论的 void main() 和 int main() 问题 🤣 C99/C11 5.1.2.2.1 Program startup The function called at program startup is named main. The implementation declares no prototype for this function. It shall be defined with a return type of int and with no parameters: int main(void) { /* ... */ } or with two parameters (referred to here as argc and argv, though any names may be used, as they are local to the function in which they are declared): int main(int argc, char *argv[]) { /* ... */ } or equivalent; or in some other implementation-defined manner. Thus, int can be replaced by a typedef name defined as int, or the type of argv can be written as char ** argv, and so on. ","date":"2024-02-28","objectID":"/posts/c-standards/:2:1","tags":["Sysprog","C","Standard"],"title":"你所不知道的 C 语言: 开发工具和规格标准","uri":"/posts/c-standards/"},{"categories":["C","Linux Kernel Internals"],"content":"incomplete type C99 6.2.5 Types incomplete types (types that describe objects but lack information needed to determine their sizes). ","date":"2024-02-28","objectID":"/posts/c-standards/:2:2","tags":["Sysprog","C","Standard"],"title":"你所不知道的 C 语言: 开发工具和规格标准","uri":"/posts/c-standards/"},{"categories":["C","Linux Kernel Internals"],"content":"规格不仅要看最新的,过往的也要熟悉 因为很多 (嵌入式) 设备上运行的 Linux 可能是很旧的版本,那时 Linux 使用的是更旧的 C 语言规格。例如空中巴士 330 客机的娱乐系统里执行的是十几年前的 Red Hat Linux,总有人要为这些“古董”负责 🤣 ","date":"2024-02-28","objectID":"/posts/c-standards/:2:3","tags":["Sysprog","C","Standard"],"title":"你所不知道的 C 语言: 开发工具和规格标准","uri":"/posts/c-standards/"},{"categories":["C","Linux Kernel Internals"],"content":"GDB 使用 GDB 这类调试工具可以大幅度提升我们编写代码、除错的能力 🐶 video: Linux basic anti-debug video: C Programming, Disassembly, Debugging, Linux, GDB rr (Record and Replay Framework) video: Quick demo video: Record and replay debugging with “rr” ","date":"2024-02-28","objectID":"/posts/c-standards/:3:0","tags":["Sysprog","C","Standard"],"title":"你所不知道的 C 语言: 开发工具和规格标准","uri":"/posts/c-standards/"},{"categories":["C","Linux Kernel Internals"],"content":"C23 上一个 C 语言标准是 C17,正式名称为 ISO/IEC 9899:2018,是 2017 年准备,2018年正式发布的标准规范。C23 则是目前正在开发的规格,其预计新增特性如下: typeof: 由 GNU extension 转正,用于实作 container_of 宏 call_once: 保证在 concurrent 环境中,某段程式码只会执行 1 次 char8_t: Unicode friendly u8\"💣\"[0] unreachable(): 由 GNU extension 转正,提示允许编译器对某段程式码进行更激进的最佳化 = {}: 取代 memset 函数调用 ISO/IEC 60559:2020: 最新的 IEEE 754 浮点数运算标准 _Static_assert: 扩充 C11 允许单一参数 吸收 C++11 风格的 attribute 语法,例如 nodiscard, maybe_unused, deprecated, fallthrough 新的函数: memccpy(), strdup(), strndup() ——— 类似于 POSIX、SVID中 C 函数库的扩充 强制规范使用二补数表示整数 不支援 K\u0026R 风格的函数定义 二进制表示法: 0b10101010 以及对应 printf() 的 %b (在此之前 C 语言是不支援二进制表示法的 🤣) Type generic functions for performing checked integer arithmetic (Integer overflow) _BitInt(N) and UnsignedBitInt(N) types for bit-precise integers #elifdef and #elifndef 支持在数值中间加入分隔符,易于阅读,例如 0xFFFF'FFFF 信息 Ever Closer - C23 Draws Nearer C23 is Finished: Here is What is on the Menu ","date":"2024-02-28","objectID":"/posts/c-standards/:4:0","tags":["Sysprog","C","Standard"],"title":"你所不知道的 C 语言: 开发工具和规格标准","uri":"/posts/c-standards/"},{"categories":["C","Linux Kernel Internals"],"content":" 不少 C/C++ 开发者听过 “内存对齐” (memory alignment),但不易掌握概念及规则,遑论其在执行时期的冲击。内存管理像是 malloc/free 函数的使用,是每个 C 语言程序设计开发者都会接触到,但却难保充分排除错误的难题。本讲座尝试从硬体的行为开始探讨,希望消除观众对于 alignment, padding, memory allocator 的误解,并且探讨高效能 memory pool 的设计,如何改善整体程序的效能和可靠度。也会探讨 C11 标准的 aligned_alloc。 原文地址 ","date":"2024-02-27","objectID":"/posts/c-memory/:0:0","tags":["Sysprog","C","Memory"],"title":"你所不知道的 C 语言: 记忆体管理、对齐及硬体特性","uri":"/posts/c-memory/"},{"categories":["C","Linux Kernel Internals"],"content":"背景知识 ","date":"2024-02-27","objectID":"/posts/c-memory/:1:0","tags":["Sysprog","C","Memory"],"title":"你所不知道的 C 语言: 记忆体管理、对齐及硬体特性","uri":"/posts/c-memory/"},{"categories":["C","Linux Kernel Internals"],"content":"你所不知道的 C 语言: 指针篇 C99/C11 6.2.5 Types (28) A pointer to void shall have the same representation and alignment requirements as a pointer to a character type. C99/C11 6.3.2.3 Pointers (1) A pointer to void may be converted to or from a pointer to any object type. A pointer to any object type may be converted to a pointer to void and back again; the result shall compare equal to the original pointer. 使用 void * 必须通过 explict (显式) 或强制转型,才能存取最终的 object,因为 void 无法判断 object 的大小信息。 ","date":"2024-02-27","objectID":"/posts/c-memory/:1:1","tags":["Sysprog","C","Memory"],"title":"你所不知道的 C 语言: 记忆体管理、对齐及硬体特性","uri":"/posts/c-memory/"},{"categories":["C","Linux Kernel Internals"],"content":"你所不知道的 C 语言: 函数呼叫篇 glibc 提供了 malloc_stats() 和 malloc_info() 这两个函数,可以查询 process 的 heap 空间使用情况信息。 ","date":"2024-02-27","objectID":"/posts/c-memory/:1:2","tags":["Sysprog","C","Memory"],"title":"你所不知道的 C 语言: 记忆体管理、对齐及硬体特性","uri":"/posts/c-memory/"},{"categories":["C","Linux Kernel Internals"],"content":"Memory 金字塔 这个金字塔的层级图提示我们,善用 Cache locality 可以有效提高程式效能。 技巧 What a C programmer should know about memory (简记) ","date":"2024-02-27","objectID":"/posts/c-memory/:2:0","tags":["Sysprog","C","Memory"],"title":"你所不知道的 C 语言: 记忆体管理、对齐及硬体特性","uri":"/posts/c-memory/"},{"categories":["C","Linux Kernel Internals"],"content":"Understanding virtual memory - the plot thickens The virtual memory allocator (VMA) may give you a memory it doesn’t have, all in a vain hope that you’re not going to use it. Just like banks today 虚拟内存的管理类似于银行,返回的分配空间未必可以立即使用。memory allocator 和银行类似,可用空间就类似于银行的现金储备金,银行可以开很多支票,但是这些支票可以兑现的前提是这些支票不会在同一时间来兑现,虚拟内存管理也类似,分配空间也期望用户不会立即全部使用。 ","date":"2024-02-27","objectID":"/posts/c-memory/:2:1","tags":["Sysprog","C","Memory"],"title":"你所不知道的 C 语言: 记忆体管理、对齐及硬体特性","uri":"/posts/c-memory/"},{"categories":["C","Linux Kernel Internals"],"content":"Understanding stack allocation This is how variable-length arrays (VLA), and also alloca() work, with one difference - VLA validity is limited by the scope, alloca’d memory persists until the current function returns (or unwinds if you’re feeling sophisticated). VLA 和 alloca 分配的都是栈 (stack) 空间,只需将栈指针 (sp) 按需求加减一下即可实现空间分配。因为 stack 空间是有限的,所以 Linux 核心中禁止使用 VLA,防止 Stack Overflow 🤣 ","date":"2024-02-27","objectID":"/posts/c-memory/:2:2","tags":["Sysprog","C","Memory"],"title":"你所不知道的 C 语言: 记忆体管理、对齐及硬体特性","uri":"/posts/c-memory/"},{"categories":["C","Linux Kernel Internals"],"content":"Slab allocator The principle of slab allocation was described by Bonwick for a kernel object cache, but it applies for the user-space as well. Oh-kay, we’re not interested in pinning slabs to CPUs, but back to the gist — you ask the allocator for a slab of memory, let’s say a whole page, and you cut it into many fixed-size pieces. Presuming each piece can hold at least a pointer or an integer, you can link them into a list, where the list head points to the first free element. 在使用 alloc 的内存空间时,这些空间很有可能是不连续的。所以此时对于系统就会存在一些问题,一个是内存空间碎片 fragment,因为分配的空间未必会全部使用到,另一个是因为不连续,所以无法利用 Cache locality 来提升效能。 ","date":"2024-02-27","objectID":"/posts/c-memory/:2:3","tags":["Sysprog","C","Memory"],"title":"你所不知道的 C 语言: 记忆体管理、对齐及硬体特性","uri":"/posts/c-memory/"},{"categories":["C","Linux Kernel Internals"],"content":"Demand paging explained Linux 系统会提供一些内存管理的 API 和机制: mlock() - lock/unlock memory 禁止某个区域的内存被 swapped out 到磁盘 (只是向 OS 建议,OS 可能不会理会) madvise() - give advice about use of memory (同样只是向 OS 建议,OS 可能不会理会) lazy loading - 利用缺页异常 (page-fault) 来实现 copy on write 信息 現代處理器設計: Cache 原理和實際影響 Cache 原理和實際影響: 進行 CPU caches 中文重點提示並且重現對應的實驗 針對多執行緒環境設計的 Memory allocator rpmalloc 探討 ","date":"2024-02-27","objectID":"/posts/c-memory/:2:4","tags":["Sysprog","C","Memory"],"title":"你所不知道的 C 语言: 记忆体管理、对齐及硬体特性","uri":"/posts/c-memory/"},{"categories":["C","Linux Kernel Internals"],"content":"堆 Heap Stack Overflow: Why are two different concepts both called “heap”? Several authors began about 1975 to call the pool of available memory a “heap.” ","date":"2024-02-27","objectID":"/posts/c-memory/:3:0","tags":["Sysprog","C","Memory"],"title":"你所不知道的 C 语言: 记忆体管理、对齐及硬体特性","uri":"/posts/c-memory/"},{"categories":["C","Linux Kernel Internals"],"content":"Data alignment 一个 data object 具有两个特性: value storage location (address) ","date":"2024-02-27","objectID":"/posts/c-memory/:4:0","tags":["Sysprog","C","Memory"],"title":"你所不知道的 C 语言: 记忆体管理、对齐及硬体特性","uri":"/posts/c-memory/"},{"categories":["C","Linux Kernel Internals"],"content":"alignment vs unalignment 假设硬体要求 4 Bytes alignment,CPU 存取数据时的操作如下: alignment unalignment Source 除此之外,unalignment 也可能会无法充分利用 cache 效能,即存取的数据一部分 cache hit,另一部分 cache miss。当然对于这种情况,cache 也是采用类似上面的 merge 机制来进行存取,只是效能低下。 GCC: 6.60.8 Structure-Packing Pragmas The n value below always is required to be a small power of two and specifies the new alignment in bytes. #pragma pack(push[,n]) pushes the current alignment setting on an internal stack and then optionally sets the new alignment. #pragma pack(pop) restores the alignment setting to the one saved at the top of the internal stack (and removes that stack entry). Note that #pragma pack([n]) does not influence this internal stack; thus it is possible to have #pragma pack(push) followed by multiple #pragma pack(n) instances and finalized by a single #pragma pack(pop). alignment 与 unalignment 的效能分布: ","date":"2024-02-27","objectID":"/posts/c-memory/:4:1","tags":["Sysprog","C","Memory"],"title":"你所不知道的 C 语言: 记忆体管理、对齐及硬体特性","uri":"/posts/c-memory/"},{"categories":["C","Linux Kernel Internals"],"content":"malloc malloc 分配的空间是 alignment 的: man malloc The malloc() and calloc() functions return a pointer to the allocated memory, which is suitably aligned for any built-in type. The GNU C Library - Malloc Example The block that malloc gives you is guaranteed to be aligned so that it can hold any type of data. On GNU systems, the address is always a multiple of eight on 32-bit systems, and a multiple of 16 on 64-bit systems. 使用 GDB 进行测试,确定在 Linux x86_64 上 malloc 分配的内存以 16 Bytes 对齐,即地址以 16 进制显示时最后一个数为 0。 ","date":"2024-02-27","objectID":"/posts/c-memory/:4:2","tags":["Sysprog","C","Memory"],"title":"你所不知道的 C 语言: 记忆体管理、对齐及硬体特性","uri":"/posts/c-memory/"},{"categories":["C","Linux Kernel Internals"],"content":"unalignment get \u0026 set 如上面所述,在 32-bit 架构上进行 8 bytes 对齐的存取效能比较高 (远比单纯访问一个 byte 高),所以原文利用这一特性实作了 unaligned_get8 这一函数。 csrc \u0026 0xfffffffc 向下取整到最近的 8 bytes alignment 的地址 v \u003e\u003e (((uint32_t) csrc \u0026 0x3) * 8) 将获取的 alignment 的 32-bit 进行位移以获取我们想要的那个字节 而在 你所不知道的 C 语言: 指针篇 中实作的 16-bit integer 在 unalignment 情况下的存取,并没有考虑到上面利用 alignment 来提升效能。 参考原文 32-bit integer 存取,实作 64-bit integer 的 get \u0026 set: ","date":"2024-02-27","objectID":"/posts/c-memory/:4:3","tags":["Sysprog","C","Memory"],"title":"你所不知道的 C 语言: 记忆体管理、对齐及硬体特性","uri":"/posts/c-memory/"},{"categories":["C","Linux Kernel Internals"],"content":"oncurrent-II 源码: concurrent-ll 论文: A Pragmatic Implementation of Non-Blocking Linked Lists ","date":"2024-02-27","objectID":"/posts/c-memory/:5:0","tags":["Sysprog","C","Memory"],"title":"你所不知道的 C 语言: 记忆体管理、对齐及硬体特性","uri":"/posts/c-memory/"},{"categories":["C","Linux Kernel Internals"],"content":" Linux 核心原始程式码存在大量 bit(-wise) operations (简称 bitops),颇多乍看像是魔法的 C 程式码就是 bitops 的组合。 原文地址 ","date":"2024-02-23","objectID":"/posts/c-bitwise/:0:0","tags":["Sysprog","C","Bitwise"],"title":"你所不知道的 C 语言: bitwise 操作","uri":"/posts/c-bitwise/"},{"categories":["C","Linux Kernel Internals"],"content":"复习数值系统 YouTube: 十进制,十二进制,六十进制从何而来?阿拉伯人成就了文艺复兴?[数学大师] 你所不知道的 C 语言: 数值系统 解读计算机编码 ","date":"2024-02-23","objectID":"/posts/c-bitwise/:1:0","tags":["Sysprog","C","Bitwise"],"title":"你所不知道的 C 语言: bitwise 操作","uri":"/posts/c-bitwise/"},{"categories":["C","Linux Kernel Internals"],"content":"位元组合 一些位元组合表示特定的意义,而不是表示数值,这些组合被称为 trap representation C11 6.2.6.2 Integer types For unsigned integer types other than unsigned char, the bits of the object representation shall be divided into two groups: value bits and padding bits (there need not be any of the latter). If there are N value bits, each bit shall represent a different power of 2 between 1 and 2N−1, so that objects of that type shall be capable of representing values from 0 to 2N−1 using a pure binary representation; this shall be known as the value representation. The values of any padding bits are unspecified. uintN_t 和 intN_t 保证没有填充位元 (padding bits),且 intN_t 是二补数编码,所以对这两种类型进行位操作是安全的。 C99 7.18.1.1 Exact-width integer types The typedef name intN_t designates a signed integer type with width N, no padding bits, and a two’s complement representation. 信息 有符号整数上也有可能产生陷阱表示法 (trap representation) 补充资讯: CS:APP Web Aside DATA:TMIN: Writing TMin in C ","date":"2024-02-23","objectID":"/posts/c-bitwise/:1:1","tags":["Sysprog","C","Bitwise"],"title":"你所不知道的 C 语言: bitwise 操作","uri":"/posts/c-bitwise/"},{"categories":["C","Linux Kernel Internals"],"content":"位移运算 位移运算的未定义情况: C99 6.5.7 Bitwise shift operators 左移超过变量长度,则运算结果未定义 If the value of the right operand is negative or is greater than or equal to the width of the promoted left operand, the behavior is undefined. 对一个负数进行右移,C 语言规格未定义,作为 implementation-defined,GCC 实作为算术位移 (arithmetic shift) If E1 has a signed type and a negative value, the resulting value is implementation-defined. ","date":"2024-02-23","objectID":"/posts/c-bitwise/:1:2","tags":["Sysprog","C","Bitwise"],"title":"你所不知道的 C 语言: bitwise 操作","uri":"/posts/c-bitwise/"},{"categories":["C","Linux Kernel Internals"],"content":"Signed \u0026 Unsigned 当 Unsigned 和 Signed 混合在同一表达式时,Signed 会被转换成 Unsigned,运算结果可能不符合我们的预期 (这里大赞 Rust,这种情况会编译失败🤣)。案例请参考原文,这里举一个比较常见的例子: int n = 10; for (int i = n - 1 ; i - sizeof(char) \u003e= 0; i--) printf(\"i: 0x%x\\n\",i); 这段程式码会导致无限循环,因为条件判断语句 i - sizeof(char) \u003e= 0 恒为真 (变量 i 被转换成 Unsigned 了)。 6.5.3.4 The sizeof operator The value of the result is implementation-defined, and its type (an unsigned integer type) is size_t, defined in \u003cstddef.h\u003e (and other headers). 7.17 Common definitions \u003cstddef.h\u003e size_t which is the unsigned integer type of the result of the sizeof operator ","date":"2024-02-23","objectID":"/posts/c-bitwise/:1:3","tags":["Sysprog","C","Bitwise"],"title":"你所不知道的 C 语言: bitwise 操作","uri":"/posts/c-bitwise/"},{"categories":["C","Linux Kernel Internals"],"content":"Sign Extension 将 w bit signed integer 扩展为 w+k bit signed integer,只需将 sign bit 补充至扩展的 bits。 数值等价性推导: positive: 显然是正确的,sign bit 为 0,扩展后数值仍等于原数值 negitive: 将 w bit 情形时的除开 sign bit 的数值设为 U,则原数值为 $2^{-(w-1)} + U$,则扩展为 w+k bit 后数值为 $2^{-(w+k-1)} + 2^{w+k-2} + … + 2^{-(w-1)} + U$,因为 $2^{-(w+k-1)} + 2^{w+k-2} + … + 2^{w-1} = 2^{-(w-1)}$,所以数值依然等价。 $2^{-(w+k-1)} + 2^{w+k-2} + … + 2^{w-1}$ 可以考虑从左往右的运算,每次都是将原先的数值减半,所以最后的数值为 $2^{-(w+k-1)}$ 所以如果 n 是 signed 32-bit,则 n \u003e\u003e 31 等价于 n == 0 ? 0 : -1。在这个的基础上,请重新阅读 解读计算机编码 中的 abs 和 min/max 的常数时间实作。 ","date":"2024-02-23","objectID":"/posts/c-bitwise/:1:4","tags":["Sysprog","C","Bitwise"],"title":"你所不知道的 C 语言: bitwise 操作","uri":"/posts/c-bitwise/"},{"categories":["C","Linux Kernel Internals"],"content":"Bitwise Operator Bitwise Operators Quiz Answers Practice with bit operations Bitwise Practice Each lowercase letter is 32 + uppercase equivalent. This means simply flipping the bit at position 5 (counting from least significant bit at position 0) inverts the case of a letter. The gdb print command (shortened p) defaults to decimal format. Use p/format to instead select other formats such as x for hex, t for binary, and c for char. // unsigned integer `mine`, `yours` remove yours from mine mine = mine \u0026 ~yours test if mine has both of two lowest bits on (mine \u0026 0x3) == 0x3 n least significant bits on, all others off (1 \u003c\u003c n) - 1 k most significant bits on, all others off (~0 \u003c\u003c (32 - k)) or ~(~0U \u003e\u003e k) // unsigned integer `x`, `y` (right-shift: arithmetic shift) x \u0026= (x - 1) clears lowest \"on\" bit in x (x ^ y) \u003c 0 true if x and y have opposite signs 程序语言只提供最小粒度为 Byte 的操作,但是不直接提供 Bit 粒度的操作,这与字节顺序相关。假设提供以 Bit 为粒度的操作,这就需要在编程时考虑 大端/小端模式,极其繁琐。 ","date":"2024-02-23","objectID":"/posts/c-bitwise/:2:0","tags":["Sysprog","C","Bitwise"],"title":"你所不知道的 C 语言: bitwise 操作","uri":"/posts/c-bitwise/"},{"categories":["C","Linux Kernel Internals"],"content":"bitwise \u0026 logical 位运算满足交换律,但逻辑运算并不满足交换律,因为短路机制。考虑 Linked list 中的情形: // list_head *head if (!head || list_empty(head)) if (list_empty(head) || !head) 第二条语句在执行时会报错,因为 list_empty 要求传入的参数不为 NULL。 逻辑运算符 ! 相当有效,C99 并没有完全支持 bool 类型,对于整数,它是将非零整数视为 true,零视为 false。所以如果你需要保证某一表达式的结果不仅是 true of false,还要求对应 0 or 1 时,可以使用 !!(expr) 来实现。 C99 6.5.3.3 Unary arithmetic operators The result of the logical negation operator ! is 0 if the value of its operand compares unequal to 0, 1 if the value of its operand compares equal to 0. The result has type int. The expression !E is equivalent to (0==E). 所以 !!(expr) 的结果为 int 并且数值只有 0 或 1。 ","date":"2024-02-23","objectID":"/posts/c-bitwise/:2:1","tags":["Sysprog","C","Bitwise"],"title":"你所不知道的 C 语言: bitwise 操作","uri":"/posts/c-bitwise/"},{"categories":["C","Linux Kernel Internals"],"content":"Right Shifts 对于 Unsigned 或 positive sign integer 做右移运算时 x \u003e\u003e n,其最终结果值为 $\\lfloor x / 2^n \\rfloor$。 因为这种情况的右移操作相当于对每个 bit 表示的 power 加上 $-n$,再考虑有些 bit 表示的 power 加上 $-n$ 后会小于 0,此时直接将这些 bit 所表示的值去除即可 (因为在 integer 中 bit 的 power 最小为 0,如果 power 小于 0 表示的是小数值),这个操作对应于向下取整。 00010111 \u003e\u003e 2 (23 \u003e\u003e 4) -\u003e 000101.11 (5.75) -\u003e 000101 (5) ","date":"2024-02-23","objectID":"/posts/c-bitwise/:2:2","tags":["Sysprog","C","Bitwise"],"title":"你所不知道的 C 语言: bitwise 操作","uri":"/posts/c-bitwise/"},{"categories":["C","Linux Kernel Internals"],"content":"bitwise 实作 Vi/Vim 为什么使用 hjkl 作为移动字符? 當我們回顧 1967 年 ASCII 的編碼規範,可發現前 32 個字元都是控制碼,讓人們得以透過這些特別字元來控制畫面和相關 I/O,早期鍵盤的 “control” 按鍵就搭配這些特別字元使用。“control” 組合按鍵會將原本字元的第 1 個 bit 進行 XOR,於是 H 字元對應 ASCII 編碼為 100_1000 (過去僅用 7 bit 編碼),組合 “control” 後 (即 Ctrl+H) 會得到 000_1000,也就是 backspace 的編碼,這也是為何在某些程式中按下 backspace 按鍵會得到 ^H 輸出的原因。相似地,當按下 Ctrl+J 時會得到 000_1010,即 linefeed 注意 where n is the bit number, and 0 is the least significant bit Source ","date":"2024-02-23","objectID":"/posts/c-bitwise/:3:0","tags":["Sysprog","C","Bitwise"],"title":"你所不知道的 C 语言: bitwise 操作","uri":"/posts/c-bitwise/"},{"categories":["C","Linux Kernel Internals"],"content":"Set a bit unsigned char a |= (1 \u003c\u003c n); ","date":"2024-02-23","objectID":"/posts/c-bitwise/:3:1","tags":["Sysprog","C","Bitwise"],"title":"你所不知道的 C 语言: bitwise 操作","uri":"/posts/c-bitwise/"},{"categories":["C","Linux Kernel Internals"],"content":"Clear a bit unsigned char a \u0026= ~(1 \u003c\u003c n); ","date":"2024-02-23","objectID":"/posts/c-bitwise/:3:2","tags":["Sysprog","C","Bitwise"],"title":"你所不知道的 C 语言: bitwise 操作","uri":"/posts/c-bitwise/"},{"categories":["C","Linux Kernel Internals"],"content":"Toggle a bit unsigned char a ^= (1 \u003c\u003c n); ","date":"2024-02-23","objectID":"/posts/c-bitwise/:3:3","tags":["Sysprog","C","Bitwise"],"title":"你所不知道的 C 语言: bitwise 操作","uri":"/posts/c-bitwise/"},{"categories":["C","Linux Kernel Internals"],"content":"Test a bit bool a = (val \u0026 (1 \u003c\u003c n)) \u003e 0; ","date":"2024-02-23","objectID":"/posts/c-bitwise/:3:4","tags":["Sysprog","C","Bitwise"],"title":"你所不知道的 C 语言: bitwise 操作","uri":"/posts/c-bitwise/"},{"categories":["C","Linux Kernel Internals"],"content":"The right/left most byte // assuming 16 bit, 2-byte short integer unsigned short right = val \u0026 0xff; /* right most (least significant) byte */ unsigned short left = (val \u003e\u003e 8) \u0026 0xff; /* left most (most significant) byte */ // assuming 32 bit, 4-byte int integer unsigned int right = val \u0026 0xff; /* right most (least significant) byte */ unsigned int left = (val \u003e\u003e 24) \u0026 0xff; /* left most (most significant) byte */ ","date":"2024-02-23","objectID":"/posts/c-bitwise/:3:5","tags":["Sysprog","C","Bitwise"],"title":"你所不知道的 C 语言: bitwise 操作","uri":"/posts/c-bitwise/"},{"categories":["C","Linux Kernel Internals"],"content":"Sign bit // assuming 16 bit, 2-byte short integer, two's complement bool sign = val \u0026 0x8000; // assuming 32 bit, 4-byte int integer, two's complement bool sign = val \u0026 0x80000000; ","date":"2024-02-23","objectID":"/posts/c-bitwise/:3:6","tags":["Sysprog","C","Bitwise"],"title":"你所不知道的 C 语言: bitwise 操作","uri":"/posts/c-bitwise/"},{"categories":["C","Linux Kernel Internals"],"content":"Uses of Bitwise Operations or Why to Study Bits Compression Set operations Encryption 最常见的就是位图 (bitmap),常用于文件系统 (file system),可以节省空间 (每个元素只用一个 bit 来表示),可以很方便的进行集合操作 (通过 bitwise operator)。 x ^ y = (~x \u0026 y) | (x \u0026 ~y) ","date":"2024-02-23","objectID":"/posts/c-bitwise/:3:7","tags":["Sysprog","C","Bitwise"],"title":"你所不知道的 C 语言: bitwise 操作","uri":"/posts/c-bitwise/"},{"categories":["C","Linux Kernel Internals"],"content":"影像处理 Stack Overflow: what (r+1 + (r » 8)) » 8 does? 在图形引擎中将除法运算 x / 255 用位运算 (x+1 + (x \u003e\u003e 8)) \u003e\u003e 8 来实作,可以大幅度提升计算效能。 ","date":"2024-02-23","objectID":"/posts/c-bitwise/:4:0","tags":["Sysprog","C","Bitwise"],"title":"你所不知道的 C 语言: bitwise 操作","uri":"/posts/c-bitwise/"},{"categories":["C","Linux Kernel Internals"],"content":"案例分析 实作程式码: RGBAtoBW 给定每个 pixel 为 32-bit 的 RGBA 的 bitmap,提升效能的方案: 建立表格加速浮点运算 减少位运算: 可以使用 pointer 的 offset 取代原本复杂的 bitwise operation bwPixel = table[rgbPixel \u0026 0x00ffffff] + rgbPixel \u0026 0xff000000; 只需对 RGB 部分建立浮点数表,因为 rgbPixel \u0026 0xff00000 获取的是 A,无需参与浮点运算。这样建立的表最大下标应为 0x00ffffff,所以这个表占用 $2^{24} Bytes = 16MB$,显然这个表太大了 not cache friendly bw = (uint32_t) mul_299[r] + (uint32_t) mul_587[g] + (uint32_t) mul_144[b]; bwPixel = (a \u003c\u003c 24) + (bw \u003c\u003c 16) + (bw \u003c\u003c 8) + bw; 分别对 R, G, B 建立对应的浮点数表,则这三个表总共占用 $3 \\times 2^8 Bytes \u003c 32KB$ cache friendly ","date":"2024-02-23","objectID":"/posts/c-bitwise/:4:1","tags":["Sysprog","C","Bitwise"],"title":"你所不知道的 C 语言: bitwise 操作","uri":"/posts/c-bitwise/"},{"categories":["C","Linux Kernel Internals"],"content":"案例探讨 信息 位元旋转实作和 Linux 核心案例 reverse bit 原理和案例分析 ","date":"2024-02-23","objectID":"/posts/c-bitwise/:5:0","tags":["Sysprog","C","Bitwise"],"title":"你所不知道的 C 语言: bitwise 操作","uri":"/posts/c-bitwise/"},{"categories":["C","Linux Kernel Internals"],"content":"类神经网络的 ReLU 极其常数时间复杂度实作 原文地址 ReLU 定义如下: $$ ReLU(x) = \\begin{cases} x \u0026 \\text{if } x \\geq 0 \\newline 0 \u0026 \\text{if } x \\lt 0 \\end{cases} $$ 显然如果 $x$ 是 32-bit 的二补数,可以使用上面提到的 x \u003e\u003e 31 的技巧来实作 constant-time function: int32_t ReLU(int32_t x) { return ~(x \u003e\u003e 31) \u0026 x; } 但是在深度学习中,浮点数使用更加常见,对于浮点数进行位移运算是不允许的 C99 6.5.7 Bitwise shift operators Each of the operands shall have integer type. 所以这里以 32-bit float 浮点数类型为例,利用 32-bit 二补数和 32-bit float 的 MSB 都是 sign bit,以及 C 语言类型 union 的特性 C99 6.5.2.3 (82) If the member used to access the contents of a union object is not the same as the member last used to store a value in the object, the appropriate part of the object representation of the value is reinterpreted as an object representation in the new type as described in 6.2.6 (a process sometimes called “type punning”). This might be a trap representation. 即 union 所有成员是共用一块内存的,所以访问成员时会将这块内存存储的 object 按照成员的类型进行解释。利用 int32_t 和 float 的 MSB 都是 sign bit 的特性,可以巧妙绕开对浮点数进行位移运算的限制,并且因为 union 成员内存的共用性质,保证结果的数值符合预期。 float ReLU(float x) { union { float f; int32_t i; } u = {.f = x}; u.i \u0026= ~(u.i \u003e\u003e 31); return u.f; } 同理可以完成 64-bit 浮点数的 ReLU 常数时间实作。 double ReLU(float x) { union { double f; int64_t i; } u = {.f = x}; u.i \u0026= ~(u.i \u003e\u003e 63); return u.f; } ","date":"2024-02-23","objectID":"/posts/c-bitwise/:6:0","tags":["Sysprog","C","Bitwise"],"title":"你所不知道的 C 语言: bitwise 操作","uri":"/posts/c-bitwise/"},{"categories":["C","Linux Kernel Internals"],"content":"从 $\\sqrt 2$ 的存在谈开平方根的快速运算 原文地址 注意 这一部分需要学员对现代数学观点有一些了解,强烈推荐修台大齐震宇老师的「数学潜水艇/新生营演讲」,齐老师的这些讲座难度和广度大致相当于其它院校开设的「数学导论」一课。 ","date":"2024-02-23","objectID":"/posts/c-bitwise/:7:0","tags":["Sysprog","C","Bitwise"],"title":"你所不知道的 C 语言: bitwise 操作","uri":"/posts/c-bitwise/"},{"categories":["C","Linux Kernel Internals"],"content":"$\\sqrt 2$ 的缘起和计算 YouTube: 第一次数学危机,天才是怎么被扼杀的? 可以通过「十分逼近法」来求得近似精确的 $\\sqrt 2$ 的数值,这也是「数值计算/分析」领域的一个应用,除此之外还可以使用「二分逼近法」进行求值。十分逼近法和二分逼近法的主要区别在于:十分逼近法的收敛速度比二分逼近法快很多,即会更快求得理想范围精度对应的数值。 在数组方法的分析中,主要关心两件事: 收敛速度 误差分析 由逼近法的过程不难看出,它们非常适合使用递归来实作: YouTube: 二分逼近法和十分逼近法求平方根 ","date":"2024-02-23","objectID":"/posts/c-bitwise/:7:1","tags":["Sysprog","C","Bitwise"],"title":"你所不知道的 C 语言: bitwise 操作","uri":"/posts/c-bitwise/"},{"categories":["C","Linux Kernel Internals"],"content":"固定点和牛顿法 固定点定理相关的证明挺有意思的 (前提是你修过数学导论这类具备现代数学观点和思维的课 🤣): 存在性 (分类讨论) 唯一性 (反证法) 固定点定理 (逻辑推理) 可以将求方程的根转换成求固定点的问题,然后利用收敛数列进行求解: $f(x) = 0$ $g(x) = p - f(x)$ and $g(p) = p$ 牛顿法公式: $$ p \\approx p_0 -\\dfrac{f(p_0)}{f’(p_0)} $$ 后面利用 $f(x) = x^2 - N = 0$ 求平方根的公式可以根据这个推导而来的,图形化解释 (切线) 也符合这个公式,自行推导: $$ \\begin{split} f(x) \u0026= x^2-N = 0 \\\\ b \u0026= a - \\frac{f(a)}{f'(a)} = a - \\frac{a^2 - N}{2a} \\\\ \u0026= \\frac{a^2+N}{2a} = (a+\\frac{N}{a}) \\div 2 \\end{split} $$ int mySqrt(int n) { if (n == 0) return 0; if (n \u003c 4) return 1; unsigned int ans = n / 2; if (ans \u003e 65535) // 65535 = 2^16 - 1 ans = 65535; while (ans * ans \u003e n || (ans + 1) * (ans + 1) \u003c= n) ans = (ans + n / ans) / 2; return ans; } 这个方法的流程是,选定一个不小于目标值 $x$ 的初始值 $a$,这样依据牛顿法,$a_i,\\ a_{i-1},\\ …$ 会递减逼近 $x$。因为是递减的,所以防止第 12 行的乘法溢出只需要考虑初始值 $a$ 即可,这也是第 9~10 行的逻辑。那么只剩下一个问题:如何保证初始值 $a$ 不小于目标值 $x$ 呢?其实很简单,只需要根据当 $n \\geq 2$ 时满足 $n=x^2 \\geq 2x$,即 $\\frac{n}{2} \\geq x$,便可推断出 $\\frac{n}{2}$ 在 $n \\geq 2$ 时必然是满足大等于目标值 $x$,所以可以使用其作为初始值 $a$,这也是第 8 行的逻辑。 因为求解的目标精度是整数,所以第 12 行的判断是否求得平方根的逻辑是合理的 (结合中间值 $a_i$ 递减的特性思考)。 LeetCode 69. Sqrt(x) ","date":"2024-02-23","objectID":"/posts/c-bitwise/:7:2","tags":["Sysprog","C","Bitwise"],"title":"你所不知道的 C 语言: bitwise 操作","uri":"/posts/c-bitwise/"},{"categories":["C","Linux Kernel Internals"],"content":"二进位的平方根 在使用位运算计算平方根的程式码中,我们又见到了的 union 和 do {...} whil (0) 的运用。位运算求解平方根的核心在于: $n$ 可以根据 IEEE 754 表示为 $S\\times1.Frac\\times2^{127+E}$,这种表示法下求解平方根只需计算 $\\sqrt{1.Frac}$ 和 $\\sqrt{2^{(127+E)}}$ 两部分 (只考虑非负数的平方根)。虽然描述起来简单,但由于 IEEE 754 编码的复杂性,需要考虑很多情况,例如 $E$ 全 0 或全 1,因为此时对应的数值就不是之前表示的那样了 (指 $S\\times1.Frac\\times2^{127+E}$),需要额外考量。 sign: 1 bit 0x80000000 exponent: 8 bits 0x7f800000 fraction: 23 bits 0x007fffff 原文给出的程式码是用于计算 $n$ 在 IEEE 754 编码下的指数部分在平方根的结果,虽然看起来只需要除以 2 即右移 1 位即可,但其实不然,例如上面所说的考虑指数部分全为 0 的情况,所以这个程式码是精心设计用于通用计算的。 在原始程式码的基础上,加上对 ix0 (对应 $1.Frac$) 使用牛顿法求平方根的逻辑,即可完成对 n 的平方根的求解。 当然这里要求和之前一样,平方根只需要整数精度即可,所以只需求出指数部分的平方根,然后通过二分法进行逼近即可满足要求 (因为剩余部分是 $1.Frac$ 并不影响平方根的整数精度,但是会导致一定误差,所以需要对指数部分进行二分逼近求值)。 先求出整數 n 開根號後結果的 $1.FRACTION \\times 2^{EXPONENT}$ 的 $EXPONENT$ 部份,則我們知道 n 開根號後的結果 $k$ 應滿足 $2^{EXPONENT} \\leq k \u003c 2^{EXPONENT+1}$,因此後續可以用二分搜尋法找出結果。 注意 这段程式码可以再一次看到 枚举 union 和 宏 do {...} while (0) 的应用之外,主要是根据 IEEE 754 编码规范进行求解,所以需要对浮点数的编码格式有一定认知,可以参考阅读: 你所不知道的 C 语言: 浮点数运算。 ","date":"2024-02-23","objectID":"/posts/c-bitwise/:7:3","tags":["Sysprog","C","Bitwise"],"title":"你所不知道的 C 语言: bitwise 操作","uri":"/posts/c-bitwise/"},{"categories":["C","Linux Kernel Internals"],"content":"Fast Inverse Square Root (平方根倒数) 下面的录影解说了程式码中的黑魔法 0x5f3759df 的原理,本质也是牛顿法,只不过是 选择一个接近目标值的初始值,从而只需要一次牛顿法即可求解出相对高精度的目标平方根 (例如将 $1.4$ 作为牛顿法求 $\\sqrt 2$ 的初始值,只需一次迭代求解出的精度已经相当惊人了),除此之外还运用到了对数求平方根倒数的技巧: YouTube: 没有显卡的年代,这群程序员用4行代码优化游戏 使用 IEEE 754 表示任意单精度浮点数为: $x = (1 + \\frac{M}{2^{23}}) \\times 2^{E-127}$,则该 $x$ 对应的对数为 $$ \\begin{split} \\log x \u0026= \\log{(1 + \\frac{M}{2^{23}}) \\times 2^{E-127}} \\\\ \u0026= \\log{(1 + \\frac{M}{2^{23}})} + \\log{2^{E-127}} \\\\ \u0026= \\frac{M}{2^{23}} + E - 127 \\\\ \u0026 = \\frac{1}{2^{23}}(2^{23} \\times E + M) - 127 \\\\ \u0026 = \\frac{1}{2^{23}}X - 127 \\end{split} $$ 注意上面处理 $\\log{(1 + \\frac{M}{2^{23}})}$ 部分时使用近似函数 $f(x) = x$ 代替了,当然会有一些误差,但由于我们后面计算的是平方根倒数的近似值,所以有一些误差没有关系。最后的 $2^{23} \\times E + M$ 部分只和浮点数的表示域相关,并且 这个运算的结果值和以二补数编码解释浮点数的数值相同 (参考上面的 IEEE 754 浮点数结构图,以及二补数的数值计算规则),我们用一个大写标识 $X$ 来标记其只与浮点数编码相关,并且对应二补数编码下的数值。 推导出对数的通用公式后,接下来就可以推导 平方根倒数的近似值 了,即求得对应数值的 $-\\frac{1}{2}$ 次方。假设 $a$ 是 $y$ 的平方根倒数,则有等式: $$ \\begin{split} \\log a \u0026= \\log{y^{-\\frac{1}{2}}} \\\\ \\log a \u0026= -\\frac{1}{2} \\log y \\\\ -\\frac{1}{2^{23}}A - 127 \u0026= -\\frac{1}{2}(-\\frac{1}{2^{23}} - 127) \\\\ A \u0026= 381 \\times 2^{22} - \\frac{1}{2} Y \\end{split} $$ 中间将数值由浮点数转换成二补数编码表示,并求得最终的浮点数表示为 $381 \\times 2^{22} - \\frac{1}{2} Y$,其中的 $381 \\times 2^{22}$ 对应的 16 进制恰好为 0x5f400000,这已经很接近我们看到的魔数了,但还有一点偏差。 这是因为在计算 $\\log{(1 + \\frac{M}{2^{23}})}$ 时直接使用 $f(x) = x$ 导致的总体误差还是较大,但是只需要将其稍微往 $y$ 轴正方向偏移一些就可以减少总体误差 (机器学习中常用的技巧 🤣),所以使用 $\\frac{M}{2^{23}} + \\lambda$ 代替原先的 $\\frac{M}{2^{23}}$ ($\\lambda$ 为修正的误差且 $\\lambda \u003e 0$),这会导致最终结果的 381 发生稍微一些变化 (因为二补数编码解释浮点数格式部分 $X$ 不能动,只能影响常数 $127$,而常数 $127$ 又直接影响最终结果的 $381$ 这类常数部分),进而产生魔数 0x5f3759df。 float InvSqrt(float x) { float xhalf = 0.5f * x; int i = *(int *) \u0026x; i = 0x5f3759df - (i \u003e\u003e 1); x = *(float *) \u0026i; x = x * (1.5f - xhalf * x * x); // only once newton iteration return x; } ","date":"2024-02-23","objectID":"/posts/c-bitwise/:7:4","tags":["Sysprog","C","Bitwise"],"title":"你所不知道的 C 语言: bitwise 操作","uri":"/posts/c-bitwise/"},{"categories":["C","Linux Kernel Internals"],"content":"C 语言的 bit-field 原文地址 #include \u003cstdbool.h\u003e #include \u003cstdio.h\u003e bool is_one(int i) { return i == 1; } int main() { struct { signed int a : 1; } obj = { .a = 1 }; puts(is_one(obj.a) ? \"one\" : \"not one\"); return 0; } C99 6.7.2.1 Structure and union specifiers A bit-field shall have a type that is a qualified or unqualified version of _Bool, signed int, unsigned int, or some other implementation-defined type. A bit-field is interpreted as a signed or unsigned integer type consisting of the specified number of bits. 将 a 这个 1-bit 的位域 (bit-field) 声明成 signed int,即将 a 视为一个 1-bit 的二补数,所以 a 的数值只有 0,-1。接下来将 1 赋值给 a 会使得 a 的数值为 -1,然后将 a 作为参数传入 is_one 时会进行符号扩展 (sign extension) 为 32-bit 的二补数 (假设编译器会将 int 视为 signed int),所以数值仍然为 -1。因此最终会输出 “not one”. ","date":"2024-02-23","objectID":"/posts/c-bitwise/:8:0","tags":["Sysprog","C","Bitwise"],"title":"你所不知道的 C 语言: bitwise 操作","uri":"/posts/c-bitwise/"},{"categories":["C","Linux Kernel Internals"],"content":"Linux 核心: BUILD_BUG_ON_ZERO() /* * Force a compilation error if condition is true, but also produce a * result (of value 0 and type size_t), so the expression can be used * e.g. in a structure initializer (or where-ever else comma expressions * aren't permitted). */ #define BUILD_BUG_ON_ZERO(e) (sizeof(struct { int:(-!!(e)); })) 这个宏运用了上面所说的 !! 技巧将 -!!(e) 的数值限定在 0 和 -1。 这个宏的功能是: 当 e 为 true 时,-!!(e) 为 -1,即 bit-field 的 size 为负数 当 e 为 false 时,-!!(e) 为 0,即 bit-field 的 size 为 0 C99 6.7.2.1 Structure and union specifiers The expression that specifies the width of a bit-field shall be an integer constant expression with a nonnegative value that does not exceed the width of an object of the type that would be specified were the colon and expression omitted. If the value is zero, the declaration shall have no declarator. A bit-field declaration with no declarator, but only a colon and a width, indicates an unnamed bit-field. As a special case, a bit-field structure member with a width of 0 indicates that no further bit-field is to be packed into the unit in which the previous bitfield, if any, was placed. (108) An unnamed bit-field structure member is useful for padding to conform to externally imposed layouts. 根据上面 C99 标准的说明,当 bit-feild 的 size 为负数时会编译失败 (只允许 integer constant expression with a nonnegative value),当 bit-field 为 0 时,会进行 alignment (以之前的 bit-field 成员所在的 unit 为单位)。 struct foo { int a : 3; int b : 2; int : 0; /* Force alignment to next boundary */ int c : 4; int d : 3; }; int main() { int i = 0xFFFF; struct foo *f = (struct foo *) \u0026i; printf(\"a=%d\\nb=%d\\nc=%d\\nd=%d\\n\", f-\u003ea, f-\u003eb, f-\u003ec, f-\u003ed); return 0; } 这里使用了 size 为 0 的 bit-field,其内存布局如下: i = 1111 1111 1111 1111 X stand for unknown value assume little endian padding \u0026 start from here ↓ 1111 1111 1111 1111XXXX XXXX XXXX XXXX b baaa ddd cccc |← int 32 bits →||← int 32 bits →| zero size bit-field 使得这里在 a, b 和 c, d 之间进行 sizeof(int) 的 alignment,所以 c, d 位于 i 这个 object 范围之外,因此 c, d 每次执行时的数值是不确定的,当然这也依赖于编译器,可以使用 gcc 和 clang 进行测试 🤣 C11 3.14 1 memory location (NOTE 2) A bit-field and an adjacent non-bit-field member are in separate memory locations. The same applies to two bit-fields, if one is declared inside a nested structure declaration and the other is not, or if the two are separated by a zero-length bit-field declaration, or if they are separated by a non-bit-field member declaration. It is not safe to concurrently update two non-atomic bit-fields in the same structure if all members declared between them are also (non-zero-length) bit-fields, no matter what the sizes of those intervening bit-fields happen to be. 所以 BUILD_BUG_ON_ZERO 宏相当于编译时期的 assert,因为 assert 是在执行时期才会触发的,对于 Linux 核心来说代价太大了 (想象一下核心运行着突然触发一个 assert 导致当掉 🤣),所以采用了 BUILD_BUG_ON_ZERO 宏在编译时期就进行检查 (莫名有一种 Rust 的风格 🤣) 对于 BUILD_BUG_ON_ZERO 这个宏,C11 提供了 _Static_assert 语法达到相同效果,但是 Linux kernel 自己维护了一套编译工具链 (这个工具链 gcc 版本可能还没接纳 C11 🤣),所以还是使用自己编写的 BUILD_BUG_ON_ZERO 宏。 ","date":"2024-02-23","objectID":"/posts/c-bitwise/:8:1","tags":["Sysprog","C","Bitwise"],"title":"你所不知道的 C 语言: bitwise 操作","uri":"/posts/c-bitwise/"},{"categories":["Rust"],"content":" In this fourth Crust of Rust video, we cover smart pointers and interior mutability, by re-implementing the Cell, RefCell, and Rc types from the standard library. As part of that, we cover when those types are useful, how they work, and what the equivalent thread-safe versions of these types are. In the process, we go over some of the finer details of Rust's ownership model, and the UnsafeCell type. We also dive briefly into the Drop Check rabbit hole (https://doc.rust-lang.org/nightly/nomicon/dropck.html) before coming back up for air. 整理自 John Gjengset 的影片 ","date":"2024-02-20","objectID":"/posts/smart-pointers-and-interior-mutability/:0:0","tags":["Rust","Smart pointer","Interior Mutability"],"title":"Crust of Rust: Smart Pointers and Interior Mutability","uri":"/posts/smart-pointers-and-interior-mutability/"},{"categories":["Rust"],"content":"影片注解 ","date":"2024-02-20","objectID":"/posts/smart-pointers-and-interior-mutability/:1:0","tags":["Rust","Smart pointer","Interior Mutability"],"title":"Crust of Rust: Smart Pointers and Interior Mutability","uri":"/posts/smart-pointers-and-interior-mutability/"},{"categories":["Rust"],"content":"Interior Mutability Module std::cell Rust memory safety is based on this rule: Given an object T, it is only possible to have one of the following: Having several immutable references (\u0026T) to the object (also known as aliasing). Having one mutable reference (\u0026mut T) to the object (also known as mutability). Values of the Cell\u003cT\u003e, RefCell\u003cT\u003e, and OnceCell\u003cT\u003e types may be mutated through shared references (i.e. the common \u0026T type), whereas most Rust types can only be mutated through unique (\u0026mut T) references. We say these cell types provide ‘interior mutability’ (mutable via \u0026T), in contrast with typical Rust types that exhibit ‘inherited mutability’ (mutable only via \u0026mut T). We can use (several) immutable references of a cell to mutate the thing inside of the cell. There is (virtually) no way for you to get a reference to the thing inside of a cell. Because if no one else has a pointer to it (the thing inside of a cell), the changing it is fine. Struct std::cell::UnsafeCell If you have a reference \u0026T, then normally in Rust the compiler performs optimizations based on the knowledge that \u0026T points to immutable data. Mutating that data, for example through an alias or by transmuting an \u0026T into an \u0026mut T, is considered undefined behavior. UnsafeCell\u003cT\u003e opts-out of the immutability guarantee for \u0026T: a shared reference \u0026UnsafeCell\u003cT\u003e may point to data that is being mutated. This is called “interior mutability”. The UnsafeCell API itself is technically very simple: .get() gives you a raw pointer *mut T to its contents. It is up to you as the abstraction designer to use that raw pointer correctly. ","date":"2024-02-20","objectID":"/posts/smart-pointers-and-interior-mutability/:1:1","tags":["Rust","Smart pointer","Interior Mutability"],"title":"Crust of Rust: Smart Pointers and Interior Mutability","uri":"/posts/smart-pointers-and-interior-mutability/"},{"categories":["Rust"],"content":"Cell Module std::cell Cell\u003cT\u003e Cell\u003cT\u003e implements interior mutability by moving values in and out of the cell. That is, an \u0026mut T to the inner value can never be obtained, and the value itself cannot be directly obtained without replacing it with something else. Both of these rules ensure that there is never more than one reference pointing to the inner value. This type provides the following methods: Cell 在 Rust 中对一个变量 (T),在已存在其 immutable references (\u0026T) 时使用 mutable reference (\u0026mut T) 是禁止的,因为这样会因为编译器优化而导致程序的行为不一定符合我们的预期。考虑以下的代码: let x = 3; let r1 = \u0026x, r2 = \u0026x; let r3 = \u0026mut x; println!(\"{}\", r1); r3 = 5; println!(\"{}\", r2); 假设以上的代码可以通过编译,那么程序执行到第 6 行打印出来的可能是 3 而不是我们预期的 5,这是因为编译器会对 immtuable references 进行激进的优化,例如进行预取,所以在第 6 行时对于 r2 使用的还是先前预取的值 3 而不是内存中最新的值 5。这也是 Rust 制定对 immutable reference 和 mutable reference 的规则的原因之一。 为了达到我们的预期行为,可以使用 UnsafeCell 来实现: let x = 3; let uc = UnsafeCell::new(x); let r1 = \u0026uc, r2 = \u0026uc, r3 = \u0026uc; unsafe { println!(\"{}\", *uc.get()); } unsafe { *uc.get() = 5; } unsafe { println!(\"{}\", *uc.get()); } 上面的代码可以通过编译,并且在第 6 行时打印出来的是预期的 5。这是因为编译器会对 UnsafeCell 进行特判,而避免进行一些激进的优化 (例如预取),从而使程序行为符合我们的预期。并且 UnsafeCell 的 get() 方法只需要接受 \u0026self 参数,所以可以对 UnsafeCell 进行多个 immutable references,这并不违反 Rust 的内存安全准则。同时每个对于 UnsafeCel 的 immutable references 都可以通过所引用的 UnsafeCell 来实现内部可变性 (interior mutability)。 上述代码片段存在大量的 unsafe 片段 (因为 UnsafeCell),将这些 unsafe 操作封装一下就实现了 Cell。但是因为 Cell 的方法 get() 和 set() 都需要转移所有权,所以 Cell 只能用于实现了 Copy trait 的类型的内部可变性。但是对于 concurrent 情形,UnsafeCell 就是一个临界区,无法保证内部修改是同步的,所以 Cell 不是 thread safe 的。 Cell is typically used for more simple types where copying or moving values isn’t too resource intensive (e.g. numbers) 注意 Cell 提供了这样一个“内部可变性”机制: 在拥有对一个 object 多个引用时,可以通过任意一个引用对 object 进行内部可变,并保证在此之后其他引用对于该 object 的信息是最新的。 ","date":"2024-02-20","objectID":"/posts/smart-pointers-and-interior-mutability/:1:2","tags":["Rust","Smart pointer","Interior Mutability"],"title":"Crust of Rust: Smart Pointers and Interior Mutability","uri":"/posts/smart-pointers-and-interior-mutability/"},{"categories":["Rust"],"content":"RefCell Module std::cell RefCell\u003cT\u003e RefCell\u003cT\u003e uses Rust\\’s lifetimes to implement “dynamic borrowing”, a process whereby one can claim temporary, exclusive, mutable access to the inner value. Borrows for RefCell\u003cT\u003e\\s are tracked at runtime, unlike Rust’s native reference types which are entirely tracked statically, at compile time. Runtime Borrow Check RefCell RefCell 也提供了之前所提的“内部可变性”机制,但是是通过提供 引用 而不是转移所有权来实现。所以它常用于 Tree, Graph 这类数据结构,因为这些数据结构的节点 “很大”,不大可能实现 Copy 的 Trait (因为开销太大了),所以一般使用 RefCell 来实现节点的相互引用关系。 ","date":"2024-02-20","objectID":"/posts/smart-pointers-and-interior-mutability/:1:3","tags":["Rust","Smart pointer","Interior Mutability"],"title":"Crust of Rust: Smart Pointers and Interior Mutability","uri":"/posts/smart-pointers-and-interior-mutability/"},{"categories":["Rust"],"content":"Rc method std::boxed::Box::into_raw After calling this function, the caller is responsible for the memory previously managed by the Box. In particular, the caller should properly destroy T and release the memory, taking into account the memory layout used by Box. The easiest way to do this is to convert the raw pointer back into a Box with the Box::from_raw function, allowing the Box destructor to perform the cleanup. Rc ","date":"2024-02-20","objectID":"/posts/smart-pointers-and-interior-mutability/:1:4","tags":["Rust","Smart pointer","Interior Mutability"],"title":"Crust of Rust: Smart Pointers and Interior Mutability","uri":"/posts/smart-pointers-and-interior-mutability/"},{"categories":["Rust"],"content":"Raw pointers vs references * mut and * const are not references, they are raw pointers. In Rust, there are a bunch of semantics you have to follow when you using references. Like if you use \u0026 symbol, an \u0026 alone means a shared reference, and you have to guarantee that there are no exclusive references to that thing. And similarly, if you have a \u0026mut, a exclusive reference, you know that there are not shared references. The * version of these, like * mut and * const, do not have these guarantees. If you have a * mut, there may be other * muts to the same thing. There might be * const to the same thing. You have no guarantee, but you also cann't do much with a *. If you have a raw pointer, the only thing you can really do to it is use an unsafe block to dereference it and turn it into reference. But that is unsafe, you need to document wht it is safe. You're not able to go from a const pointer to an exclusive reference. But you can go from a mutable pointer to an exclusive reference. To guarantee that you have to follow onwership semantics in Rust. ","date":"2024-02-20","objectID":"/posts/smart-pointers-and-interior-mutability/:1:5","tags":["Rust","Smart pointer","Interior Mutability"],"title":"Crust of Rust: Smart Pointers and Interior Mutability","uri":"/posts/smart-pointers-and-interior-mutability/"},{"categories":["Rust"],"content":"PhantomData \u0026 Drop check The Rustonomicon: Drop Check Medium: Rust Notes: PhantomData struct Foo\u003c'a, T: Default\u003e { v: \u0026'a mut T, } impl\u003cT\u003e Drop for Foo\u003c'_, T: Default\u003e { fn drop(\u0026mut self) { let _ = std::mem::replace(self.v, T::default()); } } fn main() { let mut t = String::from(\"hello\"); let foo = Foo { v: \u0026mut t }; drop(t); drop(foo); } 最后的 2 行 drop 语句会导致编译失败,因为编译器知道 foo 引用了 t,所以会进行 drop check,保证 t 的 lifetime 至少和 foo 一样长,因为 drop 时会按照从内到外的顺序对结构体的成员及其本身进行 drop。但是对于我们实现的 Rc 使用的是 raw pointer,如果不加 PhantomData,那么在对 Rc 进行 drop 时并不会检查 raw pointer 所指向的 RcInner 的 lifetime 是否满足要求,即在 drop Rc 之前 drop RcInner 并不会导致编译失败。简单来说,PhantomData 就是让编译器以为 Rc 拥有 RcInner 的所有权或引用,由此进行期望的 drop check 行为。 ","date":"2024-02-20","objectID":"/posts/smart-pointers-and-interior-mutability/:1:6","tags":["Rust","Smart pointer","Interior Mutability"],"title":"Crust of Rust: Smart Pointers and Interior Mutability","uri":"/posts/smart-pointers-and-interior-mutability/"},{"categories":["Rust"],"content":"Thread Safety Cell Because even though you're not giving out references to things, having two threads modify the same value at the same time is just not okay. There actually is o thread-safe version of Cell. (Think it as pointer in C 🤣) RefCell You could totally implement a thread-safe version of RefCell, one that uses an atomic counter instead of Cell for these numbers. So it turns out that the CPU has built-in instructions that can, in a thread-safe way, increment and decrement counters. Rc The thread-safe version of Rc is Arc, or Atomic Reference Count. ","date":"2024-02-20","objectID":"/posts/smart-pointers-and-interior-mutability/:1:7","tags":["Rust","Smart pointer","Interior Mutability"],"title":"Crust of Rust: Smart Pointers and Interior Mutability","uri":"/posts/smart-pointers-and-interior-mutability/"},{"categories":["Rust"],"content":"Copy-on-Write (COW) Struct std::borrow::Cow The type Cow is a smart pointer providing clone-on-write functionality: it can enclose and provide immutable access to borrowed data, and clone the data lazily when mutation or ownership is required. The type is designed to work with general borrowed data via the Borrow trait. ","date":"2024-02-20","objectID":"/posts/smart-pointers-and-interior-mutability/:1:8","tags":["Rust","Smart pointer","Interior Mutability"],"title":"Crust of Rust: Smart Pointers and Interior Mutability","uri":"/posts/smart-pointers-and-interior-mutability/"},{"categories":["Rust"],"content":"Homework 信息 实作说明: 尝试使用 RefCell 来实现 Linux kernel 风格的 linked list 数据结构为 circular doubly linked list 实现 insert_head, remove_head 方法 实现 insert_tail, remove_tail 方法 实现 list_size, list_empty, list_is_singular 方法 实现迭代器 (Iterator),支持双向迭代 (DoubleEndedIterator) 参考资料: sysprog21/linux-list linux/list.h ","date":"2024-02-20","objectID":"/posts/smart-pointers-and-interior-mutability/:2:0","tags":["Rust","Smart pointer","Interior Mutability"],"title":"Crust of Rust: Smart Pointers and Interior Mutability","uri":"/posts/smart-pointers-and-interior-mutability/"},{"categories":["Rust"],"content":"Documentations 这里列举视频中一些概念相关的 documentation 学习的一手资料是官方文档,请务必自主学会阅读规格书之类的资料 ","date":"2024-02-20","objectID":"/posts/smart-pointers-and-interior-mutability/:3:0","tags":["Rust","Smart pointer","Interior Mutability"],"title":"Crust of Rust: Smart Pointers and Interior Mutability","uri":"/posts/smart-pointers-and-interior-mutability/"},{"categories":["Rust"],"content":"Crate std 可以使用这里提供的搜素栏进行搜索 (BTW 不要浪费时间在 Google 搜寻上!) Module std::cell Struct std::cell::UnsafeCell Struct std::cell::Cell Struct std::cell::RefCell Module std::rc Module std::sync Struct std::sync::Mutex Struct std::sync::RwLock Struct std::sync::Arc Struct std::boxed::Box method std::boxed::Box::into_raw method std::boxed::Box::from_raw Struct std::ptr::NonNull method std::ptr::NonNull::new_unchecked method std::ptr::NonNull::as_ref method std::ptr::NonNull::as_ptr Struct std::marker::PhantomData Struct std::borrow::Cow Trait std::ops::Drop Trait std::ops::Deref Trait std::ops::DerefMut Trait std::marker::Sized Function std::thread::spawn Function std::mem::replace Function std::mem::drop ","date":"2024-02-20","objectID":"/posts/smart-pointers-and-interior-mutability/:3:1","tags":["Rust","Smart pointer","Interior Mutability"],"title":"Crust of Rust: Smart Pointers and Interior Mutability","uri":"/posts/smart-pointers-and-interior-mutability/"},{"categories":["Rust"],"content":"References 可能不是你看过最无聊的 Rust 入门喜剧102 (2) 智能指针 [bilibili] ","date":"2024-02-20","objectID":"/posts/smart-pointers-and-interior-mutability/:4:0","tags":["Rust","Smart pointer","Interior Mutability"],"title":"Crust of Rust: Smart Pointers and Interior Mutability","uri":"/posts/smart-pointers-and-interior-mutability/"},{"categories":["C","Linux Kernel Internals"],"content":" 尽管数值系统并非 C 语言所持有,但在 Linux 核心大量存在 u8/u16/u32/u64 这样通过 typedef 所定义的类型,伴随着各种 alignment 存取,如果对数值系统的认知不够充分,可能立即就被阻拦在探索 Linux 核心之外——毕竟你完全搞不清楚,为何 Linux 核心存取特定资料需要绕一大圈。 原文地址 ","date":"2024-02-20","objectID":"/posts/c-numerics/:0:0","tags":["Sysprog","C","Numerics"],"title":"你所不知道的 C 语言: 数值系统篇","uri":"/posts/c-numerics/"},{"categories":["C","Linux Kernel Internals"],"content":"Balanced ternary balanced ternary 三进制中 -, 0, + 在数学上具备对称性质。它相对于二进制编码的优势在于,其本身就可以表示正负数 (通过 +-, 0, +),而二进制需要考虑 unsigned 和 signed 的情况,从而决定最高位所表示的数值。 相关的运算规则: + add - = 0 0 add + = + 0 add - = - 以上运算规则都比较直观,这也决定了 balanced ternary 在编码上的对称性 (减法等价于加上逆元,逆元非常容易获得)。但是需要注意,上面的运算规则并没有涉及到相同位运算的规则,例如 $+\\ (add)\\ +$,这种运算也是 balanced ternary 相对于二进制编码的劣势,可以自行推导一下这种运算的规则。 The Balanced Ternary Machines of Soviet Russia ","date":"2024-02-20","objectID":"/posts/c-numerics/:1:0","tags":["Sysprog","C","Numerics"],"title":"你所不知道的 C 语言: 数值系统篇","uri":"/posts/c-numerics/"},{"categories":["C","Linux Kernel Internals"],"content":"数值编码与阿贝尔群 阿贝尔群也用于指示为什么使用二补数编码来表示整数: 存在唯一的单位元 (二补数中单位元 0 的编码是唯一的) 每个元素都有逆元 (在二补数中几乎每个数都有逆元) 浮点数 IEEE 754: An example of a layout for 32-bit floating point is Conversión de un número binario a formato IEEE 754 单精度浮点数相对于整数 在某些情況下不满足結合律和交换律,所以不构成 阿贝尔群,在编写程序时需要注意这一点。即使编写程序时谨慎处理了单精度浮点数运算,但是编译器优化可能会将我们的处理破划掉。所以涉及到单精度浮点数,都需要注意其运算。 信息 你所不知道的 C 语言: 浮点数运算 你所不知道的 C 语言: 编译器和最佳化原理篇 ","date":"2024-02-20","objectID":"/posts/c-numerics/:2:0","tags":["Sysprog","C","Numerics"],"title":"你所不知道的 C 语言: 数值系统篇","uri":"/posts/c-numerics/"},{"categories":["C","Linux Kernel Internals"],"content":"Integer Overflow ","date":"2024-02-20","objectID":"/posts/c-numerics/:3:0","tags":["Sysprog","C","Numerics"],"title":"你所不知道的 C 语言: 数值系统篇","uri":"/posts/c-numerics/"},{"categories":["C","Linux Kernel Internals"],"content":"2002 年 FreeBSD [53] #define KSIZE 1024 char kbuf[KSIZE]; int copy_from_kernel(void *user_dest, int maxlen) { int len = KSIZE \u003c maxlen ? KSIZE : maxlen; memcpy(user_dest, kbuf, len); return len; } 假设将“负”的数值带入 maxlen,那么在上述的程式码第 4 行时 len 会被赋值为 maxlen,在第 5 行中,根据 memcpy 的原型声明 void *memcpy(void *dest, const void *src, size_t n); 会将 len (=maxlen) 解释为 size_t 类型,关于 size_t 类型 C99 [7.17 Common definitions \u003cstddef.h\u003e] size_t which is the unsigned integer type of the result of the sizeof operator; 所以在 5 行中 memcpy 会将 len 这个“负“的数值按照无符号数的编码进行解释,这会导致将 len 解释为一个超级大的无符号数,可能远比 KSIZE 这个限制大。copy_from_kernel 这个函数是运行在 kernel 中的,这样可能会造成潜在的 kernel 信息数据泄露问题。 ","date":"2024-02-20","objectID":"/posts/c-numerics/:3:1","tags":["Sysprog","C","Numerics"],"title":"你所不知道的 C 语言: 数值系统篇","uri":"/posts/c-numerics/"},{"categories":["C","Linux Kernel Internals"],"content":"2002 年 External data representation (XDR) [62] void *copy_elements(void *ele_src[], int ele_cnt, int ele_size) { void *result = malloc(ele_cnt * ele_size); if (result==NULL) return NULL; void *next = result; for (int i = 0; i \u003c ele_cnt; i++) { memcpy(next, ele_src[i], ele_size); next += ele_size; } return result; } 假设将 ele_cnt = $2^{20}+1$, ele_size=$2^{12}$ 代入,显然在第 2 行的 ele_cnt * ele_size 会超出 32 位整数表示的最大值,导致 overflow。又因为 malloc 的原型声明 void *malloc(size_t size); malloc 会将 ele_cnt * ele_size 溢出后保留的值解释为 size_t,这会导致 malloc 分配的内存空间远小于 ele_cnt * ele_size Bytes (这是 malloc 成功的情况,malloc 也有可能会失败,返回 NULL)。 因为 malloc 成功分配空间,所以会通过第 3 行的测试。在第 5~8 行的 for 循环,根据 ele_cnt 和 ele_size 的值进行 memcpy,但是因为分配的空间远远小于 ele_cnt * ele_size,所以这样会覆写被分配空间外的内存区域,可能会造成 kernel 的信息数据被覆盖。 ","date":"2024-02-20","objectID":"/posts/c-numerics/:3:2","tags":["Sysprog","C","Numerics"],"title":"你所不知道的 C 语言: 数值系统篇","uri":"/posts/c-numerics/"},{"categories":["C","Linux Kernel Internals"],"content":"Bitwise 3Blue1Brown: How to count to 1000 on two hands [YouTube] 本质上是使用无符号数的二进制编码来进行计数,将手指/脚趾视为数值的 bit 信息 解读计算机编码 ","date":"2024-02-20","objectID":"/posts/c-numerics/:4:0","tags":["Sysprog","C","Numerics"],"title":"你所不知道的 C 语言: 数值系统篇","uri":"/posts/c-numerics/"},{"categories":["C","Linux Kernel Internals"],"content":"Power of two 通过以下程式码可以判断 x 是否为 2 的次方 x \u0026 (x - 1) == 0 通过值为 1 的最低位来进行归纳法证明,例如,对 0b00000001, 0b00000010, 0b00000100, … 来进行归纳证明 (还需要证明 x 中只能有一个 bit 为值 1,不过这个比较简单)。另一种思路,通过 LSBO 以及 $X$ 和 $-X$ 的特性来证明。 LSBO: Least Significant bit of value One $-X = ~(X - 1)$ $-X$ 的编码等价于 $X$ 的编码中比 LSBO 更高的 bits 进行反转,LSBO 及更低的 bits 保持不变 ","date":"2024-02-20","objectID":"/posts/c-numerics/:4:1","tags":["Sysprog","C","Numerics"],"title":"你所不知道的 C 语言: 数值系统篇","uri":"/posts/c-numerics/"},{"categories":["C","Linux Kernel Internals"],"content":"ASCII table 通过 ASCII table 中对 ASCII 编码的分布规律,可以实现大小写转换的 constant-time function 也可以通过命令 man ascii 来输出精美的 ASCII table // 字符转小写 (x | ' ') // 字符转大写 (x \u0026 ' ') // 大小写互转 (x ^ ' ') Each lowercase letter is 32 + uppercase equivalent. This means simply flipping the bit at position 5 (counting from least significant bit at position 0) inverts the case of a letter. ","date":"2024-02-20","objectID":"/posts/c-numerics/:4:2","tags":["Sysprog","C","Numerics"],"title":"你所不知道的 C 语言: 数值系统篇","uri":"/posts/c-numerics/"},{"categories":["C","Linux Kernel Internals"],"content":"XOR swap 通过 xor 运算符可以实现无需临时变量的,交换两个数值的程式码 void xorSwap(int *x, int *y) { *x ^= *y; *y ^= *x; *x ^= *y; } 第 3 行的 *y ^= *x 的结果等价于 *y ^ *x ^ *y,整数满足交换律和结合律,所以结果为 *x 第 4 行的 *x ^= *y 的结果等价于 *x ^ *y ^ *x,整数满足交换律和结合律,所以结果为 *y 这个实作方法常用于没有额外空间的情形,例如 Bootloader ","date":"2024-02-20","objectID":"/posts/c-numerics/:4:3","tags":["Sysprog","C","Numerics"],"title":"你所不知道的 C 语言: 数值系统篇","uri":"/posts/c-numerics/"},{"categories":["C","Linux Kernel Internals"],"content":"避免 overflow 整数运算 (x + y) / 2 可能会导致 overflow (如果 x, y 数值都接近 UINT32_MAX),可以改写为以下不会导致 overflow 的程式码 (x \u0026 y) + (x ^ y) \u003e\u003e 1 使用加法器来思考: 对于 x + y,x \u0026 y 表示进位,x ^ y 表示位元和,所以 x + y 等价于 (x \u0026 y) \u003c\u003c 1 + (x ^ y) 这个运算不会导致 overflow (因为使用了 bitwise 运算)。因此 (x + y) / 2 等价于 ((x \u0026 y) \u003c\u003c 1 + (x ^ y)) \u003e\u003e 1 = ((x \u0026 y) \u003c\u003c 1) \u003e\u003e 1 + (x ^ y) \u003e\u003e 1 = (x \u0026 y) + (x ^ y) \u003e\u003e 1 整数满足交换律和结合律 ","date":"2024-02-20","objectID":"/posts/c-numerics/:4:4","tags":["Sysprog","C","Numerics"],"title":"你所不知道的 C 语言: 数值系统篇","uri":"/posts/c-numerics/"},{"categories":["C","Linux Kernel Internals"],"content":"macro DIRECT #if LONG_MAX == 2147483647L #define DETECT(X) \\ (((X) - 0x01010101) \u0026 ~(X) \u0026 0x80808080) #else #if LONG_MAX == 9223372036854775807L #define DETECT(X) \\ (((X) - 0x0101010101010101) \u0026 ~(X) \u0026 0x8080808080808080) #else #error long int is not a 32bit or 64bit type. #endif #endif DIRECT 宏的作用是侦测 32bit/64bit 中是否存在一个 Byte 为 NULL。我们以最简单的情况 1 个 Byte 时来思考这个实作的本质: ((X) - 0x01) \u0026 ~(X) \u0026 0x80 = ~(~((X) - 0x01) | X) \u0026 0x80 ~((X) - 0x01) 是 X 的取负值编码,即 -X,根据二补数编码中 -X 和 X 的特性,可得 (~((X) - 0x01) | X) 为: X 二补数编码中值为 1 的最低位 (后续称之为 LSBO) 及更低位保持不变,LSBO 更高位均为 1。则 ~(~((X) - 0x01) | X) 为: X 二补数编码中值为 1 的最低位 (后续称之为 LSBO) 的更低位翻转,LSBO 及更高位均为 0。 LSBO: Least Significant Bit with value of One X = 0x0080 (X) - 0x01 = 0xff80 ~((X) - 0x01) = 0x007f ~(~((X) - 0x01) | X) \u0026 0x80 = 0 可以自行归纳推导出: 对于任意不为 0 的数值,上述流程推导的最终值都为 0,但对于值为 0 的数值,最终值为 0x80。由此可以推导出最开始的实作 DIRECT 宏。 这个 DIRECT 宏相当实用,常用于加速字符串操作,将原先的以 1-byte 为单元的操作加速为以 32bit/64bit 为单位的操作。可以阅读相关实作并寻找其中的逻辑: newlib 的 strlen newlib 的 strcpy ","date":"2024-02-20","objectID":"/posts/c-numerics/:4:5","tags":["Sysprog","C","Numerics"],"title":"你所不知道的 C 语言: 数值系统篇","uri":"/posts/c-numerics/"},{"categories":["C","Linux Kernel Internals"],"content":"Count Leading Zero 计算 $\\log_2N$ 可以通过计算数值对应的编码,高位有多少连续的 0’bits,再用 31 减去即可。可以通过 0x0001, 0x0010, 0x0002, … 等编码来进行归纳推导出该结论。 iteration version int clz(uint32_t x) { int n = 32, c = 16; do { uint32_t y = x \u003e\u003e c; if (y) { n -= c; x = y; } c \u003e\u003e= 1; } while (c); return (n - x); } binary search technique int clz(uint32_t x) { if (x == 0) return 32; int n = 0; if (x \u003c= 0x0000FFFF) { n += 16; x \u003c\u003c= 16; } if (x \u003c= 0x00FFFFFF) { n += 8; x \u003c\u003c= 8; } if (x \u003c= 0x0FFFFFFF) { n += 4; x \u003c\u003c= 4; } if (x \u003c= 0x3FFFFFFF) { n += 2; x \u003c\u003c= 2; } if (x \u003c= 0x7FFFFFFF) { n += 1; x \u003c\u003c= 1; } return n; } byte-shift version int clz(uint32_t x) { if (x == 0) return 32; int n = 1; if ((x \u003e\u003e 16) == 0) { n += 16; x \u003c\u003c= 16; } if ((x \u003e\u003e 24) == 0) { n += 8; x \u003c\u003c= 8; } if ((x \u003e\u003e 28) == 0) { n += 4; x \u003c\u003c= 4; } if ((x \u003e\u003e 30) == 0) { n += 2; x \u003c\u003c= 2; } n = n - (x \u003e\u003e 31); return n; } 在这些实作中,循环是比较直观的,但是比较低效;可以利用编码的特性,使用二分法或位运算来加速实作。 ","date":"2024-02-20","objectID":"/posts/c-numerics/:5:0","tags":["Sysprog","C","Numerics"],"title":"你所不知道的 C 语言: 数值系统篇","uri":"/posts/c-numerics/"},{"categories":["C","Linux Kernel Internals"],"content":"避免循环 int func(unsigned int x) { int val = 0; int i = 0; for (i = 0; i \u003c 32; i++) { val = (val \u003c\u003c 1) | (x \u0026 0x1); x \u003e\u003e= 1; } return val; } 这段程式码的作用是,对一个 32bit 的数值进行逐位元反转。这个逐位元反转功能非常实用,常实作于加密算法,例如 DES、AES。 但是与上面的 Count Leading Zero 类似,上面程式码使用了循环,非常低效,可以通过位运算来加速。 int func(unsigned int x) { int val = 0; val = num; val = ((val \u0026 0xffff0000) \u003e\u003e 16) | ((val \u0026 0x0000ffff) \u003c\u003c 16); val = ((val \u0026 0xff00ff00) \u003e\u003e 8) | ((val \u0026 0x00ff00ff) \u003c\u003c 8); val = ((val \u0026 0xf0f0f0f0) \u003e\u003e 4) | ((val \u0026 0x0f0f0f0f) \u003c\u003c 4); val = ((val \u0026 0xcccccccc) \u003e\u003e 2) | ((val \u0026 0x33333333) \u003c\u003c 2); val = ((val \u0026 0xaaaaaaaa) \u003e\u003e 1) | ((val \u0026 0x55555555) \u003c\u003c 1); return val; } Reverse integer bitwise without using loop [Stack Overflow] 技巧 Bits Twiddling Hacks 解析: (一), (二), (三) ","date":"2024-02-20","objectID":"/posts/c-numerics/:6:0","tags":["Sysprog","C","Numerics"],"title":"你所不知道的 C 语言: 数值系统篇","uri":"/posts/c-numerics/"},{"categories":["C","Linux Kernel Internals"],"content":"加解密的应用 假設有一張黑白的相片是由很多個0 ~255 的 pixel 組成 (0 是黑色,255 是白色),這時候可以用任意的 KEY (000000002 - 111111112) 跟原本的每個 pixel 做運算,如果使用 AND (每個 bit 有 75% 機率會變成 0),所以圖會變暗。如果使用 OR (每個 bit 有 75% 機率會變 1),圖就會變亮。這兩種幾乎都還是看的出原本的圖片,但若是用 XOR 的話,每個 bit 變成 0 或 1 的機率都是 50%,所以圖片就會變成看不出東西的雜訊。 上圖左 1 是原圖,左 2 是用 AND 做運算之後,右 2 是用 OR 做運算之後,右 1 是用 XOR,可見使用 XOR 的加密效果最好。 这就是在密码学领域偏爱 XOR 的原因之一。除此之外,XOR 在概率统计上的优异特性也是另一个原因,具体证明推导请查看原文的说明。 ","date":"2024-02-20","objectID":"/posts/c-numerics/:7:0","tags":["Sysprog","C","Numerics"],"title":"你所不知道的 C 语言: 数值系统篇","uri":"/posts/c-numerics/"},{"categories":["Linux Kernel Internals"],"content":"程序分析工具 Cppcheck 是 静态 程序分析工具,即无需运行程序就可以分析出程序潜在的问题,当然会有一定的误差,类似的工具有 cargo-check Valgrind 是 动态 程序分析工具,即需要将程序运行起来再进行分析,通常用于检测内存泄漏 (memory leak) ","date":"2024-02-19","objectID":"/posts/linux2023-lab0/:1:0","tags":["Sysprog","Linux","C"],"title":"Linux 核心设计: C Programming Lab","uri":"/posts/linux2023-lab0/"},{"categories":["Linux Kernel Internals"],"content":"Queue ","date":"2024-02-19","objectID":"/posts/linux2023-lab0/:2:0","tags":["Sysprog","Linux","C"],"title":"Linux 核心设计: C Programming Lab","uri":"/posts/linux2023-lab0/"},{"categories":["Linux Kernel Internals"],"content":"leetcode 相关的 LeetCode 题目的实作情况: LeetCode 2095. Delete the Middle Node of a Linked List LeetCode 82. Remove Duplicates from Sorted List II LeetCode 24. Swap Nodes in Pairs LeetCode 25. Reverse Nodes in k-Group LeetCode 2487. Remove Nodes From Linked List / 参考题解 LeetCode 23. Merge k Sorted Lists ","date":"2024-02-19","objectID":"/posts/linux2023-lab0/:2:1","tags":["Sysprog","Linux","C"],"title":"Linux 核心设计: C Programming Lab","uri":"/posts/linux2023-lab0/"},{"categories":["Linux Kernel Internals"],"content":"q_new \u0026 q_free q_new 使用 malloc 分配空间,并使用 INIT_LIST_HEAD 进行初始化。 q_free 遍历 queue 进行逐个节点释放,所以需要使用 _safe 后缀的 for_each 宏,释放时需要先释放成员 value,再释放节点 (回想一下 C++ 的析构函数),可以直接使用 q_release_element 函数。 q_free 在遍历时需要释放当前节点所在元素的空间,所以需要使用 list_for_each_entry_safe,而 q_size 无需在遍历时修改当前节点,所以使用 list_for_each 就足够了。 ","date":"2024-02-19","objectID":"/posts/linux2023-lab0/:2:2","tags":["Sysprog","Linux","C"],"title":"Linux 核心设计: C Programming Lab","uri":"/posts/linux2023-lab0/"},{"categories":["Linux Kernel Internals"],"content":"q_insert \u0026 q_remove insert 时需要特判 head 是否为 NULL 以及 malloc 分配是否成功,接下来需要使用 strdup 对所给参数进行复制 (strdup 内部是通过 malloc 来实现的,所以之前 q_free 时也需要是否 value),最后根据插入的位置调用 list_add 或 list_add_tail 进行插入。 remove 时需要特判 head 是否为 NULL 以及 queue 是否为空,接下来根据需要 remove 的节点调用 list_first_entry 或 list_last_entry 获取节点对应的元素,通过 list_del_init 来清除出 queue,最后如果 value 字段不为 NULL,则通过 memcpy 将 value 字段对应的字符串复制到指定位置。 ","date":"2024-02-19","objectID":"/posts/linux2023-lab0/:2:3","tags":["Sysprog","Linux","C"],"title":"Linux 核心设计: C Programming Lab","uri":"/posts/linux2023-lab0/"},{"categories":["Linux Kernel Internals"],"content":"Valgrind 2007 年的论文: Valgrind: A Framework for Heavyweight Dynamic Binary Instrumentation 繁体中文版本的 论文导读 memory lost: definitely lost indirectly lost possibly lost still readchable 运行 valgrind 和 gdb 类似,都需要使用 -g 参数来编译 C/C++ 源程序以生成调试信息,然后还可以通过 -q 参数指示 valgrind 进入 quite 模式,减少启动时信息的输出。 $ valgrind -q --leak-check=full ./case1 --leak-check=full: 启用全面的内存泄漏检查,valgrind 将会报告所有的内存泄漏情况,包括详细的堆栈跟踪信息 --show-possibly-lost=no: 不输出 possibly lost 相关报告 --track-fds=yes: 侦测 file descriptor 开了没关的情况 valgrind 输出的报告 invalid write/read 这类的单位是 Byte,即 size of X (bytes) 程序运行时内存布局: 信息 Valgrind User Manual Massif: a heap profiler ","date":"2024-02-19","objectID":"/posts/linux2023-lab0/:3:0","tags":["Sysprog","Linux","C"],"title":"Linux 核心设计: C Programming Lab","uri":"/posts/linux2023-lab0/"},{"categories":["Linux Kernel Internals"],"content":"自动测试 ","date":"2024-02-19","objectID":"/posts/linux2023-lab0/:4:0","tags":["Sysprog","Linux","C"],"title":"Linux 核心设计: C Programming Lab","uri":"/posts/linux2023-lab0/"},{"categories":["Linux Kernel Internals"],"content":"追踪内存的分配和释放 Wikipedia: Hooking Wikipedia: Test harness GCC: Arrays of Length Zero The alignment of a zero-length array is the same as the alignment of its elements. 相关源代码阅读 (harness.h, harness.c): test_malloc() test_free() test_calloc() find_footer() find_header() ","date":"2024-02-19","objectID":"/posts/linux2023-lab0/:4:1","tags":["Sysprog","Linux","C"],"title":"Linux 核心设计: C Programming Lab","uri":"/posts/linux2023-lab0/"},{"categories":["Linux Kernel Internals"],"content":"qtest 命令解释器 新增指令 hello,用于打印 Hello, world\" 的信息。调用流程: main → run_console → cmd_select → interpret_cmd → parse_args → interpret_cmda → do_hello 相关源代码阅读 (console.h, console.c): init_cmd() ADD_COMMADN add_cmd() ","date":"2024-02-19","objectID":"/posts/linux2023-lab0/:4:2","tags":["Sysprog","Linux","C"],"title":"Linux 核心设计: C Programming Lab","uri":"/posts/linux2023-lab0/"},{"categories":["Linux Kernel Internals"],"content":"Signal signal(2) — Linux manual page signal() sets the disposition of the signal signum to handler, which is either SIG_IGN, SIG_DFL, or the address of a programmer-defined func‐ tion (a “signal handler”). setjmp(3) — Linux manual page The functions described on this page are used for performing “nonlocal gotos”: transferring execution from one function to a predetermined location in another function. The setjmp() function dynamically establishes the target to which control will later be transferred, and longjmp() performs the transfer of execution. Why use sigsetjmp()/siglongjmp() instead of setjmp()/longjmp()? The Linux Programming Interface The sa_mask field allows us to specify a set of signals that aren’t permitted to interrupt execution of this handler. In addition, the signal that caused the handler to be invoked is automatically added to the process signal mask. This means that a signal handler won’t recursively interrupt itself if a second instance of the same signal arrives while the handler is executing. However, there is a problem with using the standard longjmp() function to exit from a signal handler. We noted earlier that, upon entry to the signal handler, the kernel automatically adds the invoking signal, as well as any signals specified in the act.sa_mask field, to the process signal mask, and then removes these signals from the mask when the handler does a normal return. What happens to the signal mask if we exit the signal handler using longjmp()? The answer depends on the genealogy of the particular UNIX implementation. jmp_ready 技巧 (用于保证在 siglongjmp() 之前必然执行过一次 sigsetjmp()): Because a signal can be generated at any time, it may actually occur before the target of the goto has been set up by sigsetjmp() (or setjmp()). To prevent this possibility (which would cause the handler to perform a nonlocal goto using an uninitialized env buffer), we employ a guard variable, canJump, to indicate whether the env buffer has been initialized. If canJump is false, then instead of doing a nonlocal goto, the handler simply returns. 相关源代码阅读: qtest.c q_init() sigsegv_handler() sigalrm_handler() harness.c trigger_exception() exception_setup() ","date":"2024-02-19","objectID":"/posts/linux2023-lab0/:4:3","tags":["Sysprog","Linux","C"],"title":"Linux 核心设计: C Programming Lab","uri":"/posts/linux2023-lab0/"},{"categories":["Rust","Network"],"content":" In this stream, we started implementing the ubiquitous TCP protocol that underlies much of the traffic on the internet! In particular, we followed RFC 793 — https://tools.ietf.org/html/rfc793 — which describes the original protocol, with the goal of being able to set up and tear down a connection with a “real” TCP stack at the other end (netcat in particular). We’re writing it using a user-space networking interface (see https://www.kernel.org/doc/Documentation/networking/tuntap.txt and the Rust bindings at https://docs.rs/tun-tap/). 整理自 John Gjengset 的影片: Part 1 ","date":"2024-02-17","objectID":"/posts/rust-tcp/:0:0","tags":["Rust","TCP","Network"],"title":"Impl Rust: TCP/IP","uri":"/posts/rust-tcp/"},{"categories":["Rust","Network"],"content":"影片注解 Part 1 ","date":"2024-02-17","objectID":"/posts/rust-tcp/:1:0","tags":["Rust","TCP","Network"],"title":"Impl Rust: TCP/IP","uri":"/posts/rust-tcp/"},{"categories":["Rust","Network"],"content":"Raw socket vs TUN/TAP device Raw sockets [Wikipedia] TUN/TAP [Wikipedia] Raw socket vs TUN device [Stack Overflow] Universal TUN/TAP device driver [Linux kernel documentation] Raw socket: Internet –\u003e NIC –\u003e kernel –\u003e user space Internet \u003c– NIC \u003c– kernel \u003c– user space Host interact with other hosts in Internet. TUN/TAP device: kernel –\u003e | TUN/TAP | –\u003e user space kernel \u003c– | TUN/TAP | \u003c– user space Kernel interact with programs in user space in the same host. 和其他物理网卡一样,用户进程创建的 TUN/TAP 设备仍然是被 kernel 所拥有的 (kernel 可以使用设备进行发送/接收),只不过用户进程也可以像操作 管道 (pipe) 那样,操作所创建的 TUN/TAP 设备 (可以使用该设备进行发送/接收),从而与 kernel 的物理网卡进行通信。 Universal TUN/TAP device driver [Linux kernel documentation] 3.2 Frame format: If flag IFF_NO_PI is not set each frame format is: Flags [2 bytes] Proto [2 bytes] Raw protocol(IP, IPv6, etc) frame. 通过 TUN/TAP 设备接收的封包,会拥有 Flags 和 Proto 这两个字段 (共 4 个字节,这也是 iface 的 without_packet_info 和 recv 方法所描述的 prepended packet info),然后才是原始协议的 frame。其中的 Proto 字段是 EtherType [Wikipedia],可以根据里面的 values 来判断接受封包的协议类型 (0x0800 表示 IPv4,0x86DD 表示 IPv6)。 ","date":"2024-02-17","objectID":"/posts/rust-tcp/:1:1","tags":["Rust","TCP","Network"],"title":"Impl Rust: TCP/IP","uri":"/posts/rust-tcp/"},{"categories":["Rust","Network"],"content":"setcap setcap [Linux manual page] cap_from_text [Linux manual page] 因为 TUN/TAP 是由 kernel 提供的,所以需要赋予我们项目的可执行文件权限,使它能访问我们创建的 TUN/TAP 设备 (为了简单起见,下面只列出 release 版本的方法,debug 版本的方法类似)。 # 编译 $ cargo build --release # 设置文件权限 $ sudo setcap cap_net_admin=eip target/release/trust # 运行 $ cargo run --release 在另一终端输入命令 ip addr 就可以看到此时会多出一个名为 tun0 的设备,这正是我们创建的 TUN 设备。 ip-address [Linux manual page] ip-link [Linux manual page] 在另一个终端中输入: # 列出当前所有的网络设备 $ ip addr # 配置设备 tun0 的 IP 地址 $ sudo ip addr add 192.168.0.1/24 dev tun0 # 启动设备 tun0 $ sudo ip link set up dev tun0 每次编译后都需要执行一遍这个流程 (因为重新编译生成的可执行文件需要重新设置权限),我们将这些流程的逻辑写成一个脚本 run.sh。这个脚本为了输出的美观性增加了额外逻辑,例如将 trust 放在后台执行,将脚本设置为等待 trust 执行完成后才结束执行。 ","date":"2024-02-17","objectID":"/posts/rust-tcp/:1:2","tags":["Rust","TCP","Network"],"title":"Impl Rust: TCP/IP","uri":"/posts/rust-tcp/"},{"categories":["Rust","Network"],"content":"Endianness Endianness [Wikipedia] Why is network-byte-order defined to be big-endian? [Stack Overflow] Rust 提供了 Trait std::simd::ToBytes 用于大小端字节序之间的相互转换,方法 from_be_bytes 是将大端字节序的一系列字节转换成对应表示的数值。 ","date":"2024-02-17","objectID":"/posts/rust-tcp/:1:3","tags":["Rust","TCP","Network"],"title":"Impl Rust: TCP/IP","uri":"/posts/rust-tcp/"},{"categories":["Rust","Network"],"content":"IP 因为 TUN 只是在 Network layer 的虚拟设备 (TAP 则是 Data link layer 层),所以需要手动解析 IP 封包。 RFC 791 3.1. Internet Header Format List of IP protocol numbers [Wikipedia] 可以按照上面的格式来解析封包头,也可以引入 Crate etherparse 来解析 IP 封包头。 ping 命令使用的是 Network layer 上的 ICMP 协议,可以用于测试 TUN 是否成功配置并能接收封包。 $ ping -I tun0 192.168.0.2 ping (networking utility) [Wikipedia] ping [Linux man page] nc 命令用于发送 TCP 封包 $ nc 192.168.0.2 80 nc [Linux man page] 注意 ping, nc 这些命令使用的都是 kernel 的协议栈来实现,所以在创建虚拟设备 tun0 之后,使用以上 ping, nc 命令表示 kernel 发送相应的 ICMP, TCP 封包给创建 tun0 的进程 (process)。 可以使用 tshark (Terminal Wireshark) 工具来抓包,配合 ping,nc 命令可以分析 tun0 的封包传送。 $ sudo apt install tshark $ sudo tshark -i tun0 Wireshark [Wikipedia] tshark [Manual Page] ","date":"2024-02-17","objectID":"/posts/rust-tcp/:1:4","tags":["Rust","TCP","Network"],"title":"Impl Rust: TCP/IP","uri":"/posts/rust-tcp/"},{"categories":["Rust","Network"],"content":"TCP [RFC 793] 3.2 Terminology The state diagram in figure 6 illustrates only state changes, together with the causing events and resulting actions, but addresses neither error conditions nor actions which are not connected with state changes. 这里面提到的 Figure 6. TCP Connection State Diagram 在其中我们可以看到 TCP 的状态转换,非常有利于直观理解 TCP 建立连接时的三次握手过程。 警告 NOTE BENE: this diagram is only a summary and must not be taken as the total specification. Time to live [Wikipedia] In the IPv4 header, TTL is the 9th octet of 20. In the IPv6 header, it is the 8th octet of 40. The maximum TTL value is 255, the maximum value of a single octet. A recommended initial value is 64. SND.WL1 and SND.WL2 Note that SND.WND is an offset from SND.UNA, that SND.WL1 records the sequence number of the last segment used to update SND.WND, and that SND.WL2 records the acknowledgment number of the last segment used to update SND.WND. The check here prevents using old segments to update the window. ","date":"2024-02-17","objectID":"/posts/rust-tcp/:1:5","tags":["Rust","TCP","Network"],"title":"Impl Rust: TCP/IP","uri":"/posts/rust-tcp/"},{"categories":["Rust","Network"],"content":"Documentations 这里列举视频中一些概念相关的 documentation 学习的一手资料是官方文档,请务必自主学会阅读规格书之类的资料 ","date":"2024-02-17","objectID":"/posts/rust-tcp/:2:0","tags":["Rust","TCP","Network"],"title":"Impl Rust: TCP/IP","uri":"/posts/rust-tcp/"},{"categories":["Rust","Network"],"content":"Crate std Module std::io Type Alias std::io::Result Module std::collections::hash_map method std::collections::hash_map::HashMap::entry method std::collections::hash_map::Entry::or_default Trait std::default::Default Module std::net Macro std::eprintln method std::result::Result::expect method u16::from_be_bytes ","date":"2024-02-17","objectID":"/posts/rust-tcp/:2:1","tags":["Rust","TCP","Network"],"title":"Impl Rust: TCP/IP","uri":"/posts/rust-tcp/"},{"categories":["Rust","Network"],"content":"Crate tun_tap Enum tun_tap::Mode ","date":"2024-02-17","objectID":"/posts/rust-tcp/:2:2","tags":["Rust","TCP","Network"],"title":"Impl Rust: TCP/IP","uri":"/posts/rust-tcp/"},{"categories":["Rust","Network"],"content":"Crate etherparse Struct etherparse::Ipv4HeaderSlice Struct etherparse::Ipv4Header Struct etherparse::TcpHeaderSlice Struct etherparse::TcpHeader ","date":"2024-02-17","objectID":"/posts/rust-tcp/:2:3","tags":["Rust","TCP","Network"],"title":"Impl Rust: TCP/IP","uri":"/posts/rust-tcp/"},{"categories":["Rust","Network"],"content":"References https://datatracker.ietf.org/doc/html/rfc793 https://datatracker.ietf.org/doc/html/rfc1122 https://datatracker.ietf.org/doc/html/rfc7414#section-2 https://datatracker.ietf.org/doc/html/rfc2398 https://datatracker.ietf.org/doc/html/rfc2525 https://datatracker.ietf.org/doc/html/rfc791 https://www.saminiir.com/lets-code-tcp-ip-stack-3-tcp-handshake/ https://www.saminiir.com/lets-code-tcp-ip-stack-4-tcp-data-flow-socket-api/ https://www.saminiir.com/lets-code-tcp-ip-stack-5-tcp-retransmission/ 注意 RFC 793 描述了原始的 TCP 协议的内容 (重点阅读 3.FUNCTIONAL SPECIFICATION ) RFC 1122 则是对原始的 TCP 功能的一些扩展进行说明 RFC 7414 的 Section 2 则对 TCP 的核心功能进行了简要描述 RFC 2398 描述了对实现的 TCP 的一些测试方法和工具 RFC 2525 说明了在实现 TCP 过程中可能会出现的错误,并指出可能导致错误的潜在问题 RFC 791 描述了 IP 协议 的内容 最后 3 篇博客介绍了 TCP 协议相关术语和概念,可以搭配 RFC 793 阅读 ","date":"2024-02-17","objectID":"/posts/rust-tcp/:3:0","tags":["Rust","TCP","Network"],"title":"Impl Rust: TCP/IP","uri":"/posts/rust-tcp/"},{"categories":["Linux Kernel Internals"],"content":"Source ","date":"2024-02-16","objectID":"/posts/linux-quiz1/:0:0","tags":["Sysprog","Linux","C"],"title":"Linux 核心设计: 第 1 周测验题 linked list","uri":"/posts/linux-quiz1/"},{"categories":["Linux Kernel Internals"],"content":"2018q1 第 4 週測驗題 ","date":"2024-02-16","objectID":"/posts/linux-quiz1/:1:0","tags":["Sysprog","Linux","C"],"title":"Linux 核心设计: 第 1 周测验题 linked list","uri":"/posts/linux-quiz1/"},{"categories":["Linux Kernel Internals"],"content":"测验 1 FuncA 的作用是 (e) 建立新節點,內容是 value,並安插在結尾 FuncB 的作用是 (d) 建立新節點,內容是 value,並安插在開頭 FuncC 的作用是 (e) 找到節點內容為 value2 的節點,並在之後插入新節點,內容為 value1 在 main 函数调用 display 函数之前,链表分布为: 48 -\u003e 51 -\u003e 63 -\u003e 72 -\u003e 86 在程式輸出中,訊息 Traversal in forward direction 後依序印出哪幾個數字呢? (d) 48 (c) 51 (a) 63 (e) 72 (b) 86 在程式輸出中,訊息 Traversal in reverse direction 後依序印出哪幾個數字呢? (b) 86 (e) 72 (a) 63 (c) 51 (d) 48 技巧 延伸題目: 在上述 doubly-linked list 實作氣泡排序和合併排序,並提出需要額外實作哪些函示才足以達成目標 引入統計模型,隨機新增和刪除節點,然後評估上述合併排序程式的時間複雜度和效能分佈 (需要製圖和數學分析) ","date":"2024-02-16","objectID":"/posts/linux-quiz1/:1:1","tags":["Sysprog","Linux","C"],"title":"Linux 核心设计: 第 1 周测验题 linked list","uri":"/posts/linux-quiz1/"},{"categories":["Linux Kernel Internals"],"content":"测验 2 FuncX 的作用是 (涵蓋程式執行行為的正確描述最多者) (f) 判斷是否為 circular linked list,若為 circular 則回傳 0,其他非零值,過程中計算走訪的節點總數 K1 » 後面接的輸出為何 (b) Yes K2 » 後面接的輸出為何 (a) No K3 » 後面接的輸出為何 (a) No K4 » 後面接的輸出為何 (a) No K5 » 後面接的輸出為何 (f) 0 count » 後面接的輸出為何 (f) 0 ","date":"2024-02-16","objectID":"/posts/linux-quiz1/:1:2","tags":["Sysprog","Linux","C"],"title":"Linux 核心设计: 第 1 周测验题 linked list","uri":"/posts/linux-quiz1/"},{"categories":["Linux Kernel Internals"],"content":"2020q1 第 1 週測驗題 ","date":"2024-02-16","objectID":"/posts/linux-quiz1/:2:0","tags":["Sysprog","Linux","C"],"title":"Linux 核心设计: 第 1 周测验题 linked list","uri":"/posts/linux-quiz1/"},{"categories":["Linux Kernel Internals"],"content":"测验 1 本题使用的是单向 linked list typedef struct __list { int data; struct __list *next; } list; 一开始的 if 语句用于判断 start 是否为 NULL 或是否只有一个节点,如果是则直接返回无需排序 接下来使用 mergesort 来对 linked list 进行从小到大排序,并且每次左侧链表只划分一个节点,剩余节点全部划为右侧链表 list *left = start; list *right = left-\u003enext; left-\u003enext = NULL; // LL0; 再来就是归并操作,将 left 和 right 进行归并,如果 merge 为 NULL,则将对应的节点赋值给它和 start,否则需要迭代 left 或 right 以及 merge 以完成归并操作 for (list *merge = NULL; left || right; ) { if (!right || (left \u0026\u0026 left-\u003edata \u003c right-\u003edata)) { if (!merge) { start = merge = left; // LL1; } else { merge-\u003enext = left; // LL2; merge = merge-\u003enext; } left = left-\u003enext; // LL3; } else { if (!merge) { start = merge = right; // LL4; } else { merge-\u003enext = right; // LL5; merge = merge-\u003enext; } right = right-\u003enext; // LL6; } } 技巧 延伸問題: 解釋上述程式運作原理; 指出程式改進空間,特別是考慮到 Optimizing merge sort; 將上述 singly-linked list 擴充為 circular doubly-linked list 並重新實作對應的 sort; 依循 Linux 核心 include/linux/list.h 程式碼的方式,改寫上述排序程式; 嘗試將原本遞迴的程式改寫為 iterative 版本; ","date":"2024-02-16","objectID":"/posts/linux-quiz1/:2:1","tags":["Sysprog","Linux","C"],"title":"Linux 核心设计: 第 1 周测验题 linked list","uri":"/posts/linux-quiz1/"},{"categories":["Linux","Linux Kernel Internals"],"content":" 面對原始程式碼超越 3 千萬行規模的 Linux 核心 (2023 年),最令人感到挫折的,絕非缺乏程式註解,而是就算見到滿滿的註解,自己卻有如文盲,全然無從理解起。為什麼呢?往往是因為對作業系統的認知太侷限。 原文地址 ","date":"2024-02-15","objectID":"/posts/linux-concepts/:0:0","tags":["Sysprog","Linux"],"title":"Linux 核心设计: 操作系统术语及概念","uri":"/posts/linux-concepts/"},{"categories":["Linux","Linux Kernel Internals"],"content":"Linux 核心发展 虚拟化 (Virtualization) 技术分为 CPU 层级的虚拟化技术,例如 KVM 和 RVM,也有操作系统层级的虚拟化技术,例如 Docker。 Plan 9 from Bell Labs [Wikipedia] LXC [Wikipedia] 信息 從 Revolution OS 看作業系統生態變化 Linux 核心設計: 透過 eBPF 觀察作業系統行為 ","date":"2024-02-15","objectID":"/posts/linux-concepts/:1:0","tags":["Sysprog","Linux"],"title":"Linux 核心设计: 操作系统术语及概念","uri":"/posts/linux-concepts/"},{"categories":["Linux","Linux Kernel Internals"],"content":"看漫画学 Linux 原文地址 inside the linux kernel 整理上图,可以得到 自底向上 的 Linux 系统结构: 地下层: 文件系统 (File System) 中央大厅层: 进程表 (process table) 内存管理 (memory management) 信息安全 (security) 看门狗 (watchdog) httpd cron 管道 (pipe) FTP SSH Wine GNOME 最上层 tty / terminal wiki: Pipeline (Unix) [Wikipedia] Process identifier [Wikipedia] watchdog [Linux man page] init [Wikipedia] systemd [Wikipedia] fork [Linux man page] clone [Linux man page] Project Genie [Wikipedia] posix_spawn [Linux man page] Native POSIX Thread Library [Wikipedia] 极客漫画: 不要使用 SIGKILL 的原因 wait [Linux man page] signal [Linux man page] TUX web server [Wikipedia] -[x] cron 技巧 Multics 采用了当时背景下的几乎所有的先进技术,可以参考该系统获取系统领域的灵感。 虚拟内存管理与现代银行的运行逻辑类似,通过 malloc 分配的有效虚拟地址并不能保证真正可用,类似于支票得去银行兑现时才知道银行真正的现金储备。但是根据统计学公式,虚拟地址和银行现金可以保证在大部分情况下,都可以满足需求,当然突发的大规模虚拟内存使用、现金兑现时就无法保证了。这部分的原理推导需要学习概率论、统计学等数理课程。 信息 Linux 核心设计: Linux 核心設計: 檔案系統概念及實作手法 Linux 核心設計: 不僅是個執行單元的 Process Linux 核心設計: 不只挑選任務的排程器 UNIX 作業系統 fork/exec 系統呼叫的前世今生 Linux 核心設計: 記憶體管理 Linux 核心設計: 發展動態回顧 Linux 核心設計: 針對事件驅動的 I/O 模型演化 Linux 核心設計: Scalability 議題 Effective System Call Aggregation (ESCA) 你所不知道的 C 語言: Stream I/O, EOF 和例外處理 Unix-like 工具使用技巧: Mastering UNIX pipes, Part 1 Mastering UNIX pipes, Part 2 ","date":"2024-02-15","objectID":"/posts/linux-concepts/:2:0","tags":["Sysprog","Linux"],"title":"Linux 核心设计: 操作系统术语及概念","uri":"/posts/linux-concepts/"},{"categories":["Linux","Linux Kernel Internals"],"content":"高阶观点 投影片: Linux Kernel: Introduction ✅ 对投影片的 重点描述 一些概念理解: 1963 Timesharing: A Solution to Computer Bottlenecks [YouTube] Supervisory program [Wikipedia] ","date":"2024-02-15","objectID":"/posts/linux-concepts/:3:0","tags":["Sysprog","Linux"],"title":"Linux 核心设计: 操作系统术语及概念","uri":"/posts/linux-concepts/"},{"categories":["Linux","Linux Kernel Internals"],"content":"Monolithic kernel vs Microkernel 淺談 Microkernel 設計和真實世界中的應用 Hybrid kernel [wikipedia] “As to the whole ‘hybrid kernel’ thing - it’s just marketing. It’s ‘oh, those microkernels had good PR, how can we try to get good PR for our working kernel? Oh, I know, let’s use a cool name and try to imply that it has all the PR advantages that that other system has’.” —— Linus Torvalds ","date":"2024-02-15","objectID":"/posts/linux-concepts/:3:1","tags":["Sysprog","Linux"],"title":"Linux 核心设计: 操作系统术语及概念","uri":"/posts/linux-concepts/"},{"categories":["Linux","Linux Kernel Internals"],"content":"虚拟化 MicroVM 和 Unikernel 都是使用 CPU 层级的虚拟化技术,在 Host OS 上面构建的 GuestOS: MicroVM 会减少硬件驱动方面的初始化,从而加快启动和服务速度 (在云服务器方面很常见,服务器端并不需要进行硬件驱动)。 Unikernel 则更激进,将 programs 和 kernel 一起进行动态编译,并且限制只能运行一个 process (例如只运行一个数据库进程,这样云服务器很常见),这样就减少了一些系统调用的呼叫,例如 fork (因为只能运行一个 process),提升了安全性 (因为 fork 系统调用可能会造成一些漏洞)。Unikernel 又叫 Library OS,可以理解为分时多人多工操作系统的另一个对立面,拥有极高的运行速度 (因为只有一个 process)。 Container Sandbox 使用的是 OS 层级的虚拟化技术,即它是将一组进程隔离起来构建为容器,这样可能会导致这一组进程就耗尽了系统的资源,其他进程无法使用系统的资源。同时因为是进程级的隔离,所以安全性不及 CPU 层级的 MicroVM 和 Unikernel。 信息 相关演讲、录影: YouTube: Inside the Mac OS X Kernel YouTube: What Are MicroVMs? And Why Should I Care? YouTube: From the Ground Up: How We Built the Nanos Unikernel 相关论文阅读: ","date":"2024-02-15","objectID":"/posts/linux-concepts/:3:2","tags":["Sysprog","Linux"],"title":"Linux 核心设计: 操作系统术语及概念","uri":"/posts/linux-concepts/"},{"categories":["Linux","Linux Kernel Internals"],"content":"Scalability Wikipedia: scalability A system whose performance improves after adding hardware, proportionally to the capacity added, is said to be a scalable system. lock-free sequence lock RCU algorithm complexity ","date":"2024-02-15","objectID":"/posts/linux-concepts/:3:3","tags":["Sysprog","Linux"],"title":"Linux 核心设计: 操作系统术语及概念","uri":"/posts/linux-concepts/"},{"categories":["Linux","Linux Kernel Internals"],"content":"eBPF 透过 eBPF 可将 Monolithic kernel 的 Linux 取得 microkernel 的特性 The Beginners Guide to eBPF Programming, Liza RIce (live programming + source code) A thorough introduction to eBPF (four articles in lwn.net), Matt FLeming, December 2017 ","date":"2024-02-15","objectID":"/posts/linux-concepts/:3:4","tags":["Sysprog","Linux"],"title":"Linux 核心设计: 操作系统术语及概念","uri":"/posts/linux-concepts/"},{"categories":["Linux","Linux Kernel Internals"],"content":"细节切入点 CPU 和 OS 的基本概念科普网站: Putting the “You” in CPU 相当于科普版 CSAPP 信息 UNSW COMP9242: Advanced Operating Systems (2023/T3) YouTube: 2022: UNSW’s COMP9242 Advanced Operating Systems 这门课可以作为辅助材料,讲得深入浅出,可以作为进阶材料阅读。 ","date":"2024-02-15","objectID":"/posts/linux-concepts/:4:0","tags":["Sysprog","Linux"],"title":"Linux 核心设计: 操作系统术语及概念","uri":"/posts/linux-concepts/"},{"categories":["Linux","Linux Kernel Internals"],"content":"系统软件开发思维 ","date":"2024-02-15","objectID":"/posts/linux-concepts/:5:0","tags":["Sysprog","Linux"],"title":"Linux 核心设计: 操作系统术语及概念","uri":"/posts/linux-concepts/"},{"categories":["Linux","Linux Kernel Internals"],"content":"Maslow’s pyramid of code review Maslow’s pyramid of code review ","date":"2024-02-15","objectID":"/posts/linux-concepts/:5:1","tags":["Sysprog","Linux"],"title":"Linux 核心设计: 操作系统术语及概念","uri":"/posts/linux-concepts/"},{"categories":["Linux","Linux Kernel Internals"],"content":"Benchmark / Profiling Benchmark / Profiling ","date":"2024-02-15","objectID":"/posts/linux-concepts/:5:2","tags":["Sysprog","Linux"],"title":"Linux 核心设计: 操作系统术语及概念","uri":"/posts/linux-concepts/"},{"categories":["Rust"],"content":" In this third Crust of Rust video, we cover iterators and trait bounds, by re-implementing the “flatten” Iterator method from the standard library. As part of that, we cover some of the weirder trait bounds that are required, including what’s needed to extend the implementation to support backwards iteration. 整理自 John Gjengset 的影片 ","date":"2024-02-05","objectID":"/posts/iterators/:0:0","tags":["Rust","Iterator"],"title":"Crust of Rust: Iterators","uri":"/posts/iterators/"},{"categories":["Rust"],"content":"影片注解 ","date":"2024-02-05","objectID":"/posts/iterators/:1:0","tags":["Rust","Iterator"],"title":"Crust of Rust: Iterators","uri":"/posts/iterators/"},{"categories":["Rust"],"content":"Generic traits vs associated types trait Iterator { type Item; fn next(\u0026mut self) -\u003e Option\u003cSelf::Item\u003e; } trait Iterator\u003cItem\u003e { fn next(\u0026mut self) -\u003e Option\u003cSelf::Item\u003e; } 为什么使用上面的 associated type 而不是下面的 generic 来实现 Iterator?因为使用 generic 来实现的话,可以对一个类型实现多个 Iterator trait 例如 Iterator\u003ci32\u003e, Iterator\u003cf64,而从语言表达上讲,我们希望一个类型只能实现一个 Iterator trait,所以使用 associated type 来实现 Iterator trait,防止二义性。 for v in vs.iter() { // borrow vs, \u0026 to v } for v in \u0026vs { // equivalent to vs.iter() } 这两条 for 语句虽然效果一样,但是后者是使用 \u003c\u0026vs\u003e into_iter 讲 \u0026vs 转为 iterator,而不是调用 iter() 方法。 ","date":"2024-02-05","objectID":"/posts/iterators/:1:1","tags":["Rust","Iterator"],"title":"Crust of Rust: Iterators","uri":"/posts/iterators/"},{"categories":["Rust"],"content":"Iterator::flatten method std::iter::Iterator::flatten Creates an iterator that flattens nested structure. This is useful when you have an iterator of iterators or an iterator of things that can be turned into iterators and you want to remove one level of indirection. flatten() 的本质是将一种 Iterator 类型转换成另一种 Iterator 类型,所以调用者和返回值 Flatten 都满足 trait Iterator,因为都是迭代器,只是将原先的 n-level 压扁为 1-level 的 Iterator 了。录影视频里只考虑 2-level 的情况。 ","date":"2024-02-05","objectID":"/posts/iterators/:1:2","tags":["Rust","Iterator"],"title":"Crust of Rust: Iterators","uri":"/posts/iterators/"},{"categories":["Rust"],"content":"DoubleEndedIterator Trait std::iter::DoubleEndedIterator It is important to note that both back and forth work on the same range, and do not cross: iteration is over when they meet in the middle. 也就是说,back 和 front 的迭代器类似于双指针,但是这两个迭代器并不会越过对方。 ","date":"2024-02-05","objectID":"/posts/iterators/:1:3","tags":["Rust","Iterator"],"title":"Crust of Rust: Iterators","uri":"/posts/iterators/"},{"categories":["Rust"],"content":"Homework 信息 实作说明: 尝试实现 Iterator 的 flat_map 方法 (Github: My Implementation) 参考资料: method std::iter::Iterator::flat_map struct std::iter::FlatMap ","date":"2024-02-05","objectID":"/posts/iterators/:2:0","tags":["Rust","Iterator"],"title":"Crust of Rust: Iterators","uri":"/posts/iterators/"},{"categories":["Rust"],"content":"Documentations 这里列举视频中一些概念相关的 documentation 学习的一手资料是官方文档,请务必自主学会阅读规格书之类的资料 ","date":"2024-02-05","objectID":"/posts/iterators/:3:0","tags":["Rust","Iterator"],"title":"Crust of Rust: Iterators","uri":"/posts/iterators/"},{"categories":["Rust"],"content":"Crate std 可以使用这里提供的搜素栏进行搜索 (BTW 不要浪费时间在 Google 搜寻上!) Trait std::iter::Iterator method std::iter::Iterator::flatten method std::iter::Iterator::rev method std::iter::Iterator::flat_map Trait std::iter::IntoIterator Struct std::iter::Flatten function std::iter::empty Struct std::iter::Empty function std::iter::once Struct std::iter::Once Trait std::iter::DoubleEndedIterator Enum std::option::Option method std::option::Option::and_then method std::option::Option::as_mut Trait std::marker::Sized ","date":"2024-02-05","objectID":"/posts/iterators/:3:1","tags":["Rust","Iterator"],"title":"Crust of Rust: Iterators","uri":"/posts/iterators/"},{"categories":["Rust"],"content":"References What is the difference between iter and into_iter? [Stack Overflow] How to run a specific unit test in Rust? [Stack Overflow] How do I implement a trait with a generic method? [Stack Overflow] 可能不是你看过最无聊的 Rust 入门喜剧 102 (1) 闭包与迭代器 [bilibili] ","date":"2024-02-05","objectID":"/posts/iterators/:4:0","tags":["Rust","Iterator"],"title":"Crust of Rust: Iterators","uri":"/posts/iterators/"},{"categories":["C","Linux Kernel Internals"],"content":" 无论是操作系统核心、C 语言函数库内部、程序开发框架,到应用程序,都不难见到 linked list 的身影,包含多种针对性能和安全议题所做的 linked list 变形,又还要考虑应用程序的泛用性 (generic programming),是很好的进阶题材。 原文地址 ","date":"2024-02-03","objectID":"/posts/c-linked-list/:0:0","tags":["Sysprog","C","Linked List"],"title":"你所不知道的 C 语言: linked list 和非连续记忆体","uri":"/posts/c-linked-list/"},{"categories":["C","Linux Kernel Internals"],"content":"Linux 核心的艺术 YouTube: The mind behind Linux | Linus Torvalds | TED 事实上 special case 和 indirect pointer 这两种写法在 clang 的最佳优化下效能并没有什么区别,我们可以不使用 indirect pointer 来写程序,但是我们需要学习 indirect pointer 这种思维方式,即 good taste。 把握程序的本质,即本质上是修改指针的值,所以可以使用指针的指针来实现,无需进行特判。 在 Unix-like 的操作系统中,类型名带有后缀 _t 表示这个类型是由 typedef 定义的,而不是语言原生的类型名,e.g. typedef struct list_entry { int value; struct list_entry *next; } list_entry_t; ","date":"2024-02-03","objectID":"/posts/c-linked-list/:1:0","tags":["Sysprog","C","Linked List"],"title":"你所不知道的 C 语言: linked list 和非连续记忆体","uri":"/posts/c-linked-list/"},{"categories":["C","Linux Kernel Internals"],"content":"linked list append \u0026 remove Source 信息 The mind behind Linux Linus on Understanding Pointers ","date":"2024-02-03","objectID":"/posts/c-linked-list/:1:1","tags":["Sysprog","C","Linked List"],"title":"你所不知道的 C 语言: linked list 和非连续记忆体","uri":"/posts/c-linked-list/"},{"categories":["C","Linux Kernel Internals"],"content":"LeetCode Source LeetCode 21. Merge Two Sorted Lists LeetCode 23. Merge k Sorted Lists Leetcode 2095. Delete the Middle Node of a Linked List LeetCode 86. Partition List 注意 原文对于 LeetCode 23. Merge k Sorted Lists 给出了 3 种解法,其时间复杂度分别为: $O(m \\cdot n)$ $O(m \\cdot n)$ $O(m \\cdot logn)$ $n$ 为 listsSize,$m$ 为 merge linked list 过程中产生的 linked list 的最大长度。 如果你对第 3 种解法的时间复杂度感到疑惑,请参考 Josh Hug 在 CS61B 的 Merge Sort 复杂度讲解。 ","date":"2024-02-03","objectID":"/posts/c-linked-list/:1:2","tags":["Sysprog","C","Linked List"],"title":"你所不知道的 C 语言: linked list 和非连续记忆体","uri":"/posts/c-linked-list/"},{"categories":["C","Linux Kernel Internals"],"content":"Circular linked list 单向 linked list 相对于双向 linked list 的优势在于,一个 cache line 可以容纳更多的 list node,而且很容易进行反向查询,这弥补了反向查询时的效能差距。例如在 64 位处理器上,地址为 64 Bit 即 8 Byte,如果 list node 的数据域存放一个 2 Byte 的整数,那么一个单向的 list node 大小为 10 Byte,双向的则为 18 Byte,又因为一般的 cache line 的大小为 64 Byte,则对于单向的 node 来说,cache line 可以存放 $64 / 10 = 6$ 个 list node,但是仅能存放 $64 / 18 = 3$ 个 list node,cache 效率明显降低。 这部分内容可以参考 jserv 的讲座 \u003c現代處理器設計: Cache 原理和實際影響\u003e ","date":"2024-02-03","objectID":"/posts/c-linked-list/:2:0","tags":["Sysprog","C","Linked List"],"title":"你所不知道的 C 语言: linked list 和非连续记忆体","uri":"/posts/c-linked-list/"},{"categories":["C","Linux Kernel Internals"],"content":"Floyd’s Cycle detection 这个“龟兔赛跑”算法保证兔子在跑两次循环圈后,一定会和刚完成一次循环圈的乌龟相遇。因为已知乌龟每次移动一步,兔子每次移动两步,可以假设在相遇点处乌龟移动的 $X$ 步,则兔子移动了 $2X$ 步,$2X$ 必为偶数,所以兔子必能在移动了 $2X$ 步后与乌龟相遇,不会出现兔子因为每次移动两步而刚好越过乌龟一步的情况。 $\\lambda$ is the length of the loop to be found, $\\mu$ is the index of the first element of the cycle. Source LeetCode 141. Linked List Cycle LeetCode 142. Linked List Cycle II LeetCode 146. LRU Cache 金刀的算法小册子 Linked List 专题 LeetCode 206. Reverse Linked List 信息 探索 Floyd Cycle Detection Algorithm ","date":"2024-02-03","objectID":"/posts/c-linked-list/:2:1","tags":["Sysprog","C","Linked List"],"title":"你所不知道的 C 语言: linked list 和非连续记忆体","uri":"/posts/c-linked-list/"},{"categories":["C","Linux Kernel Internals"],"content":"Merge Sort 实现了 recursion, non-recursion 的 merge sort Source 技巧 Merge Sort 与它的变化 不论是这里的 non-recursion 版本的 merge sort,还是后面的 non-recursion 版本的 quick sort,本质上都是通过模拟栈 (stack) 操作来实现的,关于这个模拟 stack 方法,可以参考蒋炎岩老师的录影 应用视角的操作系统 (程序的状态机模型;编译优化)。 ","date":"2024-02-03","objectID":"/posts/c-linked-list/:3:0","tags":["Sysprog","C","Linked List"],"title":"你所不知道的 C 语言: linked list 和非连续记忆体","uri":"/posts/c-linked-list/"},{"categories":["C","Linux Kernel Internals"],"content":"Linux 核心的 linked list Linux 核心使用的 linked list 是通过 Intrusive linked lists 搭配 contain_of 宏,来实现自定义的 linked list node。 sysprog21/linux-list 这个仓库将 Linux kernel 中 linked list 部分抽离出来,并改写为 user mode 的实作。本人对该仓库进行了一些改写,对 insert sort 和 quick sort 增加了 makefile 支持。 上面的仓库与 Linux kernel 的实作差异主要在于 WRITE_ONCE 宏。WRITE_ONCE 的原理简单来说是,通过 union 产生两个引用同一地址的引用 (即 __val 和 __c),然后因为对同一地址有多个引用,所以编译器进行最佳化时不会过于激进的重排序,从而达到顺序执行效果。 Source ","date":"2024-02-03","objectID":"/posts/c-linked-list/:4:0","tags":["Sysprog","C","Linked List"],"title":"你所不知道的 C 语言: linked list 和非连续记忆体","uri":"/posts/c-linked-list/"},{"categories":["C","Linux Kernel Internals"],"content":"Intrusive linked lists Intrusive linked lists 这篇文章对于 Intrusive linked list 说明的非常好,解释了其在 memory allocations 和 cache thrashing 的优势,还搭配 Linux kernel 讲解了场景应用。 ","date":"2024-02-03","objectID":"/posts/c-linked-list/:4:1","tags":["Sysprog","C","Linked List"],"title":"你所不知道的 C 语言: linked list 和非连续记忆体","uri":"/posts/c-linked-list/"},{"categories":["C","Linux Kernel Internals"],"content":"container_of Linux 核心原始程式碼巨集: container_of ","date":"2024-02-03","objectID":"/posts/c-linked-list/:4:2","tags":["Sysprog","C","Linked List"],"title":"你所不知道的 C 语言: linked list 和非连续记忆体","uri":"/posts/c-linked-list/"},{"categories":["C","Linux Kernel Internals"],"content":"Optimized QuickSort Optimized QuickSort: C Implementation (Non-Recursive) 这篇文章介绍了 non-recursion 的 quick sort 在 array 上的实作,参考该文章完成 linked list 上的 non-recursion 版本的 quick sort 实作。 非递归的快速排序中 if (L != R \u0026\u0026 \u0026begin[i]-\u003elist != head) { 其中的 \u0026begin[i]-\u003elist != head 条件判断用于空链表情况,数组版本中使用的是下标比较 L \u003c R 来判断,但是链表中使用 L != R 不足以完全表示 L \u003c R 这个条件,还需要 \u0026begin[i]-\u003elist != head 来判断链表是否为空。 ","date":"2024-02-03","objectID":"/posts/c-linked-list/:4:3","tags":["Sysprog","C","Linked List"],"title":"你所不知道的 C 语言: linked list 和非连续记忆体","uri":"/posts/c-linked-list/"},{"categories":["C","Linux Kernel Internals"],"content":"Linux 核心的 list_sort 实作 linux/list_sort.c 先将双向循环链表转换成单向链表,然后利用链表节点的 prev 来挂载 pending list (因为单向链表中 prev 没有作用,但是链表节点仍然存在 prev 字段,所以进行充分利用)。 假设 count 对应的 bits 第 k 个 bit 值为 0 且 $\u003e k$ 的 bits 都为 0,$\u003c k$ 的 bits 都为 1,则 $\u003c k $ 的这些 1 可以表示 pending list 中分别有 $2^{k-1}, 2^{k-2}, …, 2^0$ 大小的 list 各一个。 如果第 k 个 bit 值为 0 且 $\u003e k$ 的 bits 中存在值为 1 的 bit,$\u003c k$ 的 bits 均为 1,则只有 $\u003c k$ 的 bits 可以表示 pending list 中分别有 $2^{k-1}, 2^{k-2}, …, 2^0$ 大小的 list 各一个,\u003e k 的 1 表示需要进行 merge 以获得对应大小的 list。 这样也刚好能使得 merge 时是 $2: 1$ 的长度比例,因为 2 的指数之间的比例是 $2: 1$。 技巧 这部分内容在 Lab0: Linux 核心的链表排序 中有更详细的解释和讨论。 信息 List, HList, and Hash Table hash table What is the strict aliasing rule? [Stack Overflow] Unions and type-punning [Stack Overflow] Nine ways to break your systems code using volatile [Stack Overflow] WRITE_ONCE in linux kernel lists [Stack Overflow] lib/list_sort: Optimize number of calls to comparison function ","date":"2024-02-03","objectID":"/posts/c-linked-list/:4:4","tags":["Sysprog","C","Linked List"],"title":"你所不知道的 C 语言: linked list 和非连续记忆体","uri":"/posts/c-linked-list/"},{"categories":["C","Linux Kernel Internals"],"content":"Fisher–Yates shuffle Wikipedia Fisher–Yates shuffle The Fisher–Yates shuffle is an algorithm for shuffling a finite sequence. 原文所说的事件复杂度,是考虑关于构造结果链表时的复杂度,并不考虑寻找指定节点的复杂度,所以对于原始方法复杂度为 $1 + 2 + … + n = O(n^2)$,对于 modern method 复杂度为 $1 + 1 + … + 1 = O(n)$ 原文实作虽然使用了 pointer to pointer,但是使用上并没有体现 linus 所说的 good taste,重新实作如下: void shuffle(node_t **head) { srand(time(NULL)); // First, we have to know how long is the linked list int len = 0; node_t **indirect = head; while (*indirect) { len++; indirect = \u0026(*indirect)-\u003enext; } // Append shuffling result to another linked list node_t *new = NULL; node_t **new_tail = \u0026new; while (len) { int random = rand() % len; indirect = head; while (random--) indirect = \u0026(*indirect)-\u003enext; node_t *tmp = *indirect; *indirect = (*indirect)-\u003enext; tmp-\u003enext = NULL; *new_tail = tmp; new_tail = \u0026(*new_tail)-\u003enext; len--; } *head = new; } 主要是修改了新链表 new 那一部分,只需要一个 pointer to pinter new_tail 就可以避免条件判断。 ","date":"2024-02-03","objectID":"/posts/c-linked-list/:5:0","tags":["Sysprog","C","Linked List"],"title":"你所不知道的 C 语言: linked list 和非连续记忆体","uri":"/posts/c-linked-list/"},{"categories":["Rust"],"content":" In this second Crust of Rust video, we cover declarative macros, macro_rules!, by re-implementing the vec! macro from the standard library. As part of that, we cover not only how to write these, but some of the gotchas and tricks you’ll run into, and some common use-cases. 整理自 John Gjengset 的影片 ","date":"2024-01-31","objectID":"/posts/declarative-macros/:0:0","tags":["Rust","Macro","Declarative Macros"],"title":"Crust of Rust: Declarative Macros","uri":"/posts/declarative-macros/"},{"categories":["Rust"],"content":"影片注解 ","date":"2024-01-31","objectID":"/posts/declarative-macros/:1:0","tags":["Rust","Macro","Declarative Macros"],"title":"Crust of Rust: Declarative Macros","uri":"/posts/declarative-macros/"},{"categories":["Rust"],"content":"regex macro 可以使用以下 3 种分隔符来传入参数 (注意花括号 {} 的需要与 macro 名之间进行空格,末尾不需要分号,这是因为 {} 会被编译器视为一个 statement,无需使用 ; 来进行分隔): macro_rules! avec { () =\u003e {}; ... } avec!(); avec![]; avec! {} macro 定义内的 () 和 {} 也都可以使用 (), [], {} 之间的任意一种,并不影响调研 macro 的分隔符的使用(都是 3 任选 1 即可),不过推荐在 macro 定义内使用 () 和 {} 搭配。 如果需要在 macro 传入的 synatx 中使用正则表达式 (regex),则需要在外面使用 $() 进行包装: ($($elem:expr),* $(,)?) =\u003e {{ let mut v = Vec::new(); $(v.push($elem);)* v }}; 同样的,可以在 macro 体内使用 regex 对参数进行解包装,语法是相同的: $(...)[delimiter](+|*|?) 其中分隔符 (delimiter) 是可选的。它会根据内部所包含的参数 $(...) (本例中是 $(elem)) 来进行自动解包装,生成对应次数的 statement,如果有分隔符 (delimiter) 也会生成对应的符号。 ","date":"2024-01-31","objectID":"/posts/declarative-macros/:1:1","tags":["Rust","Macro","Declarative Macros"],"title":"Crust of Rust: Declarative Macros","uri":"/posts/declarative-macros/"},{"categories":["Rust"],"content":"cargo expand cargo-expand 可以将宏展开,对于宏的除错非常方便,可以以下命令来安装: $ cargo install cargo-expand 然后可以通过以下命令对 macro 进行展开: $ cargo expand 使用以下命令可以将 unit tests 与 cargo expand 结合起来,即展开的是 unit tests 之后的完整代码: $ cargo expand --lib tests ","date":"2024-01-31","objectID":"/posts/declarative-macros/:1:2","tags":["Rust","Macro","Declarative Macros"],"title":"Crust of Rust: Declarative Macros","uri":"/posts/declarative-macros/"},{"categories":["Rust"],"content":"scope 由于 Rust 中 macro 和 normal code 的作用域不一致,所以像 C 语言那种在 macro 中定义变量或在 macro 中直接修改已有变量是不可行的,操作这种 lvalue 的情况需要使用 macro 参数进行传入,否则无法通过编译。 // cannot compile macro_rules! avec { () =\u003e { let x = 1; } } // cannot compile macro_rules! avec { () =\u003e { x = 42; } } // can compile macro_rules! avec { ($x: ident) =\u003e { $x += 1; } } ","date":"2024-01-31","objectID":"/posts/declarative-macros/:1:3","tags":["Rust","Macro","Declarative Macros"],"title":"Crust of Rust: Declarative Macros","uri":"/posts/declarative-macros/"},{"categories":["Rust"],"content":"statements 在 Rust macro 中,如果需要将传入的 syntax 转换成多个 statements,需要使用 {} 进行包装: () =\u003e {{ ... }} 其中第一对 {} 是 macro 语法所要求的的,第二对 {} 则是用于包装 statements 的 {},使用 cargo expand 进行查看会更直观。 ","date":"2024-01-31","objectID":"/posts/declarative-macros/:1:4","tags":["Rust","Macro","Declarative Macros"],"title":"Crust of Rust: Declarative Macros","uri":"/posts/declarative-macros/"},{"categories":["Rust"],"content":"delimiter 注意 macro 中传入的 syntax,其使用的类似于 =\u003e 的分隔符是有限的,例如不能使用 -\u003e 作为分隔符,具体可以查阅手册。 ($arg1:ty =\u003e $arg2:ident) =\u003e { type $arg2 = $arg1; }; 技巧 当 declarative macros 变得复杂时,它的可读性会变得很差,这时候需要使用 procedural macros。但是 procedural macros 需要多花费一些编译周期 (compilition cycle),因为需要先对 procedural macros 进行编译,再编译 lib/bin 对应的源文件。 ","date":"2024-01-31","objectID":"/posts/declarative-macros/:1:5","tags":["Rust","Macro","Declarative Macros"],"title":"Crust of Rust: Declarative Macros","uri":"/posts/declarative-macros/"},{"categories":["Rust"],"content":"calculating 编写 macro 时传入的参数如果是 expression,需要先对其进行计算,然后使用 clone 方法来对该计算结果进行拷贝,这样能最大限度的避免打破 Rust 所有权制度的限制。 ($elem:expr; $count:expr) =\u003e {{ let mut v = Vec::new(); let x = $elem; for _ in 0..$count { v.push(x.clone()); } v }}; 这样传入 y.take().unwrap() 作为宏的 elem 参数就不会产生 panic。 技巧 对于会导致 compile fail 的 unit test,无法使用通常的 unit test 来测试,但是有一个技巧:可以使用 Doc-tests 的方式来构建(需要标记 compile_fail,如果不标记则默认该测试需要 compile success) /// ```compile_fail /// let v: Vec\u003cu32\u003e = vecmac::avec![42; \"foo\"]; /// ``` #[allow(dead_code)] struct CompileFailTest; ","date":"2024-01-31","objectID":"/posts/declarative-macros/:1:6","tags":["Rust","Macro","Declarative Macros"],"title":"Crust of Rust: Declarative Macros","uri":"/posts/declarative-macros/"},{"categories":["Rust"],"content":"trait Rust 中的 macro 无法限制传入参数的 Trait,例如不能限制参数必须实现 Clone 这个 Trait。 ::std::iter 带有前置双冒号 :: 的语法,是在没有显式引入 use std::iter 模块的情况下访问该模块的方式。在这种情况下,::std::iter 表示全局命名空间 (global namespace) 中的 std::iter 模块,即标准库中的 iter 模块。由于 macro 需要进行 export 建议编写 macro 时尽量使用 :: 这类语法。 技巧 计算 vector 的元素个数时使用 () 引用 [()] 进行计数是一个常见技巧,因为 () 是 zero size 的,所以并不会占用栈空间。其他的元素计数方法可以参考 The Little Book of Rust Macros 的 2.5.2 Counting 一节。 ","date":"2024-01-31","objectID":"/posts/declarative-macros/:1:7","tags":["Rust","Macro","Declarative Macros"],"title":"Crust of Rust: Declarative Macros","uri":"/posts/declarative-macros/"},{"categories":["Rust"],"content":"Homework 信息 实作说明: 尝试使用 declarative macro 来实现 HashMap 的初始化语法 (Github: My Implementation) 尝试阅读 vec macro 在 std 库的实现 Macro std::vec 参考资料: Struct std::collections::HashMap ","date":"2024-01-31","objectID":"/posts/declarative-macros/:2:0","tags":["Rust","Macro","Declarative Macros"],"title":"Crust of Rust: Declarative Macros","uri":"/posts/declarative-macros/"},{"categories":["Rust"],"content":"Documentations 这里列举视频中一些概念相关的 documentation 学习的一手资料是官方文档,请务必自主学会阅读规格书之类的资料 ","date":"2024-01-31","objectID":"/posts/declarative-macros/:3:0","tags":["Rust","Macro","Declarative Macros"],"title":"Crust of Rust: Declarative Macros","uri":"/posts/declarative-macros/"},{"categories":["Rust"],"content":"Crate std 可以使用这里提供的搜素栏进行搜索 (BTW 不要浪费时间在 Google 搜寻上!) Macro std::vec Struct std::vec::Vec Method std::vec::Vec::with_capacity method std::vec::Vec::extend method std::vec::Vec::resize Module std::iter Function std::iter::repeat method std::iter::Iterator::take method std::option::Option::take ","date":"2024-01-31","objectID":"/posts/declarative-macros/:3:1","tags":["Rust","Macro","Declarative Macros"],"title":"Crust of Rust: Declarative Macros","uri":"/posts/declarative-macros/"},{"categories":["Rust"],"content":"References 原版的 The Little Book of Rust Macros 在 Rust 更新新版本后没有持续更新,另一位大牛对这本小册子进行了相应的更新: The Little Book of Rust Macros Rust语言中文社区也翻译了该小册子: Rust 宏小册 ","date":"2024-01-31","objectID":"/posts/declarative-macros/:4:0","tags":["Rust","Macro","Declarative Macros"],"title":"Crust of Rust: Declarative Macros","uri":"/posts/declarative-macros/"},{"categories":["Rust"],"content":" We’re going to investigate a case where you need multiple explicit lifetime annotations. We explore why they are needed, and why we need more than one in this particular case. We also talk about some of the differences between the string types and introduce generics over a self-defined trait in the process. 整理自 John Gjengset 的影片 ","date":"2024-01-25","objectID":"/posts/lifetime-annotations/:0:0","tags":["Rust","Lifetime"],"title":"Crust of Rust: Lifetime Annotations","uri":"/posts/lifetime-annotations/"},{"categories":["Rust"],"content":"C 语言中的 lifetime Rust 中的 lifetime 一向是一个难点,为了更好地理解这一难点的本质,建议阅读 C 语言规格书关于 lifetime 的部分,相信你会对 Rust 的 lifetime 有不同的看法。 C11 [6.2.4] Storage durations of objects An object has a storage duration that determines its lifetime. There are four storage durations: static, thread, automatic, and allocated. ","date":"2024-01-25","objectID":"/posts/lifetime-annotations/:1:0","tags":["Rust","Lifetime"],"title":"Crust of Rust: Lifetime Annotations","uri":"/posts/lifetime-annotations/"},{"categories":["Rust"],"content":"影片注解 ","date":"2024-01-25","objectID":"/posts/lifetime-annotations/:2:0","tags":["Rust","Lifetime"],"title":"Crust of Rust: Lifetime Annotations","uri":"/posts/lifetime-annotations/"},{"categories":["Rust"],"content":"cargo check cargo check 可以给出更简洁的提示,例如相对于编译器给出的错误信息,它会整合相同的错误信息,从而提供简洁切要的提示信息。而且它是一个静态分析工具,不需要进行编译即可给出提示,所以速度会比编译快很多,在大型项目上尤为明显。 ","date":"2024-01-25","objectID":"/posts/lifetime-annotations/:2:1","tags":["Rust","Lifetime"],"title":"Crust of Rust: Lifetime Annotations","uri":"/posts/lifetime-annotations/"},{"categories":["Rust"],"content":"ref 影片大概 49 分时提到了 if let Some(ref mut remainder) = self.remainder {...} ref 的作用配合 if let 语句体的逻辑可以体会到 pointer of pointer 的美妙之处。 因为在 pattern match 中形如 \u0026mut 这类也是用于 pattern match 的,不能用于获取 reference,这也是为什么需要使用 ref mut 这类语法来获取 reference 的原因。 ","date":"2024-01-25","objectID":"/posts/lifetime-annotations/:2:2","tags":["Rust","Lifetime"],"title":"Crust of Rust: Lifetime Annotations","uri":"/posts/lifetime-annotations/"},{"categories":["Rust"],"content":"operator ? 影片大概 56 分时提到了 let remainder = self.remainder.as_mut()?; 为什么使用之前所提的 let remainder = \u0026mut self.remainder?; 这是因为使用 ? 运算符返回的是内部值的 copy,所以这种情况 remainder 里是 self.remainder? 返回的值 (是原有 self.remainder 内部值的 copy) 的 reference ","date":"2024-01-25","objectID":"/posts/lifetime-annotations/:2:3","tags":["Rust","Lifetime"],"title":"Crust of Rust: Lifetime Annotations","uri":"/posts/lifetime-annotations/"},{"categories":["Rust"],"content":"\u0026str vs String 影片大概 1:03 时提到了 str 与 String 的区别,个人觉得讲的很好: str -\u003e [char] \u0026str -\u003e \u0026[char] // fat pointer (address and size) String -\u003e Vec\u003cchar\u003e String -\u003e \u0026str (cheap -- AsRef) \u0026str -\u003e String (expensive -- memcpy) 对于 String 使用 \u0026* 可以保证将其转换成 \u0026str,因为 * 会先将 String 转换成 str。当然对于函数参数的 \u0026str,只需传入 \u0026String 即可自动转换类型。 ","date":"2024-01-25","objectID":"/posts/lifetime-annotations/:2:4","tags":["Rust","Lifetime"],"title":"Crust of Rust: Lifetime Annotations","uri":"/posts/lifetime-annotations/"},{"categories":["Rust"],"content":"lifetime 可以将结构体的 lifetime 的第一个 (一般为 'a) 视为实例的 lifetime,其它的可以表示与实例 lifetime 无关的 lifetime。由于 compiler 不够智能,所以它会将实例化时传入参数的 lifetime 中相关联的最小 lifetime 视为实例的 lifetime 约束 (即实例的 lifetime 包含于该 lifetime 内)。 当在实现结构体的方法或 Trait 时,如果在实现方法时无需使用 lifetime 的名称,则可以使用匿名 lifetime '_,或者在编译器可以推推导出 lifetime 时也可以使用匿名 lifetime '_。 only lifetime struct Apple\u003c'a\u003e { owner: \u0026'a Human, } impl Apple\u003c'_\u003e { ... } lifetime and generic struct Apple\u003c'a, T\u003e { owner: \u0026'a T, } impl\u003cT\u003e Apple\u003c'_, T\u003e { ... } compiler can know lifetime pun fn func(\u0026self) -\u003e Apple\u003c'_, T\u003e { ... } ","date":"2024-01-25","objectID":"/posts/lifetime-annotations/:2:5","tags":["Rust","Lifetime"],"title":"Crust of Rust: Lifetime Annotations","uri":"/posts/lifetime-annotations/"},{"categories":["Rust"],"content":"Documentations 这里列举视频中一些概念相关的 documentation 学习的一手资料是官方文档,请务必自主学会阅读规格书之类的资料 ","date":"2024-01-25","objectID":"/posts/lifetime-annotations/:3:0","tags":["Rust","Lifetime"],"title":"Crust of Rust: Lifetime Annotations","uri":"/posts/lifetime-annotations/"},{"categories":["Rust"],"content":"Crate std 可以使用这里提供的搜素栏进行搜索 (BTW 不要浪费时间在 Google 搜寻上!) Keywords Keyword SelfTy Keyword ref Trait std::iter::Iterator method std::iter::Iterator::eq method std::iter::Iterator::collect method std::iter::Iterator::position method std::iter::Iterator::find Enum std::option::Option method std::option::Option::take method std::option::Option::as_mut method std::option::Option::expect Primitive Type str method str::find method str::char_indices Trait std::ops::Try Macro std::try method char::len_utf8 ","date":"2024-01-25","objectID":"/posts/lifetime-annotations/:3:1","tags":["Rust","Lifetime"],"title":"Crust of Rust: Lifetime Annotations","uri":"/posts/lifetime-annotations/"},{"categories":["Toolkit"],"content":"记录一下折腾 Deepin 20.9 的物理机的过程与相关的配置。 ","date":"2024-01-24","objectID":"/posts/deepin20.9/:0:0","tags":["Linux","Deepin"],"title":"深度操作系统 Deepin 20.9 安装配置","uri":"/posts/deepin20.9/"},{"categories":["Toolkit"],"content":"安装与配置 新手教学影片: 深度操作系统deepin下载安装 (附双系统安装及分区指引) [bilibili] 安装完deepin之后该做的事情 [bilibili] ","date":"2024-01-24","objectID":"/posts/deepin20.9/:1:0","tags":["Linux","Deepin"],"title":"深度操作系统 Deepin 20.9 安装配置","uri":"/posts/deepin20.9/"},{"categories":["Toolkit"],"content":"网络代理 新手教学文档: Ubuntu 22.04LTS 相关配置 在境内可以使用 gitclone 镜像站来加快 clone 的速度。 ","date":"2024-01-24","objectID":"/posts/deepin20.9/:2:0","tags":["Linux","Deepin"],"title":"深度操作系统 Deepin 20.9 安装配置","uri":"/posts/deepin20.9/"},{"categories":["Toolkit"],"content":"编辑器: VS Code 新手教学文档: 编辑器: Visual Studio Code [HackMD] 本人的一些注解: GNU/Linux 开发工具 这里列举一下本人配置的插件: Even Better TOML CodeLLDB 用于调试 Rust Git History Native Debug 用于调试 C/C++ rust-analyzer Tokyo Night 挺好看的一个主题 Vim VSCode Great Icons 文件图标主题 问题 rust5-analyzer 插件可能会因为新版本要求 glibc 2.29 而导致启动失败,请参考这个 issue 来解决。 ","date":"2024-01-24","objectID":"/posts/deepin20.9/:3:0","tags":["Linux","Deepin"],"title":"深度操作系统 Deepin 20.9 安装配置","uri":"/posts/deepin20.9/"},{"categories":["Toolkit"],"content":"终端和 Vim 新手教学文档: 終端機和 Vim 設定 [HackMD] 本人的一些注解: GNU/Linux 开发工具 本人的终端提示符配置: \\u@\\h\\W 本人使用 Minimalist Vim Plugin Manager 来管理 Vim 插件,配置如下: \" Specify a directory for plugins (for Neovim: ~/.local/share/nvim/plugged) call plug#begin('~/.vim/plugged') Plug 'Shougo/neocomplcache' Plug 'scrooloose/nerdtree' map \u003cF5\u003e :NERDTreeToggle\u003cCR\u003e call plug#end() let g:neocomplcache_enable_at_startup = 1 let g:neocomplcache_enable_smart_case = 1 inoremap \u003cexpr\u003e\u003cTAB\u003e pumvisible()?\"\\\u003cC-n\u003e\" : \"\\\u003cTAB\u003e\" syntax on set number set cursorline colorscheme default set bg=dark set tabstop=4 set expandtab set shiftwidth=4 set ai set hlsearch set smartindent map \u003cF4\u003e : set nu!\u003cBAR\u003eset nonu?\u003cCR\u003e \" autocomplete dropdown list colorscheme hi Pmenu ctermfg=0 ctermbg=7 hi PmenuSel ctermfg=7 ctermbg=4 ","date":"2024-01-24","objectID":"/posts/deepin20.9/:4:0","tags":["Linux","Deepin"],"title":"深度操作系统 Deepin 20.9 安装配置","uri":"/posts/deepin20.9/"},{"categories":["Toolkit"],"content":"系统语言: Rust 安装教程: Installation [The book] 安装 Rust [Rust course] Channels [The rustup book] # install rust $ curl --proto '=https' --tlsv1.2 https://sh.rustup.rs -sSf | sh # install nightly toolchain $ rustup toolchain install nightly # change to nightly toolchain $ rustup default nightly # list installed toolchain $ rustup toolchain list # update installed toolchain $ rustup update 个人偏向于使用 nightly toolchain ","date":"2024-01-24","objectID":"/posts/deepin20.9/:5:0","tags":["Linux","Deepin"],"title":"深度操作系统 Deepin 20.9 安装配置","uri":"/posts/deepin20.9/"},{"categories":["Toolkit"],"content":"tldr The tldr-pages project is a collection of community-maintained help pages for command-line tools, that aims to be a simpler, more approachable complement to traditional man pages. 安装 tldr: $ sudo apt install tldr ","date":"2024-01-24","objectID":"/posts/deepin20.9/:6:0","tags":["Linux","Deepin"],"title":"深度操作系统 Deepin 20.9 安装配置","uri":"/posts/deepin20.9/"},{"categories":["Toolkit"],"content":"效果展示 Deepin Terminial Vim Deepin DDE Desktop ","date":"2024-01-24","objectID":"/posts/deepin20.9/:7:0","tags":["Linux","Deepin"],"title":"深度操作系统 Deepin 20.9 安装配置","uri":"/posts/deepin20.9/"},{"categories":["Toolkit"],"content":"FAQ 问题 重启后可能会出现,输入密码无法进入图形界面重新返回登录界面,这一循环状况。这个是 deepin 的默认 shell 是 dash 造成的,只需将默认的 shell 改为 bash 即可解决问题: $ ls -l /bin/sh lrwxrwxrwx 1 root root 9 xx月 xx xx:xx /bin/sh -\u003e /bin/dash $ sudo rm /bin/sh $ sudo ln -s /bin/bash /bin/sh 如果你已经处于无限登录界面循环这一状况,可以通过 Ctrl + Alt + \u003cF2\u003e 进入 tty2 界面进行修改: # 先查看问题日志,判断是不是 shell 导致的问题 $ cat .xsession-errors # 如果是,则重复上面的操作即可 ","date":"2024-01-24","objectID":"/posts/deepin20.9/:8:0","tags":["Linux","Deepin"],"title":"深度操作系统 Deepin 20.9 安装配置","uri":"/posts/deepin20.9/"},{"categories":["Toolkit"],"content":"在 deepin 20.9 上根据 DragonOS 构建文档 的 bootstrap.sh 的方式来构建 DragonOS 时,如果没有事先安装 Qemu 会出现 KVM 相关的依赖问题。本文记录解决这一问题的过程。 如果事先没有安装 Qemu,在使用 bootstrap.sh 时会出现如下报错: $ bash bootstrap.sh ... 下列软件包有未满足的依赖关系: qemu-kvm : 依赖: qemu-system-x86 E: 无法修正错误,因为您要求某些软件包保持现状,就是它们破坏了软件包间的依赖关系。 查询 deepin 论坛上的相关内容:qemu-kvm无法安装,可以得知是因为 qemu-kvm 在 debian 发行版上只是一个虚包,所以对于 x86 架构的机器可以直接安装 qemu-systerm-x86 Debian qemu-kvm https://packages.debian.org/search?keywords=qemu-kvm 安装 qemu-systerm-x86: $ sudo apt install qemu-systerm-x86 $ $ qemu-system-x86_64 --version QEMU emulator version 5.2.0 (Debian 1:5.2+dfsg-11+deb11u1) Copyright (c) 2003-2020 Fabrice Bellard and the QEMU Project developers 安装的 qemu 版本看起来有点低,但是先使用 bootstrap.sh 快速安装其它依赖项,然后尝试编译运行一下 DragonOS: $ bash bootstrap.sh ... |-----------Congratulations!---------------| | | | 你成功安装了DragonOS所需的依赖项! | | | | 请关闭当前终端, 并重新打开一个终端 | | 然后通过以下命令运行: | | | | make run | | | |------------------------------------------| 新开一个终端或刷新一下 ~/.bashrc: $ cd DragonOS $ make run 运行 DragonOS Ok 可以成功运行 注意 如果需要使用 RISC-V 的 Qemu 模拟器,安装 qemu-system-misc 即可: $ sudo apt install qemu-system-misc ","date":"2024-01-22","objectID":"/posts/deepin-dragonos/:0:0","tags":["Deepin","Linux","DragonOS"],"title":"Deepin 20.9 构建 DragonOS","uri":"/posts/deepin-dragonos/"},{"categories":["Toolkit","Linux Kernel Internals"],"content":"大型开源项目的规模十分庞大,例如使用 Rust 编写的 Servo 浏览器,这个项目有近十万行代码。在开发规模如此庞大的项目时,了解如何通过正确的方式进行调试非常重要,因为这样可以帮助开发者快速地找到瓶颈。 原文地址 | 教学录影 ","date":"2024-01-16","objectID":"/posts/debug-gdb/:0:0","tags":["Debug","GDB","Rust","C/C++","Sysprog"],"title":"程序调试工具 GDB","uri":"/posts/debug-gdb/"},{"categories":["Toolkit","Linux Kernel Internals"],"content":"GDB 调试 观看教学视频 拯救資工系學生的基本素養—使用 GDB 除錯基本教學 和搭配博文 ==[How to debug Rust/C/C++ via GDB][debug-gdb]==,学习 GDB 的基本操作和熟悉使用 GDB 调试 Rust/C/C++ 程序。 掌握 run/r, break/b, print/p, continue/c, step/s info/i, delete/d, backtrace/bt, frame/f, up/down, exit/q 等命令的用法。以及 GBD 的一些特性,例如 GDB 会将空白行的断点自动下移到下一代码行;使用 break 命令时可以输入源文件路径,也可以只输入源文件名称。 相关的测试文件: test.c hello_cargo/ ","date":"2024-01-16","objectID":"/posts/debug-gdb/:1:0","tags":["Debug","GDB","Rust","C/C++","Sysprog"],"title":"程序调试工具 GDB","uri":"/posts/debug-gdb/"},{"categories":["Toolkit","Linux Kernel Internals"],"content":"GDB 基本介绍 引用 “GDB, the GNU Project debugger, allows you to see what is going on ‘inside’ another program while it executes — or what another program was doing at the moment it crashed.” — from gnu.org 安装 GDB: $ sudo apt install gdb 启动 GDB 时可以加入 -q 参数 (quite),表示减少或不输出一些提示或信息。 LLDB 与 GDB 的命令类似,本文也可用于 LLDB 的入门学习。 ","date":"2024-01-16","objectID":"/posts/debug-gdb/:2:0","tags":["Debug","GDB","Rust","C/C++","Sysprog"],"title":"程序调试工具 GDB","uri":"/posts/debug-gdb/"},{"categories":["Toolkit","Linux Kernel Internals"],"content":"GDB 调试 C/C++ 要使用 GDB 来调试 C/C++,需要在编译时加上 -g 参数(必需),也可以使用 -Og 参数来对 debug 进行优化(但使用 -Og 后 compiler 可能会把一些东西移除掉,所以 debug 时可能不会符合预期),例如: $ gcc test.c -Og -g -o test $ gdb -q ./test Source ","date":"2024-01-16","objectID":"/posts/debug-gdb/:3:0","tags":["Debug","GDB","Rust","C/C++","Sysprog"],"title":"程序调试工具 GDB","uri":"/posts/debug-gdb/"},{"categories":["Toolkit","Linux Kernel Internals"],"content":"GDB 调试 Rust 在使用 build 命令构建 debug 目标文件(即位于 target/debug 目录下的目标文件,与 package 同名)后,就可以通过 gdb 来进行调试: $ cargo build $ gdb -q ./target/debug/\u003cpackage name\u003e 但是如果是使用 cargo build --release 构建的 release 目标文件(即位于 target/release 目录下的目标文件),则无法使用 GDB 进行调试,因为 release 目标未包含任何调试信息,类似于未使用 -g 参数编译 C/C++ 源代码。 Source ","date":"2024-01-16","objectID":"/posts/debug-gdb/:4:0","tags":["Debug","GDB","Rust","C/C++","Sysprog"],"title":"程序调试工具 GDB","uri":"/posts/debug-gdb/"},{"categories":["Toolkit","Linux Kernel Internals"],"content":"GDB 基本命令 ","date":"2024-01-16","objectID":"/posts/debug-gdb/:5:0","tags":["Debug","GDB","Rust","C/C++","Sysprog"],"title":"程序调试工具 GDB","uri":"/posts/debug-gdb/"},{"categories":["Toolkit","Linux Kernel Internals"],"content":"run run (r) 命令用于从程序的执行起始点开始执行,直到遇到下一个断点或者程序结束。 ","date":"2024-01-16","objectID":"/posts/debug-gdb/:5:1","tags":["Debug","GDB","Rust","C/C++","Sysprog"],"title":"程序调试工具 GDB","uri":"/posts/debug-gdb/"},{"categories":["Toolkit","Linux Kernel Internals"],"content":"continue continue (c) 命令用于从当前停止的断点位置处继续执行程序,直到遇到下一个断点或者程序结束。 注意 run 和 continue 的区别在于 run 是将程序从头开始执行。例如如果未设置任何断点,使用 run 可以反复执行程序,而如果使用 continue 则会提示 The program is not being run。 ","date":"2024-01-16","objectID":"/posts/debug-gdb/:5:2","tags":["Debug","GDB","Rust","C/C++","Sysprog"],"title":"程序调试工具 GDB","uri":"/posts/debug-gdb/"},{"categories":["Toolkit","Linux Kernel Internals"],"content":"step step (s) 命令用于 逐行 执行程序,在遇到函数调用时进入对应函数,并在函数内部的第一行暂停。step 命令以 单步方式 执行程序的每一行代码,并跟踪函数调用的进入和退出。 (gdb) step 6 bar += 3; (gdb) step 7 printf(\"bar = %d\\n\", bar); 注意 step 命令与 continue 命令相同,只能在程序处于运行态(即停留在断点处)时才能使用。 ","date":"2024-01-16","objectID":"/posts/debug-gdb/:5:3","tags":["Debug","GDB","Rust","C/C++","Sysprog"],"title":"程序调试工具 GDB","uri":"/posts/debug-gdb/"},{"categories":["Toolkit","Linux Kernel Internals"],"content":"next next (n) 命令用于执行当前行并移动到 下一行,它用于逐行执行程序,但不会进入函数调用。 ","date":"2024-01-16","objectID":"/posts/debug-gdb/:5:4","tags":["Debug","GDB","Rust","C/C++","Sysprog"],"title":"程序调试工具 GDB","uri":"/posts/debug-gdb/"},{"categories":["Toolkit","Linux Kernel Internals"],"content":"break break (b) 命令用于在可执行问卷对应的源程序中加入断点,可以在程序处于 未运行态/运行态 时加入断点(运行态是指程序停留在断点处但未执行完毕的姿态)。 可以通过指定 源文件对应的 行数/函数名 来加入断点(源文件名可以省略): (gdb) break test.c:7 (gdb) break test.c:foo 如果可执行文件由多个源文件编译链接得到,可以通过指定 源文件名字 的方式来加入断点,无需源文件路径,但如果不同路径有重名源文件,则需要指定路径来区分: (gdb) break test1.c:7 (gdb) break test2.c:main ","date":"2024-01-16","objectID":"/posts/debug-gdb/:5:5","tags":["Debug","GDB","Rust","C/C++","Sysprog"],"title":"程序调试工具 GDB","uri":"/posts/debug-gdb/"},{"categories":["Toolkit","Linux Kernel Internals"],"content":"print print (p) 命令用于在调试过程中打印 变量的值或 表达式 的结果,帮助开发者检查程序状态并查看特定变量的当前值。 # Assume x: 3, y: 4 (gdb) print x $1 = 3 (gdb) print x + y $2 = 7 使用 p 命令打印变量值时,会在左侧显示一个 $\u003cnumber\u003e,这个可以理解成临时变量,后续也可以通过这个标志来复用这些值。例如在上面的例子中: (gdb) print $1 $3 = 3 (gdb) print $1 + $3 $4 = 4 Use p/format to instead select other formats such as x for hex, t for binary, and c for char. ","date":"2024-01-16","objectID":"/posts/debug-gdb/:5:6","tags":["Debug","GDB","Rust","C/C++","Sysprog"],"title":"程序调试工具 GDB","uri":"/posts/debug-gdb/"},{"categories":["Toolkit","Linux Kernel Internals"],"content":"backtrace backtrace (bt) 命令用于打印当前调用栈的信息,也称为堆栈回溯 (backtrace)。它显示了程序在执行过程中经过的函数调用序列,以及每个函数调用的位置和参数,即可以获取以下信息: 函数调用序列:显示程序当前的函数调用序列,以及每个函数的名称和所在的源代码文件。 栈帧信息:对于每个函数调用,显示该函数的栈帧信息,包括栈帧的地址和栈帧的大小。 (gdb) backtrace (gdb) backtrace #0 foo () at test.c:7 #1 0x00005555555551d2 in main () at test.c:14 技巧 backtrace 命令对于跟踪程序的执行路径、检查函数调用的顺序以及定位错误非常有用。在实际中,一般会搭配其他GDB命令(如 up、down 和 frame)结合使用,以查看特定栈帧的更多详细信息或切换到不同的栈帧。在上面的例子中,#0 和 #1 表示栈帧的编号,可以通过 frame 配合这些编号来切换栈帧。 ","date":"2024-01-16","objectID":"/posts/debug-gdb/:5:7","tags":["Debug","GDB","Rust","C/C++","Sysprog"],"title":"程序调试工具 GDB","uri":"/posts/debug-gdb/"},{"categories":["Toolkit","Linux Kernel Internals"],"content":"where where 和 backtrace 命令都用于显示程序的调用栈信息。backtrace 提供更详细的调用栈信息,包括函数名称、文件名、行号、参数和局部变量的值。而 where 命令可以理解为 backtrace 的一个简化版本,它提供的是较为紧凑的调用栈信息,通常只包含函数名称、文件名和行号。 ","date":"2024-01-16","objectID":"/posts/debug-gdb/:5:8","tags":["Debug","GDB","Rust","C/C++","Sysprog"],"title":"程序调试工具 GDB","uri":"/posts/debug-gdb/"},{"categories":["Toolkit","Linux Kernel Internals"],"content":"frame frame (f) 命令用于选择特定的栈帧 (stack frame),从而切换到不同的函数调用上下文,每个栈帧对应于程序中的一个函数调用。 接着上一个例子,切换到 main 函数所在的栈帧: (gdb) frame 1 #1 0x00005555555551d2 in main () at test.c:14 14 int result = foo(); ","date":"2024-01-16","objectID":"/posts/debug-gdb/:5:9","tags":["Debug","GDB","Rust","C/C++","Sysprog"],"title":"程序调试工具 GDB","uri":"/posts/debug-gdb/"},{"categories":["Toolkit","Linux Kernel Internals"],"content":"up/down up 和 down 命令用于在调试过程中在不同的栈帧之间进行切换: up 用于在调用栈中向上移动到较高的栈帧,即进入调用当前函数的函数。每次执行 up 命令,GDB 将切换到上一个(更高层次)的栈帧。这可以用于查看调用当前函数的上层函数的执行上下文。 down 用于在调用栈中向下移动到较低的栈帧,即返回到当前函数调用的函数。每次执行 down 命令,GDB 将切换到下一个(较低层次)的栈帧。这可以用于返回到调用当前函数的函数的执行上下文。 这两个命令需要开发者对应函数调用堆栈的布局有一定程度的了解。 接着上一个例子: (gdb) up #1 0x00005555555551d2 in main () at test.c:14 14 int result = foo(); (gdb) down #0 foo () at test.c:7 7 printf(\"bar = %d\\n\", bar); ","date":"2024-01-16","objectID":"/posts/debug-gdb/:5:10","tags":["Debug","GDB","Rust","C/C++","Sysprog"],"title":"程序调试工具 GDB","uri":"/posts/debug-gdb/"},{"categories":["Toolkit","Linux Kernel Internals"],"content":"info info (i) 命令用于获取程序状态和调试环境的相关信息,该命令后面可以跟随不同的子命令,用于获取特定类型的信息。 一些常用的 info 子命令: info breakpoints 显示已设置的所有断点 (breakpoint) 信息,包括断点编号、断点类型、断点位置等。 info watchpoints 显示已设置的所有监视点 (watchpoint) 信息,包括监视点编号、监视点类型、监视的表达式等。 info locals 显示当前函数的局部变量的值和名称。 info args 显示当前函数的参数的值和名称。 info registers 显示当前 CPU 寄存器的值。 info threads 显示当前正在调试的所有线程 (thread) 信息,包括线程编号、线程状态等。 info frame 显示当前栈帧 (stack frame) 的信息,包括函数名称、参数、局部变量等。 info program 显示被调试程序的相关信息,例如程序入口地址、程序的加载地址等。 (gdb) info breakpoints # or simply: i b Num Type Disp Enb Address What 1 breakpoint keep y 0x000055555555518f in foo at test.c:7 2 breakpoint keep y 0x0000555555555175 in foo at test.c:4 ","date":"2024-01-16","objectID":"/posts/debug-gdb/:5:11","tags":["Debug","GDB","Rust","C/C++","Sysprog"],"title":"程序调试工具 GDB","uri":"/posts/debug-gdb/"},{"categories":["Toolkit","Linux Kernel Internals"],"content":"delete delete (d) 命令用于删除断点 (breakpoint) 或观察点 (watchpoint)。断点是在程序执行期间暂停执行的特定位置,而观察点是在特定条件满足时暂停执行的位置。 可以通过指定 断点 / 观察点 的编号或使用 delete 命令相关的参数,来删除已设置的断点 / 观察点。断点 / 观察点编号可以在使用 info breakpoints / info watchpoints 命令时获得。 ","date":"2024-01-16","objectID":"/posts/debug-gdb/:5:12","tags":["Debug","GDB","Rust","C/C++","Sysprog"],"title":"程序调试工具 GDB","uri":"/posts/debug-gdb/"},{"categories":["Toolkit","Linux Kernel Internals"],"content":"quit quit (q) 命令用于退出 GDB,返回终端页面。 (gdb) quit $ # Now, in the terminial ","date":"2024-01-16","objectID":"/posts/debug-gdb/:5:13","tags":["Debug","GDB","Rust","C/C++","Sysprog"],"title":"程序调试工具 GDB","uri":"/posts/debug-gdb/"},{"categories":["Toolkit","Linux Kernel Internals"],"content":"list list 命令用于显示当前位置的代码片段,默认情况下,它会显示当前位置的前后10行代码。 list 命令也可以显示指定范围的代码,使用 list \u003cstart\u003e,\u003cend\u003e 命令将显示从 start 行到 end 行的源代码。 ","date":"2024-01-16","objectID":"/posts/debug-gdb/:5:14","tags":["Debug","GDB","Rust","C/C++","Sysprog"],"title":"程序调试工具 GDB","uri":"/posts/debug-gdb/"},{"categories":["Toolkit","Linux Kernel Internals"],"content":"whatis whatis 命令用于获取给定标识符(如变量、函数或类型)的类型信息。 // in source code int calendar[12][31]; // in gdb (gdb) whatis calendar type = int [12][31] ","date":"2024-01-16","objectID":"/posts/debug-gdb/:5:15","tags":["Debug","GDB","Rust","C/C++","Sysprog"],"title":"程序调试工具 GDB","uri":"/posts/debug-gdb/"},{"categories":["Toolkit","Linux Kernel Internals"],"content":"x x 命令用于查看内存中的数据,使用 x 命令搭配不同的格式来显示内存中的数据,也可以搭配 / 后跟数字来指定要显示的内存单元数量。例如,x/4 \u003caddress\u003e 表示显示地址 address 开始的连续 4 个内存单元的内容。 ","date":"2024-01-16","objectID":"/posts/debug-gdb/:5:16","tags":["Debug","GDB","Rust","C/C++","Sysprog"],"title":"程序调试工具 GDB","uri":"/posts/debug-gdb/"},{"categories":["Toolkit","Linux Kernel Internals"],"content":"其他 如果被调试程序正处于运行态(即已经通过 run 命令来运行程序),此时可以通过 Ctrl+C 来中断 GDB,程序将被立即中断,并在中断时所运行到的地方暂停。这种方式被称为 手动断点,手动断点可以理解为一个临时断点,只会在该处暂停一次。 GDB 会将空白行的断点自动下移到下一非空的代码行。 set print pretty 命令可以以更易读和格式化的方式显示结构化数据,以更友好的方式输出结构体、类、数组等复杂类型的数据,更易于阅读和理解。 ","date":"2024-01-16","objectID":"/posts/debug-gdb/:5:17","tags":["Debug","GDB","Rust","C/C++","Sysprog"],"title":"程序调试工具 GDB","uri":"/posts/debug-gdb/"},{"categories":["Toolkit","Linux Kernel Internals"],"content":"References video: Linux basic anti-debug video: C Programming, Disassembly, Debugging, Linux, GDB rr (Record and Replay Framework) video: Quick demo video: Record and replay debugging with “rr” ","date":"2024-01-16","objectID":"/posts/debug-gdb/:6:0","tags":["Debug","GDB","Rust","C/C++","Sysprog"],"title":"程序调试工具 GDB","uri":"/posts/debug-gdb/"},{"categories":["C","Linux Kernel Internals"],"content":" 「指针」 扮演 「记忆体」 和 「物件」 之间的桥梁 原文地址 ","date":"2024-01-14","objectID":"/posts/c-pointer/:0:0","tags":["Sysprog","C","Pointer"],"title":"你所不知道的 C 语言: 指针篇","uri":"/posts/c-pointer/"},{"categories":["C","Linux Kernel Internals"],"content":"前言杂谈 Let’s learn programming by inventing it [CppCon 2018] ✅ 在 K\u0026R 一书中,直到 93 页才开始谈论 pointer,而全书总计 185 页,所以大概是在全书 $50.27\\%$ 的位置才开始讲 pointer。所以即使不学 pointer,你还是能够掌握 $~50\\%$ 的 C 语言的内容,但是 C 语言的核心正是 pointer,所以 Good Luck 🤣 godbolt 可以直接在网页上看到,源代码由各类 compiler 生成的 Assembly Code How to read this prototype? [Stack Overflow] ✅ Note 这个问题是关于 signal 系统调用的函数原型解读,里面的回答页给出了很多对于指针,特别是 函数指针 的说明,下面节选一些特别有意思的回答: 引用 The whole thing declares a function called signal: signal takes an int and a function pointer this function pointer takes an int and returns void signal returns a function pointer this function pointer takes an intand returns avoid` That’s where the last int comes in. You can use the spiral rule to make sense of such declarations, or the program cdecl(1). The whole thing declares a function called signal: 这里面提到了 the spiral rule 这是一个用于解析 C 语言中声明 (declaration) 的方法;另外还提到了 cdecl 这一程序,它也有类似的作用,可以使用英文进行声明或者解释。 引用 Find the leftmost identifier and work your way out, remembering that [] and () bind before *; IOW, *a[] is an array of pointers, (*a)[] is a pointer to an array, *f() is a function returning a pointer, and (*f)() is a pointer to a function. Thus, void ( *signal(int sig, void (*handler)(int)) ) (int); breaks down as signal -- signal signal( ) -- is a function signal( sig ) -- with a parameter named sig signal(int sig, ) -- of type int signal(int sig, handler ) -- and a parameter named handler signal(int sig, *handler ) -- which is a pointer signal(int sig, (*handler)( )) ) -- to a function signal(int sig, (*handler)(int)) ) -- taking an int parameter signal(int sig, void (*handler)(int)) ) -- and returning void *signal(int sig, void (*handler)(int)) ) -- returning a pointer ( *signal(int sig, void (*handler)(int)) )( ) -- to a function ( *signal(int sig, void (*handler)(int)) )(int) -- taking an int parameter void ( *signal(int sig, void (*handler)(int)) )(int); -- and returning void 这一回答强调了 * 和 []、() 优先级的关系,这在判断数组指针、函数指针时是个非常好用的技巧。 Rob Pike 于 2009/10/30 的 Golang Talk [PDF] David Brailsford 教授解说影片 Essentials: Pointer Power! - Computerphile [YouTube] ","date":"2024-01-14","objectID":"/posts/c-pointer/:1:0","tags":["Sysprog","C","Pointer"],"title":"你所不知道的 C 语言: 指针篇","uri":"/posts/c-pointer/"},{"categories":["C","Linux Kernel Internals"],"content":"阅读 C 语言规格书 一手资料的重要性毋庸置疑,对于 C 语言中的核心概念 指针,借助官方规格书清晰概念是非常重要的。 C99 [6.2.5] Types An array type of unknown size is an incomplete type. It is completed, for an identifier of that type, by specifying the size in a later declaration (with internal or external linkage). A structure or union type of unknown content is an incomplete type. It is completed, for all declarations of that type, by declaring the same structure or union tag with its defining content later in the same scope. incomplete type 和 linkage 配合可以进行 forward declaration,如果搭配 pointer 则可以进一步,在无需知道 object 内部细节即可进行程序开发。 Array, function, and pointer types are collectively called derived declarator types. A declarator type derivation from a type T is the construction of a derived declarator type from T by the application of an array-type, a function-type, or a pointer-type derivation to T. 注意 derived declarator types 表示衍生的声明类型,因为 array, function, pointer 本质都是地址,而它们的类型都是由其它类型衍生而来的,所以可以使用这些所谓的 derived declarator types 来提前声明 object,表示在某个地址会存储一个 object,这也是为什么这些类型被规格书定义为 derived declarator types。 lvalue: Locator value 危险 C 语言里只有 call by value ","date":"2024-01-14","objectID":"/posts/c-pointer/:2:0","tags":["Sysprog","C","Pointer"],"title":"你所不知道的 C 语言: 指针篇","uri":"/posts/c-pointer/"},{"categories":["C","Linux Kernel Internals"],"content":"void \u0026 void * C89 之前,函数如果没有标注返回类型,则默认返回类型 int,返回值 0。但由于这样既可以表示返回值不重要,也可以表示返回值为 0,这会造成歧义,所以引进了 void。 void * 只能表示地址,而不能对所指向的地址区域的内容进行操作。因为通过 void * 无法知道所指向区域的 size,所以无法对区域的内容进行操作,必须对 void * 进行 显示转换 才能操作指向的内容。(除此之外,针对于 gcc,对于指针本身的操作,void * 与 char * 是等价的,即对于 +/- 1 这类的操作,二者的偏移量是一致的 (这是 GNU extensions 并不是 C 语言标准);对于其它的编译器,建议将 void * 转换成 char * 再进行指针的加减运算) ","date":"2024-01-14","objectID":"/posts/c-pointer/:3:0","tags":["Sysprog","C","Pointer"],"title":"你所不知道的 C 语言: 指针篇","uri":"/posts/c-pointer/"},{"categories":["C","Linux Kernel Internals"],"content":"Alignment 这部分原文描述不是很清晰,2-byte aligned 图示如下: Alignment 如果是 2-byte aligned 且是 little-endian 的处理器,对于左边,可以直接使用 *(uint16_t *) ptr,但对于右边就无法这样(不符合 alignment): /* may receive wrong value if ptr is not 2-byte aligned */ uint16_t value = *(uint16_t *) ptr; /* portable way of reading a little-endian value */ uint16_t value = *(uint8_t *) ptr | ((*(uint8_t *) (ptr + 1)) \u003c\u003c 8); 因为内存寻址的最小粒度是 Byte,所以使用 (uint_8 *) 不需要担心 alignment 的问题。原文并没有给出 32-bit aligned 的 portable way,我们来写一下: /* may receive wrong value if ptr is not 2-byte aligned */ uint32_t value = *(uint32_t *) ptr; /* portable way of reading a little-endian value */ uint32_t value = *(uint8_t *) ptr | ((*(uint8_t *) (ptr + 1)) \u003c\u003c 8) | ((*(uint8_t *) (ptr + 2)) \u003c\u003c 16) | ((*(uint8_t *) (ptr + 3)) \u003c\u003c 24); 信息 The Lost Art of Structure Packing ","date":"2024-01-14","objectID":"/posts/c-pointer/:3:1","tags":["Sysprog","C","Pointer"],"title":"你所不知道的 C 语言: 指针篇","uri":"/posts/c-pointer/"},{"categories":["C","Linux Kernel Internals"],"content":"规格书中的 Pointer C99 [6.3.2.3] Pointers A pointer to a function of one type may be converted to a pointer to a function of another type and back again; the result shall compare equal to the original pointer. Ifaconverted pointer is used to call a function whose type is not compatible with the pointed-to type, the behavior is undefined. C11 [6.3.2.3] Pointers A pointer to a function of one type may be converted to a pointer to a function of another type and back again; the result shall compare equal to the original pointer. If a converted pointer is used to call a function whose type is not compatible with the referenced type, the behavior is undefined. C99 和 C11 都不保证 pointers (whose type is not compatible with the pointed-to / referenced type) 之间的转换是正确的。 导致这个的原因正是之前所提的 Alignment,转换后的指针类型不一定满足原有类型的 Alignment 要求,这种情况下进行 dereference 会导致异常。例如将一个 char * 指针转换成 int * 指针,然后进行 deference 有可能会产生异常。 ","date":"2024-01-14","objectID":"/posts/c-pointer/:3:2","tags":["Sysprog","C","Pointer"],"title":"你所不知道的 C 语言: 指针篇","uri":"/posts/c-pointer/"},{"categories":["C","Linux Kernel Internals"],"content":"Pointers vs. Arrays C99 6.3.2.1 Except when it is the operand of the sizeof operator or the unary \u0026 operator, or is a string literal used to initialize an array, an expression that has type ‘‘array of type’’ is converted to an expression with type ‘‘pointer to type’’ that points to the initial element of the array object and is not an lvalue. Array 只有在表示其自身为数组时才不会被 converted to Pointer,例如 // case 1: extern declaration of array extern char a[]; // case 2: defintion of array char a[10]; // case 3: size of array sizeof(a); // case 4: address of array \u0026a 在其他情况则会倍 converted to Pointer,这时 Array 可以和 Pointer 互换进行表示或操作,例如 // case 1: function parameter void func(char a[]); void func(char *a); // case 2: operation in expression char c = a[2]; char c = *(a + 2); 这也是为什么对于一个 Array a,\u0026a 和 \u0026a[0] 值虽然相同,但 \u0026a + 1 和 \u0026a[0] + 1 的结果大部分时候是大不相同的,这件事乍一看是非常惊人的,但其实不然,在了解 Array 和 Pointer 之后,也就那么一回事 🤣 Source ","date":"2024-01-14","objectID":"/posts/c-pointer/:4:0","tags":["Sysprog","C","Pointer"],"title":"你所不知道的 C 语言: 指针篇","uri":"/posts/c-pointer/"},{"categories":["C","Linux Kernel Internals"],"content":"GDB 实作 char a[10]; int main() { return 0; }; 我们以上面这个例子,通过 GDB 来对 Array 和 Pointer 进行深入研究: (gdb) print \u0026a $1 = (char (*)[10]) 0x555555558018 \u003ca\u003e (gdb) print \u0026a[0] $2 = 0x555555558018 \u003ca\u003e \"\" 符合预期,\u0026a 和 \u0026a[0] 得到的值是相同的,虽然类型看起来不同,但是现在先放到一边。 (gdb) print \u0026a + 1 $3 = (char (*)[10]) 0x555555558022 (gdb) print \u0026a[0] + 1 $4 = 0x555555558019 \u003ca+1\u003e \"\" (gdb) print a + 1 $5 = 0x555555558019 \u003ca+1\u003e \"\" Oh! 正如我们之前所说的 \u0026a + 1 与 \u0026a[0] + 1 结果并不相同(而 \u0026a[0] + 1 和 a + 1 结果相同正是我们所提到的 Array 退化为 Pointer),虽然如此,GDB 所给的信息提示我们可能是二者 Pointer 类型不相同导致的。 (gdb) whatis \u0026a type = char (*)[10] (gdb) whatis \u0026a[0] type = char * Great! 果然是 Pointer 类型不同导致的,我们可以看到 \u0026a 的类型是 char (*)[10] 一个指向 Array 的指针,\u0026a[0] 则是 char *。所以这两个 Pointer 在进行 +/- 运算时的偏移量是不同的,\u0026a[0] 的偏移量为 sizeof(a[0]) 即一个 char 的宽度 ($0x18 + 1 = 0x19$),而 \u0026a 的偏移量为 sizeof(a) 即 10 个 char 的宽度 ($0x18 + 10 = 0x22$)。 警告 在 GDB 中使用 memcpy 后直接打印可能会出现以下错误: (gdb) p memcpy(calendar, b, sizeof(b[0])) 'memcpy' has unknown return type; cast the call to its declared return type 只需加入 void * 进行类型转换即可解决该问题: (gdb) p (void *) memcpy(calendar, b, sizeof(b[0])) ... 技巧 遇到陌生的函数,可以使用 man 来快速查阅手册,例如 man strcpy, man strcat,手册可以让我们快速查询函数的一些信息,从而进入实作。 ","date":"2024-01-14","objectID":"/posts/c-pointer/:4:1","tags":["Sysprog","C","Pointer"],"title":"你所不知道的 C 语言: 指针篇","uri":"/posts/c-pointer/"},{"categories":["C","Linux Kernel Internals"],"content":"Runtime Environment 根据 Zero size arrays in C ,原文中的 char (*argv)[0] 在函数参数传递时会被转换成 char **argv。而为什么在查看地址 ((char **) argv)[0] 开始的连续 4 个 char * 内容时,会打印出 envp 中的内容,可以参考以下的进入 main 函数时的栈布局: argv 和 envp 所指的字符串区域是相连的,所以在越过 argv 字符串区域的边界后,会继续打印 envp 区域的字符串。这也是为什么打印出的字符串之间地址增长于其长度相匹配。所以从地址 (char **) argv 开始的区域只是一个 char * 数组,使用 x/4s 对这部分进行字符串格式打印显然是看不懂的。 注意 argv 和 envp 都是在 shell 进行 exec 系统调用之前进行传递(事实上是以 arguments 的形式传递给 exec) man 2 execve int execve(const char *pathname, char *const argv[], char *const envp[]); execve 实际上在内部调用了 fork,所以 argv 和 envp 的传递是在 fork 之前。(设想如果是在 fork 之后传递,可能会出现 fork 后 child process 先执行,这种情况 child process 显然无法获得这些被传递的信息) 注意到 execve 只传递了 argv 而没有传递 argc,这也很容易理解,argc 是 argv 的计数,只需 argv 即可推导出 argc。 ","date":"2024-01-14","objectID":"/posts/c-pointer/:4:2","tags":["Sysprog","C","Pointer"],"title":"你所不知道的 C 语言: 指针篇","uri":"/posts/c-pointer/"},{"categories":["C","Linux Kernel Internals"],"content":"Function Pointer 危险 与 Array 类似,Function 只有在表示自身时不会被 converted to Function Pointer (即除 sizeof 和 \u0026 运算之外),其它情况、运算时都会被 convert to Function Pointer 理解 C 语言中的 Function 以及 Function Pointer 的核心在于理解 Function Designator 这个概念,函数名字必然是 Function Designator,其它的 designator 则是根据以下两条规则进行推导得来。 C99 [ 6.3.2.1 ] A function designator is an expression that has function type. Except when it is the operand of the sizeof operator or the unary \u0026 operator, a function designator with type ‘‘function returning type’’ is converted to an expression that has type ‘‘pointer to function returning type’’. C99 [6.5.3.2-4] The unary * operator denotes indirection. If the operand points to a function, the result is a function designator. ","date":"2024-01-14","objectID":"/posts/c-pointer/:5:0","tags":["Sysprog","C","Pointer"],"title":"你所不知道的 C 语言: 指针篇","uri":"/posts/c-pointer/"},{"categories":["C","Linux Kernel Internals"],"content":"指针的修饰符 指针 p 自身不能变更,既不能改变 p 自身所存储的地址。const 在 * 之后: char * const p; 指针 p 所指向的内容不能变更,即不能通过 p 来更改所指向的内容。const 在 * 之前: const char * p; char const * p; 指针 p 自身于所指向的内容都不能变更: const char * const p; char const * const p; ","date":"2024-01-14","objectID":"/posts/c-pointer/:6:0","tags":["Sysprog","C","Pointer"],"title":"你所不知道的 C 语言: 指针篇","uri":"/posts/c-pointer/"},{"categories":["C","Linux Kernel Internals"],"content":"字符串 对于函数内部的 char *p = \"hello\"; char p[] = \"hello\"; 这两个是不一样的,因为 string literals 是必须放在 “static storage” 中,而 char p[] 则表示将资料分配在 stack 內,所以这会造成编译器隐式地生成额外代码,在执行时 (runtime) 将 string literals 从 static storage 拷贝到 stack 中,所以此时 return p 会造成 UB。而 char *p 的情形不同,此时 p 只是一个指向 static storage 的指针,进行 return p 是合法的。除此之外,无法对第一种方法的字符串进行修改操作,因为它指向的字符串存放的区域的资料是无法修改的,否则会造成 segmentationfalut 🤣 在大部分情况下,null pointer 并不是一个有效的字符串,所以在 glibc 中字符相关的大部分函数也不会对 null pointer 进行特判 (特判会增加分支,从而影响程序效能),所以在调用这些函数时需要用户自己判断是否为 null pointer,否则会造成 UB。 ","date":"2024-01-14","objectID":"/posts/c-pointer/:7:0","tags":["Sysprog","C","Pointer"],"title":"你所不知道的 C 语言: 指针篇","uri":"/posts/c-pointer/"},{"categories":["C","Linux Kernel Internals"],"content":"Linus 的“教导” Linus 親自教你 C 語言 array argument 的使用 because array arguments in C don’t actually exist. Sadly, compilers accept it for various bad historical reasons, and silently turn it into just a pointer argument. There are arguments for them, but they are from weak minds. The “array as function argument” syntax is occasionally useful (particularly for the multi-dimensional array case), so I very much understand why it exists, I just think that in the kernel we’d be better off with the rule that it’s against our coding practices. array argument 应该只用于多维数组 (multi-dimensional arrays) 的情形,这样可以保证使用下标表示时 offset 是正确的,但对于一维数组则不应该使用数组表示作为函数参数,因为这会对函数体内的 sizeof 用法误解 (以为会获得数组的 size,实际上获得的只是指针的 size)。 技巧 一个常用于计算数组中元素个数的宏: #define ARRAY_SIZE(x) (sizeof(x) / sizeof((x)[0])) 这个宏非常有用,xv6 中使用到了这个宏。 但是需要注意,使用时必须保证 x 是一个数组,而不是函数参数中由数组退化而来的指针,以及保证数组必须至少拥有一个元素的长度 (这个很容易满足,毕竟 x[0] 编译器会抛出警告)。 ","date":"2024-01-14","objectID":"/posts/c-pointer/:8:0","tags":["Sysprog","C","Pointer"],"title":"你所不知道的 C 语言: 指针篇","uri":"/posts/c-pointer/"},{"categories":["C","Linux Kernel Internals"],"content":"Lvalue \u0026 Rvalue Lvalue: locator value Rvalue: Read-only value C99 6.3.2.1 footnote The name “lvalue” comes originally from the assignment expression E1 = E2, in which the left operand E1 is required to be a (modifiable) lvalue. It is perhaps better considered as representing an object “locator value”. What is sometimes called “rvalue” is in this International Standard described as the “value of an expression”. An obvious example of an lvalue is an identifier of an object. As a further example, if E is a unary expression that is a pointer to an object, *E is an lvalue that designates the object to which E points. 即在 C 语言中 lvalue 是必须能在内存 (memory) 中可以定位 (locator) 的东西,因为可以定位 (locator) 所以才可以在表达式左边从而修改值。想像一下,在 C 语言中修改一个常数的值显然是不可能的,因为常数无法在内存 (memory) 定位 (locator) 所以常数在 C 语言中不是 lvalue。C 语言中除了 lvalue 之外的 value 都是 rvalue (这与 C++ 有些不同,C++ 的 lvalue 和 rvalue 的定义请参考 C++ 的规格书)。 ","date":"2024-01-14","objectID":"/posts/c-pointer/:9:0","tags":["Sysprog","C","Pointer"],"title":"你所不知道的 C 语言: 指针篇","uri":"/posts/c-pointer/"},{"categories":["Network"],"content":"之前学校的计网理论课学得云里雾里,对于物理层和数据链路层并没有清晰的逻辑框架,而这学期的计网课设内容为数据链路层和网络层的相关内容,写起来还是云里雾里。虽然最终艰难地把课设水过去了,但是个人认为网络对于 CSer 非常重要,特别是在互联网行业,网络知识是必不可少的。 所以决定寒假重学计网,于是在 HackMD 上冲浪寻找相关资料。然后发现了这篇笔记 110-1 計算機網路 (清大開放式課程),里面提到清大计网主要介绍 L2 ~ L4 一些著名的协议和算法,这完美符合个人的需求,而且该笔记还补充了一些额外的内容,例如 IPv6,所以当即决定搭配这篇笔记来学习清大的计算机网络概论。 ","date":"2024-01-14","objectID":"/posts/nthu-computer-network/:0:0","tags":["Network"],"title":"國立清華大學 計算機網路 重點提示","uri":"/posts/nthu-computer-network/"},{"categories":["Network"],"content":"清大计算机网络概论 本課程將介紹計算機網路的基本運作原理與標準的網路七層結構,由淺入深,可以讓我們對於計算機網路的運作有最基本的認識,本課程還會介紹全球建置最多的有線網路──IEEE 802.3 Ethernet 的基本運作原理, 還有全球建置最多的無線區域網路──IEEE 802.11 Wireless LAN 的基本運作原理, 想知道網路交換機(switches) 是如何運作的嗎 ? 想知道網際網路最重要也最關鍵的通訊協議 ── TCP/IP 是如何運作的嗎 ? 想知道網際網路最重要的路由器 (Routers) 是如何運作的嗎 ? 在本課程裡您都可以學到這些重要的基本知識。 开课学校 课程主页 课程资料 课程影片 國立清華大學 計算機網路概論 課程講義與練習題 Youtube ","date":"2024-01-14","objectID":"/posts/nthu-computer-network/:1:0","tags":["Network"],"title":"國立清華大學 計算機網路 重點提示","uri":"/posts/nthu-computer-network/"},{"categories":["Network"],"content":"Foundation Outline: Applications Network Connectivity Network Architecture Network Performance ","date":"2024-01-14","objectID":"/posts/nthu-computer-network/:2:0","tags":["Network"],"title":"國立清華大學 計算機網路 重點提示","uri":"/posts/nthu-computer-network/"},{"categories":["Network"],"content":"Applications Foundation - 5 进行 1 次 URL request 需要进行 17 次的讯息交换: 6 次讯息交换用于查询 URL 对应的 IP Address 3 次讯息交换用于建立 TCP 连接(TCP 的 3 次握手) 4 次讯息交换用于 HTTP 协议的请求和回复 4 次讯息交换用于关闭 TCP 连接(TCP 的 4 次握手) ","date":"2024-01-14","objectID":"/posts/nthu-computer-network/:2:1","tags":["Network"],"title":"國立清華大學 計算機網路 重點提示","uri":"/posts/nthu-computer-network/"},{"categories":["Network"],"content":"Network Connectivity Foundation - 8 交换机 (Switches) 可以分为很多层级,即可以有不同层级的交换机,例如 L2 层的交换机,L3 层的交换机以及 L4 层的交换机。如何判断交换机是哪个层级?很简单,只需要根据交换机所处理的讯息,L2 层交换机处理的是 MAC Address,L3 层交换机处理的是 IP Address,而 L4 层交换机处理的是 TCP 或者 UDP 相关的讯息。 交换机 (Switches) 用于网络 (Network) 内部的连接,路由 (Router) 用于连接不同的网络 (Network),从而形成 Internetwork。 地址 (Address),对于网卡来说是指 MAC Address,对于主机来说是指 IP Address。Host-to-Host connectivity 是指不同网络 (Network) 的主机,即位于 Internetwork 的不同主机之间,进行连接。 ","date":"2024-01-14","objectID":"/posts/nthu-computer-network/:2:2","tags":["Network"],"title":"國立清華大學 計算機網路 重點提示","uri":"/posts/nthu-computer-network/"},{"categories":["Network"],"content":"Network Architecture Foundation - 22 Physical Layer: 如何将原始资料在 link 上传输,例如不同介质、信息编码。(P25) Data Link Layer: 在 Physical Layer 基础上,如何将 frame 传给直接相连的主机或设备,核心是通过 Media Access Control Protocol 解决 Multiple access 产生的碰撞问题。这一层交换的数据被称为 frame。(P26) Network Layer: 在 Data Link Layer 基础上,如何将 packet 通过 Internet 送给目的地主机。核心是通过 Routing Protocols 动态转发 packet。这一层交换的数据被称为 packet。(P27) Transport Layer: 在 Network Layer 基础上,提供不同主机 processes 之间的资料传送。由于 Networkd Layer 是主机间进行资料传送,所以在 Transport Layer 不论是可靠还是不可靠的传输协议,都必须要实现最基本的机制:主机与 process 之间数据的复用和分解。这一层交换的数据被称为 message。(P28) 注意 Switch 一般处于 L2 Layer,Router 一般处于 L3 Layer。L4 Layer 及以上的 layers 通常只存在于 hosts,switches 和 routers 内部一般不具有这些 layers。(P29) Internet Architecture 的层级并不是严格的,Host 可以略过 Application Layer 而直接使用 Transport Layer、Network Layer 中的协议。(P30) Internet Architecture 的核心是 IP 协议,它作为沙漏形状的中心位置,为处于其上层的协议与处于其下层协议之间提供了一个映射关系。(P31) ","date":"2024-01-14","objectID":"/posts/nthu-computer-network/:2:3","tags":["Network"],"title":"國立清華大學 計算機網路 重點提示","uri":"/posts/nthu-computer-network/"},{"categories":["Network"],"content":"Network Performance Foundation - 36 Foundation - 37 Bandwidth: Number of bits per second (P34) Delay 可以近似理解为 Propagation time。有效利用 network 的标志是在接收对方的回应之前,发送方传送的资料充满了 pipe,即发送了 Delay $\\times$ Bandwitdh bits 的资料量。(P39) Foundation - 40 RTT 可以近似理解为 2 $\\times$ Propagation time,因为一个来回需要从 sender 到 reciever,再从 reciever 到 sender。 ","date":"2024-01-14","objectID":"/posts/nthu-computer-network/:2:4","tags":["Network"],"title":"國立清華大學 計算機網路 重點提示","uri":"/posts/nthu-computer-network/"},{"categories":["Network"],"content":"Homework Redis 作者 Salvatore Sanfilippo 的聊天室项目: smallchat,通过该项目可以入门学习网络编程 (Network Programming),请复现该项目。 Salvatore Sanfilippo 在 YouTube 上对 smallchat 的讲解: Smallchat intro smallchat client \u0026 raw line input GitHub 上也有使用 Go 和 Rust 实现该项目的仓库,如果你对 Go 或 Rust 的网络编程 (Network Programming) 感兴趣,可以参考这个仓库。 ","date":"2024-01-14","objectID":"/posts/nthu-computer-network/:2:5","tags":["Network"],"title":"國立清華大學 計算機網路 重點提示","uri":"/posts/nthu-computer-network/"},{"categories":["Network"],"content":"IEEE 802.3 Ethernet Outline: Introduction Ethernet Topologies Ethernet Frame Format Ethernet MAC Protocol – CSMA/CD 802.3 Ethernet Standards Summary: MAC Protocol – CSMA/CD Connection less, unreliable transmission Topology from Bus to Star (switches) Half-duplex transmission in Bus topology Work best under lightly loaded conditions Too much collision under heavy load Full-duplex transmission in Switch topology (point-to-point) No more collisions !! Excellent performance (wired speed) ","date":"2024-01-14","objectID":"/posts/nthu-computer-network/:3:0","tags":["Network"],"title":"國立清華大學 計算機網路 重點提示","uri":"/posts/nthu-computer-network/"},{"categories":["Network"],"content":"Introduction Ethernet - 03 Ethernet 发展过程: 传输速度从 10Mb 发展到 100Gb (P4) Ethernet 的特点: Unreliable, Connectionless, CSMA/CD (P5) ","date":"2024-01-14","objectID":"/posts/nthu-computer-network/:3:1","tags":["Network"],"title":"國立清華大學 計算機網路 重點提示","uri":"/posts/nthu-computer-network/"},{"categories":["Network"],"content":"Ethernet Topologies Ethernet - 07 Ethernet - 18 10Base5: 10Mbps, segment up to 500m (P8) 10Base2: 10Mbps, segment up to 200m (P8) 10BaseT: 10Mbps, Twisted pair, segment up to 100m (P16) Repeater, Hub 都是 physical layer 的设备,只负责 转发信号,无法防止 collision (P12, P16) Switch 则是 data-link layer 的设备,内置芯片进行 数据转发,可以防止 collision (P19) Manchester Encoding (P11): Ethernet 下层的 physical layer 使用的编码方式是 Manchester Encoding: 在一个时钟周期内,信号从低到高表示 1,从高到低表示 0 注意 Manchester Encoding 发送方在进行数据传输之前需要发送一些 bits 来进行时钟同步 (例如 P22 的 Preamble 部分),接收方完成时钟同步后,可以对一个时钟周期进行两次采样:一次前半段,一次后半段,然后可以通过两次取样电位信号的变化来获取对应的 bit (低到高表示 1,高到低表示 0)。 有些读者可能会疑惑,既然都进行时钟同步了,为什么不直接使用高电位信号表示 1,低电位信号表示 0 这样直观的编码方式?这是因为如果采取这种编码方式,那么在一个时钟周期内信号不会有变化,如果接收的是一系列的 1 或 0,信号也不会变化。这样可能会导致漏采样,或者编码出错却无法及时侦测。而采用 Manchester Encoding 接收方每个时钟周期内信号都会变化,如果接收方在一次时钟周期内的两次采样,信号没有发生变化,那么可以立即侦测到出错了 (要么是漏采样了,要么是编码出错了)。 ","date":"2024-01-14","objectID":"/posts/nthu-computer-network/:3:2","tags":["Network"],"title":"國立清華大學 計算機網路 重點提示","uri":"/posts/nthu-computer-network/"},{"categories":["Network"],"content":"Ethernet Frame Format Ethernet - 23 除开 Preamble, SFD 之外,一个 Frame 的大小为 $64 \\sim 1518$ bytes。因为 DA, SA, TYPE, FCS 占据了 $6 + 6 + 2 + 4 = 18$ bytes,所以 Data 部分的大小为 $48 ~\\sim 1500$ bytes (P43) MAC Address 是 unique 并且是与 Adaptor 相关的,所以一个主机可能没有 MAC Address (没有 Adaptor),可能有两个 MAC Address (有两个 Adaptor)。MAC Address 是由 Adaptor 的生产商来决定的。(P24) unicast address, broadcast address, multicast address (P26) ","date":"2024-01-14","objectID":"/posts/nthu-computer-network/:3:3","tags":["Network"],"title":"國立清華大學 計算機網路 重點提示","uri":"/posts/nthu-computer-network/"},{"categories":["Network"],"content":"CSMA/CD Ethernet - 46 Ethernet - 41 Ethernet - 45 Ethernet - 49 关于 CSMA/CD 的详细介绍可以查看 P34 ~ P38 关于 Ethernet Frame 的大小限制设计可以查看 P39 ~ P43 关于 CSMA/CD Collision Handling 的策略机制可以查看 P44 ~ P45, P47 ~ P48 注意 Host 在 detect collision 之后进行 backoff random delay,delay 结束后按照 1-persistent protocol (P35) 继续等待到 busy channel goes idle 后立刻进行传输。 ","date":"2024-01-14","objectID":"/posts/nthu-computer-network/:3:4","tags":["Network"],"title":"國立清華大學 計算機網路 重點提示","uri":"/posts/nthu-computer-network/"},{"categories":["Network"],"content":"IEEE 802.11 Wireless LAN 无线网络这章太难了,战术性放弃 ","date":"2024-01-14","objectID":"/posts/nthu-computer-network/:4:0","tags":["Network"],"title":"國立清華大學 計算機網路 重點提示","uri":"/posts/nthu-computer-network/"},{"categories":["Network"],"content":"IEEE 802.1D Spanning Tree Algorithm ","date":"2024-01-14","objectID":"/posts/nthu-computer-network/:5:0","tags":["Network"],"title":"國立清華大學 計算機網路 重點提示","uri":"/posts/nthu-computer-network/"},{"categories":["Network"],"content":"Referenecs 110-1 計算機網路 (清大開放式課程) 小菜学网络 NUDT 高级计算机网络实验: 基于UDP的可靠传输 可靠 UDP 的实现 (KCP over UDP) 基于 UDP 的可靠传输 [bilibili] 实现基于 UDP 的网络文件传输器,程序员的经验大礼包项目 [bilibili] ping 命令但是用来通信,学习计算机网络好项目,也可能是校园网福利 [bilibili] Implementing TCP in Rust [YouTube] Let's code a TCP/IP stack ","date":"2024-01-14","objectID":"/posts/nthu-computer-network/:6:0","tags":["Network"],"title":"國立清華大學 計算機網路 重點提示","uri":"/posts/nthu-computer-network/"},{"categories":["C","Linux Kernel Internals"],"content":"C 语言规格书阅读学习记录。 规格书草案版本为 n1256,对应 C99 标准,对应的 PDF 下载地址。 也配合 C11 标准来阅读,草案版本 n1570,对应的 PDF 下载地址。 阅读规格书需要一定的体系结构、编译原理的相关知识,但不需要很高的程度。请善用检索工具,在阅读规格书时遇到术语时,请先在规格书中进行检索,因为极大可能是规格书自己定义的术语。 ","date":"2024-01-06","objectID":"/posts/c-specification/:0:0","tags":["C","Sysprog"],"title":"C 语言规格书 重点提示","uri":"/posts/c-specification/"},{"categories":["C","Linux Kernel Internals"],"content":"6. Language ","date":"2024-01-06","objectID":"/posts/c-specification/:1:0","tags":["C","Sysprog"],"title":"C 语言规格书 重点提示","uri":"/posts/c-specification/"},{"categories":["C","Linux Kernel Internals"],"content":"6.2 Concepts ","date":"2024-01-06","objectID":"/posts/c-specification/:2:0","tags":["C","Sysprog"],"title":"C 语言规格书 重点提示","uri":"/posts/c-specification/"},{"categories":["C","Linux Kernel Internals"],"content":"6.2.2 Linkages of identifiers linkage: external internal none 一个拥有 file scope 并且关于 object 或 function 的 identifier 声明,如果使用 static 修饰,则该 identifer 有 internal linkage,e.g. // file scope static int a; static void f(); int main() {} 一个 scope 内使用 static 修饰的 identifier 声明,如果在同一 scope 内已存在该 identifier 声明,则该 identifier 的 linkage 取决于先前的 identifier 声明。如果该 identifier 不存在先前声明或者先前声明 no linkage,则该 identifier 是 external linkage,e.g. // Example 1 static int a; // a is internal linkage extern int a; // linkage is the same as prior // Example 2 extern int b; // no prior, a is external linkage extern int b; // linkage is the same as prior 如果一个 function identifier 声明没有 storage-class 修饰符,则其 linkage 等价于加上 extern 修饰的声明的 linkage,e.g. int func(int a, int b); // equal to `extern int func(int a. int b);` // and then no prior, it is external linkage 如果一个 object identifier 声明没有 storage-class 修饰符,且拥有 file scope,则其拥有 external linkage,e.g. // file scope int a; // external linkage int main() {} ","date":"2024-01-06","objectID":"/posts/c-specification/:2:1","tags":["C","Sysprog"],"title":"C 语言规格书 重点提示","uri":"/posts/c-specification/"},{"categories":["C","Linux Kernel Internals"],"content":"6.5 Expressions ","date":"2024-01-06","objectID":"/posts/c-specification/:3:0","tags":["C","Sysprog"],"title":"C 语言规格书 重点提示","uri":"/posts/c-specification/"},{"categories":["C","Linux Kernel Internals"],"content":"6.5.3 Unary operators 注意 C99 [6.2.5] Types There are three real floating types, designated as float, double, and long double. The real floating and complex types are collectively called the floating types. The integer and real floating types are collectively called real types. Integer and floating types are collectively called arithmetic types. A function type describes a function with specified return type. A function type is characterized by its return type and the number and types of its parameters. A function type is said to be derived from its return type, and if its return type is T, the function type is sometimes called ‘‘function returning T’’. The construction of a function type from a return type is called ‘‘function type derivation’’. Arithmetic types and pointer types are collectively called scalar types. C99 [6.3.2.1] Lvalues, arrays, and function designators A function designator is an expression that has function type. Except when it is the operand of the sizeof operator or the unary \u0026 operator, a function designator with type ‘‘function returning type’’ is converted to an expression that has type ‘‘pointer to function returning type’’. 6.5.3.1 Prefix increment and decrement operators Constraints 前缀自增或自减运算符的操作数,必须为实数 (real types) 类型(即不能是复数)或者是指针类型,并且其值是可变的。 Semantics ++E 等价于 (E+=1) --E 等价于 (E-=1) 6.5.3.2 Address and indirection operators Constraints \u0026 运算符的操作数必须为 function designator,[] 或 * 的运算结果,或者是一个不是 bit-field 和 register 修饰的左值。 * 运算符的操作数必须为指针类型。 Semantics \u0026*E 等价于 E,即 \u0026 和 * 被直接忽略,但是它们的 constraints 仍然起作用。所以 (\u0026*(void *)0) 并不会报错。 \u0026a[i] 等价于 a + i,即忽略了 \u0026 以及 * (由 [] 隐式指代)。 其它情况 \u0026 运算的结果为一个指向 object 或 function 的指针。 如果 * 运算符的操作数是一个指向 function 的指针,则结果为对应的 function designator。 如果 * 运算符的操作数是一个指向 object 的指针,则结果为指示该 obejct 的左值。 如果 * 运算符的操作数为非法值的指针,则对该指针进行 * 运算的行为三未定义的。 6.5.3.3 Unary arithmetic operators Constraints 单目 + 或 - 运算符的操作数必须为算数类型 (arithmetic type),~ 运算符的操作数必须为整数类型 (integer type),! 运算符的操作数必须为常数类型 (scalar type)。 Semantics 在进行单目 +、-、~ 运算之前,会对操作数进行整数提升 (integer promotions),结果的类型与操作数进行整数提升后的类型一致。 !E 等价于 (E==0),结果为 int 类型。 ","date":"2024-01-06","objectID":"/posts/c-specification/:3:1","tags":["C","Sysprog"],"title":"C 语言规格书 重点提示","uri":"/posts/c-specification/"},{"categories":["C","Linux Kernel Internals"],"content":"6.5.6 Additive operators 介绍加减法运算,其中包括了指针的运算,务必阅读这部分关于指针运算的标准说明。 ","date":"2024-01-06","objectID":"/posts/c-specification/:3:2","tags":["C","Sysprog"],"title":"C 语言规格书 重点提示","uri":"/posts/c-specification/"},{"categories":["C","Linux Kernel Internals"],"content":"6.5.7 Bitwise shift operators Constraints 位运算的操作数都必须为整数类型。 Semantics 在进行位运算之前会先对操作数进行整数提升 (integer promotion),位运算结果类型与整数提升后的左操作数一致。如果右运算数是负数,或者大于等于整数提升后的左运算数的类型的宽度,那么这个位运算行为是未定义的。 假设运算结果的类型为 T $E1 \u003c\u003c E2$ 如果 E1 是无符号,则结果为 $E1 \\times 2^{E2} \\bmod (\\max[T] + 1)$。 如果 E1 是有符号,E1 不是负数,并且 T 可以表示 $E1 \\times 2^{E2}$,则结果为 $E1 \\times 2^{E2}$。 除了以上两种行为外,其他均是未定义行为。 $E1 \u003e\u003e E2$ 如果 E1 是无符号,或者 E1 是有符号并且是非负数,则结果为 $E1 / 2^{E2}$。 如果 E1 是有符号并且是负数,则结果由具体实现决定 (implementation-defined)。 ","date":"2024-01-06","objectID":"/posts/c-specification/:3:3","tags":["C","Sysprog"],"title":"C 语言规格书 重点提示","uri":"/posts/c-specification/"},{"categories":["C","Linux Kernel Internals"],"content":"7. Library ","date":"2024-01-06","objectID":"/posts/c-specification/:4:0","tags":["C","Sysprog"],"title":"C 语言规格书 重点提示","uri":"/posts/c-specification/"},{"categories":["C","Linux Kernel Internals"],"content":"7.18 Integer types \u003cstdint.h\u003e 描述了头文件 stdint.h 必须定义和实现的整数类型,以及相应的宏。 ","date":"2024-01-06","objectID":"/posts/c-specification/:5:0","tags":["C","Sysprog"],"title":"C 语言规格书 重点提示","uri":"/posts/c-specification/"},{"categories":["C","Linux Kernel Internals"],"content":"7.18.1 Integer types 7.18.1.1 Exact-width integer types 二补数编码,固定长度 N 的整数类型: 有符号数:intN_t 无符号数:uintN_t 7.18.1.2 Minimum-width integer types 至少拥有长度 N 的整数类型: 有符号数:int_leastN_t 无符号数:uint_leastN_t 7.18.1.3 Fastest minimum-width integer types 至少拥有长度 N,且操作速度最快的整数类型: 有符号数:int_fastN_t 无符号数:uint_fastN_t 7.18.1.4 Integer types capable of holding object pointers 可以将指向 void 的有效指针转换成该整数类型,也可以将该整数类型转换回指向 void 的指针类型,并且转换结果与之前的指针值保持一致: 有符号数:intptr_t 无符号数:uintptr_t 7.18.1.5 Greatest-width integer types 可以表示任意整数类型所表示的值的整数类型,即具有最大长度的整数类型: 有符号数:intmax_t 无符号数:uintmax_t ","date":"2024-01-06","objectID":"/posts/c-specification/:5:1","tags":["C","Sysprog"],"title":"C 语言规格书 重点提示","uri":"/posts/c-specification/"},{"categories":["Toolkit"],"content":"Git 中文教学 新手入门推荐,对于 Git 的入门操作讲解十分友好。 视频地址 学习记录 ","date":"2024-01-04","objectID":"/posts/git/:1:0","tags":["Git","GitHub"],"title":"Git/GitHub 资源与问题汇总","uri":"/posts/git/"},{"categories":["Toolkit"],"content":"Git 常见问题及解决 ","date":"2024-01-04","objectID":"/posts/git/:2:0","tags":["Git","GitHub"],"title":"Git/GitHub 资源与问题汇总","uri":"/posts/git/"},{"categories":["Toolkit"],"content":"git pull/push 遇到 Port 22 connect timeout 网络问题导致 22 端口被禁止,无法正常使用 ssh。切换成 443 端口并且编写配置文件即可: $ vim ~/.ssh/config # In ~/.ssh/config Host github.com HostName ssh.github.com Port 443 ","date":"2024-01-04","objectID":"/posts/git/:2:1","tags":["Git","GitHub"],"title":"Git/GitHub 资源与问题汇总","uri":"/posts/git/"},{"categories":["Toolkit"],"content":"GitHub 支持多个账户通过 ssh 连接 Using multiple github accounts with ssh keys ","date":"2024-01-04","objectID":"/posts/git/:2:2","tags":["Git","GitHub"],"title":"Git/GitHub 资源与问题汇总","uri":"/posts/git/"},{"categories":["Toolkit"],"content":"References Git 基本原理 Learn Git Branching DIY a Git ugit 动手学习GIT - 最好学习GIT的方式是从零开始做一个 ","date":"2024-01-04","objectID":"/posts/git/:3:0","tags":["Git","GitHub"],"title":"Git/GitHub 资源与问题汇总","uri":"/posts/git/"},{"categories":["Linux Kernel Internals"],"content":" 人们对数学的加减运算可轻易在脑中辨识符号并理解其结果,但电脑做任何事都受限于实体资料储存及操作方式,换言之,电脑硬体实际只认得 0 和 1,却不知道符号 + 和 - 在数学及应用场域的意义,於是工程人员引入「补数」以便在二进位系统中,表达人们认知上的正负数。但您有没有想过,为何「二补数」(2’s complement) 被电脑广泛采用呢?背後的设计考量又是什麽?本文尝试从数学观点去解读编码背後的原理,并佐以资讯安全及程式码最佳化的考量,探讨二补数这样的编码对于程式设计有何关键影响。 原文地址:解讀計算機編碼 技巧 为了更好的理解本文的一些数学概念,例如群,以及后续其他关于数值系统、浮点数的讲座,Jserv 强烈建议我们去修读数学系的 数学导论。笔者在这里分享一下台大齐震宇老师的 2015 年的新生营讲座,这个讲座覆盖了数学导论的内容。 YouTube: 臺大 2015 數學系新生營 ","date":"2023-12-31","objectID":"/posts/binary-representation/:0:0","tags":["Sysprog","Numerics","Bitwise"],"title":"解读计算机编码","uri":"/posts/binary-representation/"},{"categories":["Linux Kernel Internals"],"content":"一补数 (Ones’ complement) ","date":"2023-12-31","objectID":"/posts/binary-representation/:1:0","tags":["Sysprog","Numerics","Bitwise"],"title":"解读计算机编码","uri":"/posts/binary-representation/"},{"categories":["Linux Kernel Internals"],"content":"9 的补数 科普短片: Not just counting, but saving lives: Curta ","date":"2023-12-31","objectID":"/posts/binary-representation/:1:1","tags":["Sysprog","Numerics","Bitwise"],"title":"解读计算机编码","uri":"/posts/binary-representation/"},{"categories":["Linux Kernel Internals"],"content":"运算原理 注意 以一补数编码形式表示的运算子,在参与运算后,运算结果符合一补数的编码: $$ [X]_{一补数} + [Y]_{一补数} = [X+Y]_{一补数} $$ 接下来进行分类讨论,以 32-bit 正数 $X$, $Y$ 为例: $X + Y = X + Y$ 显然运算子和运算结果都满足一补数编码。 $X - Y = X + (2^{32} - 1 - Y)$ 如果 $X \u003e Y$,则运算结果应为 $X - Y$ 且为正数,其一补数编码为 $X - Y$。而此时 $$ 2^{32} - 1 + X - Y $$ 显然会溢出,为了使运算结果对应一补数编码,所以此时循环进位对应 $+\\ (1 - 2_{32})$。 如果 $X \u003c Y$,则运算结果应为 $X - Y$ 且为负数,其一补数编码为 $$ 2^{32} - 1 - (Y - X) = 2_{32} - 1 - X - Y $$ 而此时 $2^{32} - 1 + X - Y$ 并不会溢出,并且满足运算结果的一补数编码,所以无需进行循环进位。 如果 $X = Y$,显然 $$ X - Y = X + 2^{32} - 1 - Y = 2^{32} - 1 $$ 为 0 成立。 $-X - Y = (2^{32} - 1 - X) + (2^{32} - 1 - Y)$,显然会导致溢出。而 $-X - Y$ 的一补数编码为 $$ 2^{32} - 1 - (X + Y) = 2^{32} - 1 - X - Y $$ 所以需要在溢出时循环进位 $+\\ (1 - 2^{32})$ 来消除运算结果中的一个 $2^{32} - 1$。 ","date":"2023-12-31","objectID":"/posts/binary-representation/:1:2","tags":["Sysprog","Numerics","Bitwise"],"title":"解读计算机编码","uri":"/posts/binary-representation/"},{"categories":["Linux Kernel Internals"],"content":"二补数 (Two’s complement) ","date":"2023-12-31","objectID":"/posts/binary-representation/:2:0","tags":["Sysprog","Numerics","Bitwise"],"title":"解读计算机编码","uri":"/posts/binary-representation/"},{"categories":["Linux Kernel Internals"],"content":"正负数编码表示 假设有 n-bit 的二补数编码 $A$,$-A$ 的推导如下: 格式一: $$ \\begin{align*} A + \\neg A \u0026= 2^n - 1 \\\\ A + \\neg A + 1 \u0026\\equiv 0 \\equiv 2^n \\ (\\bmod 2^n) \\\\ -A \u0026= \\neg A + 1 \\\\ \\end{align*} $$ 格式二: $$ \\begin{align*} A + \\neg A \u0026= 2^n - 1 \\\\ A + \\neg A - 1 \u0026= 2^n - 2 \\\\ A - 1 \u0026= 2^n - 1 - (\\neg A + 1) \\\\ \\neg (A - 1) \u0026= \\neg A + 1 \\\\ \\neg (A - 1) \u0026= -A \\\\ \\end{align*} $$ 也可以通过一补数和二补数,在时钟表上的对称轴偏差,来理解上述两种方式是等价的。 Twos’ complement 注意 在二补数编码中,将一个整数转换成其逆元,也可以依据以下的方法: 以 LSB 到 MSB 的顺序,寻找第一个值为 1 的 bit,将这个 bit 以及比其更低的 bits (包含该 bit) 都保持不变,将比该 bit 更高的 bits (不包括该 bit) 进行取反操作。下面是一些例子 (以 32-bit 为例): 0x0080 \u003c-\u003e 0xff80 0x0001 \u003c-\u003e 0xffff 0x0002 \u003c-\u003e 0xfffe ","date":"2023-12-31","objectID":"/posts/binary-representation/:2:1","tags":["Sysprog","Numerics","Bitwise"],"title":"解读计算机编码","uri":"/posts/binary-representation/"},{"categories":["Linux Kernel Internals"],"content":"加 / 减法器设计 科普短片: See How Computers Add Numbers In One Lesson ✅ 了解晶体管的原理 了解基本逻辑门元件,例如 OR, AND 逻辑门的设计 了解加法器的原理和工作流程。 ","date":"2023-12-31","objectID":"/posts/binary-representation/:2:2","tags":["Sysprog","Numerics","Bitwise"],"title":"解读计算机编码","uri":"/posts/binary-representation/"},{"categories":["Linux Kernel Internals"],"content":"阿贝尔群及对称性 技巧 群论的最大用途是关于「对称性」的研究;所有具有对称性质,群论都可派上用场。只要发生变换后仍有什么东西还维持不变,那符合对称的性质。 一个圆左右翻转后还是圆,它在这种变换下是对称的,而这刚好与群的 封闭性 (Closure) 对应。 一个时钟的时刻,从 0 时刻开始,两边的时刻相加模 12 的结果均为 0,这与群的 单位元 (Identity element) 和 逆元 (Inverse element) 对应。 上述两个例子反映了群论的性质,对于对称性研究的重要性和原理依据。 科普影片: 从五次方程到伽罗瓦理论 ","date":"2023-12-31","objectID":"/posts/binary-representation/:3:0","tags":["Sysprog","Numerics","Bitwise"],"title":"解读计算机编码","uri":"/posts/binary-representation/"},{"categories":["Linux Kernel Internals"],"content":"旁路攻击 观看科普视频: 我听得到你打了什么字 ✅ 阅读相关论文 Keyboard Acoustic Emanations 体验使用相关工具 kbd-audio 借由 Wikipedia 了解旁路攻击 (Side-channel attack) 和时序攻击 (Timing attack) 的基本概念 ✅ Black-box testing Row hammer Cold boot attack Rubber-hose cryptanalysis 延伸阅读 The password guessing bug in Tenex Side Channel Attack By Using Hidden Markov Model One\u0026Done: A Single-Decryption EM-Based Attack on OpenSSL’s Constant-Time Blinded RSA ","date":"2023-12-31","objectID":"/posts/binary-representation/:4:0","tags":["Sysprog","Numerics","Bitwise"],"title":"解读计算机编码","uri":"/posts/binary-representation/"},{"categories":["Linux Kernel Internals"],"content":"Constant-Time Functions 比较常见的常数时间实作方法是,消除分支。因为不同分支的执行时间可能会不同,这会被利用进行时序攻击。这个方法需要对 C 语言中的编码和位运算有一定的了解。 C99 7.18.1.1 Exact-width integer types C99 6.5.7.5 Bitwise shift operators Source Branchless abs 如果 n 是 signed 32-bit,则 n \u003e\u003e 31 等价于 n == 0 ? 0 : -1 方法一,原理为 $-A = \\neg (A - 1)$: #include \u003cstdint.h\u003e int32_t abs(int32_t x) { int32_t mask = (x \u003e\u003e 31); return (x + mask) ^ mask; } 方法二,原理为 $-A = \\neg A + 1$: #include \u003cstdint.h\u003e int32_t abs(int32_t x) { int32_t mask = (x \u003e\u003e 31); return (x ^ mask) - mask; } Branchless min/max Min: #include \u003cstdint.h\u003e int32_t min(int32_t a, int32_t b) { int32_t diff = (a - b); return b + (diff \u0026 (diff \u003e\u003e 31)); } 如果 diff \u003e 0 即 b 小,那么 (diff \u003e\u003e 31) == 0,则 b + (diff \u0026 (diff \u003e\u003e 31)) == b 如果 diff \u003c 0 即 a 小,那么 (diff \u003e\u003e 31) == -1,则 b + (diff \u0026 (diff \u003e\u003e 31)) == b + (a - b) == a Max: #include \u003cstdint.h\u003e int32_t max(int32_t a, int32_t b) { int32_t diff = (b - a); return b - (diff \u0026 (diff \u003e\u003e 31)); } 如果 diff \u003e 0 即 b 大, 那么 (diff \u003e\u003e 31) == 0,则 b - (diff \u0026 (diff \u003e\u003e 31)) == b 如果 diff \u003c 0 即 a 大,那么 (diff \u003e\u003e 31) == -1,则 b - (diff \u0026 (diff \u003e\u003e 31)) == b - (b - a) == a 信息 基于 C 语言标准研究与系统程序安全议题 ","date":"2023-12-31","objectID":"/posts/binary-representation/:4:1","tags":["Sysprog","Numerics","Bitwise"],"title":"解读计算机编码","uri":"/posts/binary-representation/"},{"categories":["Linux Kernel Internals","Rust"],"content":"Rust in 100 Seconds 观看短片: Rust in 100 Seconds ✅ 了解 Rust,初步了解其安全性原理 所有权 (ownership) 借用 (borrow) 警告 0:55 This is wrong, value mutability doesn’t have anything to do with the value being stored on the stack or the heap (and the example let mut hello = \"hi mom\" will be stored on the stack since it’s type is \u0026'static str), it depends on the type of the value (if it’s Sized or not). ","date":"2023-12-28","objectID":"/posts/why-rust-/:1:0","tags":["Rust","Sysprog"],"title":"Why Rust?","uri":"/posts/why-rust-/"},{"categories":["Linux Kernel Internals","Rust"],"content":"The adoption of Rust in Business (2022) 阅读报告: The adoption of Rust in Business (2022) ✅ Rust 目前蓬勃发展,预测未来是很难的,但是 Rust 已经是进行时的未来了 🤣 ","date":"2023-12-28","objectID":"/posts/why-rust-/:2:0","tags":["Rust","Sysprog"],"title":"Why Rust?","uri":"/posts/why-rust-/"},{"categories":["Linux Kernel Internals","Rust"],"content":"Visualizing memory layout of Rust’s data types YouTube: Visualizing memory layout of Rust’s data types 影片的中文翻译: 可视化 Rust 各数据结构的内存布局 [bilibili] 可搭配阅读相关的文档: [2022-05-04] 可视化 Rust 各数据类型的内存布局 ","date":"2023-12-28","objectID":"/posts/why-rust-/:3:0","tags":["Rust","Sysprog"],"title":"Why Rust?","uri":"/posts/why-rust-/"},{"categories":["Toolkit","Linux Kernel Internals"],"content":"教学影片:Git 中文教学 ","date":"2023-12-27","objectID":"/posts/git-learn/:0:0","tags":["Git","GitHub"],"title":"Git 学习记录","uri":"/posts/git-learn/"},{"categories":["Toolkit","Linux Kernel Internals"],"content":"安装与设定 注意 ✅ 观看影片 Git 教学系列 - 安装与配置,完成常用的 Git 设置。 设置 Git 的编辑器为 vim,主要用于 commit 时的编辑: $ git config --global core.editor vim 设置 Git 的合并解决冲突工具为 vimdiff: $ git config --global merge.tool vimdiff 启用 Git 命令行界面的颜色显示: $ git config --global color.ui true 设置常用命令的别名: $ git config --global alias.st status $ git config --global alias.ch checkout $ git config --global alias.rst reset HEAD 效果为:命令 git st 等价于 git status,其余的类似。 设置 Windows 和 Mac/Linux 的换行符同步: # In Windows $ git config --global core.autocrlf true # In Mac/Linux $ git config --global core.autocrlf input 效果为:在 Windows 提交时自动将 CRLF 转为 LF,检出代码时将 LF 转换成 CRLF。在 Mac/Linux 提交时将 CRLF转为 LF,检出代码时不转换。这是因为 Windows 的换行符为 \\r\\n,而 Mac/Linux 的换行符仅为 \\n。 ","date":"2023-12-27","objectID":"/posts/git-learn/:1:0","tags":["Git","GitHub"],"title":"Git 学习记录","uri":"/posts/git-learn/"},{"categories":["Toolkit","Linux Kernel Internals"],"content":"Add 和 Commit ","date":"2023-12-27","objectID":"/posts/git-learn/:2:0","tags":["Git","GitHub"],"title":"Git 学习记录","uri":"/posts/git-learn/"},{"categories":["Toolkit","Linux Kernel Internals"],"content":"指定 Commit 注意 ✅ 观看影片 Git 教学系列 - 指定 Commit,掌握 git log、git show、git diff 的常用方法。理解 Hash Value 和 commit 对于 Git 版本控制的核心作用。 只要 commit 了,资料基本不可能丢失,即使误操作了也是可以补救回来的(除非把 .git/ 文件夹也删除了)。 ","date":"2023-12-27","objectID":"/posts/git-learn/:3:0","tags":["Git","GitHub"],"title":"Git 学习记录","uri":"/posts/git-learn/"},{"categories":["Toolkit","Linux Kernel Internals"],"content":"Hash Value Every commit has a unique hash value. Calculate by SHA1 Hash value can indicate a commit absolutely. ","date":"2023-12-27","objectID":"/posts/git-learn/:3:1","tags":["Git","GitHub"],"title":"Git 学习记录","uri":"/posts/git-learn/"},{"categories":["Toolkit","Linux Kernel Internals"],"content":"Indicate Commit git manage references to commit HEAD Branch Tag Remote Also, We can indicate commit by ^, ~ 通俗地将,不论是 HEAD、Branch、Tag、Remote,其本质都是使用 Hash Value 进行索引的 commit,所以 ~ 和 ^ 也可以作用于它们。 可以通过 git log 来查看 commit 以及对应的 Hash 值。事实上,这个命令十分灵活,举个例子: git log 4a6ebc -n1 这个命令的效果是从 Hash 值为 4a6bc 的 commit 开始打印 1 条 commit 记录(没错,对应的是 -n1),因为 Git 十分聪明,所以 commit 对应的 Hash 值只需前 6 位即可(因为这样已经几乎不会发生 Hash 冲突)。 Examples 打印 master 分支的最新一个 commit: git log master -n1 打印 master 分支的最新一个 commit(仅使用一行打印 commit 信息): git log master -n1 --oneline 打印 HEAD 所指向的 commit: git log HEAD -n1 --oneline 打印 HEAD 所指向的 commit 的前一个 commit: git log HEAD^ -n1 --oneline ^ 可以持续使用,比如 HEAD^^ 表示 HEAD 所指向的 commit 的前两个 commit。当 ^ 数量过多时,可以使用 ~ 搭配数字来达到相同效果。例如: git log HEAD^^^^^ -n1 --oneline git log HEAD~5 -n1 --oneline 一般来说,使用 ^ 就已经足够了,几乎不会遇到使用 ~ 的场景,因为这种场景一般会去找图形化界面吧。🤣 打印与文件 README.md 相关的 commits(仅使用一行显示): git log --oneline README.md 打印与文件 README.md 相关的 commits(显示详细信息,包括文件内容的增减统计): git log --stat README.md 打印与文件 README.md 相关的 commits(显示详细信息,包括文件内容的增减细节): git log --patch README.md 在打印的 commit 信息中抓取与 README 符合的信息(可以与 --stat 或 --patch 配合使用): git log -S README ","date":"2023-12-27","objectID":"/posts/git-learn/:3:2","tags":["Git","GitHub"],"title":"Git 学习记录","uri":"/posts/git-learn/"},{"categories":["Toolkit","Linux Kernel Internals"],"content":"View History git log \u003cpath\u003e|\u003ccommit\u003e -n: limit number --oneline: view hash and commit summary --stat: view files change --patch: view lines change -S or --grep: find modification ","date":"2023-12-27","objectID":"/posts/git-learn/:3:3","tags":["Git","GitHub"],"title":"Git 学习记录","uri":"/posts/git-learn/"},{"categories":["Toolkit","Linux Kernel Internals"],"content":"View Commit git show \u003ccommit\u003e Equal to log -n1 ","date":"2023-12-27","objectID":"/posts/git-learn/:3:4","tags":["Git","GitHub"],"title":"Git 学习记录","uri":"/posts/git-learn/"},{"categories":["Toolkit","Linux Kernel Internals"],"content":"See Difference 查看当前的修改,可以查看已经修改但没有 staged 文件的变化: git diff 查看当前的修改,可以查看已经修改且 staged 文件的变化: git diff --staged 查看当前与指定的 commit 的差异: git diff \u003ccommit\u003e # e.g. git diff master^ 查两个指定的 commit 之间的差异: git diff \u003ccommit\u003e \u003ccommit\u003e # e.g. git diff master^ master^^ ","date":"2023-12-27","objectID":"/posts/git-learn/:3:5","tags":["Git","GitHub"],"title":"Git 学习记录","uri":"/posts/git-learn/"},{"categories":["Toolkit","Linux Kernel Internals"],"content":"Path Add and Amend 注意 ✅ 观看影片 Git 教学系列 - Patch Add and Amend,掌握 git add -p、git checkout -p、git add ---amend 的用法,使用 add 和 checkout 时强烈建议使用 -p,掌握修改 commit 的两种方法。 ","date":"2023-12-27","objectID":"/posts/git-learn/:4:0","tags":["Git","GitHub"],"title":"Git 学习记录","uri":"/posts/git-learn/"},{"categories":["Toolkit","Linux Kernel Internals"],"content":"Only Add Related git add -p 推荐尽量使用这个 git add -p 而不是单纯的 git add。 使用 git add -p 后,Git 会帮我们把涉及的修改分成 section,然后我们就可以对每一个 section 涉及的修改进行 review,选择 y(yes) 表示采纳该 sction 对应的修改,选择 n(no) 表示不采纳。 如果觉得 section 切割的粒度太大了,可以选择 s(split) 来进行更细粒度的划分。如果这样仍然觉得粒度不够,可以选择 e(edit) 对 section 涉及的修改,进行以行为粒度的 review,具体操作可以查阅此时给出的提示。 还有一些其它的选项,比如 j、J、k、K,这些是类似 vim,用于切换进行 review 的 section,不太常用。q(quit) 表示退出。 由于可以针对一个文件的不同 section 进行 review,所以在进行 git add -p 之后,使用 git status 可以发现同一个文件会同时处于两种状态。 ","date":"2023-12-27","objectID":"/posts/git-learn/:4:1","tags":["Git","GitHub"],"title":"Git 学习记录","uri":"/posts/git-learn/"},{"categories":["Toolkit","Linux Kernel Internals"],"content":"Checkout Also git checkout -p 这个操作比较危险,因为这个操作的效果与 git add -p 相反,如果选择 y 的话,文件涉及的修改就会消失,如果涉及的修改没有 commit 的话,那么涉及的修改是无法救回的。但是怎么说,这个操作还是比直接使用 git checkout 稍微保险一点,因为会先进入 review 界面,而不是直接撤销修改。所以,请一定要使用 git checkout -p! ","date":"2023-12-27","objectID":"/posts/git-learn/:4:2","tags":["Git","GitHub"],"title":"Git 学习记录","uri":"/posts/git-learn/"},{"categories":["Toolkit","Linux Kernel Internals"],"content":"Modify Commit 有两种方式来修改最新的 commit: # 1. Use git commit --amend git commit --amend # 2. Use reset HEAD^ then re-commit git reset HEAD^ git add -p git commit git commit --amend 并不是直接替换原有的 commit,而是创建了一个新的 commit 并重新设置了 HEAD 的指向。所以,新旧两个 commit 的 Hash Value 并不相同,事实上,如果你拥有旧 commit 的 Hash Value,是可以通过 git checkout \u003ccommit\u003e 切换到那个 commit 的。其原理如下图: 但是注意,git reset HEAD^ 是会撤销原先的 commit(仅限于本地 Git 存储库)。 ","date":"2023-12-27","objectID":"/posts/git-learn/:4:3","tags":["Git","GitHub"],"title":"Git 学习记录","uri":"/posts/git-learn/"},{"categories":["Toolkit","Linux Kernel Internals"],"content":"Branch and Merge 注意 ✅ 观看影片 Git 教学系列 - Branch and Merge,掌握创建、删除、切换分支的用法,掌握合并分支、解决冲突的方法。 git checkout \u003ccommit\u003e git branch \u003cname\u003e git branch \u003cname\u003e \u003ccommit\u003e git branch [-d|-D] \u003cname\u003e git merge \u003cname\u003e --no-ff ","date":"2023-12-27","objectID":"/posts/git-learn/:5:0","tags":["Git","GitHub"],"title":"Git 学习记录","uri":"/posts/git-learn/"},{"categories":["Toolkit","Linux Kernel Internals"],"content":"Move and Create Branch Checkout: move HEAD git checkout \u003ccommit\u003e: Move HEAD to commit git checkout \u003cpath\u003e: WARNING: discard change 可以将路径上的文件复原到之前 commit 的状态。 Branch: git branch: List branch git branch \u003cname\u003e: Create branch Or just: git checkout -b Examples 修改一个文件并恢复: # modify file load.cpp git status git checkout load.cpp git status 删除一个文件并恢复: rm load.cpp git status git checkout load.cpp git status 正如上一节所说的,git checkout 尽量带上 -p 参数,因为如果一不小心输入了 git checkout .,那就前功尽弃了。 显示分支: # only show name git branch # show more infomation git branch -v 切换分支: # switch to branch 'main' git checkout main 创建分支: # 1. using `git branch` git branch cload # 2. using `git checkout -b` git checkout -b asmload # 3. create a new branch in \u003ccommit\u003e git branch cload \u003ccommit\u003e 切换到任一 commit: git checkout \u003ccommit\u003e 直接 checkout 到任一 commit 会有警告,这是因为,当你以该 commit 为基点进行一系列的 commit,这些新的 commit 会在你切换分支后消失,因为没有 branch 来引用它们。之前可以被引用是因为 HEAD 引用,切换分支后 HEAD 不再引用这些 commit,所以就会消失。在这种情况,Git 会在发出警告的同时建议我们使用 git branch 来创建分支进行引用。 ","date":"2023-12-27","objectID":"/posts/git-learn/:5:1","tags":["Git","GitHub"],"title":"Git 学习记录","uri":"/posts/git-learn/"},{"categories":["Toolkit","Linux Kernel Internals"],"content":"View Branch 列出仓库的所有分支: git branch 也可以通过 log 来查看分支: git log --decorate: 在 log 的首行显示所有的 references(可能需要通过 git config log.decorate auto 来开启) --graph: 以图形化的方式显示 branch 的关系(主要是 commit 的引用) ","date":"2023-12-27","objectID":"/posts/git-learn/:5:2","tags":["Git","GitHub"],"title":"Git 学习记录","uri":"/posts/git-learn/"},{"categories":["Toolkit","Linux Kernel Internals"],"content":"Delete Branch 删除分支: git branch -d \u003cname\u003e 对于有没有 merge 的 commit 的分支,Git 会警告,需要使用 -D 来强制删除: git branch -D \u003cname\u003e for no-merge commit WARNING: Discard Commit Git 会发出警告的原因同样是 no-merge commit 在删除分支后就无法被引用,所以会发出警告。 ","date":"2023-12-27","objectID":"/posts/git-learn/:5:3","tags":["Git","GitHub"],"title":"Git 学习记录","uri":"/posts/git-learn/"},{"categories":["Toolkit","Linux Kernel Internals"],"content":"Merge 合并分支。默认使用 fast-forward,即如果没有冲突,直接将要合并的分支提前到被合并分支的 commit 处,而不会另外生成一个 merge commit。但这样会使得被合并的分支在合并后,没有历史痕迹。可以通过 --no-ff (no fast forward) 来强制生成 merge commit。推荐使用 merge 时加上 --no-ff 这个参数。 git merge \u003cbranch\u003e 通常是 main/master 这类主分支合并其它分支: git checkout main/master git merge \u003cbranch\u003e ","date":"2023-12-27","objectID":"/posts/git-learn/:5:4","tags":["Git","GitHub"],"title":"Git 学习记录","uri":"/posts/git-learn/"},{"categories":["Toolkit","Linux Kernel Internals"],"content":"Resolve Conflict Manually resolve: Check every codes between \u003c\u003c\u003c\u003c\u003c\u003c\u003c, \u003e\u003e\u003e\u003e\u003e\u003e\u003e Edit code to what it should be Use mergetool like vimdiff: It shows: local, base, remote, file to be edited Edit “file ro be edited” to what is should be Add and Commit # 1. 合并分支 git merge \u003cbranch\u003e # 2. 检查状态,查看 unmerged 的文件 git status # 3. 编辑 unmerged 文件,编辑冲突区域代码即可 vim \u003cfile\u003e # 4. 添加解决完冲突的文件 git add \u003cfile\u003e # 5. 进行 merge commit git commit 冲突区域就是 \u003c\u003c\u003c\u003c\u003c\u003c\u003c 和 \u003e\u003e\u003e\u003e\u003e\u003e\u003e 内的区域,在 merge 操作后,Git 已经帮我们把 unmerged 文件修改为待解决冲突的状态,直接编辑文件即可。在编辑完成后,需要手动进行 add 和 commit,此次 commit 的信息 Git 已经帮我们写好了,一般不需要修改。 如果使用的是 mergetool,以 vimdiff 为例,只需将第 3 步的 vim \u003cfile\u003e 改为 git mergetool 即可。vimdiff 会提供 4 个视窗:底部视窗是我们的编辑区,顶部左边是当前合并分支的状态,顶部中间是 base (合并分支和被合并的共同父节点) 的状态,顶部右边是 remote 的状态,按需要选择、编辑。 vimdiff 在编辑完后会保留 *.orig 的文件,这个文件是待解决冲突的文件副本。 ","date":"2023-12-27","objectID":"/posts/git-learn/:5:5","tags":["Git","GitHub"],"title":"Git 学习记录","uri":"/posts/git-learn/"},{"categories":["Toolkit","Linux Kernel Internals"],"content":"Merge Conflict Prevent very long development branch. Split source code clearly. ","date":"2023-12-27","objectID":"/posts/git-learn/:5:6","tags":["Git","GitHub"],"title":"Git 学习记录","uri":"/posts/git-learn/"},{"categories":["Toolkit","Linux Kernel Internals"],"content":"Rebase 注意 ✅ 观看影片 Git 教学系列 - Branch and Merge,掌握 TODO 的方法。git rebase 是 Git 的精华,可以让我们实现更细粒度的操作,可以说学会了 rebase 才算真正入门了 Git。 这个视频讲得比较乱,所以推荐配合视频给出的参考文章 Git-rebase 小笔记 来学习。 ","date":"2023-12-27","objectID":"/posts/git-learn/:6:0","tags":["Git","GitHub"],"title":"Git 学习记录","uri":"/posts/git-learn/"},{"categories":["Toolkit"],"content":"网络代理 根据项目 clash-for-linux-backup 来配置 Ubuntu 的网络代理。 $ git clone https://github.com/Elegybackup/clash-for-linux-backup.git clash-for-linux 过程当中可能需要安装 curl 和 net-tools,根据提示进行安装即可: sudo apt install curl sudo apt install net-tools 安装并启动完成后,可以通过 localhost:9090/ui 来访问 Dashboard。 启动代理: $ cd clash-for-linux $ sudo bash start.sh $ source /etc/profile.d/clash.sh $ proxy_on 关闭代理: $ cd clash-for-linux $ sudo bash shutdown.sh $ proxy_off ","date":"2023-12-27","objectID":"/posts/ubuntu22.04lts/:1:0","tags":["Linux","Ubuntu"],"title":"Ubuntu 22.04LTS 相关配置","uri":"/posts/ubuntu22.04lts/"},{"categories":["Toolkit"],"content":"搜狗输入法 根据 搜狗输入法 Linux 安装指导 来安装搜狗输入法。 安装时无需卸载系统 ibus 输入法框架 (与上面的安装指导不一致) 通过 Ctrl + space 唤醒搜狗输入法 通过 Ctrl + Shift + Z 呼出特殊符号表 ","date":"2023-12-27","objectID":"/posts/ubuntu22.04lts/:2:0","tags":["Linux","Ubuntu"],"title":"Ubuntu 22.04LTS 相关配置","uri":"/posts/ubuntu22.04lts/"},{"categories":["Toolkit"],"content":"快捷键 新建终端: Ctrl + Alt + T 锁屏: Super + L:锁定屏幕并熄屏。 显示桌面: Super + d 或者 Ctrl + Alt + d 最小化所有运行的窗口并显示桌面,再次键入则重新打开之前的窗口。 显示所有的应用程序: Super + a 可以通过 ESC 来退出该显示。 显示当前运行的所有应用程序: Super 移动窗口位置: Super + 左箭头:当前窗口移动到屏幕左半边区域 Super + 右箭头:当前窗口移动到屏幕右半边区域 Super + 上箭头:当前窗口最大化 Super + 下箭头:当前窗口恢复正常 隐藏当前窗口到任务栏: Super + h 切换当前的应用程序: Super + Tab:以应用程序为粒度显示切换选项 Alt + Tab:以窗口为粒度显示切换选项 切换虚拟桌面/工作区: Ctrl + Alt + 左/右方向键 自定义键盘快捷键: Settings -\u003e Keyboard -\u003e Keyboard Shortcus | View and Customize Shortcuts -\u003e Custom Shortcuts ","date":"2023-12-27","objectID":"/posts/ubuntu22.04lts/:3:0","tags":["Linux","Ubuntu"],"title":"Ubuntu 22.04LTS 相关配置","uri":"/posts/ubuntu22.04lts/"},{"categories":["Toolkit","Linux Kernel Internals"],"content":" 摘要 GNU/Linux 开发工具,几乎从硬件到软件,Linux 平台能够自下而上提供各类触及“灵魂”的学习案例,让所有课程从纸上谈兵转变成沙场实战,会极大地提升工程实践的效率和技能。 原文地址 ","date":"2023-12-25","objectID":"/posts/gnu-linux-dev/:0:0","tags":["Sysprog","Linux"],"title":"GNU/Linux 开发工具","uri":"/posts/gnu-linux-dev/"},{"categories":["Toolkit","Linux Kernel Internals"],"content":"安装 Windows / Ubuntu 双系统 因为有些操作必须在物理硬件上才能执行。 ","date":"2023-12-25","objectID":"/posts/gnu-linux-dev/:1:0","tags":["Sysprog","Linux"],"title":"GNU/Linux 开发工具","uri":"/posts/gnu-linux-dev/"},{"categories":["Toolkit","Linux Kernel Internals"],"content":"Markdown 与 LaTeX 速览 LaTeX 语法示例一节,作为工具书册,在需要使用时知道如何查询。 速览 Markdown 语法示例一节,作为工具书册,在需要使用时知道如何查询。 注意 编写 Markdown 文本以及 LaTeX 语法表示的数学式可以通过: Hugo + FixIt ✅ VS Code + Markdown Preview Enhanced ","date":"2023-12-25","objectID":"/posts/gnu-linux-dev/:2:0","tags":["Sysprog","Linux"],"title":"GNU/Linux 开发工具","uri":"/posts/gnu-linux-dev/"},{"categories":["Toolkit","Linux Kernel Internals"],"content":"Git 和 Github 阅读 SSH key 产生方法一节,配置好 Git 和 Github 的 SSH key。同时也可作为工具书册,在需要使用时知道如何查询。 推荐通过 LearnGitBranching 来熟悉 Git 命令!!! 以下资源作为自学资源,用于补充自己不熟悉的操作,或者作为以上资料的补充工具手册。 Git 中文教学 - YouTube (学习记录) 30 天精通 Git 版本控制 - GitHub 警告 原文档中的将公钥复制到 clipboard 中使用了 clip 命令,但是这个命令在 Ubuntu 中并没有对应的命令。可以使用 xclip + alias 达到近似效果。 $ sudo apt install xclip # using alias to implement clip, you can add this to bashrc $ alias clip='xclip -sel c' $ clip \u003c ~/.ssh/id_rsa.pub ","date":"2023-12-25","objectID":"/posts/gnu-linux-dev/:3:0","tags":["Sysprog","Linux"],"title":"GNU/Linux 开发工具","uri":"/posts/gnu-linux-dev/"},{"categories":["Toolkit","Linux Kernel Internals"],"content":"编辑器: Visual Studio Code 认真阅读,跟随教学文档进行安装、设置。重点阅读 设定、除错(调试) 这两部分。更新 VS Code 部分作为手册,在需要时进行参考。 以下资源作为自学资源,用于补充自己不熟悉的操作,或者作为以上资料的补充工具手册。 开开心心学 Vistual Studio Code 完成 SSH key 的生成。 完成 VS Code 的设置。 安装 Git History 插件。 安装 Native Debug 插件,并进行 Debug (test-stopwatch.c) 操作。 安装 VSCode Great Icons 文件图标主题,另外推荐两款颜色主题:One Dark Pro, Learn with Sumit。 VS Code 控制台使用说明: 可以在面板的输出,点击 GIT 选项显示 VS Code 背后执行的 git 命令。 可以使用 ctrl + shift + P 呼出命令区,然后通过输入 Git branch 和 Git checkout 等并选择对应选项,来达到创建分支、切换分支等功能。 技巧 在 VS Code 设置中,需要在设置中打开 Open Default Settings 选项才能在左侧面板观察到预设值。键位绑定同理。 要想进行调试,需要在使用 gcc 生成目标文件时,加入 -g 参数来生产调试信息。 原文档中的 GDB 教学链接-除错程式-gdb 已失效,这是目前的有效链接。也可通过该影片 拯救资工系学生的基本素养-使用 GDB 除错基本教学 来补充学习 GDB 的操作。 ","date":"2023-12-25","objectID":"/posts/gnu-linux-dev/:4:0","tags":["Sysprog","Linux"],"title":"GNU/Linux 开发工具","uri":"/posts/gnu-linux-dev/"},{"categories":["Toolkit","Linux Kernel Internals"],"content":"终端和 Vim 认真阅读,跟随教学影片 快快乐乐学 Vim 和教学文档配置好 终端提示符、Vim。 完成命令行提示符配置 完成 Vim 的设定 安装并使用 Minial Vim Plugin Manager 来管理 Vim 插件 (neocomplcache, nerdtree) 安装并使用 byobu 来管理多个终端视图。 技巧 在 .vimrc 中增加插件后,打开 vim,执行 :PlugInstall 来安装插件,完成后在 vim 执行 :source ~/.vimrc。(可以通过 :PlugStatus 来查看插件安装状态) 使用 F4 键来[显示/不显示][行数/相对行数]。 使用 F5 键来呼入/呼出文件树(nerdtree),在文件树恻通过 ENTER 键来访问目录/文件。 使用 Ctrl-w-h/Ctrl-w-l 切换到 文件树/编辑区。 自动补全时使用 ENTER 键来选中,使用方向键或 Ctrl-N/Ctrl-U/Ctrl-P 来上下选择。 在 Vim 中可以通过 :set paste,并在 insert 模式下,将粘贴板的内容通过 Ctrl-Shift-V 进行粘贴。 byobu 使用说明: 在终端输入 byobu F2 新增 Terminial 分页。F3, F4 在 Terminial 分页中切换。Ctrl +F6 删除当前 Terminial 分页。 Shift + F2 水平切割 Terminial。Ctrl +F2 垂直切割 Terminial。Shift + 方向键 切换。 在 byobu 中暂时无法使用之前设置的 F4 或 F5 快捷键,但是可以直接通过命令 :set norelative 来关闭相对行数。 推荐观看影片 How to Do 90% of What Plugins Do (With Just Vim) 来扩展 Vim 插件的使用姿势。 以下资源为 Cheat Sheet,需要使用时回来参考即可。 Vim Cheat Sheet Bash terminal Cheat Sheet ","date":"2023-12-25","objectID":"/posts/gnu-linux-dev/:5:0","tags":["Sysprog","Linux"],"title":"GNU/Linux 开发工具","uri":"/posts/gnu-linux-dev/"},{"categories":["Toolkit","Linux Kernel Internals"],"content":"Makefile 速览教学文档,作为工具书册,在需要使用时知道如何查询。 gcc 的 -MMD 和 -MF 参数对我们编写 Makefile 是一个巨大利器。理解 Makefile 的各种变量定义的原理。 对之前的 test-stopwatch.c 编写了一个 Makefile 来自动化管理。 ","date":"2023-12-25","objectID":"/posts/gnu-linux-dev/:6:0","tags":["Sysprog","Linux"],"title":"GNU/Linux 开发工具","uri":"/posts/gnu-linux-dev/"},{"categories":["Toolkit","Linux Kernel Internals"],"content":"Linux 性能分析工具: Perf 认真阅读,复现教学文档中的所有例子,初步体验 perf 在性能分析上的强大。 安装 perf 并将 kernel.perf_event_paranoid 设置为 1。 动手使用 perf_top_example.c,体验 perf 的作用。 搭配影片: Branch Prediction 对照阅读: Fast and slow if-statements: branch prediction in modern processors 编译器提供的辅助机制: Branch Patterns, Using GCC 动手使用 perf_top_while.c,体验 perf top 的作用。 动手使用 perf_stat_cache_miss.c,体验 perf stat 的作用。(原文的结果有些不直观,务必亲自动手验证) 动手使用 perf_record_example.c,体验 perf record 的作用。(原文的操作不是很详细,可以参考下面的 Success) Source 成功 $ perf record -e branch-misses:u,branch-instructions:u ./perf_record_example [ perf record: Woken up 1 times to write data ] [ perf record: Captured and wrote 0.009 MB perf.data (94 samples) ] 输出第一行表示 perf 工具在收集性能数据时被唤醒了 1 次,以将数据写入输出文件。 输出第二行表示 perf 工具已经取样并写入了一个名为 perf.data 的二进制文件,文件大小为 0.009 MB,其中包含了 94 个采样。(可以通过 ls 命令来检查 perf.data 文件是否存在) 接下来通过 perf report 对之前输出的二进制文件 perf.data 进行分析。可以通过方向键选择,并通过 ENTER 进入下一层查看分析结果。 $ perf report Available samples 5 branch-misses:u 89 branch-instructions:u 技巧 perf 需要在 root 下进行性能分析。 perf top 是对于哪个程序是性能瓶颈没有头绪时使用,可以查看哪个程序(以及程序的哪个部分)是热度点。 在 perf top 时可以通过 h 键呼出帮助列表。 可以通过方向键选择需要进一步分析的部分,并通过 a 键来查看指令级别粒度的热点。 perf stat 是对某一个要优化的程序进行性能分析,对该程序涉及的一系列 events 进行取样检查。 perf record 的精度比 perf stat 更高,可以对取样的 events 进行函数粒度的分析。 ","date":"2023-12-25","objectID":"/posts/gnu-linux-dev/:7:0","tags":["Sysprog","Linux"],"title":"GNU/Linux 开发工具","uri":"/posts/gnu-linux-dev/"},{"categories":["Toolkit","Linux Kernel Internals"],"content":"Linux 绘图工具: gnuplot 阅读教程,搭配教学影片 轻轻松松学 gnuplot,使用 gnuplot 完成所给例子相应图像的绘制。 使用 runtime.gp 完成 runtime.png 的绘制生成。 使用 statistic.gp 完成降雨量折线图 statistic.png 的绘制生成。 注意 原文所给的 statistic.gp 是使用 Times_New_Roman 来显示中文的,但笔者的 Ubuntu 中并没有这个字体,所以会显示乱码。可以通过 fc-list :lang=zh 命令来查询当前系统中的已安装的中文字体。 Source 安装 gnuplot: $ sudo apt-get install gnuplot gnuplot script 的使用流程: # 创建并编写一个后缀名为 .gp 的文件 $ vim script.gp # 根据 script 内的指令进行绘图 $ gnuplot script.gp # 根据 script 指定的图片保存路径打开图片 $ eog [name of picture] 下面以一个 script 进行常用指令的说明: reset set ylabel 'time(sec)' set style fill solid set title 'performance comparison' set term png enhanced font 'Verdana,10' set output 'runtime.png' plot [:][:0.100]'output.txt' using 2:xtic(1) with histogram title 'original', \\ '' using ($0-0.06):($2+0.001):2 with labels title ' ', \\ '' using 3:xtic(1) with histogram title 'optimized' , \\ '' using 4:xtic(1) with histogram title 'hash' , \\ '' using ($0+0.3):($3+0.0015):3 with labels title ' ', \\ '' using ($0+0.4):($4+0.0015):4 with labels title ' ' reset 指令的作用为,将之前 set 指令设置过的内容全部重置。 set style fill solid 将绘制出的柱形或区域使用实心方式填充。 set term png enhanced font 'Verdana,10' term png 生成的图像以 png 格式进行保存。(term 是 terminial 的缩写) enhanced 启用增强文本模式,允许在标签和注释中使用特殊的文本格式,如上下标、斜体、下划线等。 font 'Verdana,10' 指定所使用的字体为 Verdana,字号为10。可进行自定义设置。 其它指令查询原文或手册即可。 $0 在 gnuplot 中表示伪列,可以简单理解为行号,以下为相应图示: 原始数据集: append() 0.048240 0.040298 0.057908 findName() 0.006495 0.002938 0.000001 (人为)增加了 伪列 表示的数据集(最左边 0, 1 即为伪列): 0 append() 0.048240 0.040298 0.057908 1 findName() 0.006495 0.002938 0.000001 技巧 gnuplot 在绘制生成图像时是安装指令的顺序进行的,并且和一般的画图软件类似,在最上层进行绘制。所以在编写 script 的指令时需要注意顺序,否则生成图像的部分可能并不像预期一样位于最上层。(思考上面 script 的 3, 4 列的 label 的绘制顺序) gnuplot script 中的字符串可以使用 '' 或者 \"\" 来包裹,同样类似于 Python。 直接在终端输入 gnuplot 会进入交互式的命令界面,也可以使用 gnulpot 指令来绘图(类似与 Python)。在这种交互式界面环境中,如果需要在输入完指令后立即显示图像到新窗口,而不是保存图像再打开,只需输入进行指令: set term wxt ehanced persist raise term wxt 将图形终端类型设置为WXT,这会在新窗口中显示绘图。 ersist 该选项使绘图窗口保持打开状态,即使脚本执行完毕也不会自动关闭。 raise 该选项将绘图窗口置于其他窗口的前面,以确保它在屏幕上的可见性。 一些额外的教程: Youtube - gnuplot Tutorlal 这个教程有五部影片,到发布者的主页搜寻即可。 ","date":"2023-12-25","objectID":"/posts/gnu-linux-dev/:8:0","tags":["Sysprog","Linux"],"title":"GNU/Linux 开发工具","uri":"/posts/gnu-linux-dev/"},{"categories":["Toolkit","Linux Kernel Internals"],"content":"Linux 绘图工具: Graphviz 官方网站 一小时实践入门 Graphviz 安装: $ sudo apt install graphviz 查看安装版本: $ dot -V dot - graphviz version 2.43.0 (0) 通过脚本生成图像: $ dot -Tpng example.dot -o example.png Graphviz 在每周测验题的原理解释分析时会大量使用到,请务必掌握以实作出 Readable 的报告。 ","date":"2023-12-25","objectID":"/posts/gnu-linux-dev/:9:0","tags":["Sysprog","Linux"],"title":"GNU/Linux 开发工具","uri":"/posts/gnu-linux-dev/"},{"categories":["Toolkit","Linux Kernel Internals"],"content":"其它工具 ","date":"2023-12-25","objectID":"/posts/gnu-linux-dev/:10:0","tags":["Sysprog","Linux"],"title":"GNU/Linux 开发工具","uri":"/posts/gnu-linux-dev/"},{"categories":["Toolkit","Linux Kernel Internals"],"content":"man $ man man The table below shows the section numbers of the manual followed by the types of pages they contain. 1 Executable programs or shell commands 2 System calls (functions provided by the kernel) 3 Library calls (functions within program libraries) 4 Special files (usually found in /dev) 5 File formats and conventions, e.g. /etc/passwd 6 Games 7 Miscellaneous (including macro packages and conventions), e.g. man(7), groff(7), man-pages(7) 8 System administration commands (usually only for root) 9 Kernel routines [Non standard] ","date":"2023-12-25","objectID":"/posts/gnu-linux-dev/:10:1","tags":["Sysprog","Linux"],"title":"GNU/Linux 开发工具","uri":"/posts/gnu-linux-dev/"},{"categories":["Toolkit","Linux Kernel Internals"],"content":"cloc man 1 cloc cloc - Count, or compute differences of, lines of source code and comments. ","date":"2023-12-25","objectID":"/posts/gnu-linux-dev/:10:2","tags":["Sysprog","Linux"],"title":"GNU/Linux 开发工具","uri":"/posts/gnu-linux-dev/"},{"categories":["Toolkit","Linux Kernel Internals"],"content":"top man 1 top top - display Linux processes ","date":"2023-12-25","objectID":"/posts/gnu-linux-dev/:10:3","tags":["Sysprog","Linux"],"title":"GNU/Linux 开发工具","uri":"/posts/gnu-linux-dev/"},{"categories":["Toolkit","Linux Kernel Internals"],"content":"htop man 1 htop htop - interactive process viewer ","date":"2023-12-25","objectID":"/posts/gnu-linux-dev/:10:4","tags":["Sysprog","Linux"],"title":"GNU/Linux 开发工具","uri":"/posts/gnu-linux-dev/"},{"categories":["Mathematics"],"content":"这里记录一些收集到的数学开放式课程学校资料。 中大數學系開放式課程 國立台灣大學 齊震宇 數學導論:相關講義 | 教學錄影 數學潛水艇:綫性代數、拓撲 微積分 分析 國立台灣大學 謝銘倫 綫性代數 ","date":"2023-12-23","objectID":"/posts/math/:0:0","tags":["Mathematics"],"title":"数学开放式课程学习指引","uri":"/posts/math/"},{"categories":["Mathematics"],"content":"逻辑、集合论 ","date":"2023-12-23","objectID":"/posts/math/:1:0","tags":["Mathematics"],"title":"数学开放式课程学习指引","uri":"/posts/math/"},{"categories":["Mathematics"],"content":"簡易邏輯 在联结词构成的复合命题中,命题的语义和真值需要分开考虑,特别是联结词 $\\implies$。以 $p \\implies q$ 为例 (注意 $p$ 和 $q$ 都是抽象命题,可指代任意命题,类似于未知数 $x$),如果从 $p$ 和 $q$ 的语义考虑,很容易就陷入语义的 “如果 $p$ 则 $q$” 这类语义混淆中,导致强加因果,但因为在逻辑上 $p$ 和 $q$ 可以没有任何关系,所以此时忽略它们的语义,而只根据它们的真值和相关定义上推断该复合命题的真值 ($p \\implies q$ 等价于 $(\\neg p) \\lor q$)。简单来说,逻辑上的蕴涵式包括语义上的因果关系,即因果关系是蕴涵式的真子集。 第 16 页的趣味问题可以通过以下 “标准” 方式 (这里的 “标准” 指的是一种通用方法思路,并非应试教育中的得分点) 来解决: 令悟空、八戒、悟净和龙马分别为 $a, b, c, d$,令命题「$X$ 第 $Y$」为 $XY$,例如 “悟空第一” 则表示为命题 $a1$,则有以下命题成立: $$ \\begin{split} \u0026 (c1 \\land (\\neg b2)) \\lor ((\\neg c1) \\land b2) \\\\ \\land\\ \u0026 (c2 \\land (\\neg d3)) \\lor ((\\neg c2) \\land d3) \\\\ \\land\\ \u0026 (d4 \\land (\\neg a2)) \\lor ((\\neg d4) \\land a2) \\\\ \\end{split} $$ 然后化简该表达式即可得到结果 (因为这个问题是精心设计过的,所以会有一个唯一解)。 性质也是命题,即其真假值可以谈论 (但未必能确定)。但正如第 22 页的注一所说,一般不讨论性质 $A(x)$ 的真假值,只有将具体成代入时才有可能谈论其真假值 (这很好理解,抽象的不定元 $x$ 可以代表无限多的东西,代入性质 $A(x)$ 的真假值可能并不相同)。但是需要注意后面集合论中虽然也使用了性质来定义群体,但是此时的性质表示 $A(x)$ 这个命题为真,即 $x$ 满足 $A$ 这个性质。所以需要细心看待逻辑学和集合论的性质一次,它们有共同点也有不同点。 处理更多不定元的性质时,按照第 24 ~ 27 页的注二的方法,将其转换成简单形式的单一不定元的性质进行处理。 ","date":"2023-12-23","objectID":"/posts/math/:1:1","tags":["Mathematics"],"title":"数学开放式课程学习指引","uri":"/posts/math/"},{"categories":["Mathematics"],"content":"集合概念簡介 第 42 页上运用了类似的化简不定元技巧,通过括号将同一时间处理不定元数量减少为 1.除此之外,这里还有一个不等式和区间/射线符号的技巧: 不带等号的不等式和区间/射线符号搭配使用时,需要反转区间/射线符号,这个技巧可以从区间/射线符号的定义推导而来。以该页最后的例子为例: $$ (\\bigcup_{m \\in (0,1)}(1-\\frac{1}{k}, 9-m]) \\land (8 \u003c 9-m \u003c 9) \\\\ \\begin{split} (1-\\frac{1}{k}, 9-m] \u0026= \\{x \\in \\mathbb{R} | 1-\\frac{1}{k} \u003c x \u003c= 9-m \u003c 9 \\} \\\\ \u0026= \\{x \\in \\mathbb{R} | 1-\\frac{1}{k} \u003c x \u003c 9 \\} \\\\ \u0026= (1-\\frac{1}{k}, 9) \\\\ \\end{split} $$ 倒数第二个例子也类似: $$ (\\bigcap_{j \\in \\mathbb{R},\\ j\u003e0}(-j, 9)) \\land (-j\u003c0) \\\\ \\begin{split} (-j, 9) \u0026= \\{x \\in \\mathbb{R} | -j \u003c 0 \u003c= x \u003c 9 \\} \\\\ \u0026= \\{x \\in \\mathbb{R} | 0 \u003c= x \u003c 9 \\} \\\\ \u0026= [0, 9) \\\\ \\end{split} $$ 第 45 ~ 46 页分别展示了例行公事式的证明和受过教育似的证明这两种方法,需要注意的是第一钟方法使用的是 逻辑上等价 进行推导 (因为它一次推导即可证明两个集合等价),而第二种方法使用的是 蕴涵 进行推导 (因为它是通过两个方向分别证明包含关系,不需要等价性推导)。例行公事式的证明的要点在于,事先说明后续证明涉及的元素 $x$ 的性质,然后在后续证明过程中某一步将这个性质加入,进而构造出另一个集合的形式。 练习一: 例行公事式的证明 练习二: 例行公事式的证明 ","date":"2023-12-23","objectID":"/posts/math/:1:2","tags":["Mathematics"],"title":"数学开放式课程学习指引","uri":"/posts/math/"},{"categories":["Mathematics"],"content":"初等整數論 第 13 页 (反转了的) 辗转相除法的阅读顺序是:先阅读左边,在阅读右边,右边的推导是将上面的式子代入到下面的式子得来。 ","date":"2023-12-23","objectID":"/posts/math/:2:0","tags":["Mathematics"],"title":"数学开放式课程学习指引","uri":"/posts/math/"},{"categories":["Mathematics"],"content":"群、群作用與 Burnside 引理 第 16 页的 $Perm(X)$ 表示 $X$ 的元素进行置换对应的所有映射构成的集合,这个集合的基数为 $8!$。表示这个集合的某个元素 (也就是置换对应的映射),可以用投影片上的形如 $(1\\ 4\\ 2)(3\\ 7)(5)(6)$ 来表示,比较直观的体现这个映射的效果。 第 18 页子群定义的结合律一般不需要特别考虑,因为子群的任意元素属于群,而群的元素都满足结合律,所以子群的任意元素都满足结合律。 第 18 页的证明提示「消去律」,是指在证明子群性质时利用群的 可逆 和 单位元 性质进行证明。因为依据定义,群的单位元可作用的范围比子群的单位元作用范围广。 ","date":"2023-12-23","objectID":"/posts/math/:3:0","tags":["Mathematics"],"title":"数学开放式课程学习指引","uri":"/posts/math/"},{"categories":["Mathematics"],"content":"綫性代數","date":"2023-12-23","objectID":"/posts/math/:4:0","tags":["Mathematics"],"title":"数学开放式课程学习指引","uri":"/posts/math/"},{"categories":["draft"],"content":"博客(英语:Blog)是一种在线日记型式的个人网站,借由张帖子章、图片或视频来记录生活、抒发情感或分享信息。博客上的文章通常根据张贴时间,以倒序方式由新到旧排列。 ","date":"2023-12-23","objectID":"/posts/hello_world/:0:0","tags":["draft"],"title":"Hello, World","uri":"/posts/hello_world/"},{"categories":["draft"],"content":"数学公式 行内公式:$N(b,d)=(b-1)M$ 公式块: $$ \\int_{a}^{b}x(t)dt = \\dfrac{b - a}{N} \\\\ =\\sum_{k=1}^{N}x(t_k)\\cdot\\dfrac{b-a}{N} $$ $$ \\begin{aligned} \\int_{a}^{b}x(t)dt \u0026= \\dfrac{b - a}{N} \\\\ \u0026=\\sum_{k=1}^{N}x(t_k)\\cdot\\dfrac{b-a}{N} \\\\ \\end{aligned} $$ $$ \\mathrm{Integrals\\ are\\ numerically\\ approximated\\ as\\ finite\\ series}:\\\\ \\begin{split} \\int_{a}^{b}x(t)dt \u0026= \\dfrac{b - a}{N} \\\\ \u0026=\\sum_{k=1}^{N}x(t_k)\\cdot\\dfrac{b-a}{N} \\end{split} \\\\ where\\ t_k = a + (b-a)\\cdot k/N $$ $$ \\begin{align*} p(x) = 3x^6 + 14x^5y \u0026+ 590x^4y^2 + 19x^3y^3 \\\\ \u0026- 12x^2y^4 - 12xy^5 + 2y^6 - a^3b^3 - a^2b - ab + c^5d^3 + c^4d^3 - cd \\end{align*} $$ $$ \\begin{split} \u0026(X \\in B) = X^{-1}(B) = {s \\in S: X(s) \\in B} \\subset S \\\\ \u0026\\Rightarrow P(x \\in B) = P({s \\in S: X(s) \\in B}) \\end{split} $$ ","date":"2023-12-23","objectID":"/posts/hello_world/:1:0","tags":["draft"],"title":"Hello, World","uri":"/posts/hello_world/"},{"categories":["draft"],"content":"代码块 let i: i32 = 13; let v = vec![1, 2, 3, 4, 5, 65]; for x in v.iter() { println!(\"{}\", x); } typedef struct Block_t { int head; int data; } Block_t; ","date":"2023-12-23","objectID":"/posts/hello_world/:2:0","tags":["draft"],"title":"Hello, World","uri":"/posts/hello_world/"},{"categories":["draft"],"content":"Admonition 注意 111年度資工所心得 摘要 Georgia Tech - Advanced Operating Systems: Part 1 / Part 2 / Part 3 / Part 4 信息 Reddit: Best book to learn in-depth knowledge about the Linux Kernel? Project: Linux From Scratch Book: Linux Kernel Development Video: Steven Rostedt - Learning the Linux Kernel with tracing 技巧 WIkipedia: Xenix / Multics / Plan9 / FreeBSD 成功 一个 成功 横幅 问题 一个 问题 横幅 警告 一个 警告 横幅 失败 一个 失败 横幅 危险 一个 危险 横幅 Bug 一个 Bug 横幅 示例 一个 示例 横幅 引用 一个 引用 横幅 ","date":"2023-12-23","objectID":"/posts/hello_world/:3:0","tags":["draft"],"title":"Hello, World","uri":"/posts/hello_world/"},{"categories":["draft"],"content":"References FixIt 快速上手 使用 Hugo + Github 搭建个人博客 Markdown 基本语法 Emoji 支持 扩展 Shortcodes 概述 图表支持 URL management ","date":"2023-12-23","objectID":"/posts/hello_world/:4:0","tags":["draft"],"title":"Hello, World","uri":"/posts/hello_world/"}] \ No newline at end of file diff --git a/docs/posts/c-compiler-construction/index.html b/docs/posts/c-compiler-construction/index.html index 7ff5cb11..7d361b9f 100644 --- a/docs/posts/c-compiler-construction/index.html +++ b/docs/posts/c-compiler-construction/index.html @@ -15,8 +15,8 @@ AMaCC 是由成功大學師生開發的 self-compiling 的 C 語言編譯器,可產生 Arm 架構的執行檔 (ELF 格式,運作在 GNU/Linux)、也支援 just-in-time (JIT) 編譯和執行,原始程式碼僅 1500 行,在這次講座中,我們就來揭開 AMaCC 背後的原理和實作議題。 預期會接觸到 IR (Intermediate representation), dynamic linking, relocation, symbol table, parsing tree, language frontend, Arm 指令編碼和 ABI 等等。 "> - - + + - + @@ -47,8 +47,8 @@ "mainEntityOfPage": { "@type": "WebPage", "@id": "https:\/\/ccrysisa.github.io\/posts\/c-compiler-construction\/" - },"genre": "posts","keywords": "Sysprog, C, Compiler","wordcount": 782 , - "url": "https:\/\/ccrysisa.github.io\/posts\/c-compiler-construction\/","datePublished": "2024-04-23T15:17:38+08:00","dateModified": "2024-04-25T19:49:18+08:00","publisher": { + },"genre": "posts","keywords": "Sysprog, C, Compiler","wordcount": 1659 , + "url": "https:\/\/ccrysisa.github.io\/posts\/c-compiler-construction\/","datePublished": "2024-04-23T15:17:38+08:00","dateModified": "2024-04-30T17:00:03+08:00","publisher": { "@type": "Organization", "name": ""},"author": { "@type": "Person", @@ -176,7 +176,7 @@
AMaCC 是由成功大學師生開發的 self-compiling 的 C 語言編譯器,可產生 Arm 架構的執行檔 (ELF 格式,運作在 GNU/Linux)、也支援 just-in-time (JIT) 編譯和執行,原始程式碼僅 1500 行,在這次講座中,我們就來揭開 AMaCC 背後的原理和實作議題。
+預期會接觸到 IR (Intermediate representation), dynamic linking, relocation, symbol table, parsing tree, language frontend, Arm 指令編碼和 ABI 等等。
+编译原理课程教导的是如何完成一个「编译器的编译器」 即 Compiler-compiler,这个难度比较大,因为需要考虑通用性,但是实作一个简单的编译器并没有这么难。
一般而言,编译器的编写分为 3 个步骤:
+在该项目的虚拟机设计中,函数调用时 callee 的参数位于 caller 的栈帧 (frame) 内,并且函数调用需要使用到 4 条指令:
+CALL
指令将 callee 的返回地址压入栈,然后跳转到 callee 的入口处ENT
指令保存 ebp 寄存器的值并为 callee 的栈帧设置 esp 和 ebp 寄存器,在栈中给 callee 的局部变量分配空间ADJ
指令在 callee 逻辑执行完毕后,释放之前分配给局部变量的栈空间LEV
指令将 ebp 和 esp 寄存器恢复为对应 caller 栈帧,并跳转到之前保存的 callee 的返回地址处除此之外,在 callee 执行期间,可能需要通过 LEA
指令来访问函数参数和局部变量。
+
|
+
+
|
PRTF
means?
+
+
|
+
+
|
这里 c4 对于 PRTF
指令的处理暂时没看明白…
-m32
error
+ gcc 通过 -m32
参数编译本节代码时可能会遇到以下报错:
+
|
+
+
|
这是因为当前安装的 gcc 只有 64 位的库而没有 32 位的库,通过以下命令安装 32 位库解决问题:
+
+
|
+
+
|
Stack Overflow:
+ +JIT (Just in time) 表示“即时”,形象描述就是“及时雨” 🤣 原理是将解释执行的“热点“编译成位于一个内存区域的 machine code,从而减轻内存的压力 (解释执行时会在内存中跳来跳去,而一个区域的 machine code 是连续执行,内存压力没这么大并且可以充分利用 cache 从而提高效能)。
+JIT (Just in time) 表示“即时”,形象描述就是“及时雨” 🤣 原理是将解释执行的“热点“编译成位于一个内存区域的 machine code,从而减轻内存的压力。因为解释执行时会在内存中跳来跳去,而一个区域的 machine code 是连续执行,内存压力没这么大并且可以充分利用 cache 从而提高效能。另一个因素可以参考 你所不知道的 C 語言: goto 和流程控制篇,从 VM 的 swith-case
和 computed goto
在效能差异的主要因素「分支预测」进行思考。
最后两个链接对于提高系统编程 (System programming) 能力非常有益,Just do it!
系统编程 (System Programming) 的入门项目,阅读过程需要查询搭配 man 手册,以熟悉库函数和系统调用的原型和作用。
+Linux manual page: +fflush +/ elf +/ exec +/ perror +/ getline +/ strchr +/ waitpid +/ fprintf +/ pipe +/ dup +/ close
+总计约 96.12K 字
总计约 97.00K 字
总计约 96.12K 字
总计约 97.00K 字
总计约 96.12K 字
总计约 97.00K 字