lupyuen/lupyuen.org
1
0
Fork 0
mirror of https://github.com/lupyuen/lupyuen.github.io.git synced 2025-01-13 10:18:33 +08:00
Code Issues Projects Releases Packages Wiki Activity Actions 1
lupyuen.org/articles/riscv.html
Lup Yuen Lee 6e2f142414
Some checks are pending
Build Articles / build (push) Waiting to run
Details
Commit from GitHub Actions
2025-01-04 06:09:02 +00:00

867 lines
No EOL
55 KiB
HTML
Raw Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<meta name="generator" content="rustdoc">
<title>64-bit RISC-V with Apache NuttX Real-Time Operating System</title>
<!-- Begin scripts/articles/*-header.html: Article Header for Custom Markdown files processed by rustdoc, like chip8.md -->
<meta property="og:title"
content="64-bit RISC-V with Apache NuttX Real-Time Operating System"
data-rh="true">
<meta property="og:description"
content="Let's boot Apache NuttX Real-Time Operating System on a 64-bit RISC-V Device (QEMU Emulator) and explore the Boot Code inside NuttX"
data-rh="true">
<meta name="description"
content="Let's boot Apache NuttX Real-Time Operating System on a 64-bit RISC-V Device (QEMU Emulator) and explore the Boot Code inside NuttX">
<meta property="og:image"
content="https://lupyuen.github.io/images/riscv-title.png">
<meta property="og:type"
content="article" data-rh="true">
<link rel="canonical"
href="https://lupyuen.org/articles/riscv.html" />
<!-- End scripts/articles/*-header.html -->
<!-- Begin scripts/rustdoc-header.html: Header for Custom Markdown files processed by rustdoc, like chip8.md -->
<link rel="alternate" type="application/rss+xml" title="RSS Feed for lupyuen" href="/rss.xml" />
<link rel="stylesheet" type="text/css" href="../normalize.css">
<link rel="stylesheet" type="text/css" href="../rustdoc.css" id="mainThemeStyle">
<link rel="stylesheet" type="text/css" href="../dark.css">
<link rel="stylesheet" type="text/css" href="../light.css" id="themeStyle">
<link rel="stylesheet" type="text/css" href="../prism.css">
<script src="../storage.js"></script><noscript>
<link rel="stylesheet" href="../noscript.css"></noscript>
<link rel="shortcut icon" href="../favicon.ico">
<style type="text/css">
#crate-search {
background-image: url("../down-arrow.svg");
}
</style>
<!-- End scripts/rustdoc-header.html -->
</head>
<body class="rustdoc">
<!--[if lte IE 8]>
<div class="warning">
This old browser is unsupported and will most likely display funky
things.
</div>
<![endif]-->
<!-- Begin scripts/rustdoc-before.html: Pre-HTML for Custom Markdown files processed by rustdoc, like chip8.md -->
<!-- Begin Theme Picker -->
<div class="theme-picker" style="left: 0"><button id="theme-picker" aria-label="Pick another theme!"><img src="../brush.svg"
width="18" alt="Pick another theme!"></button>
<div id="theme-choices"></div>
</div>
<!-- Theme Picker -->
<!-- End scripts/rustdoc-before.html -->
<h1 class="title">64-bit RISC-V with Apache NuttX Real-Time Operating System</h1>
<nav id="rustdoc"><ul>
<li><a href="#boot-nuttx-on-64-bit-risc-v-qemu" title="Boot NuttX on 64-bit RISC-V QEMU">1 Boot NuttX on 64-bit RISC-V QEMU</a><ul></ul></li>
<li><a href="#qemu-emulator-for-risc-v" title="QEMU Emulator for RISC-V">2 QEMU Emulator for RISC-V</a><ul></ul></li>
<li><a href="#qemu-starts-nuttx" title="QEMU Starts NuttX">3 QEMU Starts NuttX</a><ul></ul></li>
<li><a href="#risc-v-boot-code-in-nuttx" title="RISC-V Boot Code in NuttX">4 RISC-V Boot Code in NuttX</a><ul>
<li><a href="#get-cpu-id" title="Get CPU ID">4.1 Get CPU ID</a><ul></ul></li>
<li><a href="#disable-interrupts" title="Disable Interrupts">4.2 Disable Interrupts</a><ul></ul></li>
<li><a href="#wait-for-interrupt" title="Wait for Interrupt">4.3 Wait for Interrupt</a><ul></ul></li>
<li><a href="#load-interrupt-vector" title="Load Interrupt Vector">4.4 Load Interrupt Vector</a><ul></ul></li>
<li><a href="#32-bit-vs-64-bit-risc-v" title="32-bit vs 64-bit RISC-V">4.5 32-bit vs 64-bit RISC-V</a><ul></ul></li>
<li><a href="#other-instructions" title="Other Instructions">4.6 Other Instructions</a><ul></ul></li></ul></li>
<li><a href="#jump-to-start" title="Jump to Start">5 Jump to Start</a><ul></ul></li>
<li><a href="#whats-next" title="What’s Next">6 What’s Next</a><ul></ul></li>
<li><a href="#notes" title="Notes">7 Notes</a><ul></ul></li>
<li><a href="#appendix-analysis-of-nuttx-boot-code" title="Appendix: Analysis of NuttX Boot Code">8 Appendix: Analysis of NuttX Boot Code</a><ul></ul></li>
<li><a href="#appendix-build-apache-nuttx-rtos-for-64-bit-risc-v-qemu" title="Appendix: Build Apache NuttX RTOS for 64-bit RISC-V QEMU">9 Appendix: Build Apache NuttX RTOS for 64-bit RISC-V QEMU</a><ul></ul></li>
<li><a href="#appendix-compile-apache-nuttx-rtos-for-64-bit-risc-v-qemu" title="Appendix: Compile Apache NuttX RTOS for 64-bit RISC-V QEMU">10 Appendix: Compile Apache NuttX RTOS for 64-bit RISC-V QEMU</a><ul></ul></li>
<li><a href="#appendix-download-toolchain-for-64-bit-risc-v" title="Appendix: Download Toolchain for 64-bit RISC-V">11 Appendix: Download Toolchain for 64-bit RISC-V</a><ul></ul></li>
<li><a href="#appendix-xpack-gnu-risc-v-embedded-gcc-toolchain-for-64-bit-risc-v" title="Appendix: xPack GNU RISC-V Embedded GCC Toolchain for 64-bit RISC-V">12 Appendix: xPack GNU RISC-V Embedded GCC Toolchain for 64-bit RISC-V</a><ul></ul></li></ul></nav><p>📝 <em>25 Jun 2023</em></p>
<p><img src="https://lupyuen.github.io/images/riscv-title.png" alt="Apache NuttX RTOS on 64-bit QEMU RISC-V Emulator" /></p>
<p><a href="https://nuttx.apache.org/docs/latest/index.html"><strong>Apache NuttX</strong></a> is a <strong>Real-Time Operating System (RTOS)</strong> that runs on many kinds of devices, from 8-bit to 64-bit.</p>
<p>(Think Linux, but a lot smaller and simpler)</p>
<p>In this article we’ll…</p>
<ul>
<li>
<p>Boot NuttX RTOS on a <strong>64-bit RISC-V</strong> device</p>
</li>
<li>
<p>Explore the <strong>Boot Code</strong> that starts NuttX on RISC-V</p>
</li>
<li>
<p>And learn a little <strong>RISC-V Assembly</strong>!</p>
</li>
</ul>
<p><em>But we need RISC-V Hardware?</em></p>
<p>No worries! We’ll run NuttX on the <strong>QEMU Emulator</strong> for 64-bit RISC-V.</p>
<p>(Which will work on Linux, macOS and Windows machines)</p>
<p><img src="https://lupyuen.github.io/images/riscv-build.png" alt="Building Apache NuttX RTOS in 4 minutes" /></p>
<p><a href="https://lupyuen.github.io/articles/riscv#appendix-build-apache-nuttx-rtos-for-64-bit-risc-v-qemu"><em>Building Apache NuttX RTOS in 4 minutes</em></a></p>
<h1 id="boot-nuttx-on-64-bit-risc-v-qemu"><a class="doc-anchor" href="#boot-nuttx-on-64-bit-risc-v-qemu">§</a>1 Boot NuttX on 64-bit RISC-V QEMU</h1>
<p>We begin by <strong>booting NuttX RTOS</strong> on RISC-V QEMU Emulator (64-bit)…</p>
<ol>
<li>
<p>Download and install <a href="https://www.qemu.org/download/"><strong>QEMU Emulator</strong></a>.</p>
<div class="example-wrap"><pre class="language-bash"><code>## For macOS:
brew install qemu
## For Debian and Ubuntu:
sudo apt install qemu-system-riscv64</code></pre></div></li>
<li>
<p>Download <strong><code>nuttx</code></strong> from the <a href="https://github.com/lupyuen/lupyuen.github.io/releases/tag/nuttx-riscv64"><strong>NuttX Release</strong></a>…</p>
<p><a href="https://github.com/lupyuen/lupyuen.github.io/releases/download/nuttx-riscv64/nuttx"><strong>nuttx: NuttX Image for 64-bit RISC-V QEMU</strong></a></p>
<p>If we prefer to <strong>build NuttX</strong> ourselves: <a href="https://lupyuen.github.io/articles/riscv#appendix-build-apache-nuttx-rtos-for-64-bit-risc-v-qemu"><strong>Follow these steps</strong></a></p>
</li>
<li>
<p>Start the <strong>QEMU RISC-V Emulator</strong> (64-bit) with NuttX RTOS…</p>
<div class="example-wrap"><pre class="language-bash"><code>qemu-system-riscv64 \
-semihosting \
-M virt,aclint=on \
-cpu rv64 \
-bios none \
-kernel nuttx \
-nographic</code></pre></div></li>
<li>
<p>NuttX is now running in the QEMU Emulator! (Pic below)</p>
<div class="example-wrap"><pre class="language-text"><code>uart_register: Registering /dev/console
uart_register: Registering /dev/ttyS0
nx_start_application: Starting init thread
NuttShell (NSH) NuttX-12.1.0-RC0
nsh&gt; nx_start: CPU0: Beginning Idle Loop
nsh&gt;</code></pre></div>
<p><a href="https://gist.github.com/lupyuen/93ad51d49e5f02ad79bb40b0a57e3ac8">(See the Complete Log)</a></p>
</li>
<li>
<p>Enter “<strong>help</strong>” to see the available commands…</p>
<div class="example-wrap"><pre class="language-text"><code>nsh&gt; help
help usage: help [-v] [&lt;cmd&gt;]
. break dd exit ls ps source umount
[ cat df false mkdir pwd test unset
? cd dmesg free mkrd rm time uptime
alias cp echo help mount rmdir true usleep
unalias cmp env hexdump mv set truncate xd
basename dirname exec kill printf sleep uname
Builtin Apps:
nsh ostest sh</code></pre></div></li>
<li>
<p>NuttX works like a tiny version of Linux, so the commands will look familiar…</p>
<div class="example-wrap"><pre class="language-text"><code>nsh&gt; uname -a
NuttX 12.1.0-RC0 275db39 Jun 16 2023 20:22:08 risc-v rv-virt
nsh&gt; ls /dev
/dev:
console
null
ttyS0
zero
nsh&gt; ps
PID GROUP PRI POLICY TYPE NPX STATE EVENT SIGMASK STACK USED FILLED COMMAND
0 0 0 FIFO Kthread N-- Ready 0000000000000000 002000 001224 61.2% Idle Task
1 1 100 RR Task --- Running 0000000000000000 002992 002024 67.6% nsh_main</code></pre></div>
<p><a href="https://gist.github.com/lupyuen/93ad51d49e5f02ad79bb40b0a57e3ac8">(See the Complete Log)</a></p>
</li>
<li>
<p>To Exit QEMU: Press <strong><code>Ctrl-A</code></strong> then <strong><code>x</code></strong></p>
</li>
</ol>
<p>Let’s talk about QEMU…</p>
<p><a href="https://github.com/lupyuen/lupyuen.github.io/issues/21#issuecomment-1809337352">(How to enable the <strong>Hello App</strong>)</a></p>
<p><a href="https://github.com/lupyuen/lupyuen.github.io/issues/21#issuecomment-1814305271">(<strong>NuttX Kernel Mode</strong> works OK with QEMU)</a></p>
<p><img src="https://lupyuen.github.io/images/riscv-title.png" alt="Apache NuttX RTOS on RISC-V QEMU" /></p>
<p><a href="https://gist.github.com/lupyuen/93ad51d49e5f02ad79bb40b0a57e3ac8"><em>Apache NuttX RTOS on RISC-V QEMU</em></a></p>
<h1 id="qemu-emulator-for-risc-v"><a class="doc-anchor" href="#qemu-emulator-for-risc-v">§</a>2 QEMU Emulator for RISC-V</h1>
<p><em>Earlier we ran this command. What does it mean?</em></p>
<div class="example-wrap"><pre class="language-bash"><code>qemu-system-riscv64 \
-kernel nuttx \
-cpu rv64 \
-M virt,aclint=on \
-semihosting \
-bios none \
-nographic</code></pre></div>
<p>The above command starts the <a href="https://www.qemu.org/docs/master/system/target-riscv.html"><strong>QEMU Emulator for RISC-V</strong></a> (64-bit) with…</p>
<ul>
<li>
<p>Kernel Image: <strong>nuttx</strong></p>
</li>
<li>
<p>CPU: <a href="https://www.qemu.org/docs/master/system/target-riscv.html"><strong>64-bit RISC-V</strong></a></p>
</li>
<li>
<p>Machine: <a href="https://www.qemu.org/docs/master/system/riscv/virt.html"><strong>Generic Virtual Platform (virt)</strong></a></p>
</li>
<li>
<p>Handle Interrupts with <a href="https://patchwork.kernel.org/project/qemu-devel/cover/20210724122407.2486558-1-anup.patel@wdc.com/"><strong>Advanced Core Local Interruptor (ACLINT)</strong></a></p>
<p><a href="https://five-embeddev.com/baremetal/interrupts/#the-machine-mode-interrupts">(Instead of the older SiFive Core Local Interruptor CLINT)</a></p>
</li>
<li>
<p>Enable <a href="https://www.qemu.org/docs/master/about/emulation.html#semihosting"><strong>Semihosting Debugging</strong></a> without BIOS</p>
<p><a href="https://lupyuen.github.io/articles/semihost#semihosting-on-nuttx-qemu">(Why Semihosting)</a></p>
</li>
<li>
<p>Run Emulator in <strong>Console Mode</strong> (instead of Graphical Mode)</p>
</li>
</ul>
<p><em>Which RISC-V Instructions are supported by QEMU?</em></p>
<p>QEMU’s RISC-V <a href="https://www.qemu.org/docs/master/system/riscv/virt.html#supported-devices"><strong>Generic Virtual Platform (virt)</strong></a> supports <strong>RV64GC</strong>, which is equivalent to <a href="https://en.wikipedia.org/wiki/RISC-V#ISA_base_and_extensions"><strong>RV64IMAFDCZicsr_Zifencei</strong></a> (phew)…</p>
<div><table><thead><tr><th style="text-align: center"></th><th style="text-align: left"></th></tr></thead><tbody>
<tr><td style="text-align: center"><strong>RV64I</strong></td><td style="text-align: left">64-bit Base Integer Instruction Set</td></tr>
<tr><td style="text-align: center"><strong>M</strong></td><td style="text-align: left">Integer Multiplication and Division</td></tr>
<tr><td style="text-align: center"><strong>A</strong></td><td style="text-align: left">Atomic Instructions</td></tr>
<tr><td style="text-align: center"><strong>F</strong></td><td style="text-align: left">Single-Precision Floating-Point</td></tr>
<tr><td style="text-align: center"><strong>D</strong></td><td style="text-align: left">Double-Precision Floating-Point</td></tr>
<tr><td style="text-align: center"><strong>C</strong></td><td style="text-align: left">Compressed Instructions</td></tr>
<tr><td style="text-align: center"><strong>Zicsr</strong></td><td style="text-align: left">Control and Status Register (CSR) Instructions</td></tr>
<tr><td style="text-align: center"><strong>Zifencei</strong></td><td style="text-align: left">Instruction-Fetch Fence</td></tr>
</tbody></table>
</div>
<p><a href="https://en.wikipedia.org/wiki/RISC-V#ISA_base_and_extensions">(Source)</a></p>
<p>We’ll meet these instructions shortly.</p>
<h1 id="qemu-starts-nuttx"><a class="doc-anchor" href="#qemu-starts-nuttx">§</a>3 QEMU Starts NuttX</h1>
<p><em>What happens when NuttX RTOS boots on QEMU?</em></p>
<p>Let’s find out by tracing the <strong>RISC-V Boot Code</strong> in NuttX!</p>
<p>Earlier we ran this command to generate the <a href="https://lupyuen.github.io/articles/riscv#appendix-build-apache-nuttx-rtos-for-64-bit-risc-v-qemu"><strong>RISC-V Disassembly</strong></a> for the NuttX Kernel…</p>
<div class="example-wrap"><pre class="language-bash"><code>riscv64-unknown-elf-objdump \
-t -S --demangle --line-numbers --wide \
nuttx \
&gt;nuttx.S \
2&gt;&amp;1</code></pre></div>
<p>This produces <a href="https://github.com/lupyuen/lupyuen.github.io/releases/download/nuttx-riscv64/nuttx.S"><strong>nuttx.S</strong></a>, the disassembled NuttX Kernel for RISC-V.</p>
<p><a href="https://github.com/lupyuen/lupyuen.github.io/releases/download/nuttx-riscv64/nuttx.S"><strong>nuttx.S</strong></a> begins with this RISC-V code…</p>
<div class="example-wrap"><pre class="language-text"><code>0000000080000000 &lt;__start&gt;:
nuttx/arch/risc-v/src/chip/qemu_rv_head.S:46
__start:
/* Load mhartid (cpuid) */
csrr a0, mhartid
80000000: f1402573 csrr a0, mhartid</code></pre></div>
<p>This says…</p>
<ul>
<li>
<p>NuttX Boot Code is at <a href="https://github.com/apache/nuttx/blob/master/arch/risc-v/src/qemu-rv/qemu_rv_head.S#L41-L120"><strong>qemu_rv_head.S</strong></a></p>
</li>
<li>
<p>NuttX Kernel begins execution at address <strong><code>0x8000</code> <code>0000</code></strong></p>
<p>(What if NuttX is started by the U-Boot Bootloader? <a href="https://lupyuen.github.io/articles/nuttx2#start-address-of-nuttx-kernel"><strong>See this</strong></a>)</p>
</li>
</ul>
<p>Now we head into the NuttX Boot Code…</p>
<p><img src="https://lupyuen.github.io/images/riscv-code.png" alt="RISC-V Boot Code for Apache NuttX RTOS" /></p>
<p><a href="https://github.com/apache/nuttx/blob/master/arch/risc-v/src/qemu-rv/qemu_rv_head.S"><em>RISC-V Boot Code for Apache NuttX RTOS</em></a></p>
<h1 id="risc-v-boot-code-in-nuttx"><a class="doc-anchor" href="#risc-v-boot-code-in-nuttx">§</a>4 RISC-V Boot Code in NuttX</h1>
<p><em>What’s inside the NuttX Boot Code?</em></p>
<p>The RISC-V Assembly code in <a href="https://github.com/apache/nuttx/blob/master/arch/risc-v/src/qemu-rv/qemu_rv_head.S#L41-L120"><strong>qemu_rv_head.S</strong></a> will…</p>
<ol>
<li>
<p>Get the <a href="https://github.com/apache/nuttx/blob/master/arch/risc-v/src/qemu-rv/qemu_rv_head.S#L41-L47"><strong>CPU ID</strong></a></p>
</li>
<li>
<p>Check the <a href="https://github.com/apache/nuttx/blob/master/arch/risc-v/src/qemu-rv/qemu_rv_head.S#L54-L68"><strong>Number of CPUs</strong></a></p>
</li>
<li>
<p>Set the <a href="https://github.com/apache/nuttx/blob/master/arch/risc-v/src/qemu-rv/qemu_rv_head.S#L68-L98"><strong>Stack Pointer</strong></a></p>
</li>
<li>
<p>Disable <a href="https://github.com/apache/nuttx/blob/master/arch/risc-v/src/qemu-rv/qemu_rv_head.S#L96-L102"><strong>Interrupts</strong></a></p>
</li>
<li>
<p>Load the <a href="https://github.com/apache/nuttx/blob/master/arch/risc-v/src/qemu-rv/qemu_rv_head.S#L102-L105"><strong>Interrupt Vector</strong></a></p>
</li>
<li>
<p>Jump to <a href="https://github.com/apache/nuttx/blob/master/arch/risc-v/src/qemu-rv/qemu_rv_head.S#L105-L109"><strong>qemu_rv_start</strong></a></p>
</li>
</ol>
<p>Let’s decipher the RISC-V Instructions in our Boot Code…</p>
<h2 id="get-cpu-id"><a class="doc-anchor" href="#get-cpu-id">§</a>4.1 Get CPU ID</h2>
<p>This is how we fetch the <strong>CPU ID</strong> in RISC-V Assembly: <a href="https://github.com/apache/nuttx/blob/master/arch/risc-v/src/qemu-rv/qemu_rv_head.S#L43-L47">qemu_rv_head.S</a></p>
<div class="example-wrap"><pre class="language-text"><code>/* Load mhartid (cpuid) */
csrr a0, mhartid</code></pre></div>
<p>Let’s break it down…</p>
<ul>
<li>
<p><strong><code>csrr</code></strong> is the RISC-V Instruction that reads the <a href="https://five-embeddev.com/quickref/instructions.html#-csr--csr-instructions"><strong>Control and Status Register</strong></a></p>
<p>(Which contains the CPU ID)</p>
</li>
<li>
<p><strong><code>a0</code></strong> is the RISC-V Register that will be loaded with the CPU ID.</p>
<p>According to the <a href="https://github.com/riscv-non-isa/riscv-eabi-spec/blob/master/EABI.adoc#3-register-usage-and-symbolic-names"><strong>RISC-V EABI</strong></a> (Embedded Application Binary Interface), <strong>a0</strong> is actually an alias for the Official RISC-V Register <strong>x10</strong>.</p>
<p>(“a” refers to “Function Call Argument”)</p>
</li>
<li>
<p><strong><code>mhartid</code></strong> says that we’ll read from the <a href="https://five-embeddev.com/riscv-isa-manual/latest/machine.html#hart-id-register-mhartid"><strong>Hart ID Register</strong></a>, containing the ID of the Hardware Thread (“Hart”) that’s running our code.</p>
<p>(Equivalent to CPU ID)</p>
</li>
</ul>
<p>So the above code will load the CPU ID into Register <strong>x10</strong>.</p>
<p>(We’ll call it <strong>a0</strong> for convenience)</p>
<h2 id="disable-interrupts"><a class="doc-anchor" href="#disable-interrupts">§</a>4.2 Disable Interrupts</h2>
<p>To <strong>disable interrupts</strong> in RISC-V, we do this: <a href="https://github.com/apache/nuttx/blob/master/arch/risc-v/src/qemu-rv/qemu_rv_head.S#L98-L102">qemu_rv_head.S</a></p>
<div class="example-wrap"><pre class="language-text"><code>/* Disable all interrupts (i.e. timer, external) in mie */
csrw mie, zero</code></pre></div>
<p>Which means…</p>
<ul>
<li>
<p><strong><code>csrw</code></strong> will write to the <a href="https://five-embeddev.com/quickref/instructions.html#-csr--csr-instructions"><strong>Control and Status Register</strong></a></p>
<p>(Which controls interrupts and other CPU settings)</p>
</li>
<li>
<p><strong><code>mie</code></strong> says that we’ll write to the <a href="https://five-embeddev.com/riscv-isa-manual/latest/machine.html#machine-interrupt-registers-mip-and-mie"><strong>Machine Interrupt Enable Register</strong></a></p>
<p>(0 to Disable Interrupts, 1 to Enable)</p>
</li>
<li>
<p><strong><code>zero</code></strong> says that we’ll read from <a href="https://five-embeddev.com/quickref/regs_abi.html"><strong>Register x0</strong></a>…</p>
<p>Which always reads as 0!</p>
</li>
</ul>
<p>Thus the above instruction will set the Machine Interrupt Enable Register to 0, which will disable interrupts.</p>
<p>(Yeah RISC-V has a funny concept of “0”)</p>
<h2 id="wait-for-interrupt"><a class="doc-anchor" href="#wait-for-interrupt">§</a>4.3 Wait for Interrupt</h2>
<p>Now check out this curious combination of instructions: <a href="https://github.com/apache/nuttx/blob/master/arch/risc-v/src/qemu-rv/qemu_rv_head.S#L62-L68">qemu_rv_head.S</a></p>
<div class="example-wrap"><pre class="language-text"><code>/* Wait forever */
csrw mie, zero
wfi</code></pre></div>
<p>From the previous section, we know that “<strong>csrw mie, zero</strong>” will disable interrupts.</p>
<p>But <strong><code>wfi</code></strong> will <a href="https://five-embeddev.com/riscv-isa-manual/latest/machine.html#wfi"><strong>Wait for Interrupt</strong></a>…</p>
<p>Which will never happen because we <strong>disabled interrupts!</strong></p>
<p>Thus the above code will get stuck there, <strong>waiting forever</strong>. (Intentionally)</p>
<p><a href="https://developer.arm.com/documentation/ddi0596/2020-12/Base-Instructions/WFI--Wait-For-Interrupt-">(<strong><code>wfi</code></strong> is probably the only instruction common to <strong>RISC-V and Arm CPUs</strong>)</a></p>
<h2 id="load-interrupt-vector"><a class="doc-anchor" href="#load-interrupt-vector">§</a>4.4 Load Interrupt Vector</h2>
<p>RISC-V handles interrupts by looking up the <a href="https://five-embeddev.com/quickref/interrupts.html"><strong>Interrupt Vector Table</strong></a>.</p>
<p>This is how we load the <strong>Address of the Vector Table</strong> into the CPU Settings: <a href="https://github.com/apache/nuttx/blob/master/arch/risc-v/src/qemu-rv/qemu_rv_head.S#L102-L105">qemu_rv_head.S</a></p>
<div class="example-wrap"><pre class="language-text"><code>/* Load address of Interrupt Vector Table */
la t0, __trap_vec
csrw mtvec, t0</code></pre></div>
<ul>
<li>
<p><a href="https://michaeljclark.github.io/asm.html#:~:text=The%20la%20(load%20address)%20instruction,command%20line%20options%20or%20an%20."><strong><code>la</code></strong></a> loads the Address of the Vector Table into <strong>Register t0</strong></p>
<p><a href="https://github.com/riscv-non-isa/riscv-eabi-spec/blob/master/EABI.adoc#3-register-usage-and-symbolic-names">(Which is aliased to <strong>Register x5</strong>)</a></p>
<p><a href="https://github.com/apache/nuttx/blob/master/arch/risc-v/src/common/riscv_vectors.S">(<strong>trap_vec</strong> is defined here)</a></p>
</li>
<li>
<p><strong><code>csrw</code></strong> writes <strong>t0</strong> into the <a href="https://five-embeddev.com/quickref/instructions.html#-csr--csr-instructions"><strong>Control and Status Register</strong></a> at…</p>
</li>
<li>
<p><strong><code>mtvec</code></strong>, the <a href="https://five-embeddev.com/riscv-isa-manual/latest/machine.html#machine-trap-vector-base-address-register-mtvec"><strong>Machine Trap-Vector Base-Address Register</strong></a></p>
</li>
</ul>
<p>Which will load the Address of our Interrupt Vector Table into the CPU Settings.</p>
<p><a href="https://michaeljclark.github.io/asm.html#:~:text=The%20la%20(load%20address)%20instruction,command%20line%20options%20or%20an%20.">(<strong><code>la</code></strong> is actually a Pseudo-Instruction that expands to <strong><code>auipc</code></strong> and <strong><code>addi</code></strong>)</a></p>
<p><a href="https://five-embeddev.com/quickref/instructions.html#-rv32--integer-register-immediate-instructions">(<strong><code>auipc</code></strong> loads an Address Offset from the Program Counter)</a></p>
<p><a href="https://five-embeddev.com/quickref/instructions.html#-rv32--integer-register-immediate-instructions">(<strong><code>addi</code></strong> adds an Immediate Value to a Register)</a></p>
<h2 id="32-bit-vs-64-bit-risc-v"><a class="doc-anchor" href="#32-bit-vs-64-bit-risc-v">§</a>4.5 32-bit vs 64-bit RISC-V</h2>
<p>Adapting 32-bit code for 64-bit sounds hard… But it’s easy peasy for RISC-V!</p>
<p>Our Boot Code uses an Assembler Macro to figure out if we’re running <strong>32-bit or 64-bit</strong> RISC-V: <a href="https://github.com/apache/nuttx/blob/master/arch/risc-v/src/qemu-rv/qemu_rv_head.S#L73-L82">qemu_rv_head.S</a></p>
<div class="example-wrap"><pre class="language-text"><code>#ifdef CONFIG_ARCH_RV32
/* Do this for 32-bit RISC-V */
slli t1, a0, 2
#else
/* Do this for 64-bit RISC-V */
slli t1, a0, 3
#endif</code></pre></div>
<p>Which means that the exact same Boot Code will run on <strong>32-bit AND 64-bit RISC-V</strong>!</p>
<p>(<strong><code>slli</code></strong> sounds “silly”, but it’s <a href="https://five-embeddev.com/quickref/instructions.html#-rv32--integer-register-immediate-instructions"><strong>Logical Shift Left</strong></a>)</p>
<p>(<strong>CONFIG_ARCH_RV32</strong> is derived from our <a href="https://github.com/apache/nuttx/blob/master/boards/risc-v/qemu-rv/rv-virt/configs/nsh64/defconfig"><strong>NuttX Build Configuration</strong></a>)</p>
<h2 id="other-instructions"><a class="doc-anchor" href="#other-instructions">§</a>4.6 Other Instructions</h2>
<p><em>What about the other RISC-V Instructions in our Boot Code?</em></p>
<p>Let’s skim through the rest…</p>
<ul>
<li>
<p><a href="https://five-embeddev.com/quickref/instructions.html#-c--control-transfer-instructions"><strong><code>bnez</code></strong></a> branches to <strong>Label <code>1f</code></strong> if <strong>Register a0</strong> is Non-Zero</p>
<div class="example-wrap"><pre class="language-text"><code>bnez a0, 1f</code></pre></div>
<p><a href="https://github.com/apache/nuttx/blob/master/arch/risc-v/src/qemu-rv/qemu_rv_head.S#L47-L50">(Source)</a></p>
</li>
<li>
<p><a href="https://five-embeddev.com/quickref/instructions.html#-c--control-transfer-instructions"><strong><code>j</code></strong></a> jumps to <strong>Label <code>2f</code></strong></p>
<p>(We’ll explain Labels in a while)</p>
<div class="example-wrap"><pre class="language-text"><code>j 2f</code></pre></div>
<p><a href="https://github.com/apache/nuttx/blob/master/arch/risc-v/src/qemu-rv/qemu_rv_head.S#L52-L54">(Source)</a></p>
</li>
<li>
<p><a href="https://github.com/riscv-non-isa/riscv-asm-manual/blob/master/riscv-asm.md#-a-listing-of-standard-risc-v-pseudoinstructions"><strong><code>li</code></strong></a> loads the <strong>Value 1</strong> into <strong>Register t1</strong></p>
<p><a href="https://github.com/riscv-non-isa/riscv-asm-manual/blob/master/riscv-asm.md#-a-listing-of-standard-risc-v-pseudoinstructions">(<strong><code>li</code></strong> is a Pseudo-Instruction that expands to <strong><code>addi</code></strong>)</a></p>
<p><a href="https://five-embeddev.com/quickref/instructions.html#-rv32--integer-register-immediate-instructions">(<strong><code>addi</code></strong> adds an Immediate Value to a Register)</a></p>
<div class="example-wrap"><pre class="language-text"><code>li t1, 1</code></pre></div>
<p><a href="https://github.com/apache/nuttx/blob/master/arch/risc-v/src/qemu-rv/qemu_rv_head.S#L59-L62">(Source)</a></p>
</li>
<li>
<p><a href="https://five-embeddev.com/quickref/instructions.html#-rv32--conditional-branches"><strong><code>blt</code></strong></a> branches to <strong>Label <code>3f</code></strong> if <strong>Register a0</strong> is less than <strong>Register t1</strong></p>
<p>(And grabs a sandwich)</p>
<div class="example-wrap"><pre class="language-text"><code>blt a0, t1, 3f</code></pre></div>
<p><a href="https://github.com/apache/nuttx/blob/master/arch/risc-v/src/qemu-rv/qemu_rv_head.S#L62-L65">(Source)</a></p>
</li>
<li>
<p><a href="https://five-embeddev.com/quickref/instructions.html#-rv32--integer-computational-instructions"><strong><code>add</code></strong></a> sets <strong>Register t0</strong> to the value of <strong>Register t0</strong> + <strong>Register t1</strong></p>
<p><a href="https://github.com/riscv-non-isa/riscv-eabi-spec/blob/master/EABI.adoc#3-register-usage-and-symbolic-names">(<strong>t1</strong> is aliased to <strong>Register x15</strong>)</a></p>
<div class="example-wrap"><pre class="language-text"><code>add t0, t0, t1</code></pre></div>
<p><a href="https://github.com/apache/nuttx/blob/master/arch/risc-v/src/qemu-rv/qemu_rv_head.S#L80-L82">(Source)</a></p>
</li>
<li>
<p><a href="https://github.com/apache/nuttx/blob/master/arch/risc-v/src/common/riscv_internal.h#L55-L63"><strong><code>REGLOAD</code></strong></a> is an Assembly Macro that expands to <strong><code>ld</code></strong></p>
<p><a href="https://five-embeddev.com/quickref/instructions.html#-rv64--load-and-store-instructions"><strong><code>ld</code></strong></a> loads <strong>Register t0</strong> into the <strong>Stack Pointer Register</strong></p>
<p><a href="https://github.com/riscv-non-isa/riscv-eabi-spec/blob/master/EABI.adoc#3-register-usage-and-symbolic-names">(Which is aliased to <strong>Register x2</strong>)</a></p>
<div class="example-wrap"><pre class="language-text"><code>REGLOAD sp, 0(t0)</code></pre></div>
<p><a href="https://github.com/apache/nuttx/blob/master/arch/risc-v/src/qemu-rv/qemu_rv_head.S#L82-L86">(Source)</a></p>
</li>
<li>
<p><a href="https://five-embeddev.com/quickref/instructions.html#-rv32--programmers-model-for-base-integer-isa"><strong><code>jal</code></strong></a> (Jump And Link) will jump to the address <strong>qemu_rv_start</strong> and store the Return Address in <strong>Register x1</strong></p>
<p>(Works like a Function Call)</p>
<div class="example-wrap"><pre class="language-text"><code>jal x1, qemu_rv_start</code></pre></div>
<p><a href="https://github.com/apache/nuttx/blob/master/arch/risc-v/src/qemu-rv/qemu_rv_head.S#L105-L109">(Source)</a></p>
</li>
<li>
<p><a href="https://github.com/riscv-non-isa/riscv-asm-manual/blob/master/riscv-asm.md#-a-listing-of-standard-risc-v-pseudoinstructions"><strong><code>ret</code></strong></a> returns from a Function Call.</p>
<p><a href="https://github.com/riscv-non-isa/riscv-asm-manual/blob/master/riscv-asm.md#-a-listing-of-standard-risc-v-pseudoinstructions">(<strong><code>ret</code></strong> is a Pseudo-Instruction that expands to <strong><code>jalr</code></strong>)</a></p>
<p><a href="https://five-embeddev.com/quickref/instructions.html#-rv32--unconditional-jumps">(<strong><code>jalr</code></strong> “Jump And Link Register” will jump to the Return Address stored in <strong>Register x1</strong>)</a></p>
<div class="example-wrap"><pre class="language-text"><code>ret</code></pre></div>
<p><a href="https://github.com/apache/nuttx/blob/master/arch/risc-v/src/qemu-rv/qemu_rv_head.S#L117-L120">(Source)</a></p>
</li>
</ul>
<p>Here’s the complete list of RISC-V Instructions…</p>
<ul>
<li>
<p><a href="https://five-embeddev.com/quickref/instructions.html"><strong>RISC-V Instructions</strong></a></p>
</li>
<li>
<p><a href="https://github.com/riscv-non-isa/riscv-asm-manual/blob/master/riscv-asm.md#-a-listing-of-standard-risc-v-pseudoinstructions"><strong>RISC-V Pseudo-Instructions</strong></a></p>
</li>
<li>
<p><a href="https://web.archive.org/web/20230331004925/http://riscvbook.com/greencard-20181213.pdf"><strong>RISC-V Reference Card</strong></a></p>
</li>
</ul>
<p><a href="https://lupyuen.github.io/articles/riscv#appendix-analysis-of-nuttx-boot-code">(See the <strong>Detailed Analysis</strong> of the NuttX Boot Code)</a></p>
<p><em>Why are the RISC-V Labels named “1f”, “2f”, “3f”?</em></p>
<p><a href="https://github.com/apache/nuttx/blob/master/arch/risc-v/src/qemu-rv/qemu_rv_head.S#L47-L50"><strong>“<code>1f</code>”</strong></a> refers to the <a href="https://github.com/apache/nuttx/blob/master/arch/risc-v/src/qemu-rv/qemu_rv_head.S#L53-L56"><strong>Local Label “<code>1</code>”</strong></a> with a <a href="https://github.com/riscv-non-isa/riscv-asm-manual/blob/master/riscv-asm.md#labels"><strong>Forward Reference</strong></a>.</p>
<p>(Instead of a <a href="https://github.com/riscv-non-isa/riscv-asm-manual/blob/master/riscv-asm.md#labels"><strong>Backward Reference</strong></a>)</p>
<p><em>Can we write our own RISC-V Assembly Code? As a learning exercise?</em></p>
<p>Yep! Here’s how we inserted our own RISC-V Assembly Code into the NuttX Boot Code…</p>
<ul>
<li><a href="https://lupyuen.github.io/articles/nuttx2#print-to-qemu-console"><strong>“Print to QEMU Console”</strong></a></li>
</ul>
<p>Let’s jump to <strong>qemu_rv_start</strong>…</p>
<p><img src="https://lupyuen.github.io/images/riscv-start.png" alt="RISC-V Start Code for NuttX RTOS" /></p>
<p><a href="https://github.com/apache/nuttx/blob/master/arch/risc-v/src/qemu-rv/qemu_rv_start.c#L94-L151"><em>RISC-V Start Code for NuttX RTOS</em></a></p>
<h1 id="jump-to-start"><a class="doc-anchor" href="#jump-to-start">§</a>5 Jump to Start</h1>
<p><em>Our Boot Code jumps to qemu_rv_start…</em></p>
<p><em>What happens next?</em></p>
<p><a href="https://github.com/apache/nuttx/blob/master/arch/risc-v/src/qemu-rv/qemu_rv_start.c#L94-L151"><strong>qemu_rv_start</strong></a> is the very first C Function that NuttX runs when it boots on QEMU.</p>
<p>(And <a href="https://github.com/apache/nuttx/blob/master/arch/risc-v/src/jh7110/jh7110_start.c#L129-L159"><strong>jh7110_start</strong></a> for Star64 JH7110)</p>
<p>The function will…</p>
<ol>
<li>
<p>Configure the <a href="https://github.com/apache/nuttx/blob/master/arch/risc-v/src/qemu-rv/qemu_rv_start.c#L105-L108"><strong>Floating-Point Unit</strong></a></p>
</li>
<li>
<p>Clear the <a href="https://github.com/apache/nuttx/blob/master/arch/risc-v/src/qemu-rv/qemu_rv_start.c#L113-L117"><strong>BSS Memory</strong></a></p>
</li>
<li>
<p>Initialise the <a href="https://github.com/apache/nuttx/blob/master/arch/risc-v/src/qemu-rv/qemu_rv_start.c#L119-L123"><strong>Serial Port</strong></a></p>
</li>
<li>
<p>Initialise the <a href="https://github.com/apache/nuttx/blob/master/arch/risc-v/src/qemu-rv/qemu_rv_start.c#L129-L135"><strong>Memory Management Unit</strong></a></p>
<p>(For Kernel Mode only)</p>
</li>
<li>
<p>Call <a href="https://github.com/apache/nuttx/blob/master/arch/risc-v/src/qemu-rv/qemu_rv_start.c#L135-L139"><strong>nx_start</strong></a></p>
</li>
</ol>
<p><em>What happens in nx_start?</em></p>
<p><a href="https://github.com/apache/nuttx/blob/master/sched/init/nx_start.c#L297-L707"><strong>nx_start</strong></a> will initialise a whole bunch of NuttX things…</p>
<ul>
<li><a href="https://lupyuen.github.io/articles/unicorn2#after-primary-routine"><strong>“After Primary Routine: nx_start”</strong></a></li>
</ul>
<p>Which will start the NuttX Shell that we’ve seen earlier.</p>
<p>And that’s how NuttX RTOS boots on QEMU Emulator for RISC-V!</p>
<p><em>Why are we doing all this?</em></p>
<p>We’re about to port NuttX to the <a href="https://doc-en.rvspace.org/Doc_Center/jh7110.html"><strong>StarFive JH7110</strong></a> RISC-V SoC and <a href="https://wiki.pine64.org/wiki/STAR64"><strong>Pine64 Star64</strong></a> Single-Board Computer.</p>
<p>The analysis we’ve done today will be super helpful as we write the Boot Code for these RISC-V devices.</p>
<p>Stay tuned for updates in the next article!</p>
<ul>
<li>
<p><a href="https://lupyuen.github.io/articles/linux"><strong>“Booting RISC-V Linux on Star64 JH7110 SBC”</strong></a></p>
</li>
<li>
<p><a href="https://lupyuen.github.io/articles/star64"><strong>“Inspecting the RISC-V Linux Images for Star64 SBC”</strong></a></p>
</li>
<li>
<p><a href="https://github.com/lupyuen/nuttx-star64"><strong>“Apache NuttX RTOS for Pine64 Star64 64-bit RISC-V SBC (StarFive JH7110)”</strong></a></p>
</li>
</ul>
<h1 id="whats-next"><a class="doc-anchor" href="#whats-next">§</a>6 What’s Next</h1>
<p>I hope this article has been an educational exploration of Apache NuttX RTOS on 64-bit RISC-V…</p>
<ul>
<li>
<p>We booted NuttX RTOS on an emulated <strong>64-bit RISC-V</strong> device</p>
</li>
<li>
<p>We peeked at the <strong>Boot Code</strong> that starts NuttX on RISC-V</p>
</li>
<li>
<p>And hopefully we learnt a little <strong>RISC-V Assembly</strong>!</p>
</li>
</ul>
<p>As we’ve seen, NuttX is a tiny operating system that’s perfect for experimenting with RISC-V gadgets. We’ll do this and much more in the upcoming articles!</p>
<ul>
<li>
<p><a href="https://lupyuen.github.io/articles/nuttx2"><strong>“Apache NuttX RTOS on RISC-V: Star64 JH7110 SBC”</strong></a></p>
</li>
<li>
<p><a href="https://lupyuen.github.io/articles/linux"><strong>“Booting RISC-V Linux on Star64 JH7110 SBC”</strong></a></p>
</li>
<li>
<p><a href="https://lupyuen.github.io/articles/star64"><strong>“Inspecting the RISC-V Linux Images for Star64 SBC”</strong></a></p>
</li>
<li>
<p><a href="https://github.com/lupyuen/nuttx-star64"><strong>“Apache NuttX RTOS for Pine64 Star64 64-bit RISC-V SBC (StarFive JH7110)”</strong></a></p>
</li>
</ul>
<p><a href="https://lupyuen.github.io/articles/pr">(We welcome <strong>your contribution</strong> to Apache NuttX RTOS)</a></p>
<p>Many Thanks to my <a href="https://lupyuen.github.io/articles/sponsor"><strong>GitHub Sponsors</strong></a> for supporting my work! This article wouldn’t have been possible without your support.</p>
<ul>
<li>
<p><a href="https://lupyuen.github.io/articles/sponsor"><strong>Sponsor me a coffee</strong></a></p>
</li>
<li>
<p><a href="https://news.ycombinator.com/item?id=36453810"><strong>Discuss this article on Hacker News</strong></a></p>
</li>
<li>
<p><a href="https://github.com/lupyuen/nuttx-ox64"><strong>My Current Project: “Apache NuttX RTOS for Ox64 BL808”</strong></a></p>
</li>
<li>
<p><a href="https://github.com/lupyuen/nuttx-star64"><strong>My Other Project: “NuttX for Star64 JH7110”</strong></a></p>
</li>
<li>
<p><a href="https://github.com/lupyuen/pinephone-nuttx"><strong>Older Project: “NuttX for PinePhone”</strong></a></p>
</li>
<li>
<p><a href="https://lupyuen.github.io"><strong>Check out my articles</strong></a></p>
</li>
<li>
<p><a href="https://lupyuen.github.io/rss.xml"><strong>RSS Feed</strong></a></p>
</li>
</ul>
<p><em>Got a question, comment or suggestion? Create an Issue or submit a Pull Request here…</em></p>
<p><a href="https://github.com/lupyuen/lupyuen.github.io/blob/master/src/riscv.md"><strong>lupyuen.github.io/src/riscv.md</strong></a></p>
<h1 id="notes"><a class="doc-anchor" href="#notes">§</a>7 Notes</h1>
<ol>
<li>
<p>Learning about <strong>RISC-V Architecture</strong>? This book has a concise overview, it might be available from your Local Library through the Libby App…</p>
<p><a href="https://share.libbyapp.com/title/5479987"><strong>“Modern Computer Architecture and Organization” by Jim Ledin</strong></a></p>
</li>
<li>
<p><strong>Hart IDs</strong> are not guaranteed to be contiguous. One is guaranteed to be 0, the rest will all be different, but not necessarily 0, 1, 2, 3, 4, 5, …</p>
<p><a href="https://news.ycombinator.com/item?id=36453810#36455404">(Source)</a></p>
</li>
<li>
<p>To <strong>Enable Logging</strong> for RISC-V QEMU: Use the QEMU Option <strong><code>-trace "*"</code></strong></p>
<p><a href="https://lupyuen.github.io/articles/semihost#appendix-ram-disk-address-for-risc-v-qemu">(Like this)</a></p>
</li>
<li>
<p>Here’s the <a href="https://lupyuen.github.io/articles/semihost#appendix-device-tree-for-risc-v-qemu"><strong>Device Tree</strong></a> for RISC-V QEMU</p>
</li>
</ol>
<p><img src="https://lupyuen.github.io/images/riscv-code.png" alt="RISC-V Boot Code for Apache NuttX RTOS" /></p>
<p><a href="https://github.com/apache/nuttx/blob/master/arch/risc-v/src/qemu-rv/qemu_rv_head.S"><em>RISC-V Boot Code for Apache NuttX RTOS</em></a></p>
<h1 id="appendix-analysis-of-nuttx-boot-code"><a class="doc-anchor" href="#appendix-analysis-of-nuttx-boot-code">§</a>8 Appendix: Analysis of NuttX Boot Code</h1>
<p>Earlier we talked about the <strong>NuttX Boot Code</strong> for RISC-V QEMU…</p>
<ul>
<li><a href="https://lupyuen.github.io/articles/riscv#risc-v-boot-code-in-nuttx"><strong>“RISC-V Boot Code in NuttX”</strong></a></li>
</ul>
<p>Below is our Detailed Analysis of the Boot Code in…</p>
<ul>
<li><a href="https://github.com/apache/nuttx/blob/master/arch/risc-v/src/qemu-rv/qemu_rv_head.S"><strong>qemu_rv_head.S</strong></a></li>
</ul>
<p><strong>For All Hart IDs:</strong></p>
<p>Load the Hart ID (CPU ID) from the system…</p>
<div class="example-wrap"><pre class="language-text"><code>__start:
/* Load mhartid (cpuid) */
csrr a0, mhartid</code></pre></div>
<p><a href="https://github.com/apache/nuttx/blob/master/arch/risc-v/src/qemu-rv/qemu_rv_head.S#L41-L47">(Source)</a></p>
<p><a href="https://lupyuen.github.io/articles/riscv#risc-v-boot-code-in-nuttx">(<strong>RISC-V Instructions</strong> explained)</a></p>
<p><strong>If Hart ID is 0 (First CPU):</strong></p>
<p>Set Stack Pointer to the Idle Thread Stack…</p>
<div class="example-wrap"><pre class="language-text"><code> /* Set stack pointer to the idle thread stack */
bnez a0, 1f
la sp, QEMU_RV_IDLESTACK_TOP
j 2f</code></pre></div>
<p><a href="https://github.com/apache/nuttx/blob/master/arch/risc-v/src/qemu-rv/qemu_rv_head.S#L47-L54">(Source)</a></p>
<p><strong>If Hart ID is 1, 2, 3, …</strong></p>
<ul>
<li>Validate the Hart ID (Must be less than Number of CPUs)</li>
<li>Compute the Stack Base Address based on <code>g_cpu_basestack</code> and Hart ID</li>
<li>Set the Stack Pointer to the computed Stack Base Address</li>
</ul>
<div class="example-wrap"><pre class="language-text"><code>1:
/* Load the number of CPUs that the kernel supports */
#ifdef CONFIG_SMP
li t1, CONFIG_SMP_NCPUS
#else
li t1, 1
#endif
/* If a0 (mhartid) &gt;= t1 (the number of CPUs), stop here */
blt a0, t1, 3f
csrw mie, zero
wfi
3:
/* To get g_cpu_basestack[mhartid], must get g_cpu_basestack first */
la t0, g_cpu_basestack
/* Offset = pointer width * hart id */
#ifdef CONFIG_ARCH_RV32
slli t1, a0, 2
#else
slli t1, a0, 3
#endif
add t0, t0, t1
/* Load idle stack base to sp */
REGLOAD sp, 0(t0)
/*
* sp (stack top) = sp + idle stack size - XCPTCONTEXT_SIZE
*
* Note: Reserve some space used by up_initial_state since we are already
* running and using the per CPU idle stack.
*/
li t0, STACK_ALIGN_UP(CONFIG_IDLETHREAD_STACKSIZE - XCPTCONTEXT_SIZE)
add sp, sp, t0</code></pre></div>
<p><a href="https://github.com/apache/nuttx/blob/master/arch/risc-v/src/qemu-rv/qemu_rv_head.S#L53-L96">(Source)</a></p>
<p><strong>For All Hart IDs:</strong></p>
<ul>
<li>Disable Interrupts</li>
<li>Load the Trap Vector Table</li>
<li>Jump to <a href="https://lupyuen.github.io/articles/riscv#jump-to-start"><strong>qemu_rv_start</strong></a> (Or <a href="https://github.com/apache/nuttx/blob/master/arch/risc-v/src/jh7110/jh7110_start.c#L129-L159"><strong>jh7110_start</strong></a> for Star64 JH7110)</li>
</ul>
<div class="example-wrap"><pre class="language-text"><code>2:
/* Disable all interrupts (i.e. timer, external) in mie */
csrw mie, zero
/* Load the Trap Vector Table */
la t0, __trap_vec
csrw mtvec, t0
/* Jump to qemu_rv_start */
jal x1, qemu_rv_start
/* We shouldn&#39;t return from _start */</code></pre></div>
<p><a href="https://github.com/apache/nuttx/blob/master/arch/risc-v/src/qemu-rv/qemu_rv_head.S#L96-L120">(Source)</a></p>
<p><img src="https://lupyuen.github.io/images/riscv-build.png" alt="Build Apache NuttX RTOS for 64-bit RISC-V QEMU" /></p>
<h1 id="appendix-build-apache-nuttx-rtos-for-64-bit-risc-v-qemu"><a class="doc-anchor" href="#appendix-build-apache-nuttx-rtos-for-64-bit-risc-v-qemu">§</a>9 Appendix: Build Apache NuttX RTOS for 64-bit RISC-V QEMU</h1>
<p>The easiest way to run <strong>Apache NuttX RTOS on 64-bit RISC-V</strong> is to download the <strong>NuttX Image</strong> and boot it on QEMU Emulator…</p>
<ul>
<li><a href="https://lupyuen.github.io/articles/riscv#boot-nuttx-on-64-bit-risc-v-qemu"><strong>“Boot NuttX on 64-bit RISC-V QEMU”</strong></a></li>
</ul>
<p>But if we’re keen to <strong>build NuttX ourselves</strong>, here are the steps…</p>
<ol>
<li>
<p>Install the Build Prerequisites, skip the RISC-V Toolchain…</p>
<p><a href="https://lupyuen.github.io/articles/nuttx#install-prerequisites"><strong>“Install Prerequisites”</strong></a></p>
</li>
<li>
<p>Download the RISC-V Toolchain for <strong>riscv64-unknown-elf</strong>…</p>
<p><a href="https://lupyuen.github.io/articles/riscv#appendix-download-toolchain-for-64-bit-risc-v"><strong>“Download Toolchain for 64-bit RISC-V”</strong></a></p>
</li>
<li>
<p>Download and configure NuttX…</p>
<div class="example-wrap"><pre class="language-bash"><code>mkdir nuttx
cd nuttx
git clone https://github.com/apache/nuttx nuttx
git clone https://github.com/apache/nuttx-apps apps
cd nuttx
tools/configure.sh rv-virt:nsh64
make menuconfig</code></pre></div></li>
<li>
<p>In <strong>menuconfig</strong>, browse to “<strong>Device Drivers</strong> &gt; <strong>System Logging</strong>”</p>
<p>Disable this option…</p>
<div class="example-wrap"><pre class="language-text"><code>Prepend Timestamp to Syslog Message</code></pre></div></li>
<li>
<p>Browse to “<strong>Build Setup</strong> &gt; <strong>Debug Options</strong>”</p>
<p>Select the following options…</p>
<div class="example-wrap"><pre class="language-text"><code>Enable Debug Features
Enable Error Output
Enable Warnings Output
Enable Informational Debug Output
Enable Debug Assertions
Scheduler Debug Features
Scheduler Error Output
Scheduler Warnings Output
Scheduler Informational Output</code></pre></div>
<p>Save and exit <strong>menuconfig</strong>.</p>
</li>
<li>
<p>Build the NuttX Project and dump the RISC-V Disassembly…</p>
<div class="example-wrap"><pre class="language-bash"><code>make V=1 -j7
riscv64-unknown-elf-objdump \
-t -S --demangle --line-numbers --wide \
nuttx \
&gt;nuttx.S \
2&gt;&amp;1</code></pre></div>
<p><a href="https://gist.github.com/lupyuen/9d9b89dfd91b27f93459828178b83b77">(See the Build Log)</a></p>
<p><a href="https://github.com/lupyuen/lupyuen.github.io/releases/tag/nuttx-riscv64">(See the Build Outputs)</a></p>
</li>
<li>
<p>If the build fails with…</p>
<div class="example-wrap"><pre class="language-text"><code>sed: 1: &quot;/CONFIG_BASE_DEFCONFIG/ ...&quot;: bad flag in substitute command: &#39;}&#39;</code></pre></div>
<p>Please run “<strong>make menuconfig</strong> &gt; <strong>Build Setup</strong> &gt; <strong>Debug Options</strong>” and uncheck “<strong>Enable Debug Features</strong>”. Save, exit <strong>menuconfig</strong> and rebuild NuttX with <strong>make</strong>.</p>
</li>
</ol>
<p>This produces the NuttX ELF Image <strong>nuttx</strong> that we may boot on QEMU RISC-V Emulator…</p>
<ul>
<li><a href="https://lupyuen.github.io/articles/riscv#boot-nuttx-on-64-bit-risc-v-qemu"><strong>“Boot NuttX on 64-bit RISC-V QEMU”</strong></a></li>
</ul>
<p>Let’s look at the GCC Command that compiles NuttX for 64-bit RISC-V QEMU…</p>
<h1 id="appendix-compile-apache-nuttx-rtos-for-64-bit-risc-v-qemu"><a class="doc-anchor" href="#appendix-compile-apache-nuttx-rtos-for-64-bit-risc-v-qemu">§</a>10 Appendix: Compile Apache NuttX RTOS for 64-bit RISC-V QEMU</h1>
<p>From the previous section, we see that the NuttX Build compiles the source files with these <strong>GCC Options</strong>…</p>
<div class="example-wrap"><pre class="language-bash"><code>riscv64-unknown-elf-gcc \
-c \
-fno-common \
-Wall \
-Wstrict-prototypes \
-Wshadow \
-Wundef \
-Os \
-fno-strict-aliasing \
-fomit-frame-pointer \
-ffunction-sections \
-fdata-sections \
-g \
-march=rv64imac \
-mabi=lp64 \
-mcmodel=medany \
-isystem nuttx/include \
-D__NuttX__ \
-DNDEBUG \
-D__KERNEL__ \
-pipe \
-I nuttx/arch/risc-v/src/chip \
-I nuttx/arch/risc-v/src/common \
-I nuttx/sched \
chip/qemu_rv_start.c \
-o qemu_rv_start.o</code></pre></div>
<p><a href="https://gist.github.com/lupyuen/9d9b89dfd91b27f93459828178b83b77">(See the Build Log)</a></p>
<p>The <strong>RISC-V Options</strong> are…</p>
<ul>
<li>
<p><strong>march=rv64imac</strong>: This generates Integer-Only 64-bit RISC-V code, no Floating-Point.</p>
<p>Which is surprising because RISC-V QEMU actually <a href="https://lupyuen.github.io/articles/riscv#qemu-emulator-for-risc-v"><strong>supports Floating-Point</strong></a>.</p>
<p>We’ll fix this as we port NuttX to the <a href="https://doc-en.rvspace.org/Doc_Center/jh7110.html"><strong>StarFive JH7110</strong></a> RISC-V SoC and <a href="https://wiki.pine64.org/wiki/STAR64"><strong>Pine64 Star64</strong></a> SBC.</p>
</li>
<li>
<p><strong>mabi=lp64</strong>: This Application Binary Interface says that Long Pointers are 64-bit. No Floating-Point Arguments will be passed in Registers.</p>
<p>We might fix this for JH7110 SoC and Star64 SBC.</p>
<p><a href="https://gcc.gnu.org/onlinedocs/gcc-9.1.0/gcc/RISC-V-Options.html">(More about this)</a></p>
</li>
<li>
<p><strong>mcmodel=medany</strong>: Sounds like a burger (or fast-food AI model) but it actually generates code for the Medium-Any Code Model. (Instead of Medium-Low)</p>
<p><a href="https://gcc.gnu.org/onlinedocs/gcc-9.1.0/gcc/RISC-V-Options.html">(More about this)</a></p>
</li>
</ul>
<h1 id="appendix-download-toolchain-for-64-bit-risc-v"><a class="doc-anchor" href="#appendix-download-toolchain-for-64-bit-risc-v">§</a>11 Appendix: Download Toolchain for 64-bit RISC-V</h1>
<p><strong>UPDATE:</strong> We don’t recommend <a href="https://github.com/sifive/freedom-tools/releases/tag/v2020.12.0"><strong>SiFive Freedom Tools</strong></a> for building Apache NuttX RTOS on Linux, macOS or Windows. (Since it’s outdated)</p>
<p>Please download the <strong>xPack Toolchain</strong> from the next section…</p>
<h1 id="appendix-xpack-gnu-risc-v-embedded-gcc-toolchain-for-64-bit-risc-v"><a class="doc-anchor" href="#appendix-xpack-gnu-risc-v-embedded-gcc-toolchain-for-64-bit-risc-v">§</a>12 Appendix: xPack GNU RISC-V Embedded GCC Toolchain for 64-bit RISC-V</h1>
<p>To build NuttX on Linux, macOS or Windows, we download the toolchain for <a href="https://xpack.github.io/dev-tools/riscv-none-elf-gcc/install/"><strong>xPack GNU RISC-V Embedded GCC</strong></a>. Here are the steps…</p>
<div class="example-wrap"><pre class="language-bash"><code>## Install Toolchain for RISC-V Target: xPack GNU RISC-V Embedded GCC
## Based on https://xpack.github.io/dev-tools/riscv-none-elf-gcc/install/
$ sudo apt -y remove \
gcc-riscv64-unknown-elf \
binutils-riscv64-unknown-elf \
picolibc-riscv64-unknown-elf
$ ls /usr/bin/riscv*
ls: cannot access &#39;/usr/bin/riscv*&#39;: No such file or directory
## Install xPack GCC Toolchain for RISC-V (Linux x64)
## https://github.com/xpack-dev-tools/riscv-none-elf-gcc-xpack/releases
$ wget https://github.com/xpack-dev-tools/riscv-none-elf-gcc-xpack/releases/download/v13.2.0-2/xpack-riscv-none-elf-gcc-13.2.0-2-linux-x64.tar.gz
$ tar xf xpack-riscv-none-elf-gcc-13.2.0-2-linux-x64.tar.gz
$ export PATH=$PWD/xpack-riscv-none-elf-gcc-13.2.0-2/bin:$PATH
$ riscv-none-elf-gcc -v
gcc version 13.2.0 (xPack GNU RISC-V Embedded GCC x86_64)
## For macOS Arm64: Change the above to...
## wget https://github.com/xpack-dev-tools/riscv-none-elf-gcc-xpack/releases/download/v14.2.0-1/xpack-riscv-none-elf-gcc-14.2.0-1-darwin-arm64.tar.gz
## tar xf xpack-riscv-none-elf-gcc-14.2.0-1-darwin-arm64.tar.gz
## Build NuttX, based on...
## https://lupyuen.github.io/articles/release#build-nuttx-for-star64
## https://github.com/lupyuen/nuttx-star64/blob/main/.github/workflows/star64.yml
$ mkdir nuttx
$ cd nuttx
$ git clone https://github.com/apache/nuttx.git nuttx
$ git clone https://github.com/apache/nuttx-apps apps
$ cd nuttx
## Build NuttX for Star64 JH7110 SBC (or VisionFive2 SBC)
## To build NuttX for QEMU: Change &quot;star64:nsh&quot; to &quot;rv-virt:nsh64&quot;
## To build NuttX for Ox64: Change &quot;star64:nsh&quot; to &quot;ox64:nsh&quot;
$ make distclean
$ tools/configure.sh star64:nsh
$ make
LD: nuttx
CP: nuttx.hex</code></pre></div>
<p><a href="https://gist.github.com/lupyuen/8ba3de9ebba0881678b6ecab977443f5">(See the <strong>Complete Steps</strong>)</a></p>
<p>Remember to add the xPack Toolchain to the <strong><code>PATH</code></strong> Environment Variable…</p>
<div class="example-wrap"><pre class="language-text"><code>xpack-riscv-none-elf-gcc-12.3.0-1/bin</code></pre></div>
<p>xPack names the binaries differently differently from other toolchains. Everywhere we see <strong><code>riscv64-unknown-elf</code></strong>, please change to <strong><code>riscv-none-elf</code></strong>.</p>
<p>xPack Toolchain works OK with <strong>Math Functions</strong> on JH7110: <a href="https://gist.github.com/lupyuen/63bd510d7e45ceebe7443c78ed31c6c8"><strong>Source Code</strong></a> / <a href="https://gist.github.com/lupyuen/9bdb1f5478318631d0480f03f6041d83"><strong>Output Log</strong></a> / <a href="https://gist.github.com/lupyuen/24f440e14349b2ed56d1784867156378"><strong>ELF Symbols</strong></a></p>
<p><em>What about Alpine Linux and Debian Linux Containers?</em></p>
<p>Yep xPack Toolchain also works with <strong>Alpine and Debian Linux Containers</strong> (like for VSCode)…</p>
<ul>
<li>
<p><a href="https://github.com/lupyuen/nuttx-nim#build-nuttx-with-debian-container-in-vscode"><strong>Build NuttX with Debian Linux Container</strong></a></p>
<p>(Debian builds more easily than Alpine)</p>
</li>
<li>
<p><a href="https://gist.github.com/lupyuen/880caa0547378028243b8cc5cfdc50a8"><strong>Build NuttX with Alpine Linux Container</strong></a></p>
<p>(Beware of glibc-to-musl conversion!)</p>
</li>
</ul>
<p><em>Does the xPack Toolchain support <code>-mcmodel=medany</code>?</em></p>
<p>Yes the xPack Libraries are compiled with <strong><code>-mcmodel=medany</code></strong>.</p>
<p>xPack Toolchain requires applications to be compiled with <strong><code>-mcmodel=medany</code></strong>, otherwise the link might fail.</p>
<p><a href="https://xpack.github.io/blog/2023/08/25/riscv-none-elf-gcc-v12-3-0-1-released/#-mcmodelmedany">(Source)</a></p>
<p><em>What about the standard toolchain: gcc-riscv64-unknown-elf?</em></p>
<p><a href="https://github.com/sifive/freedom-tools/issues/54"><strong>According to this post</strong></a>, we might use <strong>gcc-riscv64-unknown-elf</strong> and <strong>picolibc-riscv64-unknown-elf</strong>.</p>
<p>But when we build NuttX with <strong>gcc-riscv64-unknown-elf</strong>, it fails with <a href="https://lupyuen.github.io/articles/release#appendix-missing-mathh"><strong>missing “math.h”</strong></a>…</p>
<div class="example-wrap"><pre class="language-text"><code>$ sudo apt install \
gcc-riscv64-unknown-elf \
picolibc-riscv64-unknown-elf
$ make
./stdio/lib_dtoa_engine.c:40:10:
fatal error: math.h: No such file or directory
#include &lt;math.h&gt;</code></pre></div>
<p>How do we point the <strong>NuttX Include and Lib Paths</strong> to picolibc for the NuttX Build?</p>
<p>(So that the NuttX Build will use the RISC-V “math.h” that’s bundled with picolibc)</p>
<p><strong>TODO:</strong> Point the NuttX Include and Lib Paths to picolibc, <a href="https://github.com/apache/nuttx/issues/10594#issuecomment-1722716562"><strong>like this</strong></a></p>
<p><a href="https://www.mail-archive.com/dev@nuttx.apache.org/msg10533.html">(We might need to add <strong>libm.a</strong> to <strong>LDLIBS</strong>)</a></p>
<!-- Begin scripts/rustdoc-after.html: Post-HTML for Custom Markdown files processed by rustdoc, like chip8.md -->
<!-- Begin Theme Picker and Prism Theme -->
<script src="../theme.js"></script>
<script src="../prism.js"></script>
<!-- Theme Picker and Prism Theme -->
<!-- End scripts/rustdoc-after.html -->
</body>
</html>
Reference in a new issue View git blame Copy permalink