Documentation and register location generation

build/Makefile

@@ -52,6 +52,7 @@ hardware-get:
 	docker cp upsilon-hardware:/home/user/upsilon/gateware/soc_subregions.json ../boot/
 	docker cp upsilon-hardware:/home/user/upsilon/gateware/pico0.json ../boot/
 	docker cp upsilon-hardware:/home/user/upsilon/gateware/mmio.py ../boot/
+	docker cp upsilon-hardware:/home/user/upsilon/gateware/pico0_mmio.h ../boot/
 hardware-clean:
 	-docker container stop upsilon-hardware
 	-docker container rm upsilon-hardware

doc/controller_manual.rst

@@ -34,12 +34,21 @@ The ``machine`` module contains arrays called ``mem8``, ``mem16``, and ``mem32``
 They are used to directly access memory locations on the main CPU bus. Note
 that ``mem32`` accesses must be word aligned.
 
+Example::
+
+    import machine
+    from mmio import *
+    machine.mem32[pico0_dbg_reg]
+
+This reads the first register from ``pico0_dbg_reg``.
+
 -------------------
 Accessing Registers
 -------------------
 
-At the lowest level, a program will write to and read from "registers." These
+At the lowest level, a program will write to and read from "registers" which
-registers control the operations of various parts of the system.
+are mapped to memory. These registers control the operations of various parts
+of the system.
 
 The main bus has two register buses: "CSR" (which is the LiteX default), and
 custom Wishbone code. CSR register information is in the ``csr.json`` file.
@@ -48,6 +57,33 @@ Wishbone bus registers are allocated with regions that are specified in
 ``soc_subregions.json``. These should be automatically dumped to the Micropython
 file ``mmio.py`` for easy usage.
 
+``csr.json`` is not that well documented and can change from version to version
+of LiteX.
+
+``soc_subregions.json`` is a JSON object where the keys denote ``memories`` in
+``csr.json``. If the object of that key is ``null``, then that region is
+uniform (e.g. it is RAM, which is one continuous block). The objects of each of
+these are registers that reside in the memory region. The keys of the registers
+are
+
+1. ``origin``: offset of the register from the beginning of the memory.
+2. ``size``: Size of the register in bytes. Right now this is always ``4``,
+   even if the writable space is smaller. Always access with words.
+3. ``rw``: True if writable and False if not. Sometimes this is not there
+   because the writable might be dynamic or must be inferred from other
+   properties.
+4. ``direction``: Specifcally for registers that are Main-write-Pico-read
+   or Main-read-Pico-write. ``PR`` denotes Pico-read, ``PW`` denotes Pico-write.
+
+``pico0.json`` (and other PicoRV32 JSON files) are JSON objects whose keys are
+memory regions. Their values are objects with keys:
+
+1. ``origin``: Absolute position of the memory region.
+2. ``width``: Width of the memory region in bytes.
+3. ``registers``: Either ``null`` (uniform region, like above), or an object
+   whose keys are the names of registers in the region. The values of these
+   keys have the same interpretation as ``soc_subregions.json`` above.
+
 ====================
 System Within a Chip
 ====================
@@ -58,12 +94,155 @@ programmed and controlled through Micropython.
 
 The SWiC is a RV32IMC core. Code for the SWiC needs to be compiled for a start
 address of ``0x10000`` and a IRQ handler at ``0x10010``. The default length of
+
 the SWiC region is ``0x1000`` bytes.
 
 Each core is given the name ``pico0``, ``pico1``, etc. The regions of each CPU
 are stored in ``pico0.json``, ``pico1.json``, etc. The system used to control
 slave access to the CPU bus is a CSR (and should be in ``mmio.py``).
 
+----------------------------
+Compiling and Executing Code
+----------------------------
+
+There is a Makefile in /swic/ that contains the commands to compile a source
+file (with start function ``_start``) to a binary file without static variables
+or RO data.
+
+Each CPU has a header file (for example ``pico0_mmio.h`` that contains the
+offsets where each word-sized register can be accessed.
+
+If there is only program code (no RODATA, static variables, etc.) then you can
+dump the ``.text`` section using objdump (this requires a RISC-V compiler
+installed, 64 bit is fine). Afterwards the data can be loaded by writing each
+byte into the RAM section (the start of the ram section in the main CPU
+corresponds to ``0x10000`` on the SWiC).
+
+More advanced options would require more advanced linker script knowledge.
+
+----------------
+Complete Example
+----------------
+
+The compiler can be accessed in the docker container, you can also install it
+under Ubuntu.
+
+I haven't tested this yet, but this is how the code should work::
+
+    #include "pico0_mmio.h"
+
+    void _start(void) {
+        uint32_t i = 0;
+
+        for (;;) {
+            *DAC0_TO_SLAVE = i;
+            *DAC0_ARM = 1;
+            while (*!DAC_FINISHED_OR_READY);
+
+            i += *DAC_FROM_SLAVE;
+            *DAC0_ARM = 0;
+        }
+    }
+
+This code does reads and writes to registers defined in ``pico0_mmio.h``.
+This file in this example is saved as ``test.c``.
+
+To compile it use::
+
+    riscv64-unknown-elf-gcc \
+      -march=rv32imc \
+      -mabi=ilp32 \
+      -ffreestanding \
+      -nostdlib \
+      -Os \
+      -Wl,-build-id=none,-Bstatic,-T,riscv.ld,--strip-debug \
+      -nostartfiles \
+      -lgcc \
+      test.c -o test.elf
+
+In order:
+
+1. ``-march=rv32imc`` compiles for RISC-V, 32 bit registers, multiplication,
+   and compressed instructions.
+2. ``-mabi=ilp32`` compiles for the 32 bit ABI without floating pint.
+3. ``-ffreestanding`` compiles as "Freestanding C" <https://en.cppreference.com/w/c/language/conformance>.
+4. ``-Os`` means "optimize for size."
+5. ``-Wl`` introduces linker commands, I don't know how the linker works.
+6. ``-nostartfiles`` does not include the default ``_start`` in the binary.
+7. ``-lgcc`` links the base GCC library, which is used for builtins (I think).
+8. ``test.c -o test.elf`` compiles the C file and outputs it to ``test.elf``.
+
+The resulting ELF can be inspected using ``riscv64-unknown-elf-objdump`` (look up
+the instructions). To copy the machine code to ``test.bin``, execute::
+
+    riscv64-unknown-elf-objcopy -O binary -j .text test.elf test.bin
+
+This code can now be loaded into the PicoRV32's ram. The next code sections
+will be written in Micropython.
+
+First include the Micropython MMIO locations::
+
+    from mmio import *
+
+Next import ``machine``. You will almost always need this library.::
+
+    import machine
+
+Next import the PicoRV32 standard library. (Currently it is hard-coded for pico0,
+this might change if more CPUs are required.)::
+
+    import picorv32
+
+Next load the code into Micropython::
+
+    with open('test.bin', 'rb') as f:
+        prog = f.read()
+
+This assumes that ``test.bin`` has been uploaded to the FPGA. We now turn off
+the CPU and switch control of the RAM to the main CPU::
+
+    machine.mem32[pico0_enable] = 0
+    machine.mem32[pico0ram_iface_master_select] = 0
+
+Both of these values default to ``0`` on startup.
+
+Now the program can be loaded into the PicoRV32's ram::
+
+    assert len(prog) < 0xFFF
+    for offset, b in enumerate(prog, start=pico0_ram):
+        machine.mem8[offset] = b
+
+The first line of the code makes sure that the code will overflow from the
+RAM region and write other parts of memory. (TODO: this is hardcoded, and
+the size of the region should also be written to memory.)
+
+The loop goes though each byte of the program and writes it to the RAM,
+starting at the beginning of the RAM. ``enumerate`` is just a fancy Python way
+of a for loop with an increasing counter.
+
+As a sanity test, check that the program was written correctly::
+
+    for i in range(len(prog)):
+        assert machine.mem8[pico_ram + i] == prog[i]
+
+This can detect overwrite errors (a write to a read-only area will silently
+fail) and cache mismatch errors.
+
+After the program is loaded, the CPU can finally be started::
+
+    machine.mem32[pico0ram_iface_master_select] = 1
+    assert machine.mem8[pico0_ram] == 0
+    machine.mem32[pico0_enable] = 1
+
+The first line switches control of the RAM to the PicoRV32. The second line
+checks if the switch worked. If this line fails, most likely the preemptive
+interface is not properly connected to the PicoRV32 (or my code is buggy).
+The final line starts the CPU.
+
+The state of the CPU can be inspected using ``picorv32.dump()``. This will
+tell you if the CPU is in a trap state and what registers the CPU is currently
+reading.
+
 ================
 Computer Control
 ================
@@ -71,8 +250,7 @@ Computer Control
 Micropython code can be loaded manually with SSH but this gets cumbersome.
 Python scripts on the controlling computer connected to the Upsilon FPGA can
 upload, execute, and read data back from the FPGA automatically. The code that
-does this is in /client/ . This will be updated because of the recent structural
+does this is in /client/ . They don't work right now and need to be updated.
-changes to Upsilon.
 
 ===
 FAQ

doc/docker.md

@@ -85,6 +85,11 @@ You must also change the port in `upsilon/build/Makefile` under `tftp`.
 
 Run `make images` to create all docker images.
 
+## Copy Default Config
+
+If you do not have a ``config.py`` file in ``/gateware``, copy ``config.py.def``
+to ``config.py``.
+
 ## Setup and Run Containers
 
 For `NAME` in `hardware`, `opensbi`, `buildroot`:

doc/gateware.rst

@@ -108,19 +108,40 @@ The only masters and slaves that are word-addressed are the ones that are
 from LiteX itself. Those have special code to convert to the byte-addressed
 masters/slaves.
 
+If the slave has one bus, it **must** be an attribute called ``bus``.
+
 Each class that is accessed by a wishbone bus **must** have an attribute
 called ``width`` that is the size, in bytes, of the region. This must be a power
 of 2 (exception: wrappers around slaves since they might wrap LiteX slaves
 that don't have ``width`` attributes).
 
+Each class **should** have a attribute ``public_registers`` that is a dictionary,
+keys are names of the register shown to the programmer and
+
+1. ``origin``: offset of the register in memory
+2. ``size``: size of the register in bytes (multiple of 4)
+
+are required attributes. Other attributes are ``rw``, ``direction``, that are
+explained in /doc/controller_manual.rst .
+
 -----------------------------
 Adding Slaves to the Main CPU
 -----------------------------
 
 After adding a module with an ``Interface``, the interface is connected to
-to main CPU bus by adding::
+to main CPU bus by calling one of two functions.
 
-    self.add_slave(name, iface, SoCRegion(origin=None, size=iface.width, cached=False)
+If the slave region has no special areas in it, call::
+
+    self.bus.add_slave(name, slave.bus, SoCRegion(origin=None, size=slave.width, cached=False)
+
+If the slave region has registers, add::
+
+    self.add_slave_with_registers(name, iface, SoCRegion(...), slave.public_registers)
+
+where the SoCRegion parameters are the same as before. Each slave device
+should have a ``slave.width`` and a ``slave.public_registers`` attribute,
+unless noted. Some slaves have only one bus, some have multiple.
 
 The Wishbone cache is very confusing and causes custom Wishbone bus code to
 not work properly. Since a lot of this memory is volatile you should never

gateware/Makefile

@@ -10,13 +10,23 @@ DEVICETREE_GEN_DIR=.
 
 all: build/digilent_arty/digilent_arty.bit arty.dtb mmio.py
 
-csr.json build/digilent_arty/digilent_arty.bit: soc.py
+mmio.py csr.json build/digilent_arty/digilent_arty.bit: soc.py
+	@# Litex Version Check (current version 2023.12)
+	@if ! pip show litex | awk '/Version:/ { split($$2, a, "\\."); if (a[1] < 2023 || a[2] < 12) exit(1)}'; then \
+		pip show litex; \
+		echo "You are using an old version of LiteX!"; \
+		echo "Update LiteX or remake your image."; \
+		exit 1; \
+	fi
+	
+	@if [ ! -f config.py ]; then \
+		echo "No config file found! If you are just starting, do"; \
+		echo "$$ cp config.py.def config.py"; \
+	fi
+	
 	cd rtl && make
 	python3 soc.py
 
-mmio.py: csr.json csr2mp.py
-	python3 csr2mp.py > mmio.py
-
 clean:
 	rm -rf build csr.json arty.dts arty.dtb mmio.py
 

gateware/csr2mp.py

@@ -1,34 +0,0 @@
-#!/usr/bin/python3
-# Copyright 2023 (C) Peter McGoron
-#
-# This file is a part of Upsilon, a free and open source software project.
-# For license terms, refer to the files in `doc/copying` in the Upsilon
-# source distribution.
-#######################################################################
-#
-# This file generates memory locations
-#
-# TODO: Devicetree?
-
-import collections
-import argparse
-import json
-
-with open('csr.json', 'rt') as f:
-    csrs = json.load(f)
-
-print("from micropython import const")
-
-for key in csrs["csr_registers"]:
-    if key.startswith("pico0"):
-        print(f'{key} = const({csrs["csr_registers"][key]["addr"]})')
-
-with open('soc_subregions.json', 'rt') as f:
-    subregions = json.load(f)
-
-for key in subregions:
-    if subregions[key] is None:
-        print(f'{key} = const({csrs["memories"][key]["base"]})')
-    else:
-        print(f'{key}_base = const({csrs["memorys"][key]["base"]})')
-        print(f'{key} = {subregions[key].__repr__()}')

gateware/extio.py

@@ -30,7 +30,7 @@ class SPIMaster(Module):
 
     width = 0x10
 
-    registers = {
+    public_registers = {
             "finished_or_ready": {
                 "origin" : 0,
                 "width" : 4,

gateware/region.py

@@ -61,7 +61,7 @@ class BasicRegion:
         return lambda addr: addr[rightbits:32] == (self.origin >> rightbits)
 
     def to_dict(self):
-        return {"origin" : self.origin, "size": self.size, "registers": self.registers}
+        return {"origin" : self.origin, "width": self.size, "registers": self.registers}
 
     def __str__(self):
         return str(self.to_dict())

gateware/soc.py

@@ -64,6 +64,7 @@ from util import *
 from swic import *
 from extio import *
 from region import BasicRegion
+import json
 
 """
 Keep this diagram up to date! This is the wiring diagram from the ADC to
@@ -150,14 +151,29 @@ class _CRG(Module):
 
 class UpsilonSoC(SoCCore):
     def add_ip(self, ip_str, ip_name):
+        # The IP of the FPGA and the IP of the TFTP server are stored as
+        # "constants" which turn into preprocessor defines.
+
+        # They are IPv4 addresses that are split into octets. So the local
+        # ip is LOCALIP1, LOCALIP2, etc.
         for seg_num, ip_byte in enumerate(ip_str.split('.'),start=1):
             self.add_constant(f"{ip_name}{seg_num}", int(ip_byte))
 
     def add_slave_with_registers(self, name, bus, region, registers):
+        """ Add a bus slave, and also add its registers to the subregions
+        dictionary. """
         self.bus.add_slave(name, bus, region)
         self.soc_subregions[name] = registers
 
     def add_blockram(self, name, size, connect_now=True):
+        """ Add a blockram module to the system.
+
+        :param connect_now: Connect the block ram directly to the SoC.
+           You will probably never need this, since this just adds
+           more ram to the main CPU which already has 256 MiB of RAM.
+           Only useful for testing to see if the Blockram works by poking
+           it directly from the main CPU.
+        """
         mod = SRAM(size)
         self.add_module(name, mod)
 
@@ -167,31 +183,48 @@ class UpsilonSoC(SoCCore):
         return mod
 
     def add_preemptive_interface(self, name, size, slave):
+        """ Add a preemptive interface with "size" connected to the slave's bus. """
         mod = PreemptiveInterface(size, slave)
         self.add_module(name, mod)
         return mod
 
     def add_picorv32(self, name, size=0x1000, origin=0x10000):
+
+        # Add PicoRV32 core
         pico = PicoRV32(name, origin, origin+0x10)
         self.add_module(name, pico)
+
+        # Attach the register region to the main CPU.
         self.add_slave_with_registers(name + "_dbg_reg", pico.debug_reg_read.bus,
                 SoCRegion(origin=None, size=pico.debug_reg_read.width, cached=False),
-                pico.debug_reg_read.registers)
+                pico.debug_reg_read.public_registers)
 
+        # Add a Block RAM for the PicoRV32 toexecute from.
         ram = self.add_blockram(name + "_ram", size=size, connect_now=False)
+
+        # Control access to the Block RAM from the main CPU.
         ram_iface = self.add_preemptive_interface(name + "ram_iface", 2, ram)
+
+        # Allow access from the PicoRV32 to the Block RAM.
         pico.mmap.add_region("main",
                 BasicRegion(origin=origin, size=size, bus=ram_iface.buses[1]))
 
+        # Allow access from the main CPU to the Block RAM.
         self.add_slave_with_registers(name + "_ram", ram_iface.buses[0],
                 SoCRegion(origin=None, size=size, cached=True),
                 None)
 
     def picorv32_add_cl(self, name, param_origin=0x100000):
+        """ Add a register area containing the control loop parameters to the
+            PicoRV32.
+
+            :param param_origin: The origin of the parameters in the PicoRV32's
+            address space. """
         pico = self.get_module(name)
-        param_iface = pico.add_cl_params(param_origin, name + "_cl.json")
+        params = pico.add_cl_params(param_origin, name + "_cl.json")
-        self.bus.add_slave(name + "_cl", param_iface,
+        self.add_slave_with_registers(name + "_cl", params.mainbus,
-                SoCRegion(origin=None, size=size, cached=False))
+                SoCRegion(origin=None, size=params.width, cached=False),
+                params.public_registers)
 
     def add_AD5791(self, name, **kwargs):
         args = SPIMaster.AD5791_PARAMS
@@ -304,8 +337,10 @@ class UpsilonSoC(SoCCore):
 
         # Add control loop DACs and ADCs.
         self.add_picorv32("pico0")
+        self.picorv32_add_cl("pico0")
         # XXX: I don't have the time to restructure my code to make it
         # elegant, that comes when things work
+        # If DACs don't work, comment out from here
         module_reset = platform.request("module_reset")
         self.add_AD5791("dac0",
                 rst=module_reset,
@@ -318,11 +353,11 @@ class UpsilonSoC(SoCCore):
         self.add_preemptive_interface("dac0_PI", 2, self.dac0)
         self.add_slave_with_registers("dac0", self.dac0_PI.buses[0],
                 SoCRegion(origin=None, size=self.dac0.width, cached=False),
-                self.dac0.registers)
+                self.dac0.public_registers)
         self.pico0.mmap.add_region("dac0",
                     BasicRegion(origin=0x200000, size=self.dac0.width,
                                 bus=self.dac0_PI.buses[1],
-                                registers=self.dac0.registers))
+                                registers=self.dac0.public_registers))
 
         self.add_LT_adc("adc0",
                 rst=module_reset,
@@ -334,22 +369,46 @@ class UpsilonSoC(SoCCore):
         self.add_preemptive_interface("adc0_PI", 2, self.adc0)
         self.add_slave_with_registers("adc0", self.adc0_PI.buses[0],
                 SoCRegion(origin=None, size=self.adc0.width, cached=False),
-                self.adc0.registers)
+                self.adc0.public_registers)
         self.pico0.mmap.add_region("adc0",
                     BasicRegion(origin=0x300000, size=self.adc0.width,
                                 bus=self.adc0_PI.buses[1],
-                                registers=self.adc0.registers))
+                                registers=self.adc0.public_registers))
+        # To here
 
     def do_finalize(self):
         with open('soc_subregions.json', 'wt') as f:
-            import json
             json.dump(self.soc_subregions, f)
 
+def generate_main_cpu_include(csr_file):
+    """ Generate Micropython include from a JSON file. """
+    with open('mmio.py', 'wt') as out:
+
+        print("from micropython import const", file=out)
+        with open(csr_file, 'rt') as f:
+            csrs = json.load(f)
+
+        for key in csrs["csr_registers"]:
+            if key.startswith("pico0"):
+                print(f'{key} = const({csrs["csr_registers"][key]["addr"]})', file=out)
+
+        with open('soc_subregions.json', 'rt') as f:
+            subregions = json.load(f)
+
+        for key in subregions:
+            if subregions[key] is None:
+                print(f'{key} = const({csrs["memories"][key]["base"]})', file=out)
+            else:
+                print(f'{key}_base = const({csrs["memories"][key]["base"]})', file=out)
+                print(f'{key} = {subregions[key].__repr__()}', file=out)
+
 def main():
     from config import config
     soc =UpsilonSoC(**config)
     builder = Builder(soc, csr_json="csr.json", compile_software=True)
     builder.build()
 
+    generate_main_cpu_include("csr.json")
+
 if __name__ == "__main__":
     main()

gateware/swic.py

@@ -4,6 +4,9 @@
 # For license terms, refer to the files in `doc/copying` in the Upsilon
 # source distribution.
 
+# XXX: PicoRV32 code only handles word-sized registers correctly. Memory
+# regions made up of multiple words are not supported right now.
+
 from migen import *
 from litex.soc.interconnect.csr import CSRStorage, CSRStatus
 from litex.soc.interconnect.wishbone import Interface, SRAM, Decoder
@@ -183,8 +186,8 @@ class RegisterInterface(LiteXModule):
         bus_logic(self.picobus, pico_case)
 
         # Generate addresses
-        self.addresses = {}
+        self.public_registers = {}
-        for reg, off in self.registers:
+        for reg, off in self.public_registers:
             self.addresses[reg.name] = {
                     "width" : reg.width,
                     "direction" : reg.direction,
@@ -197,7 +200,7 @@ class RegisterRead(LiteXModule):
         "s0/fp", "s1", "a0", "a1", "a2", "a3", "a4", "a5", "a6", "a7", "s2",
         "s3", "s4", "s5", "s6", "s7", "t3", "t4", "t5", "t6",
     }
-    registers = {name: {"origin" : num * 4, "size" : 4, "rw": False} for num, name in enumerate(pico_registers)}
+    public_registers = {name: {"origin" : num * 4, "width" : 4, "rw": False} for num, name in enumerate(pico_registers)}
     """ Inspect PicoRV32 registers via Wishbone bus. """
     def __init__(self):
         self.regs = [Signal(32) for i in range(1,32)]
@@ -219,6 +222,24 @@ class RegisterRead(LiteXModule):
                 )
         ]
 
+def gen_pico_header(pico_name):
+    """ Generate PicoRV32 C header for this CPU from JSON file. """
+    import json
+    with open(pico_name + "_mmio.h", "wt") as out:
+        print('#pragma once', file=out)
+
+        with open(pico_name + ".json") as f:
+            js = json.load(f)
+
+        for region in js:
+            if js[region]["registers"] is None:
+                continue
+            origin = js[region]["origin"]
+            for reg in js[region]["registers"]:
+                macname = f"{region}_{reg}".upper()
+                loc = origin + js[region]["registers"][reg]["origin"]
+                print(f"#define {macname} (volatile uint32_t *)({loc})", file=out)
+
 # Parts of this class come from LiteX.
 #
 # Copyright (c) 2016-2019 Florent Kermarrec <florent@enjoy-digital.fr>
@@ -247,8 +268,7 @@ class PicoRV32(LiteXModule):
                     ("zpos", "PW", 32),
                 ))
         self.add_module("cl_params", params)
-        self.mmap.add_region("cl_params", BasicRegion(origin, params.width, params.picobus, params.addresses))
+        self.mmap.add_region("cl_params", BasicRegion(origin, params.width, params.picobus, params.public_registers))
-        params.dump_json(dumpname)
         return params
 
     def __init__(self, name, start_addr=0x10000, irq_addr=0x10010, stackaddr=0x100FF):
@@ -344,4 +364,5 @@ class PicoRV32(LiteXModule):
 
     def do_finalize(self):
         self.mmap.dump_mmap(self.name + ".json")
+        gen_pico_header(self.name)
         self.mmap.finalize()

swic/picorv32.py

@@ -21,6 +21,9 @@ def u(n):
     # Converts possibly signed number to unsigned 32 bit.
     return hex(n & 0xFFFFFFFF)
 
+
+# XXX: Currently hardcoded for Pico0. Future versions should accept a
+# class that has fields.
 def dump():
     print("Running:", "yes" if machine.mem32[pico0_enable] else "no")
     print("Trap status:", trapcode[machine.mem32[pico0_trap]])