306 lines
12 KiB
ReStructuredText
306 lines
12 KiB
ReStructuredText
Copyright 2024 (C) Peter McGoron.
|
|
|
|
This file is a part of Upsilon, a free and open source software project.
|
|
For license terms, refer to the files in ``doc/copying`` in the Upsilon
|
|
source distribution.
|
|
|
|
***************************************************
|
|
|
|
This manual describes the hardware portion of Upsilon.
|
|
|
|
===============
|
|
LiteX and Migen
|
|
===============
|
|
|
|
Migen is a library that generates Verilog using Python. It uses Python
|
|
objects and methods as a DSL within Python.
|
|
|
|
LiteX is a SoC generator using Migen. LiteX includes RAM, CPU, bus logic,
|
|
etc. LiteX is very powerful but not well documented.
|
|
|
|
===================
|
|
System Architecture
|
|
===================
|
|
|
|
Upsilon uses a RISC-V CPU running Linux to power most operations. It currently
|
|
uses a single-core VexRISC-V CPU running mainline Linux 5.x. How the main core
|
|
communicates with the hardware is a software issue: see /doc/software.rst .
|
|
|
|
Basic configuration of the SoC is done in the /gateware/config.py file. If this
|
|
file does not exist, copy /gateware/config.py.def to /gateware/config.py .
|
|
This is the default config.
|
|
|
|
The **main CPU** is the VexRISC-V core running Linux. All other CPUs or bus
|
|
masters can be overridden by this main CPU. To avoid confusion, "master" is
|
|
used when referring to something that is the master of the Wishbone bus: other
|
|
CPUs besides the main CPU are masters, even though their actions are
|
|
subordinated by the main CPU.
|
|
|
|
------------
|
|
Wishbone Bus
|
|
------------
|
|
|
|
The bus on all CPUs is the Wishbone bus. The Wishbone bus is a relatively simple
|
|
yet powerful master-slave architecture. For this project each bus has one master
|
|
and multiple slaves, but each slave can connect to multiple buses (and hence,
|
|
multiple masters).
|
|
|
|
All of the Wishbone bus lines are connected directly to the master with the
|
|
exception of the ``cyc`` signal. The ``cyc`` signal indicates that the master
|
|
has selected that slave device for a transfer. The ``stb`` signal will then
|
|
go up (sometimes at the same time) to indicate that there is valid data on all
|
|
other bus lines of interest. The bus master waits until ``ack`` is asserted.
|
|
|
|
The main CPU has a timeout on the Wishbone bus, but other CPUs may not. This
|
|
is to simplify interconnect logic from a programming perspective.
|
|
|
|
-------------------------
|
|
The Wishbone Bus in LiteX
|
|
-------------------------
|
|
|
|
Each master and slave has a Wishbone bus ``Interface`` (under
|
|
``litex.soc.interconnect.wishbone``). To make it, do::
|
|
|
|
self.bus = Interface(data_width=32, address_width=32, addressing="byte")
|
|
|
|
The bus is always going to be 32 bit and will always be transmitting 32 bit
|
|
words. The reason for ``addressing="byte"`` will be discussed in the next
|
|
section.
|
|
|
|
The basic structure of the bus handling code is::
|
|
|
|
self.sync += [
|
|
If(self.bus.cyc & self.bus.stb & ~self.bus.ack,
|
|
Case(self.bus.adr[0:length], ...),
|
|
self.bus.ack.eq(1)
|
|
).Else(~self.bus.cyc,
|
|
self.bus.ack.eq(0)
|
|
)
|
|
]
|
|
|
|
``length`` is the length in bits that the bus code should look at. Since the
|
|
region could be anywhere in memory, the slave should never look at the entire
|
|
address (except for debugging purposes). Most of the time::
|
|
|
|
length = self.width.bit_length()
|
|
|
|
Note that Migen differs from Verilog, since all indexing is LSB-first and the
|
|
last index is excluded. Hence ``adr[0:length]`` is equivalent to ``adr[length-1:0]``
|
|
in generated Verilog.
|
|
|
|
-----------------------------------
|
|
Rules For Writing Wishbone Bus Code
|
|
-----------------------------------
|
|
|
|
The Main CPU is word-addressed. It only reads at 32-bit word boundaries, and
|
|
will specify sub-word-unit writes using ``sel``. When a *bus* is
|
|
word-addressed, that means it expects addresses to be words. For instance,
|
|
``0x0`` is word 0, ``0x1`` is word 1, etc.
|
|
|
|
Since this is confusing, **all Upsilon Wishbone bus code must be byte
|
|
addressed.** This means that ``0x0`` is byte 0 of word 0 (little endian),
|
|
``0x1`` is byte 1 of word 1, etc. and ``0x4`` is byte 0 of word 1. Even though
|
|
all masters and slaves must be byte-addressed, they are not required to handle
|
|
misaligned accesses. **Upsilon slaves can assume that all accesses are
|
|
word-aligned,** but they should give sane errors on misaligned access.
|
|
|
|
The only masters and slaves that are word-addressed are the ones that are
|
|
from LiteX itself. Those have special code to convert to the byte-addressed
|
|
masters/slaves.
|
|
|
|
If the slave has one bus, it **must** be an attribute called ``bus``.
|
|
|
|
Each class that is accessed by a wishbone bus **must** have an attribute
|
|
called ``width`` that is the size, in bytes, of the region. This must be a power
|
|
of 2 (exception: wrappers around slaves since they might wrap LiteX slaves
|
|
that don't have ``width`` attributes).
|
|
|
|
Each class **should** have a attribute ``public_registers`` that is a dictionary,
|
|
keys are names of the register shown to the programmer and
|
|
|
|
1. ``origin``: offset of the register in memory
|
|
2. ``size``: size of the register in bytes (multiple of 4)
|
|
|
|
are required attributes. Other attributes are ``rw``, ``direction``, that are
|
|
explained in /doc/controller_manual.rst .
|
|
|
|
-----------------------------
|
|
Adding Slaves to the Main CPU
|
|
-----------------------------
|
|
|
|
After adding a module with an ``Interface``, the interface is connected to
|
|
to main CPU bus by calling one of two functions.
|
|
|
|
If the slave region has no special areas in it, call::
|
|
|
|
self.bus.add_slave(name, slave.bus, SoCRegion(origin=None, size=slave.width, cached=False)
|
|
|
|
If the slave region has registers, add::
|
|
|
|
self.add_slave_with_registers(name, iface, SoCRegion(...), slave.public_registers)
|
|
|
|
where the SoCRegion parameters are the same as before. Each slave device
|
|
should have a ``slave.width`` and a ``slave.public_registers`` attribute,
|
|
unless noted. Some slaves have only one bus, some have multiple.
|
|
|
|
The Wishbone cache is very confusing and causes custom Wishbone bus code to
|
|
not work properly. Since a lot of this memory is volatile you should never
|
|
enable the cache (possible exception: SRAM).
|
|
|
|
====================
|
|
System Within a Chip
|
|
====================
|
|
|
|
A *system within a chip* (SWiC) is a SoC within a SoC. Upsilon has the
|
|
capability to add SWiCs that can be controlled by the main CPU. The CPU for
|
|
the SWiC is the PicoRV32, which is a RISC-V RV32IMC core (RISC-V, 32 bit,
|
|
standard registers, multiplication, and compressed instructions).
|
|
|
|
The main CPU controls the SWiC through a special memory region on the Wishbone
|
|
bus. (Currently there are CSRs, but I consider this a hack and they will be
|
|
removed.) There are three ways the main CPU interacts with the SWiC:
|
|
|
|
1. Direct control. The main CPU can start and reset the SWiC CPU. It can
|
|
also inspect the SWiC CPU's registers and program counter.
|
|
2. Exclusive registers. Small data can be transfered in the Main -> SWiC and
|
|
SWiC -> Main direction using *Special Registers*. They are small registers
|
|
that can be read by both CPUs but only one CPU can write to them. This is
|
|
used for sending parameters to programs without having to recompile them.
|
|
3. *Preemptive Interfaces* (PI), which connect a Wishbone slave to two or more
|
|
Wishbone buses. Only one bus has read-write access to the slave at any time.
|
|
The main CPU controls bus access. In the future, both read and write access
|
|
can be modified, instead of the both or neither.
|
|
|
|
As an example of PI, the SWiC RAM is behind a PI. The main CPU resets the SWiC
|
|
(through direct control), fills the SWiC with machine code, fills the exclusive
|
|
registers with values, and then starts the SWiC CPU. External communiciation
|
|
(such as SPI) is through PI.
|
|
|
|
---------------------------------
|
|
Adding Memory Regions to the SWiC
|
|
---------------------------------
|
|
|
|
PicoRV32 uses a byte-addressed bus. However, it looks like it will not attempt
|
|
non-word aligned accesses. Slaves written for the main CPU will work with the SWiC,
|
|
and vice-versa.
|
|
|
|
The processing for connecting a Wishbone slave to the PicoRV32 bus is slightly
|
|
different because the usual LiteX code interferes with the build process (LiteX
|
|
only expects one Wishbone bus). The code for managing the SWiC bus is in
|
|
/gateware/region.py .
|
|
|
|
To add an ``Interface`` called ``iface``::
|
|
|
|
pico.mmap.add_region(name, BasicRegion(origin=origin, size=iface.width, bus=iface))
|
|
|
|
Note that unlike in the main CPU, the origin of the region must be specified.
|
|
The origin does not have to be a power of 2 but must have enough zero bits
|
|
to completely store ``iface.width`` bytes.
|
|
|
|
=====================
|
|
Workarounds and Hacks
|
|
=====================
|
|
|
|
---------------------------------------------
|
|
LiteX Compile Times Take Too Long for Testing
|
|
---------------------------------------------
|
|
|
|
Set ``compile_software`` to ``False`` in ``soc.py`` when checking for Verilog
|
|
compile errors. Set it back when you do an actual compile run, or your program
|
|
will not boot.
|
|
|
|
If LiteX complains about not having a RiscV compiler, that is because your
|
|
system does not have compatible RISC-V compiler in your ``$PATH``. Refer to
|
|
the LiteX install instructions above to see how to set up the SiFive GCC, which
|
|
will work.
|
|
|
|
----------------------------------
|
|
F4PGA Crashes When Using Block RAM
|
|
----------------------------------
|
|
|
|
This is really a Yosys (and really, an abc bug). F4PGA defaults to using
|
|
the ABC flow, which can break, especially for block RAM. To fix, edit out
|
|
``-abc`` in the tcl script (find it before you install it...)
|
|
|
|
This is mitigated by using ``SRAM`` in LiteX directly, which seems to
|
|
magically work.
|
|
|
|
-------------------------------------------------------------
|
|
Modules Simulate Correctly, but Don't Work at All in Hardware
|
|
-------------------------------------------------------------
|
|
|
|
Yosys fails to calculate computed parameter values correctly. For instance,
|
|
|
|
parameter CTRLVAL = 5;
|
|
localparam VALUE = CTRLVAL + 1;
|
|
|
|
Yosys will *silently* fail to compile this, setting `VALUE` to be equal
|
|
to 0. The solution is to use macros.
|
|
|
|
This also seems to magically work in PicoRV32. This may work if ``localparam
|
|
integer`` is used instead.
|
|
|
|
---------------------
|
|
Reset Pins Don't Work
|
|
---------------------
|
|
|
|
On the Arty A7 there is a Reset button. This is connected to the CPU and only
|
|
resets the CPU. Possibly due to timing issues modules get screwed up if they
|
|
share a reset pin with the CPU. The code currently connects button 0 to reset
|
|
the modules seperately from the CPU.
|
|
|
|
-------------------------
|
|
Verilog Macros Don't Work
|
|
-------------------------
|
|
|
|
Verilog's preprocessor is awful. F4PGA (through yosys) barely supports it.
|
|
|
|
You should only use Verilog macros as a replacement for ``localparam``.
|
|
When you need to do so, you must preprocess the file with
|
|
Verilator. For example, if you have a file called ``mod.v`` in the folder
|
|
``firmware/rtl/mod/``, then in the file ``firmware/rtl/mod/Makefile`` add
|
|
|
|
codegen: [...] mod_preprocessed.v
|
|
|
|
(putting it after all other generated files). The file
|
|
``firmware/rtl/common.makefile`` should automatically generate the
|
|
preprocessed file for you.
|
|
|
|
If your Verilog is complex enough to need generation, consider writing
|
|
it in Migen instead.
|
|
|
|
-------------------------
|
|
RAM Check failure on Boot
|
|
-------------------------
|
|
|
|
This is most likely a bus issue. You might have overloaded the CSR bus. Move
|
|
some CSRs to a wishbone bus module. This can also happen due to timing errors
|
|
across the main CPU bus, which should be alleviated by reducing combinational
|
|
circuits and using registers through it.
|
|
|
|
--------------------------------------------------
|
|
Accesses to a Wishbone bus memory area do not work
|
|
--------------------------------------------------
|
|
|
|
Try reading 16 words (64 bytes) into the memory area and see if the
|
|
behavior changes. Many times this is due to the Wishbone Cache interfering
|
|
with volatile memory. Set the `cached` parameter in the SoCRegion to
|
|
`False` when adding the slave.
|
|
|
|
---------------------
|
|
Migen Recursion Error
|
|
---------------------
|
|
|
|
You passed the wrong value (like a string) where Migen expected a statement
|
|
or a value. For instance, instead of an assignment statement, you instead put a
|
|
string indiciating the value you want to assign.
|
|
|
|
---------------------
|
|
Sources Missing Error
|
|
---------------------
|
|
|
|
LiteX build will stop after creating the module tree. This is because you
|
|
imported a module that does not exist. LiteX will silently fail if a Verilog
|
|
source file you added does not exist, so either remove the module or add the
|
|
file.
|