f4pga/docs/f4pga/DevNotes.md

Developer's notes

Last update: 2022-05-06

:::{warning} These notes are provided as-is and they shouldn't be treated as a full-blown accurate documentation, but rather as a helpful resource for those who want to get involved with development of f4pga. These are not updated regularly.

For more detailed, up-to-date information about the code, refer to the pydoc documentation. :::

Project's structure

• __init__.py contains the logic and entry point of the build system
• argparser.py contains boring code for CLI interface
• cache.py contains code needed for tracking modifications in the project.
• common.py contains code shared by the main utility and the modules
• flow_config.py contains code for reading and accessing flow definitions and configurations
• module_inspector.py contains utilities for inspecting I/O of modules
• module_runner.py contains code required to load modules at run-time
• module.py contains definitions required for writing and using f4pga modules
• part_db.json contains mappings from part names to platform names
• setup.py contains a package installation script
• stage.py contains classes relevant to stage representation
• modules contains loadable modules
• platforms contains platform flow definitions

:::{important} Through the codebase f4pga (tool) might be often referenced as sfbuild. Similarly, F4PGA (toolchain) might get called Symbiflow. This is due to the project being written back when F4PGA was called Symbiflow. :::

Different subsystems and where to find them?

Building and dependency resolution

All the code regarding dependency resolution is located in __init__.py file. Take a look at the Flow class.

Most of the work is done in Flow._resolve_dependencies method. Basically it performs a DFS with stages (instances of f4pga modules) as its nodes which are linked using symbolic names of dependencies on inputs and outputs. It queries the modules for information regarding i/o (most importantly the paths on which they are going to produce outputs), checks whether their inputs are going to be satisfied, checks if dependencies were modified, etc.

The actual building is done using Flow._build_dep procedure. It uses a similar DFS approach to invoke modules and check their inputs and outputs.

Modification tracking

Modification tracking is done by taking, comparing and keeping track of adler32 hashes of all dependencies. Each dependency has a set of hashes associated with it. The reason for having multiple hashes is that a dependency may have multiple "consumers", ie. stages which take it as input. Each hash is associated with particular consumer. This is necessary, because the system tries to avoid rebuilds when possible and status of each file (modified/unmodified) may differ in regards to individual stages.

Keeping track of status of each file is done using F4Cache class, which is defined in cache.py file. F4Cache is used mostly inside Flow's methods.

Internal environmental variable system

f4pga exposes some data to the user as well as reads some using internal environmental variables. These can be referenced by users in platform flow definitions and project flow configurations using the ${variable_name} syntax when defining values. They can also be read inside f4pga modules by accessing the ctx.values namespace.

The core of its system is the ResolutionEnvironemt class which can be found inside the common module.

Installation

Check CMakeLists.txt.

TODO:

• Define a clear specification for entries in platform flow definitions and platform flow configurations. Which environmental variables can be accessed where, and when?

• Force "on-demand" outputs if they are required by another stage. This may require redesigning the "on-demand" feature, which currently works by producing a dependency if and only if the user explicitly provides the path. Otherwise the path is unknown.

• Make commenting style consistent

• Document writing flow definitions

• Extend the metadata system for modules, perhaps make it easier to use.

• Add missing metadata for module targets.

• (suggestion) Generate platform definitions using CMake.

Out of the current scope

• Change interfaces of some internal python scripts. This could lead to possibly merging some modules for XC7 and Quicklogic into one common module.