f4pga/sfbuild/docs/DevNotes.md

Developer's notes

Project's structure

The main script is in the sfbuild.py file. sf_cache.py contains code needed for tracking modifications in the project. sf_ugly contains some ugly workarounds.

There a are two python modules which are shared by the code of sfbuild.py and sfbuild modules: sf_common and sf_module.

sfbuild modules are extensions to the build system that wrap tools to be used within sfbuild and currently they are standalone executable scripts. All sfbuild modules are single python scripts located under directories that follow sf_*_modules/ pattern. So currently those are:

• sf_common_modules - modules which can be shared by multiple platforms.
• sf_xc7_modules - modules specific to xc7 flows.
• sf_quicklogic_modules - modules specific to Quiclogic flows.

There's also a docs directory which you are probably aware of if you are reading this. All the documentation regarding sfbuild goes here.

platforms direcotory contains JSON files with platform flow definitions. Names of those files must follow platform_name.json pattern.

Differnt subsystems and where to find them?

Building and dependency resolution

All the code regarding dependency resolution is located in sfbuild.py file. Take a look at the Flow class.

Most of the work is done in Flow._resolve_dependencies method. Basically it performs a DFS with stages (instances of sfbuild modules) as its nodes which are linked using symbolic names of dependencies on inputs and outputs. It queries the modules for information regarding i/o (most importantly the paths on which they are going to produce outputs), checks whether their inputs are going to be satisfied, checks if dependencies were modified, etc.

The actual building is done using Flow._build_dep procedure. It uses a similar DFS approach to invoke modules and check their inputs and outputs.

Modification tracking

Modification tracking is done by taking, comparing and keeping track of adler32 hashes of all dependencies. Each dependency has a set of hashes associted with it. The reason for having multiple hashes is that a dependency may have multiple "consumers", ie. stages which take it as input. Each hash is associated with particular consumer. This is necessary, because the system tries to avoid rebuilds when possible and status of each file (modified/unmodified) may differ in regards to individual stages.

Keeping track of status of each file is done using SymbiCache class, which is defined in sf_cache.py file. SymbiCache is used mostly inside Flow's methods.

Module's internals and API

sf_module contains everything that is necessary to write a module. Prticularly the Module and ModuleContext classes The do_module function currently servers as to create an instance of some Module's subtype and provide a CLI interface for it.

The CLI interface however, is not meant to be used by an end-user, especially given that it reads JSON data from stdin. A wrapper for interfacing with modules exists in sfbuild.py and it's called _run_module.

Internal environmental variable system

sfbuild exposes some data to the user as well as reads some using internal environmental variables. These can be referenced by users in platform flow definitions and project flow configurations using the ${variable_name} syntax when defining values. They can also be read inside sfbuild modules by accesing the ctx.values namespace.

The core of tis system is the ResolutionEnvironemt class which can be found inside the sf_common module.

Installation

Check CMakeLists.txt.

TODO:

Therea re a couple things that need some work:

Urgent

• Full support for Quicklogic platforms.
• Testing XC7 projects with more sophisticated setups and PCF flows.

Important

• Fix and refactor overloading mechanism in platform flow definitions and platform flow configurations. Values in the global values dict should be overloaded by those in values dict under module_options.stage_name inside platform flow definition. Values in platform flow configuration should be imported from platform flow definition and then overloaded by entries in values, platform_name.values, platform_name.stages.stage_name.values dicts respectively.

• Define a clear specification for entries in platform flow definitions and platform flow configurations. Which environmental variables can be accessed where, and when?

• Force "on-demand" outputs if they are required by another stage. This may require redesigning the "on-demand" feature, which currently works by producing a dependency if and only if the user explicitely provides the path. Otherwise the path is unknown.

• Make commenting style consistent

• Write more docs

Not very important

• Extend the metadata system for modules, perhaps make it easier to use.

• Add missing metadata for module targets.

Maybe possible in the future

• Generate platform defintions using CMake.

Out of the current scope

• Change interfaces of some internal python scripts. This could lead to possibly merging some modules for XC7 and Quicklogic into one common module.

Quicklogic

So far I've been trying to bring support to EOS-S3 platform with mixed results. Some parts of upstream Symbiflow aren't there yet. The Quicklogic scripts are incomplete.

The k4n8 family remains a mystery to me. There's zero information about any other familiar that PP3 and PP2. Neither could I find example projects for that. Symbiflow's website mentions that only briefly. Yosys complains about _DLATCH_N_ not being supported when I tried synthesisng anything. Possibly related to the fact that there's no equivalent of pp3_latches_map.v for k4n8/umc22 in Yosys.

UPDATE: Finally got the ioplace stage to work. Pulling the Quicklogic fork was necessary in order to progress. The Quicklogic EOS-S3 development is now moved into eos-s3 branch of my fork. Additionally The chandalar.pcf file in symbiflow-examples seemed to be faulty. The '()' parenthesis should be replaced by '[]' brackets. I also tried to synthesize the iir project from tool-perf, but VPR seems to be unable to fit it (at least on my installation of Symbiflow which at this point is a bit old and heavily modified).

Here's a flow configuration I've used for btn_counter on eos-s3:

{
    "dependencies": {
        "sources": ["btn_counter.v"],
        "synth_log": "${build_dir}/synth.log",
        "pack_log": "${build_dir}/pack.log"
    },
    "values": {
        "top": "top",
        "build_dir": "build/eos-s3"
    },

    "ql-eos-s3": {
        "dependencies": {
            "pcf": "chandalar.pcf",
            "build_dir": "${build_dir}"
        }
    }
}