From 2ba01e787a6389b37231b4e70045a125de667351 Mon Sep 17 00:00:00 2001 From: Unai Martinez-Corral Date: Sun, 27 Feb 2022 15:43:11 +0100 Subject: [PATCH] docs/f4pga: reorganise, add scope and references Signed-off-by: Unai Martinez-Corral --- docs/f4pga/CommonTargetsAndVariables.md | 44 ------------- docs/f4pga/{GettingStarted.md => Usage.md} | 61 ++++++++++++++++--- docs/f4pga/common/index.rst | 9 --- docs/f4pga/index.rst | 27 ++++++++ .../generic_script_wrapper.md | 0 docs/f4pga/{Module.md => modules/index.md} | 47 ++++++++------ docs/f4pga/{common => modules}/io_rename.md | 0 docs/f4pga/{common => modules}/mkdirs.md | 0 docs/f4pga/{common => modules}/synth.md | 0 docs/index.rst | 9 ++- 10 files changed, 112 insertions(+), 85 deletions(-) delete mode 100644 docs/f4pga/CommonTargetsAndVariables.md rename docs/f4pga/{GettingStarted.md => Usage.md} (86%) delete mode 100644 docs/f4pga/common/index.rst create mode 100644 docs/f4pga/index.rst rename docs/f4pga/{common => modules}/generic_script_wrapper.md (100%) rename docs/f4pga/{Module.md => modules/index.md} (95%) rename docs/f4pga/{common => modules}/io_rename.md (100%) rename docs/f4pga/{common => modules}/mkdirs.md (100%) rename docs/f4pga/{common => modules}/synth.md (100%) diff --git a/docs/f4pga/CommonTargetsAndVariables.md b/docs/f4pga/CommonTargetsAndVariables.md deleted file mode 100644 index 3e24b60..0000000 --- a/docs/f4pga/CommonTargetsAndVariables.md +++ /dev/null @@ -1,44 +0,0 @@ -# sfbuild's common targets and values - -Targets and values are named with some conventions. -Below are lists of the target and value names along with their meanings" - -## Common targets that need to be provided by the user: - -| Target name | list | Description | -|-------------|:----:|-------------| -| `sources` | yes | Verilog sources | -| `sdc` | no | Synopsys Design Constraints | -| `xdc` | yes | Xilinx Design Constraints (available only for Xilinx platforms) | -| `pcf` | no | Physical Constraints File | - -## Commonly requested targets (available in most flows): - -| Target name | list | Description | -|-------------|:----:|-------------| -| `eblif` | no | Extended blif file | -| `bitstream` | no | Bitstream | -| `net` | no | Netlist | -| `fasm` | no | Final FPGA Assembly | -| `fasm_extra` | no | Additional FPGA assembly that may be generated during synthesis | -| `build_dir` | no | A directory to put the output files in | - -## Built-in values - -| Value name | type | Description | -|------------|------|-------------| -| `shareDir` | `string` | Path to symbiflow's installation "share" directory | -| `python3` | `string` | Path to Python 3 executable | -| `noisyWarnings` | `string` | Path to noisy warnings log (should be deprecated) | -| `prjxray_db` | `string` | Path to Project X-Ray database | - -## Values commonly used in flow definitions: - -| Value name | type | Description | -|------------|------|-------------| -| `top` | `string` | Top module name | -| `build_dir` | `string` | Path to build directory (should be optional) | -| `device` | `string` | Name of the device | -| `vpr_options` | `dict[string -> string \| number]` | Named ptions passed to VPR. No `--` prefix included. | -| `part_name` | `string` | Name of the chip used. The distinction between `device` and `part_name` is ambiguous at the moment and should be addressed in the future. | -| `arch_def` | `string` | Path to an XML file containing architecture definition. | diff --git a/docs/f4pga/GettingStarted.md b/docs/f4pga/Usage.md similarity index 86% rename from docs/f4pga/GettingStarted.md rename to docs/f4pga/Usage.md index e20620a..d0e0ec1 100644 --- a/docs/f4pga/GettingStarted.md +++ b/docs/f4pga/Usage.md @@ -1,4 +1,4 @@ -# sfbuild +# Usage ## Getting started @@ -28,8 +28,6 @@ flow completes. Look for a line like this one on stdout.: Target `bitstream` -> build/arty_35/top.bit ``` -------------------------------------------------------------------------------------- - ## Fundamental concepts If you want to create a new sfbuild project, it's highly recommended that you @@ -66,7 +64,7 @@ as well as provide information about files required and produced by the tool. ### Dependecies A **dependency** is any file, directory or a list of such that a **module** takes as -its input or produces on its output. +its input or produces on its output. Modules specify their dependencies by using symbolic names instead of file paths. The files they produce are also given symbolic names and paths which are either set @@ -142,7 +140,7 @@ here and there. Typically **projects's flow configuration** will be used to resolve dependencies for _HDL source code_ and _device constraints_. -## Using sfbuild to build a target +## Build a target To build a **target** "`target_name`", use the following command: ``` @@ -188,7 +186,7 @@ not exist, `mkdirs` will create it and provide as `build_dir` dependency. building a bitstream for *x7a50t* would look like that: With this flow configuration, you can build a bitstream for arty_35 using the -following command: +following command: ``` $ python3 /path/to/sfbuild.py flow.json -p x7a50t -t bitstream @@ -282,7 +280,7 @@ to the box. dependency was necessary or not. * **O** - dependency present, unchanged. This dependency is already built and is confirmed to stay unchanged during flow execution. - * **N** - dependency present, new/changed. This dependency is already present on + * **N** - dependency present, new/changed. This dependency is already present on the persistent storage, but it was either missing earlier, or its content changed from the last time. (WARNING: it won't continue to be reported as "**N**" after a successful build of @@ -290,7 +288,7 @@ to the box. should be fixed in the future.) * **S** - depenendency not present, resolved. This dependency is not currently available on the persistent storage, however it will be produced within - flow's execution. + flow's execution. * **R** - depenendency present, resolved, requires rebuild. This dependency is currently available on the persistent storage, however it has to be rebuilt due to the changes in the project. @@ -312,4 +310,49 @@ colon: In the example above file `counter.v` has been modified and is now marked as "**N**". This couses a bunch of other dependencies to be reqbuilt ("**R**"). -`build_dir` and `xdc` were already present, so they are marked as "**O**". \ No newline at end of file +`build_dir` and `xdc` were already present, so they are marked as "**O**". + +## Common targets and values + +Targets and values are named with some conventions. +Below are lists of the target and value names along with their meanings" + +### Need to be provided by the user + +| Target name | list | Description | +|-------------|:----:|-------------| +| `sources` | yes | Verilog sources | +| `sdc` | no | Synopsys Design Constraints | +| `xdc` | yes | Xilinx Design Constraints (available only for Xilinx platforms) | +| `pcf` | no | Physical Constraints File | + +### Available in most flows + +| Target name | list | Description | +|-------------|:----:|-------------| +| `eblif` | no | Extended blif file | +| `bitstream` | no | Bitstream | +| `net` | no | Netlist | +| `fasm` | no | Final FPGA Assembly | +| `fasm_extra` | no | Additional FPGA assembly that may be generated during synthesis | +| `build_dir` | no | A directory to put the output files in | + +### Built-in values + +| Value name | type | Description | +|------------|------|-------------| +| `shareDir` | `string` | Path to symbiflow's installation "share" directory | +| `python3` | `string` | Path to Python 3 executable | +| `noisyWarnings` | `string` | Path to noisy warnings log (should be deprecated) | +| `prjxray_db` | `string` | Path to Project X-Ray database | + +### Used in flow definitions + +| Value name | type | Description | +|------------|------|-------------| +| `top` | `string` | Top module name | +| `build_dir` | `string` | Path to build directory (should be optional) | +| `device` | `string` | Name of the device | +| `vpr_options` | `dict[string -> string \| number]` | Named ptions passed to VPR. No `--` prefix included. | +| `part_name` | `string` | Name of the chip used. The distinction between `device` and `part_name` is ambiguous at the moment and should be addressed in the future. | +| `arch_def` | `string` | Path to an XML file containing architecture definition. | diff --git a/docs/f4pga/common/index.rst b/docs/f4pga/common/index.rst deleted file mode 100644 index 348dcc8..0000000 --- a/docs/f4pga/common/index.rst +++ /dev/null @@ -1,9 +0,0 @@ -Modules -####### - -.. toctree:: - - generic_script_wrapper - io_rename - mkdirs - synth diff --git a/docs/f4pga/index.rst b/docs/f4pga/index.rst new file mode 100644 index 0000000..baf587d --- /dev/null +++ b/docs/f4pga/index.rst @@ -0,0 +1,27 @@ +Overview +######## + +Python F4PGA is a package containing multiple modules to facilitate the usage of all the tools integrated in the F4PGA +ecosystem, and beyond. +The scope of Python F4PGA is threefold: + +* Provide a fine-grained *pythonic* interface to the tools and utilities available as either command-line interfaces + (CLIs) or application proggraming interfaces (APIs) (either web or through shared libraries). +* Provide a CLI entrypoint covering the whole flows for end-users to produce bitstreams from HDL and/or software sources. +* Provide a CLI entrypoint for developers contributing to bitstream documentation and testing (continuous integration). + +.. ATTENTION:: + This is work-in-progress to adapt and organize the existing shell/bash based plumbing from multiple F4PGA repositories. + Therefore, it's still a *pre-alpha* and the codebase, commands and flows are subject to change. + It is strongly suggested not to rely on Python F4PGA until this note is updated/removed. + +References +========== + +* :gh:`chipsalliance/fpga-tool-perf#390@issuecomment-1023487178 ` +* :ghsharp:`2225` +* :ghsharp:`2371` +* :ghsharp:`2455` +* `F4PGA GSoC 2022 project ideas: Generalization of wrapper scripts for installed F4PGA toolchain and making them OS agnostic `__ +* :gh:`FuseSoc ` | :gh:`Edalize ` +* `Electronic Design Automation Abstraction (EDA²) `__ diff --git a/docs/f4pga/common/generic_script_wrapper.md b/docs/f4pga/modules/generic_script_wrapper.md similarity index 100% rename from docs/f4pga/common/generic_script_wrapper.md rename to docs/f4pga/modules/generic_script_wrapper.md diff --git a/docs/f4pga/Module.md b/docs/f4pga/modules/index.md similarity index 95% rename from docs/f4pga/Module.md rename to docs/f4pga/modules/index.md index c11e933..10e172e 100644 --- a/docs/f4pga/Module.md +++ b/docs/f4pga/modules/index.md @@ -1,10 +1,12 @@ -# sfbuild modules interface +# Modules + +## Interface This document contains all the information needed to configure modules for your _**sfbuild**_ project as well as some info about the API used to write modules. -## Configuration interface: +### Configuration interface: Modules are configured through an internal API by _**sfbuild**_. The basic requirement for a module script is to expose a class with `Module` @@ -23,13 +25,13 @@ A _module configuration_ is a structure with the following fields: either singular strings or lists of strings. * `produces` = a dictionary that contains keys which are names of the dependencies produced by the module. The values are requested filenames for the - files generated by the module. They can be either singular strings or lists of + files generated by the module. They can be either singular strings or lists of strings. * `values` - a dictionary that contains other values used to configure the module. The keys are value's names and the values can have any type. * `platform` - Platform's name. This is a string. -## Platform-level configuration +### Platform-level configuration In case of **platform's flow definition**, a `values` dictionary can be defined globally and the values defined there will be passed to every module's config. @@ -38,7 +40,7 @@ Those values can be overriden per-module through `module_options` dictionary. Parameters used during module's contruction can also be defined in `module_options` as `params` (those are not a part of _module configuration_, instead they are used -during the actual construction of a module instance, before it declares any of its +during the actual construction of a module instance, before it declares any of its input/outputs etc.) Defining dictionaries for `takes` and `produces` is disallowed within @@ -47,7 +49,7 @@ Defining dictionaries for `takes` and `produces` is disallowed within For a detailed look on the concepts described here, please have a look at `sfbuild/platforms/xc7a50t` -## Project-level configuration +### Project-level configuration Similarly to **platform's flow definition**, `values` dict can be provided. The values provided there will overwrite the values from @@ -57,7 +59,7 @@ Unlike **platform's flow definition**, **project's flow configuration** may cont `dependencies` dict. This dictionary would be used to map saymbolic dependency names to actual paths. Most dependencies can have their paths resolved implicitly without the need to provide explicit paths, which is a mechanism that is described -in a later section of this document. However some dependencies must be provided +in a later section of this document. However some dependencies must be provided explicitelly, eg. paths to project's verilog source files. It should be noted that depending on the flow definition and the dependency in question, the path does not necessarily have to point to an already existing file. If the dependency is a @@ -75,11 +77,11 @@ Each of those entries may contain `dependencies`, `values` fields which will overload the `dependecies` and `values` defined in a global scope of **project's flow configuration**. Any other field under those platform entries is treated as a _stage-specific-configuration_. The key is a name of a stage within -a flow for the specified platform and the values are dicts which may contain +a flow for the specified platform and the values are dicts which may contain `dependencies` and `values` fields that overload `dependencies` and `values` repespectively, locally for the stage. -## Internal environmental variables +### Internal environmental variables It's very usefule to be able to refer to some data within **platform's flow definition** and **project's flow configuration** to @@ -129,13 +131,13 @@ Be careful when using this kind of resolution, as it's computational and memory complexity grows exponentially in ragards to the number of list variables being referenced, which is a rather obvious fact, but it's still worth mentioning. -The variables that can be referenced within a definition/configuration fall into 3 +The variables that can be referenced within a definition/configuration fall into 3 categories: * **value references** - anything declared as a `value` can be accessed by it's name * **dependency references** - any dependency path can be referenced using the name - of the dependency prefaced with a ':' prefix. Eg.: `${:eblif}` will resolve + of the dependency prefaced with a ':' prefix. Eg.: `${:eblif}` will resolve to the path of `eblif` dependency. Make sure that the dependency can be actually resolved when you are using this kind of reference. For example you can't use the a reference to `eblif` dependency in a module which does not @@ -149,7 +151,7 @@ categories: * `python3` - path to Python 3 interpreter. * `noisyWarnings` - (this one should probably get removed) -## `Module` class +### `Module` class Each nmodule is represented as a class derived from `Module` class. @@ -165,14 +167,14 @@ Each module script should expose the class by defining it's name/type alias as `ModuleClass`. sfbuild tries to access a `ModuleClass` attribute within a package when instantiating a module. -## Module's execution modes +### Module's execution modes A module ahas essentially two execution modes: * _mapping_ mode * _exec_ mode -### _mapping_ mode +#### _mapping_ mode In _mapping_ mode the module is provided with an incomplete configuration which includes: @@ -184,7 +186,7 @@ includes: The module has to provide a dictionary that will provide every output dependency that's not _on-demand_ a default path. This is basically a promise that when executed in _exec_ mode, the module will produce files for this paths. -Typically such paths would be derived from a path of one of it's input dependencies. +Typically such paths would be derived from a path of one of it's input dependencies. This mechanism allows the user to avoid specifying an explicit path for each intermediate target. @@ -192,7 +194,7 @@ It should be noted that variables referring to the output dependencies can't be accessed at this stage for the obvious reason as their values are yet to be evaluated. -### _exec_ mode +#### _exec_ mode In _exec_ mode the module does the actual work. @@ -210,7 +212,7 @@ The configuration passed into this mode is full and it includes: When the module finishes executing in _exec_ mode, all of the dependencies described in `outputs` should be present. -## Module initialization/instantiation +### Module initialization/instantiation In the the `__init__` method of module's class, the following fields should be set: @@ -222,7 +224,7 @@ set: * `prod_meta` - A dictionary which maps product names to descriptions of these products. -### Qualifiers/decorators +#### Qualifiers/decorators By default the presence of all the dependencies and values is mandatory (In case of `produces` that means that the module always has to produce the listed @@ -242,3 +244,12 @@ ways: Currently it's impossible to combine both '`!`' and '`?`' together. This limitation does not have any reason behind it other than the way the qualifier system is implemented at the moment. It might be removed in the future. + +## Common modules + +```{toctree} +generic_script_wrapper +io_rename +mkdirs +synth +``` diff --git a/docs/f4pga/common/io_rename.md b/docs/f4pga/modules/io_rename.md similarity index 100% rename from docs/f4pga/common/io_rename.md rename to docs/f4pga/modules/io_rename.md diff --git a/docs/f4pga/common/mkdirs.md b/docs/f4pga/modules/mkdirs.md similarity index 100% rename from docs/f4pga/common/mkdirs.md rename to docs/f4pga/modules/mkdirs.md diff --git a/docs/f4pga/common/synth.md b/docs/f4pga/modules/synth.md similarity index 100% rename from docs/f4pga/common/synth.md rename to docs/f4pga/modules/synth.md diff --git a/docs/index.rst b/docs/index.rst index 21b7f03..f952b51 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -44,13 +44,12 @@ The project aims to design tools that are highly extendable and multiplatform. .. toctree:: - :caption: pyF4PGA Reference + :caption: Python utils :maxdepth: 2 - f4pga/GettingStarted - f4pga/CommonTargetsAndVariables - f4pga/Module - f4pga/common/index + f4pga/index + f4pga/Usage + f4pga/modules/index f4pga/DevNotes