Rationalize and refactor build system (part 2) #1422
Conversation
whitequark
force-pushed
the
catherine/refactor-cmake-part2
branch
4 times, most recently
from
6303fbe to
0fc4fb2
Compare
whitequark
force-pushed
the
catherine/refactor-cmake-part2
branch
6 times, most recently
from
0085bdd to
a8e6330
Compare
whitequark
changed the title
whitequark
force-pushed
the
catherine/refactor-cmake-part2
branch
from
a8e6330 to
cd4f8a0
Compare
This fully preserves existing functionality, although the `embed` mode is untested and seems broken.
While it served a purpose (granting the ability to build `.bba` files separately from the rest of nextpnr), it made things excessively convoluted, especially around paths. This commit removes the ability to pre-generate chip databases. As far as I know, I was the primary user of that feature. It can be added back if there is demand for it. In exchange the per-family `CMakeLists.txt` files are now much easier to understand.
whitequark
force-pushed
the
catherine/refactor-cmake-part2
branch
from
3a4c296 to
9caffca
Compare
whitequark
force-pushed
the
catherine/refactor-cmake-part2
branch
from
9caffca to
73509b9
Compare
Primarily, this commit makes both of them use the `BBAsm` functions to build and compile `.bba` files. In addition, Himbaechel targets are now aligned with the rest in how they are configured: instead of having all uarches enabled with all of the devices disabled (the opposite of the rest of nextpnr), uarches must be enabled explicitly but they come with all devices enabled (except for Xilinx, which does not have a list of devices).
This removes the atomic rename for bbasm outputs because it embeds the resulting paths into the `.cc` files in embed mode. In any case the write should be fast enough to not be a big risk for interrupted builds. This was tested with Clang 19 only (gcc hasn't had a release that supports `#embed` yet).
Two user-visible changes were made: * `-DUSE_RUST` is replaced with `-DBUILD_RUST`, by analogy with `-DBUILD_PYTHON` * `-DCOVERAGE` was removed as it doesn't work with either modern GCC or Clang
This is added primarily for YoWASP.
The impetus for this commit is the fact that it causes rare but build-breaking race conditions when used with `make -jN` with `N > 1`. These race conditions are difficult to track down or fix because of the very rudmentary debugging tools provided by `make` and opaque semantics of CMake's Makefile generator. They break the build by running two `.bba` generation processes, then one of them renaming the `.bba.new` file once it's done, leaving the other one to fail. After reflection (as the author of this code path) and discussion with community members who use it, I've concluded that this isn't the right approach. 1. In practice, on targets where `-DSERIALIZE_CHIPDBS=` matters, you also care about other build steps, like linking nextpnr, which are not serializable this way. So you use a workaround anyway, like `make`ing individual targets instead. 2. The way to serialize the build with Make is the `-j1` option. Trying to work around `-jN` to make it work like `-j1` is inherently error prone. While there is some utility in not serializing C++ compilation this utility could be more easily achieved by providing a single target that builds all chipdbs, running `make <chipdb-target> -j1`, then running `make -jN` for the rest of the build.
whitequark
force-pushed
the
catherine/refactor-cmake-part2
branch
from
73509b9 to
fdc0198
Compare
|
@whitequark there are couple of things that are now broken (when building oss-cad-suite but some are general) minor issues (when doing in-source-tree build):
major issues :
Since generating BBA files takes time but always generate same output (for same endianess) and sometimes specific binaries/scripts only for this, this was used on oss-cad-suite build to minimize build time and lower redudancy. Using nextpnr-arch-chipdb target we can get BBA created (even at the wrong location right now) but it also lose some time on generating .cc files, splitting these two into two targets and making a way to propagate BBA_CHIPDB location for bba files instead would help fixing this. |
I would rather disallow in-source-tree builds entirely. They cause many issues and with CMake in particular there are just way too many auxiliary files to keep track of. Projects like LLVM already do that for similar reasons. @gatecat What do you think?
The change that resulted in this was done quite intentionally since the existing system for generating BBA files was unmaintainable itself and also an unmanageable obstacle to changing how the build system works. Pre-running Pre-running |
|
I'm going to add export/import of generated |
|
I've implemented this functionality in #1436, please let me know if it works. The I don't know how useful it is, but you can also do a normal build with @cr1901 You should be able to run |
Depends on #1421.
Also allows splitting
nextpnr-himbaechelper microarchitecture.