CMake + FASTBuild: Distributed, Cached, and Fast

CMake is a build system and not a build tool itself. CMake supports various “backends” like Makefiles, Visual Studio, Xcode, and Ninja. This is a powerful feature of CMake, allowing developers to use the best tool for the project. CMake has not had a new generator since 2011, when Ninja support was added. Back then, anyone with a CMake-based build system suddenly had instant access to a faster parallel build tool. The same thing just happened again: now anyone with an existing CMake build system can try FASTBuild by running cmake -GFASTBuild.

What is FASTBuild? FASTBuild is a high-performance, open-source build tool designed to accelerate software compilation on platforms like Windows, Linux, and macOS. It achieves build time reductions, often 10x or faster than traditional systems, by leveraging advanced features such as highly scalable caching and network distribution. This allows it to distribute the compilation workload across multiple machines on a network. The main consumers of FASTBuild are game developers, with adoption by both large studios and independent developers. FASTBuild is a production-ready system used for developing software across a wide range of targets, including PC, Consoles, Smartphones, and embedded systems. The new CMake integration is expected to broaden adoption beyond the gaming community.

How did we get a FASTBuild generator?

With the recent contributions of Eduard Voronkin, CMake (as of version 4.2.0) now fully supports generating FASTBuild projects. This achievement represents the culmination of a long-term effort that began with the initial feature request in 2014. The first FASTBuild merge request was submitted in 2020, followed by another attempt in 2021. In Fall 2024, Eduard successfully carried out the effort, delivering full integration and closing this multi-year initiative. A special thanks to the very supportive maintainer of FASTBuild Franta Fulin who has provided some critical performance enhancements to FASTbuild in support of this effort.

Performance Summary:

On a single machine, FASTBuild via CMake is similar to or slightly faster than Ninja, depending on the size of the project. What FASTBuild brings to the table is game-changing:

• Simple distributed builds (compile across multiple machines).
• Built-in caching of object files (skip unnecessary recompiles).
  

The following showcases a notable result from building LLVM on an M4 and two M1 Apple laptops.

 How to Use FASTBuild with CMake

1. Install FASTBuild (version 1.17 or newer to get the best experience) and put it in your PATH.
2. Install CMake 4.2 or newer 
3. Generate your build with:
   cmake [<options>] -G FASTBuild -B <path-to-build> -DCMAKE_BUILD_TYPE=Release
4. Build with:
   cmake --build <path-to-build> 

Since this is a new feature, using a master branch CMake with a dev branch of FASTBuild are likely to produce the fastest builds. During the writing of this blog, several inefficiencies were discovered and fixed on both sides, increasing performance significantly. Newer than 1.17 FASTBuild and newer than 4.2 CMake include a performance tweak that decreases configure times in some cases by 50%.

How to Enable Caching in FASTBuild

FASTBuild has built-in object file caching, which lets you reuse results across builds or machines (in order to share results across different machines, sources need to be located under consistent paths).

To enable caching you need to first tell FASTBuild where to put the cache files via one of these two options:

FASTBUILD_CACHE_PATH environment variable can be used to specify cache path via FASTBuild natively.

CMAKE_FASTBUILD_CACHE_PATH variable can be used to set up a directory which FASTBuild will use for caching (can be local directory as well as network folder). 

To enable caching during the build you pass -cache option to FASTBuild

cmake --build <path-to-build> -- -cache

See FASTBuild docs for more information.

How to Set Up Distributed Builds

Distributed builds are where FASTBuild shines: compile your project across a farm of machines. It does this by copying the compiler and its dependencies from the host machine to the remote workers. It should be noted that on Apple OSX, the clang compiler found in /usr/bin/clang and /usr/bin/clang++ will not work as a distributed compiler. This is because this is actually a thin wrapper around the actual compiler. Instead, you have to use the path to the actual compiler, something like this:

export CC=/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang

This also requires SDKROOT to be set on the main build machine for the compiler to work. This would be set to something like this:

export SDKROOT=/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk

Enabling distributed builds requires sharing the list of worker machines via one of two environment variables.

Specify either FASTBUILD_WORKERS with a semi-colon-separated list of IP addresses or machine names. Or use FASTBUILD_BROKERAGE_PATH, which is the path to a shared network location. Note: FASTBUILD_BROKERAGE_PATH variable must be set on both ends (where FBuildWorker is running as well as where the build is happening).  Make sure your firewall is set up to allow connections for port 31234

Start the workers by running FBuildWorker executable on a remote machine.

To enable distribution during the build, you pass -dist option to FBuild or pass it via the CMake –build — options.

Start the build: cmake --build <path-to-build> -- -dist

See FASTBuild docs for more information.

Here is a short video of a distributed build:

FastBuildDemo.mov

Here is a simple example using two workers on machine1 and machine2 with the build on machine3.

From a prompt on machine1

% FBuildWorker

This will pop up a GUI that looks like the following:

Repeat this on machine2

% FBuildWorker

On machine3 you run the build and in this case we will use the FASTBUILD_WORKERS env variable.

% export FASTBUILD_WORKERS="machine1;machine2"
% cd /path/to/build-tree
% fbuild –dist   or cmake –build . – -dist

This should start an amazingly fast build with compile lines scrolling on all three machines.

Loose ends and work TBD

There are still a few features missing that are supported by other CMake generators. The first is that FASTBuild can not handle C++ 20 modules and the dynamic dependency ability that they require. It lacks a multi-configuration mode, unlike Ninja. To have the build system regenerated automatically when the input CMake files are changed, you have to use cmake –build and not invoke FBuild directly. In addition, the new instrumentation feature is not supported.

The current open issues are as follows:

instrumentation: FASTBuild doesn’t support postBuild or preBuild hooks

FASTBuild: fails to build libpng

FASTBuild: fbuild.bff does not regenerate when CMake inputs are updated

Since this is very new code, you may encounter issues. If you do, please report them here: https://gitlab.kitware.com/cmake/cmake/-/issues

Conclusion

CMake + FASTBuild is a great addition to CMake! Although distributed tools like distcc and ccache have been available to CMake developers for years, the FASTBuild generator provides a single tool that provides caching and distributed builds working on Linux, Mac, and Windows! We are excited to hear reports of time savings from this new build backend. 

• FASTBuild and ninja are comparable for local builds.
  
• For distributed and cached builds, FASTBuild is a leap forward.
  

Give it a try — and let us know what you discover.

Some testing and adhoc benchmarking

For those that are interested, we gave FASTBuild a spin on CMake itself, LLVM, and VTK, with several different configurations on Windows and Mac OSX. For these tests we used the -DCMAKE_FASTBUILD_CLANG_REWRITE_INCLUDES=OFF.

The first set of graphs for CMake, VTK, and LLVM used the following specifications on the main build machine, and two M1 macs for workers:

🖥️ System Specifications

🛠️ Tool Versions

The next set of graphs for windows VTK and CMake builds used these specifications:

🖥️ System Specifications

🛠️ Tool Versions

Get Started with FASTBuild Today

Unlock faster, more efficient builds by integrating FASTBuild with your CMake projects. Take advantage of reduced compilation times, advanced caching, and network distribution to scale your development workflow across multiple platforms. To explore how FASTBuild can optimize your build system and support your team’s development goals, contact us and start accelerating your projects today.