TinyBuilder

While it seems that build automation should be an afterthought that just happens, it never is. Build scripts are hard to write, debug and maintain. The build tool chain is a source of disruption and a distraction to development. The operating system can make it difficult to test code intended to be used in a GUI process, even if the tests don't require human interaction.

TinyBuilder is a free build system intended to address these problems. All of the base product is free, without any resource or time limitations. Premium features may be used along with the base product to increase its value.

TinyBuilder has a client/server architecture. The client is simple to install; often there is no setup. The server can be tightly controlled by an independent team so all developers use exactly the same build environment. TinyBuilder supports clouds of identical build servers to support large development teams.

On-boarding a new developer simply involves installing source code versioning software and the TinyBuilder client; frequently, no other setup is required.

Developers often make use of tools that the build architects never expected. As a result, developers all too frequently install software that affects the build tool chain, creating difficult to diagnose build failures and bugs. With TinyBuilder, the tool chain is on the build server, not on the developer's machine. As a result, unexpected changes to the developer's machine will not affect the developer's builds.

When building for a product that works on multiple platforms, it is difficult for developers to share hardware, so often each developer gets redundant hardware. With TinyBuilder, this is unnecessary. Each job is assigned one or more machines; the job may run on one of a set of machines, or it may run on every member of a set of machines. Running on one of a list of machines is useful for building; large teams may share a large number of build servers while small teams may share a small number of build servers. When testing on multiple configurations, a job may run on every server in a list with each server configured differently to ensure proper test coverage.

With TinyBuilder, every developer is equal; all the work for a particular platform does not need to go to a specific developer simply because that developer is the only one with the hardware.

Any build script is easy at the beginning when there are a few source files and a single executable; good build scripts are not hard when there are a hundred executables and a thousand source files. When adding source, adding libraries or changing compilation options, the script that used to work should continue working.

The problem with many build script languages is they have little to no concept of scope; while reading the script, it is hard to predict what will be affected by a change; while adding new code will affect the rest of the script in subtle ways. The build script is like a blob where changing anything changes everything.

The TinyBuilder script is like a tree; everything is structured as a set of blocks and blocks can be included in parent blocks to any depth. To trace the effect of a change to a block, simply perform a text search through the build script files for the block name. Any block that does not directly or indirectly include the changed block will not be affected by the change.

At the root of the tree is the job. The job knows all of the input files, output files and command lines required. The input files are collected into a compressed archive that is transferred to the server over the network. On the server, the job is given its own directory, called a work area, where the commands are executed; another job will not use the same files. After the command lines complete, the output files are gathered into the compressed output archive, and transferred to the client. There, the archive is extracted and the output files are placed as if the commands had run locally.

Multiple jobs are collected into a project block. When jobs are run as part of a project, the project creates a schedule to run the jobs; a job with input from another job will run after the job generating the input. However, if the newest input file is is older than the oldest output file, then the job is considered up to date and is skipped. Since a job is explicitly informed of all of its input, it will never be skipped when it should have run. A job can do as little or as much as its command line list specifies, so scheduling can be as fine grained as desired.

Since jobs just run a list of commands, they are just as useful for testing as they are for building. Each project has a build list and a test list; the members of the test list are run even if they are up to date. Like jobs in the build list, each run only after the build jobs have generated the input executables. If all of the build jobs are up to date, only the test jobs are run.

Windows and macOS make a strong distinction between applications that run in the background and applications that interact directly with users. This can create problems for automated tests, even if the tests do not require user interaction. Other CI/CD solutions require tricking software designed to only run in the background to run as a foreground application. The configuration required is complex and fragile; as a result the build automation breaks, seemingly at random, when something in the environment changes.

For TinyBuilder, the solution is straightforward. In addition to installing the server, the TinyBuilder server install installs the agent. The agent is a foreground application by design. When a TinyBuilder job is assigned to be run by an agent, the client connects to the server and the server acts as a gateway to the agent. To start the agent, the user who installed the server simply needs to login; the agent will be available for use as long as the user is logged in.