The Pitchfork Layout (PFL)

Table of Contents

Original article: github.com/vector-of-bool/pitchfork.

Introduction
#

The phrase “it doesn’t matter, just be consistent” has really harmed the C++ community more than it has helped. “A foolish consistency”, if you will. Developers’ desire to keep their codebase unique and “special” has held back the development of tooling when every user insists that a tool support their own custom style.

For some things, like where you put the opening brace, the difference might truly be insignificant. For other things, this isn’t the case.

One such thing is the layout of a project on a filesystem.

Incongruous layouts not only harms tooling, they create unnecessary disparity, create friction during on-boarding of contributors, and present a huge burden to beginners for a task that shouldn’t even take more than a few seconds of thought.

For virtually all projects, they are some (maybe improper) subset of a few key components:

Compilable source files
Public headers
Private headers
Source files containing entry points (main() functions)
Documentation files
Tests
Samples and examples
External libraries which have been embedded within the project structure
“Add-ons” to the source (e.g. language bindings, optional plugins, platform bindings)

In addition, it is very common to see projects which subdivide themselves into submodules. These submodules may also have dependencies between each-other. Submodules greatly increase the complexity potential, but aren’t completely out of the question. A project layout standard must be able to handle subdivided projects.

Background and Terminology
#

The ideas and concepts presented herein are not novel. Rather, they are built upon the work of others. Some come from professional idea and experience, while others come from community convention, which has gradually converged on to prefer certain patterns while decrying others.

Physical vs. Logical Layout
#

This document builds heavily upon the work of John Lakos, who has spear-headed much work on the aspects of physical design. Much work has been put into evangelizing in the area of logical design, but physical design to this day remains a topic that (in this author’s opinion) is too often overlooked when designing and implementing systems, as well as the teaching thereof.

John Lakos’s 1996 book Large-Scale C++ Software Design, to this day, remains as the book on physical design in C++.

So what’s the difference between physical and logical design?

Logical design pertains closely to programming-language-level aspects of the design of a system.

Physical design pertains to the aspects of design outside of what is encoded by the language.

The source unit dealt with by the C and C++ standards is the translation unit. This corresponds to a source file which has had all preprocessor directives resolved. This is the closest that the language standards get to discussing physical design of systems.

Physical design, despite its name, still lives within the digital space, of course. It concerns the placement and naming of files and directories within a filesystem, as well as how to coordinate communication between physical units where the relationship between their filesystem locations is not deterministic.

This document does not address much in the way of logical design. Rather, it is specifically written to address many common questions and resolve common answers for physical design.

The Physical Component
#

The most fundamental unit of physical design is the Component.

A component should correspond to exactly two files: A header and a source file. For some rare cases it may be beneficial to use more than a single source file in a single physical component, but it may be a sign that further refactoring is needed.

Additionally, a component may have a test source file, but it is not considered a part of the physical component.

Logical/Physical Coherence
#

This document assumes that developers will work to maintain logical/physical coherence. There must be a coherent relationship between the logical and physical components of a system, and a single logical component must not be spread across multiple physical components. Multiple logical components should not appear within the same physical component.

Note: Logical components can include more than an individual class or function. For example, it may include classes representing private data (for the PIMPL pattern), or friend classes and functions. Blindly splitting every class and function into individual physical components is not required nor recommended.

Physical components must not form cyclic dependencies.

_{Example of friendship}

// list.hpp
namespace acme {

template <typename T>
class list;

namespace detail {

// Class implementing a "list iterator"
template <typename T>
class list_iterator {
public:
    // Public default constructor
    list_iterator() = default;

private:
    // A private constructor used to initialize the iterator to the proper state
    explicit list_iterator(list<T>&);

    // Permit our list class to construct us with the private constructor
    friend class list<T>;
};

}

template <typename T>
class list {
public:
    using iterator = detail::list_iterator<T>;

    // The iterator class is a friend, giving it access to our internals
    friend class detail::list_iterator<T>;
};

}

The list.cpp is empty other than a single include directive:

// list.cpp
#include "list.hpp"

The purpose of the otherwise empty list.cpp is to ensure that the list.hpp file will compile in isolation, ensuring that the header has the necessary #include directives and/or forward declarations.

Despite having two distinct language-level class templates, we have a single logical component. We say that these two class templates are “co-located.”

We also have two source files, a list.hpp and list.cpp, but they together form a single physical component.

_{Example using PIMPL}

// connection.hpp
#include <memory>

namespace acme {

// Forward-declare the private member data type
namespace detail { class connection_ipml; }

// The wrapper class
class connection {
private:
    // The actual PIMPL
    std::unique_ptr<detail::connection_impl> _impl;

public:
    connection(const std::string& address);
    ~connection();
};

}

And a corresponding connection.cpp:

#include "connection.hpp"

namespace acme::detail {

class connection_impl {
    // Whatever we need for the connection...
};

}

namespace {

std::unique_ptr<acme::detail::connection_impl>
init_private_data(const std::string& address) {
    // Initialize and return the connection data...
}

}

acme::connection::~connection() = default;
acme::connection::connection(const std::string& address)
    : _impl{ init_private_data(address) }
{}

Again we have two different (non-template) classes, and a single logical component. In the header, the implementation detail class is only forward declared, not fully defined. In the connection.cpp we provide the definition for the detail class.

Packages, Projects, Modules, and Submodules
#

The terms defined in prior sections were heavily borrowed from John Lakos’s work. His work also often uses the term package to refer to a collection of source components. This would roughly correspond to the entire source repository, including build files, support files, documentation, scripts, and data.

Unfortunately, the term “package” has been heavily overloaded. These days, “package” most often refers to a unit of distribution of software, rather than the software itself. For this reason, this document prefers the term “project”, and will use it instead of “package”, although they mean the same thing.

In a similar vein, the term module has also been heavily overloaded. In the wake of upcoming C++ module specifications and implementations, the word “module” will be avoided as to avoid ambiguity and confusion as a “module” corresponds with neither a package/project nor a physical component.

Still, a term is needed to refer to the subdivisions of a large project into smaller elements. For example, Qt is an enormous framework of many interconnected pieces. To refer to these pieces Qt uses the term “module”, which has already been excluded.

For lack of a better unqualified term, the best this author can find is a qualified term: Submodule. This term will be used to denote separate sections of a project which can be consumed on as-needed basis.

_{See the Submodules for more information.}

Project Files
#

PFL prescribes several files that should be present in the root of the project:

A README file should be present. It should be easily readable in plaintext, but may use “enhanced” plaintext like Markdown or similar. It should contain a description of the contents of the directory and subdirectories.
A LICENSE file must be present for projects that wish to redistribute themselves. It must be plaintext (ie. not enhanced with markup).

Tool-support files, such as .gitignore and .clang-format, may be present in this directory.

Other files in the root directory must be pertinent to the build system of the project. Other files should not appear in the root of the project.

Project Directories
#

PFL prescribes several directories that should appear at the root of the project tree. Not all of the directories are required, but they have an assigned purpose, and no other directory in the filesystem may assume the role of one of these directories. That is, these directories must be the ones used if their purpose is required.

Other directories should not appear at the root.

Note: If you have a need not fulfilled by a PFL directory listed below, that is a bug in this specification, and I would love to hear from you! Before reporting, double-check that what you need isn’t listed below and in the following sections.

build/

A special directory that should not be considered part of the source of the project. Used for storing ephemeral build results. Must not be checked into source control. If using source control, must be ignored using source control ignore-lists.

src/

Main compilable source location. Must be present for projects with compiled components that do not use submodules.

In the presence of include/, also contains private headers.

include/

Directory for public headers. May be present. May be omitted for projects that do not distinguish between private/public headers. May be omitted for projects that use submodules.

tests/

Directory for tests.

examples/

Directory for samples and examples.

external/

Directory for packages/projects to be used by the project, but not edited as part of the project.

extras/

Directory containing extra/optional submodules for the project.

data/

Directory containing non-source code aspects of the project. This might include graphics and markup files.

tools/

Directory containing development utilities, such as build and refactoring scripts

docs/

Directory for project documentation.

libs/

Directory for main project submodules.

Top-Level Directories
#

Pitchfork specifies several top-level directories. Other directories should not be present in the root directory, except for what is required by other tooling.

`build/`
#

This directory is not required, but its name should be reserved.

The build/ directory is special in that it must not be committed to a source control system. A user downloading the codebase should not see a build/ directory present in the project root, but one may be created in the course of working with the software. The _build/ directory is also reserved.

Note: Some build systems may commandeer the build/ directory for themselves. In this case, the directory _build/ should be used in place of build/.

The build/ directory may be used for ephemeral build results. Other uses of the directory are not permitted.

Creation of additional directories for build results in the root directory is not permitted.

Note: Although multiple root directories are not allowed, the structure and layout of the build/ directory is not prescribed. Multiple subdirectories of build/ may be used to hold multiple build results of different configuration.

`include/`
#

Note: The include/ and src/ directories are very closely related. Be sure to also read its section in addition to this one.

The purpose of the include/ directory is to hold public API headers.

The include/ directory should not be used if using Merged Header Placement.

_{See Library Source Layout.}

`src/`
#

Note: The src/ and include/ directories are very closely related. Be sure to also read its section in addition to this one.

The purpose and content of src/ depends on whether the project authors choose to follow Merged Header Placement or Separate Header Placement.

_{See Library Source Layout.}