From cf7eccab5eec7d68ba3f9c201c5a109bb70036dd Mon Sep 17 00:00:00 2001 From: Nate McMaster Date: Fri, 1 Dec 2017 18:15:17 -0800 Subject: [PATCH] Add a design document for the bill of materials and build caching (#688) [ci skip] --- docs/BillOfMaterials.md | 171 ++++++++++++++++++++++++++++++++++++++++ docs/BuildCaching.md | 30 +++++++ 2 files changed, 201 insertions(+) create mode 100644 docs/BillOfMaterials.md create mode 100644 docs/BuildCaching.md diff --git a/docs/BillOfMaterials.md b/docs/BillOfMaterials.md new file mode 100644 index 0000000000..d56a05995e --- /dev/null +++ b/docs/BillOfMaterials.md @@ -0,0 +1,171 @@ +Bill of Materials +================= + +The bill of materials (bom) is a list of artifacts produced by the build. + +## Generating a new bom + +A new bill of materials is generated on each build and placed in `artifacts/bom.$(Version).xml`. + +In addition, this bill of materials can be produced without building the repo by running `build.ps1 -bom`. + +## Configuration data + +:x: The bom SHOULD NOT contain data that is for configuration. Configuration data is any kind of parameter that a consumer needs to find the artifact. + +Examples of ephemeral data: + + - File paths + - Download links + - Passwords + +## Format + +The bill of materials is an XML file. Here is a minimal example of a bom. + +```xml + + + + + + + + + + + + + + + + +``` + +### Elements + +#### `` + +Describes the build output. The root element of the document. + +Optional attributes: + - `Id` - a unique identifier describing this build. + - `Created` - a timestamp of when the build was created. + +#### `` + +Describes a group of artifacts + +Optional attributes: + - `Category` - an arbitrary category name. + +#### `` + +Describes a file produced by the build. + +Required attributes: + - `Id` - the file name of the artifact. Id MUST BE unique per each bill of materials. + - `Type` - an arbitrary string describing the semantic type of the artifact. + +Optional attributes: + - Others - any other attributes on the element are considered metadata values about the artifact. + +#### `` + +Describes artifact dependencies. + +#### `` + +Describes a dependency between two artifacts. + +Requried attributes + - `Source` - the artifact that has a dependency. This artifact MUST BE defined by an `` in the same `` + - `Target` - the thing it depends on. This artifact MAY OR MAY NOT be described in this ``. + It might have been produced in a previous build or be produced by a partner team. + +## Artifact types + +These are some common artifact types produced by the build. + + - NuGetPackage + - NuGetSymbolsPackage + - DebianPackage + - RedhatPackage + - ZipArchive + - TarGzArchive + - TextFile + - LzmaArchive + - TestBundle + +## Artifact categories + +Tools may use the category information to define how they handle different files. + +These are common categories for artifacts used in aspnet/Universe. + + - ship - used for files that will be published to NuGet.org, microsoft.com, or other official publication locations + - shipoob - used for files that will be distributed in other mechanism. + - noship - these files should not be published to partner teams or released publically. This may include test artifacts. + +## A more complete example + +```xml + + + + + + + + + + + + + + + + + + + + + + + + +``` + +### Example usage: signing + +In the example above, some of the `` items were marked `ShouldBeSigned="true"`. Our signing tool could use this as a way to +determine which files should be passed on to signing. + +### Example usage: metadata + +In the example above, some of the artifacts could contain data like FileHash, CommitHash, RepositoryUrl, BranchName, and others. +It is up to the consumer of the bom to define how to interpret and use this. + diff --git a/docs/BuildCaching.md b/docs/BuildCaching.md new file mode 100644 index 0000000000..6c746807b8 --- /dev/null +++ b/docs/BuildCaching.md @@ -0,0 +1,30 @@ +Build Caching +============= + +This repository can build the entire ASP.NET Core stack from scratch. However, in most cases, we only want to build part of the stack. + +## Building with cached artifacts + +This is the default behavior when `build.ps1` without arguments. + +This repository may contains files under `releases/*.xml`. These files follow the [Bill of Materials](./BillOfMaterials.md) format. +Any artifacts described in these bom's are assumed to have been built in a previous release and are available for download. +The build may use this information to skip building certain assets when a build action would produce a duplicate asset. + +## Disabling caching + +You can for the build to skip reading the bill of materials and build everything from scratch by calling `build.ps1 -NoCache`. +This will cause the build to always rebuild assets. + +## When to commit a new bill of material + +Each build will produce a new Bill of Materials (bom). Normally, this bom isn't added to source code right away. +This bom should only be committed to source once the assets it describes are published and available from caching mechanisms. + +## Caching mechanisms + +These are some common caching mechanisms used to store build artifacts. + + - NuGet servers. NuGet.org, MyGet.org, and file shares can be used to restore previously-built nupkg files + - Debian and RPM feeds. Some .deb and .rpm installers may be cached on https://packages.microsoft.com/repos. + - Azure blob feeds. Arbitrary build artifacts may be cached on https://aspnetcore.blob.core.windows.net/.