* Fix some post-build signing issues
This fixes some post-build signing issues that are present in the aspnetcore repo
1. Add the .msi extension to be signed by Microsoft400 - Msis must be signed. With in-build signing these get handled explicitly by the wixproj infrastructure. When we do post build signing, we must sign these files.
2. Remove the strong name exclusions. These exclusions are incorrect when applied in post-build and unnecessary for in-build signing. Most importantly, the aspnetcore PKT would not end up re-strong named (it doesn't need to be strong name signed by ESRP since it's strong named in-build) because the PKT doesn't match any of the StrongNameSignInfo specified in arcade. The rest of the entries seem to be mostly about optimization. I could not find any performance difference between these entries being present and not. I am not sure whether they actually even apply to any assets. Moreover, when doing post-build signing, they would conflict with the entries in runtime and other places.
Verification - I have a tool that I wrote which unpacks every file between two directories and compares the strong name, nuget, and authenticode certs between equivalent files. This is the same tool being used to verify post-build signing. This tool shows no difference in any aspnetcore produced asset.
Baseline: https://dev.azure.com/dnceng/internal/_build/results?buildId=836183&view=results
Diff: https://dev.azure.com/dnceng/internal/_build/results?buildId=837176&view=results
* Do not push VS packages for installers when PostBuildSign == true
* Output wix command packages to the installers output path
* Don't import microbuild signing targets from wix when PostBuildSign=true
* Tweaks:
- Don't sign wixpacks when not in post-build signing
- Generate a wixpack for both the original msi name (which the wixproj generates) AND the name we use in the final outputs. This is because while these files are the same, signing differentiates the certificate based on the file name, and wixpack lookup is also based on the file names. Aspnetcore and other repos have uses the final outputs (e.g. dotnet-aspnetcore-runtime-123.5..) as well as the internal names (e.g. AspNetCoreSharedFramework_x64.msi).
- Don't sign msi's when not post-build signing.
* Avoid generating sha512 files for wixpack zips
* Don't run xplat code sign jobs if PostBuildSign == true
* Change original target names
* Conditionalize codesign operations
* Add publishing flag for linux x64 and add deb sha512 generation
* Do not push the x64 linux runtime archive more than once
This produces a bunch of warnings and causes the quarantine build to fail. Passing NoBuildNative keeps with the theme of -NoBuild also being passed to this script
Changes WiX toolset used to 3.14 to support ARM64
Generates targeting pack from the x86/x64 leg, as it gets produced using a zip that gets generated there.
The ARM64 leg now produces all the necessary msi's, exe, and wixlib needed for the installer to generate a bundle.
* Eliminate duplicates in Helix `TestRunner`
- do not add source that's already in NuGet.config
- configuration file now part of Helix content
- do not re-set an env variable that's set in `runtests.*`
nit: simplify `%Path% setting in runtests.cmd
- always quote the value
- but, keep the `SETLOCAL` command
- don't pollute environment on Helix agents
* Remove `--source` options we don't need anymore from `runtests.*`
- nit: add `--no-restore` to `dotnet run` command after `dotnet restore`
* Add `--arch arm64` to ARM64 `restore` build steps
- #25397
- was restoring for the wrong architecture
nit: add `--no-restore` to Helix project build steps
- already restored
- pin Microsoft.Net.Compilers.Toolset version to isolate us from Arcade
- the version now matches dotnet/runtime
- may move the pin to later version later in RC2 if needed
- double the macOS job max. length in our normal and quarantined PR runs
- have been seeing them timeout or come very close too often
* Grab binary logs for main Windows and Linux jobs
- in the Windows case, do not do this in official builds (logging slows build down)
nit: do not set variables with only two values three times
* Do not sign twice in Windows Code-sign build step
nit: correct wording in Signing.props
* !fixup! Don't grab the large x86 binary log
- use SignalR.Npm.FunctionalTests.npmproj to get non-stable version
- not Microsoft.AspNetCore.DeveloperCertificates.XPlat.csproj (avoid C# and F# projects)
- add `_GetPackageVersionInfo` target to all `*.npmproj` projects
- make `_GetPackageVersionInfo` target work when `yarn` is not installed
- switch codesign-xplat.yml to use `dotnet msbuild`
- above change also fixes Code-sign jobs but they're slightly faster using `dotnet msbuild`
This pull request updates the following dependencies
[marker]: <> (Begin:2f3f3ff0-d34f-49bd-50aa-08d828f9655f)
## From https://github.com/dotnet/runtime
- **Subscription**: 2f3f3ff0-d34f-49bd-50aa-08d828f9655f
- **Build**: 20200722.7
- **Date Produced**: 7/23/2020 12:04 AM
- **Commit**: 3cab6dd440a5f3763bfbd2d582b36fe51095686a
- **Branch**: refs/heads/internal/release/5.0-preview8
[DependencyUpdate]: <> (Begin)
- **Updates**:
- **System.Diagnostics.DiagnosticSource**: from 5.0.0-preview.8.20361.2 to 5.0.0-preview.8.20372.7
- **System.Diagnostics.EventLog**: from 5.0.0-preview.8.20361.2 to 5.0.0-preview.8.20372.7
- **System.Drawing.Common**: from 5.0.0-preview.8.20361.2 to 5.0.0-preview.8.20372.7
- **System.IO.Pipelines**: from 5.0.0-preview.8.20361.2 to 5.0.0-preview.8.20372.7
- **System.ComponentModel.Annotations**: from 5.0.0-preview.8.20361.2 to 5.0.0-preview.8.20372.7
- **Microsoft.Extensions.Logging.Abstractions**: from 5.0.0-preview.8.20361.2 to 5.0.0-preview.8.20372.7
- **Microsoft.Extensions.Logging.Configuration**: from 5.0.0-preview.8.20361.2 to 5.0.0-preview.8.20372.7
- **Microsoft.Extensions.Logging.Console**: from 5.0.0-preview.8.20361.2 to 5.0.0-preview.8.20372.7
- **Microsoft.Extensions.Logging.Debug**: from 5.0.0-preview.8.20361.2 to 5.0.0-preview.8.20372.7
- **Microsoft.Extensions.Logging.EventLog**: from 5.0.0-preview.8.20361.2 to 5.0.0-preview.8.20372.7
- **Microsoft.Extensions.Logging.EventSource**: from 5.0.0-preview.8.20361.2 to 5.0.0-preview.8.20372.7
- **Microsoft.Extensions.Logging.TraceSource**: from 5.0.0-preview.8.20361.2 to 5.0.0-preview.8.20372.7
- **Microsoft.Extensions.Options**: from 5.0.0-preview.8.20361.2 to 5.0.0-preview.8.20372.7
- **Microsoft.Extensions.Options.ConfigurationExtensions**: from 5.0.0-preview.8.20361.2 to 5.0.0-preview.8.20372.7
- **Microsoft.Extensions.Options.DataAnnotations**: from 5.0.0-preview.8.20361.2 to 5.0.0-preview.8.20372.7
- **Microsoft.Extensions.Primitives**: from 5.0.0-preview.8.20361.2 to 5.0.0-preview.8.20372.7
- **Microsoft.Extensions.Logging**: from 5.0.0-preview.8.20361.2 to 5.0.0-preview.8.20372.7
- **Microsoft.Extensions.Internal.Transport**: from 5.0.0-preview.8.20361.2 to 5.0.0-preview.8.20372.7
- **Microsoft.Extensions.Hosting.Abstractions**: from 5.0.0-preview.8.20361.2 to 5.0.0-preview.8.20372.7
- **Microsoft.Extensions.Caching.Abstractions**: from 5.0.0-preview.8.20361.2 to 5.0.0-preview.8.20372.7
- **Microsoft.Extensions.Caching.Memory**: from 5.0.0-preview.8.20361.2 to 5.0.0-preview.8.20372.7
- **Microsoft.Extensions.Configuration**: from 5.0.0-preview.8.20361.2 to 5.0.0-preview.8.20372.7
- **Microsoft.Extensions.Configuration.Abstractions**: from 5.0.0-preview.8.20361.2 to 5.0.0-preview.8.20372.7
- **Microsoft.Extensions.Configuration.Binder**: from 5.0.0-preview.8.20361.2 to 5.0.0-preview.8.20372.7
- **Microsoft.Extensions.Configuration.CommandLine**: from 5.0.0-preview.8.203...
- was still running in every job where explicit CD build step did not execute
- caused consistent errors in Linux MUSL x64 (Alpine) jobs
- error was ignored but made build failure diagnosis more difficult
- totally redundant in test jobs
* Remove tests from official builds
- #22787
nit: _add_ dependency on Windows ARM64 build when publishing to the BAR
- not a major problem because this particular build rarely if ever fails
- the existence of the correct manifests is much more important
* Address nit: `Windows_64_build` -> `Windows_arm64_build`
- #22243
- ignore a number of artifacts/ folders that cause task warnings but don't add value
- separately, correct overuse of `$(BuildDirectory)` variable
* Build time changes
A few changes for build time
- Don't build tests with SkipTestBuild=true and use that for official
build legs. This cuts 40%-50% off the msbuild invocations for build.
The longest build leg drops by about 30 mins.
- Skip logging of some task parameters and their metadata.
This reduces overall binlog size, which is a major contributor to
build time.
Unfortunately, this does not mean we can yet turn binlogs back on. This
change can actually increase the overall binlog size due to logging of
more project started arguments. There is another optimization for this
in progress.
Co-authored-by: Doug Bunting <6431421+dougbu@users.noreply.github.com>
- should not use `-all` and `-projects` without `-noBuildNative`
- builds fail otherwise because build.ps1 attempts to build the specified projects in desktop `msbuild`
* Make `dotnet msbuild` the default on Windows too
- add step using desktop `msbuild` when native builds may be involved
- `-All` (without `-NoBuildNative`), `-BuildNative` or `-BuildInstallers` run this step
- but `-ForceCoreMsbuild` unconditionally skips this step
nits:
- add binary log for RepoTasks build if `$BinaryLog` (echoes the `dotnet msbuild` command)
- add blank lines between build steps
* Enable building managed projects depending on native assets
- splitting native builds out confuses these projects
- use `$(BuildNative)` less, only to control actual building (not bundling)
- build both native platforms in one `msbuild` invocation
* Adjust generation scripts to explicitly choose the MSBuild engine
- ensure native assets are included in GenerateReferenceAssemblies.ps1 build
- clean up the global state that tools.ps1 corrupts
* Revert move to VS2019.Pre queues
This reverts part of commit b67d161e03
- was "[release/5.0-preview5] Update dependencies from dotnet/aspnetcore-tooling (#21710)"
* Revert "!temporary! Require `msbuild` from VS2019 16.6"
- this reverts commit 58cf2304a6
* Reduce build duplication in pipelines
- build native assets and repo tasks once per CI job
- only cleanup framework references after packing managed projects
nits:
- wrap a few long lines
- remove extra `-forceCoreMsbuild` options in SiteExtensions' build.cmd
* Fix Helix jobs
- restore.cmd doesn't work well with `-projects`; script unconditionally adds `-all`
* !fixup! Reduce duplications further
- missed a couple of places `-noBuildRepoTasks` helps
* Cleanup: Remove a few dangling binary logs
* !fixup! Correct typos in generation scripts
* !fixup! Another typo in the generation scripts
Matt Mitchell
Pranav K
Epsitha Ananth
JC Aguilera
Juan Hoyos
Doug Bunting
William Godbe
msftbot[bot]
dotnet-bot
DotNet Bot
Brennan
Hao Kung
John Luo
Safia Abdalla
Justin Kotalik