* Fix some post-build signing issues
This fixes some post-build signing issues that are present in the aspnetcore repo
1. Add the .msi extension to be signed by Microsoft400 - Msis must be signed. With in-build signing these get handled explicitly by the wixproj infrastructure. When we do post build signing, we must sign these files.
2. Remove the strong name exclusions. These exclusions are incorrect when applied in post-build and unnecessary for in-build signing. Most importantly, the aspnetcore PKT would not end up re-strong named (it doesn't need to be strong name signed by ESRP since it's strong named in-build) because the PKT doesn't match any of the StrongNameSignInfo specified in arcade. The rest of the entries seem to be mostly about optimization. I could not find any performance difference between these entries being present and not. I am not sure whether they actually even apply to any assets. Moreover, when doing post-build signing, they would conflict with the entries in runtime and other places.
Verification - I have a tool that I wrote which unpacks every file between two directories and compares the strong name, nuget, and authenticode certs between equivalent files. This is the same tool being used to verify post-build signing. This tool shows no difference in any aspnetcore produced asset.
Baseline: https://dev.azure.com/dnceng/internal/_build/results?buildId=836183&view=results
Diff: https://dev.azure.com/dnceng/internal/_build/results?buildId=837176&view=results
* Do not push VS packages for installers when PostBuildSign == true
* Output wix command packages to the installers output path
* Don't import microbuild signing targets from wix when PostBuildSign=true
* Tweaks:
- Don't sign wixpacks when not in post-build signing
- Generate a wixpack for both the original msi name (which the wixproj generates) AND the name we use in the final outputs. This is because while these files are the same, signing differentiates the certificate based on the file name, and wixpack lookup is also based on the file names. Aspnetcore and other repos have uses the final outputs (e.g. dotnet-aspnetcore-runtime-123.5..) as well as the internal names (e.g. AspNetCoreSharedFramework_x64.msi).
- Don't sign msi's when not post-build signing.
* Avoid generating sha512 files for wixpack zips
* Don't run xplat code sign jobs if PostBuildSign == true
* Change original target names
* Conditionalize codesign operations
* Add publishing flag for linux x64 and add deb sha512 generation
* Do not push the x64 linux runtime archive more than once
* !!! Correct condition controlling `public` API analysis !!!
- #26785 demonstrated analyzers were inoperable
* Remove unused `public` API files
* Update `public` API files
- three Components files were missing `#nullable enable`
- `nullable` annotations missing or out-of-date elsewhere
* Ignore `public` API files completely during source build
- do not warn about unused files
* Update more `public` API files
- otherwise need to released previous packages; that's slower and less reliable
- left escape hatches but they're not currently used
- broke in servicing exercise because repo doesn't use its own isolated feeds
- also use latest package references for non-packable implementation projects
nits:
- copy some comment and spacing improvements from release/3.1
* Fix chrome/selenium tests (#25330)
* Revert "Disable failing/hanging tests due to Chrome/Selenium issue (#25323)"
This reverts commit 332f1504128a63431cb67830703922cdeefee525.
* Update Selenium to latest
* Update API
* Try specifying a version
* Update Selenium to 4.0.0-beta5
* Disable browser log tests
* Fix components e2e tests and disable blazor standalone template test
* Disable tests using browser log
* Disable template test
* Avoid using .NET formatted strings in tests
* Annotate BasicTestApp suggesting that it needs the all globalization data
* Culture specific formatting relies on the ICU data carried by the OS. This
causes issues in our tests if WebAssembly carries a different set than the OS. Instead
updating these tests to use hardcoded strings.
* Additionally fixing an issue where some projects in the solution were using tasks from
the .dotnet SDK rather than the local copy of the SDK. This was causing issues building locally.
Co-authored-by: Pranav K <prkrishn@hotmail.com>
- include all shipping packages in Helix payloads that need runtime
- remove hard-coded `-ci` that broke Helix tests with stable versions or local builds
- for local builds, do not assume `$(Configuration)` is Release
- support `$(HelixTargetQueues)` property used in RunHelix.ps1
- lost somewhere along the lines; script ran full matrix
nits:
- clean up redundant addition of runtime and ref/ packages
- `@(HelixContent)` additions in `_CreateHelixWorkItem` target are ignored
- mention '+' separation of `-HelixQueues` argument to RunHelix.ps1
- allow `$(IsUnitTestProject)` override in case we need it in the future
- required for servicing, where e.g. `5.0.0` may be ambiguous
- also move `%DOTNET_HOME%` into the working directory
- reduces build-up in `%HELIX_CORRELATION_PAYLOAD%` and aligns w/ build.sh
nit: add more debug output to `runtests.*`
* Correct `$(SharedFxVersion)` and `$(TargetingPackVersion)` values
- Ensure `$(SharedFxVersion)` doesn't change in `$(NoSemVer20)` projects
- Ignore current project's `$(VersionSuffix)` in `$(TargetingPackVersion)`
- Never assume `$(AspNetCoreBaselineVersion)` matches released targeting pack
- Stabilize both versions correctly
- Use these properties more widely
- Remove other mechanisms to get the same values
- Reduce use of the `_GetPackageVersionInfo` target
- Reduce use of `$(SharedFxVersion)` for the targeting pack
nits:
- Correct comments about old RTMVersions.csproj project
- Fix or remove a few other comments
* Do not pass package filenames on Helix command Lines
- remove parsing of these command-line arguments from `RuntestOptions`
- instead craft the names using passed `$(SharedFxVersion)`
- restore `$(DotNetRuntimeSourceFeedKey)` on Helix command line
- lost somewhere along the line
- correct argument count in runtests.sh
- treated 11th argument as both Helix timeout and feed credential
- count was messed up somewhere alone the line
nits:
- update C# syntax in `RuntestOptions` e.g. remove unused `public` setters
- sort and group properties and their assignments
Changes WiX toolset used to 3.14 to support ARM64
Generates targeting pack from the x86/x64 leg, as it gets produced using a zip that gets generated there.
The ARM64 leg now produces all the necessary msi's, exe, and wixlib needed for the installer to generate a bundle.
- remove hard-coded `$(AppRuntimeVersion)` property entirely
- use `$(SharedFxVersion)` instead
nits:
- make `RunTests` help more clear about how `--runtime` option is used
- remove unused `--sdk` option on `RunTests` command line
- add Microsoft.AspNetCore.App.Ref package to Helix content later, ensuring it exists
- always use `$([MSBuild]::VersionXYZ(...))` for version checks
- use `$(NETCoreAppFrameworkIdentifier)` where it's available
- move `$(KnownAppHostPackOrFrameworkReferenceTfm)` setting to Directory.Build.props
- use it to correct `@(KnownFrameworkReference)` updates
- metadata of those items still uses `netcoreapp5.0`
- see also https://github.com/dotnet/efcore/pull/22279#discussion_r478674176
nits:
- do not assume `$(TargetFrameworkVersion)` starts with a 'v'; valid w/o it
- add `$(_IsMicrosoftNETCoreApp20OrOlder)` property in OpenAPI targets file
- evaluate the `Condition` once instead of three times
- mostly duplicates #25217
- update `BaselineGenerator` to produce baselines useful in 6.0 (too)
- update Baseline.Designer.props using new generator (matching 3.1.7 release)
- always suppress references expressed only in `*.nuspec` files
- needed even in servicing builds
- restore warning and errors about removed references (new for 5.0)
- adjust exclusions to handle `@(_ProjectReferenceByAssemblyName)` removal
nit: do not generate empty `<ItemGroup />` elements
* Correct `@(SuppressBaselineReference)` items
- remove out-of-date `@(SuppressBaselineReference)` items
- either 3.1.7 baselines we're using don't include reference or still using package
- fix some comments and `Condition` attributes to make remainder easy to find
- add missing `@(SuppressBaselineReference)` items
* Use BlazorWebAssemblySDK
* Update projects to use the BlazorSDK
* Remove tasks and targets from RazorSDK
* Remove workarounds from BlazorSDK
* Fixup
* Fixup
- use SignalR.Npm.FunctionalTests.npmproj to get non-stable version
- not Microsoft.AspNetCore.DeveloperCertificates.XPlat.csproj (avoid C# and F# projects)
- add `_GetPackageVersionInfo` target to all `*.npmproj` projects
- make `_GetPackageVersionInfo` target work when `yarn` is not installed
- switch codesign-xplat.yml to use `dotnet msbuild`
- above change also fixes Code-sign jobs but they're slightly faster using `dotnet msbuild`
- #20818, fix e.g. references to Microsoft.Web.Xdt.Extensions in our packages
- make `@(Reference)` items much more broadly applicable
- emit an error when `@(ProjectReference)` is used instead of `@(Reference)`
- then get rid of the errors (!!)
- rename a couple of projects to match their assembly names
- then regenerate the `@(ProjectReferenceProvider)` items
- switch intersection approach from Exclude / Exclude to Copy / Update / Copy
Projects of particular interest:
- src/DefaultBuilder/src/Microsoft.AspNetCore.csproj
- honouring metadata left e.g. Microsoft.AspNetCore.Components.WebAssembly.DevServer package unchanged
- removed redundant metadata after that confirmation
- src/Razor/tools/Microsoft.AspNetCore.Razor.Internal.Transport/
- content of this package unchanged but metadata avoids extra work
- add a comment about the extra work
- src/SiteExtensions/LoggingAggregate/src/Microsoft.AspNetCore.AzureAppServices.SiteExtension/
- success! removes Microsoft.Web.Xdt.Extensions dependency from the package
- src/SiteExtensions/Runtime/Microsoft.AspNetCore.Runtime.SiteExtension.pkgproj
- add a `Condition` to avoid an ordering issue I hit here
- src/Tools/Extensions.ApiDescription.Server/src/
- avoid errors the new build ordering and timing caused
Separately, up the timeout in the `<DownloadFile />` task
- hit repeated timeouts downloading dotnet-runtime-5.0.0-rc.1.20370.4-win-x64.zip
nits:
- remove dupe `@(Reference)` item in Microsoft.AspNetCore.Components.WebAssembly.DevServer.csproj
- remove useless `%(ProjectReference.IsImplicitlyDefined)` metadata as well as its misspellings
- remove extra spaces from ProjectReferences.props
- clean up a few comments in ResolveReferences.targets
* !fixup! Correct other references to renamed projects
This pull request updates the following dependencies
[marker]: <> (Begin:2f3f3ff0-d34f-49bd-50aa-08d828f9655f)
## From https://github.com/dotnet/runtime
- **Subscription**: 2f3f3ff0-d34f-49bd-50aa-08d828f9655f
- **Build**: 20200722.7
- **Date Produced**: 7/23/2020 12:04 AM
- **Commit**: 3cab6dd440a5f3763bfbd2d582b36fe51095686a
- **Branch**: refs/heads/internal/release/5.0-preview8
[DependencyUpdate]: <> (Begin)
- **Updates**:
- **System.Diagnostics.DiagnosticSource**: from 5.0.0-preview.8.20361.2 to 5.0.0-preview.8.20372.7
- **System.Diagnostics.EventLog**: from 5.0.0-preview.8.20361.2 to 5.0.0-preview.8.20372.7
- **System.Drawing.Common**: from 5.0.0-preview.8.20361.2 to 5.0.0-preview.8.20372.7
- **System.IO.Pipelines**: from 5.0.0-preview.8.20361.2 to 5.0.0-preview.8.20372.7
- **System.ComponentModel.Annotations**: from 5.0.0-preview.8.20361.2 to 5.0.0-preview.8.20372.7
- **Microsoft.Extensions.Logging.Abstractions**: from 5.0.0-preview.8.20361.2 to 5.0.0-preview.8.20372.7
- **Microsoft.Extensions.Logging.Configuration**: from 5.0.0-preview.8.20361.2 to 5.0.0-preview.8.20372.7
- **Microsoft.Extensions.Logging.Console**: from 5.0.0-preview.8.20361.2 to 5.0.0-preview.8.20372.7
- **Microsoft.Extensions.Logging.Debug**: from 5.0.0-preview.8.20361.2 to 5.0.0-preview.8.20372.7
- **Microsoft.Extensions.Logging.EventLog**: from 5.0.0-preview.8.20361.2 to 5.0.0-preview.8.20372.7
- **Microsoft.Extensions.Logging.EventSource**: from 5.0.0-preview.8.20361.2 to 5.0.0-preview.8.20372.7
- **Microsoft.Extensions.Logging.TraceSource**: from 5.0.0-preview.8.20361.2 to 5.0.0-preview.8.20372.7
- **Microsoft.Extensions.Options**: from 5.0.0-preview.8.20361.2 to 5.0.0-preview.8.20372.7
- **Microsoft.Extensions.Options.ConfigurationExtensions**: from 5.0.0-preview.8.20361.2 to 5.0.0-preview.8.20372.7
- **Microsoft.Extensions.Options.DataAnnotations**: from 5.0.0-preview.8.20361.2 to 5.0.0-preview.8.20372.7
- **Microsoft.Extensions.Primitives**: from 5.0.0-preview.8.20361.2 to 5.0.0-preview.8.20372.7
- **Microsoft.Extensions.Logging**: from 5.0.0-preview.8.20361.2 to 5.0.0-preview.8.20372.7
- **Microsoft.Extensions.Internal.Transport**: from 5.0.0-preview.8.20361.2 to 5.0.0-preview.8.20372.7
- **Microsoft.Extensions.Hosting.Abstractions**: from 5.0.0-preview.8.20361.2 to 5.0.0-preview.8.20372.7
- **Microsoft.Extensions.Caching.Abstractions**: from 5.0.0-preview.8.20361.2 to 5.0.0-preview.8.20372.7
- **Microsoft.Extensions.Caching.Memory**: from 5.0.0-preview.8.20361.2 to 5.0.0-preview.8.20372.7
- **Microsoft.Extensions.Configuration**: from 5.0.0-preview.8.20361.2 to 5.0.0-preview.8.20372.7
- **Microsoft.Extensions.Configuration.Abstractions**: from 5.0.0-preview.8.20361.2 to 5.0.0-preview.8.20372.7
- **Microsoft.Extensions.Configuration.Binder**: from 5.0.0-preview.8.20361.2 to 5.0.0-preview.8.20372.7
- **Microsoft.Extensions.Configuration.CommandLine**: from 5.0.0-preview.8.203...
* Use Microsoft.CodeAnalysis.PublicApiAnalyzers
- #4259 1/2
- followup 2/3 for 5266918ed2
- includes baselines for 16 MVC projects
- will automated further additions in another PR
- suppress warnings that may cause back-compat problems if fixed
nit: sort `@(LatestPackageReference)` a bit better
* Simplify ref/ assembly generation
- followup 1/2 for 5266918ed2
- correct the Razor.Tools project
- `%(Reference.Version)` metadata does not bleed through into `@(PackageReference)` items
- much more work to do so than to add this special case
- remove `$(Razor_NewtonsoftJsonPackageVersion)`
- remove RTMVersions project and use RepoTasks instead
- make it an error if RepoTasks is not restored before anything else builds
- add items and properties for System.Security.AccessControl
nits:
- remove invalid (ignored) metadata in Directory.Build.props and AzureAppServices.SiteExtension project
- improve / extend a couple of comments
- move `@(Reference)` items together in Microsoft.AspNetCore.Razor.Tools
* Remove all ref/ projects
* Remove GenAPI infrastructure
* Remove notion of a reference assembly project
- remove `$(IsReferenceAssemblyProject)`, `$(ReferenceReferenceAssemblies)` and `$(ReferenceImplementationAssemblies)`
- remove unnecessary `$(NoWarn)` settings
nits:
- remove a few misleading comments
- wrap some long lines
* Move .0 package version workaround into Versions.props
- touch up SharedFramework.External.props
* Expose `%(LatestPackageReference.RTMVersion)` metadata
- automate use of properties in the `@(LatestPackageReference)` item group to make this maintainable
- add a couple of special cases at the bottom of eng/Dependencies.props
- add one more `$(...PackageVersion)` property to avoid yet-another special case
* Enable Roslyn reference assemblies
- exclude ref/ assembly from packages other than targeting pack
- update Microsoft.AspNetCore.App.Ref.csproj
- `%(IsReferenceAssembly)` and `%(ReferenceGrouping)` metadata no longer relevant
- only ref/ assemblies are in `@(ReferencePathWithRefAssemblies)` item group
nits:
- remove now-unnecessary workaround
- issues with TFM transition are behind us
- clean up Microsoft.AspNetCore.App.Runtime.csproj slightly
- use `GeneratePathProperty="true"`
- reorder item / property settings for meta-expansion
- correct spelling errors and phrasing in comments
* Update documentation to reflect recent changes
- remove CrossRepoBreakingChanges.md; was tied to old TeamCity infrastructure
- also much less relevant given repo merges
- adjust details and examples in ReferenceResolution.md
- reflect repo merges, Dependencies.props changes, and current Maestro++ channels
- add a few more details e.g. specific files where Version.Details.xml versions are used
* !fixup! Remove another irrelevant doc file
* !fixup! Address PR review suggestions
- convert a couple of warnings to errors
- use consistent casing for Microsoft.NETCore.App.Runtime.* packages
- reduce `%(LatestPackageReference.Version)` metadata special cases
- add and improve comments e.g.
- improve comments about `$(*V0PackageVersion)` properties
- improve placement of comments about item removal in ResolveReferences.targets
- confirmed `$(*V0PackageVersion)` property list is complete
nits:
- fix solution example in ReferenceResolution.md
- remove item group definition for `@(LatestPackageReference)`
- remove `%(LatestPackageReference.VersionName)` metadata after use; large item group
- similarly, remove `%(LatestPackageReference.RTMVersion)` when not needed; just complicates `Condition`s
When I squash, I must remember this fixes
- #14801
- dotnet/aspnetcore-internal#2693
* Actually use `%(LatestPackageReference.RTMVersion)` metadata
- gather RTM package references in a new project
- a (very) separate project to work around package conflict resolution
- empty `Test` target works around Arcade's testing approach
- new target in ResolveReferences.targets updates relevant assembly paths to use the RTM packages
- done as soon as possible after `ResolvePackageAssets` determines the paths
- done for all compilation inputs, not just ref/ assemblies
All rules are currently disabled, except for one that I enabled for
src/Http via a new ".editorconfig" file I added there.
Other changes:
* Allow editorconfigs in MVC and Razor to flow to the root
* Consolidate a few editorconfig settings
* Tweak Ruleset config in Azure/AzureAD where it clashed.
Addresses the beginning of #9620, but it's a fair chunk of work to
enable most rules through the whole repo. That can be done directory by
directory and rule by rule by dropping .editorconfig files though.
* Razor SDK build ordering issues
* Build the SDK completely regardless of the MSBuild runtime type
* Split SDK integration tests into a separate project. Clean up project file
* Add project to sln
* Update Microsoft.NET.Sdk.Razor.csproj
* Fixup tests
* Avoid rebuilding dependencies if they appear up to date. Fixup tests
* Fixup
* Update CSharp.Common.props
* Cleanup the build
* Remove extra `[SkipOnHelix]` attribute
- only need the one compiled into Microsoft.AspNetCore.Testing
- update the documentation to reflect this
- nit: address Markdown warnings that VS Code showed
* Add `SuccessfulTests` to ensure something runs in every non-Helix runs
- #22241
- cleans up hundreds of warnings but leaves a couple for the Blazor tests assembly
- see comments about xUnit runner command line in the new class
* Make `dotnet msbuild` the default on Windows too
- add step using desktop `msbuild` when native builds may be involved
- `-All` (without `-NoBuildNative`), `-BuildNative` or `-BuildInstallers` run this step
- but `-ForceCoreMsbuild` unconditionally skips this step
nits:
- add binary log for RepoTasks build if `$BinaryLog` (echoes the `dotnet msbuild` command)
- add blank lines between build steps
* Enable building managed projects depending on native assets
- splitting native builds out confuses these projects
- use `$(BuildNative)` less, only to control actual building (not bundling)
- build both native platforms in one `msbuild` invocation
* Adjust generation scripts to explicitly choose the MSBuild engine
- ensure native assets are included in GenerateReferenceAssemblies.ps1 build
- clean up the global state that tools.ps1 corrupts
* Revert move to VS2019.Pre queues
This reverts part of commit b67d161e03
- was "[release/5.0-preview5] Update dependencies from dotnet/aspnetcore-tooling (#21710)"
* Revert "!temporary! Require `msbuild` from VS2019 16.6"
- this reverts commit 58cf2304a6
* Reduce build duplication in pipelines
- build native assets and repo tasks once per CI job
- only cleanup framework references after packing managed projects
nits:
- wrap a few long lines
- remove extra `-forceCoreMsbuild` options in SiteExtensions' build.cmd
* Fix Helix jobs
- restore.cmd doesn't work well with `-projects`; script unconditionally adds `-all`
* !fixup! Reduce duplications further
- missed a couple of places `-noBuildRepoTasks` helps
* Cleanup: Remove a few dangling binary logs
* !fixup! Correct typos in generation scripts
* !fixup! Another typo in the generation scripts
Matt Mitchell
Doug Bunting
John Luo
Chris Ross
Pranav K
Juan Hoyos
Brennan
William Godbe
dotnet-maestro[bot]
Hao Kung
DotNet Bot
Kevin Pilch
Javier Calvarro Nelson