* [release/5.0] Clean out Ubuntu 16.04 testing
- cherry-pick of #32894
* Clean out Ubuntu 16.04 testing
- use Ubuntu 18.04 agents for testing in Docker containers on Helix
- also move to supported version of Fedora
* Support added `msbuild` arguments in RunHelix.ps1
- bring in part of 0551cef40f5f (#32554)
- no reason to maintain major differences from main version of script
- slight difference is we still test on Windows.10.Amd64.Open by default here
nits:
- clean up a few old mentions of Centos and Fedora testing
- fix `%(Platform)` metadata for OSX `@(HelixAvailableTargetQueue)` items
- pick up a small bit of the master -> main renaming improvements
- use `[switch]` instead of `[bool]` in RunHelix.ps1 parameters
- was still running in every job where explicit CD build step did not execute
- caused consistent errors in Linux MUSL x64 (Alpine) jobs
- error was ignored but made build failure diagnosis more difficult
- totally redundant in test jobs
- #22243
- ignore a number of artifacts/ folders that cause task warnings but don't add value
- separately, correct overuse of `$(BuildDirectory)` variable
* Make `dotnet msbuild` the default on Windows too
- add step using desktop `msbuild` when native builds may be involved
- `-All` (without `-NoBuildNative`), `-BuildNative` or `-BuildInstallers` run this step
- but `-ForceCoreMsbuild` unconditionally skips this step
nits:
- add binary log for RepoTasks build if `$BinaryLog` (echoes the `dotnet msbuild` command)
- add blank lines between build steps
* Enable building managed projects depending on native assets
- splitting native builds out confuses these projects
- use `$(BuildNative)` less, only to control actual building (not bundling)
- build both native platforms in one `msbuild` invocation
* Adjust generation scripts to explicitly choose the MSBuild engine
- ensure native assets are included in GenerateReferenceAssemblies.ps1 build
- clean up the global state that tools.ps1 corrupts
* Revert move to VS2019.Pre queues
This reverts part of commit b67d161e03
- was "[release/5.0-preview5] Update dependencies from dotnet/aspnetcore-tooling (#21710)"
* Revert "!temporary! Require `msbuild` from VS2019 16.6"
- this reverts commit 58cf2304a6
* Reduce build duplication in pipelines
- build native assets and repo tasks once per CI job
- only cleanup framework references after packing managed projects
nits:
- wrap a few long lines
- remove extra `-forceCoreMsbuild` options in SiteExtensions' build.cmd
* Fix Helix jobs
- restore.cmd doesn't work well with `-projects`; script unconditionally adds `-all`
* !fixup! Reduce duplications further
- missed a couple of places `-noBuildRepoTasks` helps
* Cleanup: Remove a few dangling binary logs
* !fixup! Correct typos in generation scripts
* !fixup! Another typo in the generation scripts
* Avoid changing global state in CI runs II
- move `dotnet` tools into `$PWD/.dotnet/tools/`
- add `dotnet` tools to `$env:PATH`
- install `jq` in `$PWD/.tools/`
- install `nginx` in `$PWD/.tools/nginx/`
nits:
- remove an extra (incorrect) `$env:JAVA_HOME` setting
- build.ps1 finds a local JDK and sets this variable (and `$env:PATH`) properly
- avoid `'$(SELENIUMPROCESSTRACKINGFOLDER)' == '\artifacts\tmp\selenium\'`
- `$(BuildDirectory)` was normally the empty string (w/o fix)
- use `$(BuildDirectory)` a bit more
* Add files. disable artifact / test results publications to avoid warnings on CI
- artifacts/log is often empty when binary logs are disabled
- no tests create *.trx files
- don't have Java test results either
- only test jobs need test results published
* Update dependencies from https://github.com/dotnet/arcade build 20200511.9
- Microsoft.DotNet.Arcade.Sdk: 5.0.0-beta.20228.4 => 5.0.0-beta.20261.9
- Microsoft.DotNet.GenAPI: 5.0.0-beta.20228.4 => 5.0.0-beta.20261.9
- Microsoft.DotNet.Helix.Sdk: 5.0.0-beta.20228.4 => 5.0.0-beta.20261.9
* Update dependencies from https://github.com/dotnet/aspnetcore-tooling build 20200514.6
Microsoft.AspNetCore.Mvc.Razor.Extensions , Microsoft.AspNetCore.Razor.Language , Microsoft.CodeAnalysis.Razor , Microsoft.NET.Sdk.Razor
From Version 5.0.0-preview.5.20261.4 -> To Version 5.0.0-preview.6.20264.6
* Pre-emptively take -nobl change
* Disable binlogs in CI
* Fix build.sh to know about -nobl
* Align build.ps1|sh with latest Arcade parameters
- do not enable binary logs by default in CI builds
- leave `-binaryLog` and `-excludeCIBinaryLog` handling to eng/common/tools.ps1|sh
- was unnecessary since `-bl /bl:{some name}` worked fine, ignoring OOMs
nit: document `-excludeCIBinarylog` a bit more
* Do not pass unknown options into CodeCheck.ps1
* Pass `-ci -nobl` into remaining CI build jobs
* Switch default TFM to `net5.0`
* Update missing project templates tfms
* Add more `-ci -nobl`
- needed because _all_ builds in the pipeline are implicitly CI builds
- default-build.yml adds `-ci` when script wasn't explicit
* Default templates to net5.0
* PR feedback
* Update TFMs in explicit .nuspec files
* Update TFMs in test projects
* Update TFMs in test C# code
* Update TFMs in infrastructure files
* Future-proof a check for `net5.0` or later
- avoid comparisons involving `$(TargetFramework)` in .targets files
- fine to compare it with `''` or `$(DefaultNetCoreTargetFramework)`
* !fixup! Undo a couple of earlier fixes
- remove a duplicate `$()` setting
- correct the one remaining versioned `#if` define
- did not make it `#if NETCOREAPP` because benchmarks test numerous .NET Core TFMs
* Disable binary logs in CodeCheck.ps1
* Specify `-ci -nobl` just once when using `parameters.buildArgs`
* Restore `$binaryLog` default logic
Co-authored-by: Doug Bunting <6431421+dougbu@users.noreply.github.com>
Co-authored-by: Will Godbe <wigodbe@microsoft.com>
Co-authored-by: Viktor Hofer <viktor.hofer@microsoft.com>
- set `$env:DOTNET_CLI_HOME` because we need to install global tools in this repo
- without this, we see `dotnet-serve` installation failures on unclean machines
* Quick fix: Simplify devBuilds.yml
- remove an excess build step
* Quick fix: Move `SetupNugetSources` script invocations above `parameters.beforeBuild`
- ensure NuGet.config is ready for all internal builds
- remove now-duplicate `SetupNugetSources` invocations wherever default-build.yml is used
* Quick fix: Clean up SiteExtensions/build.cmd
- quote all rooted paths
- check `%ERRORLEVEL%` after every `CALL`
- nits:
- add a few more `ECHO` commands
- wrap long lines
* Quick fix: Ensure `$(BuildNative)` is always set correctly
- fix problems using `-all` or `/p:BuildAllProjects=true` without `-buildNative`
- ensure `$(BuildNative)` is `false` where it's not supported
- move some duplicated settings into eng/Common.props and `<Import />` the new file
- remove now-duplicated parts of conditions using `$(BuildNative)`
* Quick fix: Consistently use `--build-*`
- avoid `/p:Build*` on the command line (except with eng/scripts/ci-source-build.sh)
- nits:
- remove now-useless `-buildNative` with `-all`
- expand and correct a couple of related comments and messages
* Quick fix: Support `-all` together with `-projects`
- remove need to specify `/p:BuildAllProjects=true`
- nit: simplify some Boolean logic
* Print disk usage
* Try again
* Disk util part3
* dont look at /tmp
* Remove max-depth
* Add disk usage stats to all non-windows platforms
* Disable Ubuntu quarantined test step
* Apply suggestions from code review
- Make thingies worky on macOS
* Update .azure/pipelines/jobs/default-build.yml
* Undo skip of qurantined test run
* Update .azure/pipelines/jobs/default-build.yml
- do not reference directory that doesn't exist
* Update .azure/pipelines/jobs/default-build.yml
- reduce noise in Mac disk utilization output
Co-authored-by: Doug Bunting <6431421+dougbu@users.noreply.github.com>
- Use different Ubuntu pools for Ubuntu tests to avoid disk space issues.
- Updated the `default-build.yml` to have a new parameter specific to Ubuntu listed `useHostedUbuntu`. If we feel the need to expand the parameters usage outside of the Ubuntu configuration we can always rename to `useHosted`. Also didn't want to touch the `isTestingJob` because I wasn't sure of hte implications.
- Updated the Ubuntu test job to turn off hosted pools.
Fixes https://github.com/dotnet/aspnetcore-internal/issues/3574
- Hmm, are `queue` names case-sensitive?
- Correct `useHostedUbuntu: false` placement
- Correct placement of `useHostedUbuntu: true` default
- Try adding `--without-http_rewrite_module` to nginx configuration
- also add `set -euo pipefail` to the script for fast failure
Co-authored-by: Doug Bunting <6431421+dougbu@users.noreply.github.com>
- work around dotnet/core-eng#7970
- install Git in repo if Tar can't be found in usual locations
- use found or installed Tar in Microsoft.AspNetCore.App.Ref project
- copy into repo from wherever it's found
- add lots of `Write-Host` debugging
nit: clean up / comment on VS Code warnings about build.ps1
* Downloads and installs ProcDump as part of the build.
* Registers a scheduled job that wakes up in 160 minutes.
* Upon waking up, the background job lists all the processes it
received as candidates.
* For each process it captures a full memory dump.
* At the end of the build, a separate step checks on whether the job
ran (the build hanged) or everything is correct and displays statistics.
* If dumps are found, they are collected and made available as artifacts
under Windows(_Templates)_Tests_Logs.
- #11924
- change ci.yml to use stages and post-build.yml
- add use of publish-build-assets.yml and post-build.yml
- create manifests and push to artifacts in last build step of each job
- pass more MSBuild properties into those builds
- use Arcade to publish installers
- use distinct `$(AssetManifestFileName)` values per job
- set global property to override what's hard-coded in Publish.proj
- change codesign-xplat.yml to use empty.proj and normal Arcade signing and publication process
- remove XPlatPackageSigner.proj
- change default-build.yml to use job.yml
- remove unused parameters e.g. `matrix`, `poolName`, `variables`
- use `enableMicrobuild` and `enablePublishTestResults` to eliminate duplicate build steps
- add .dll's and .exe's as files to sign w/ Microsoft400
- add signcheck exclusions
- remove custom manifest generation i.e. the `GenerateBuildAssetManifest` target and related artifacts
- update docker infrastructure to use same paths in and out of the container
- avoids problems adding to artifacts from within the builds
- correct typo in build.sh
- use `$env:DOTNET_INSTALL_DIR` in `DotNetCommands`
- relax expectations that an arch-specific folder exists under (say) `$env:DOTNET_HOME`
- avoids need to define `$env:DOTNET_HOME` in all jobs on CI
- update dependencies from dotnet/arcade build '20190908.2'
- upgrade to eg. Arcade SDK '1.0.0-beta.19458.2' package version
- pick up dotnet/arcade@dd593acc8b fix
- enable use of `%(PublishFlatContainer)` metadata and correct signing validation issues
- use `$(DotNetFinalVersionKind)` in preparation for servicing builds
- set `$(IsStableBuild)` for use in Arcade infrastructure
- disable signing validation for now (see #13864)
nits:
- upload logs in first artifact
- remove attempts to package non-existent VSIX
- follow-up to 29cf7ecb80
- respect verbosity setting in build.sh
- add more information to Artifacts.md
- enable test signing in internal PRs
This is a setting AZDO supports to control how long you have after a
build is canceled before all of the work is terminated.
When a build reaches a timeout, usually the `build.{cmd|sh}` step will
be terminated, and then you have cancelTimeoutInMinutes number of
minutes to do everything else before everything shuts down.
We need to a value larger than the default of 5 minutes because it takes
us *about* 5 minutes to upload all of our logs and artifacts in a
success case.
This should give us a sufficient grace period to upload all of our logs
in the case that a build fails due to a test hang.
* Replace usages of $(CI) with $(ContinuousIntegrationBuild)
* Move RepoTasks to eng/tools/
* Remove dead code
* Update build.sh/ps1 to invoke Arcade instead of KoreBuild
* build/repo.props => eng/Build.props
* build/CodeSign.{props,targets} => eng/Signing.props
* build/repo.targets => eng/Build.props, eng/CodeGen.proj, and eng/AfterSolutionBuild.targets
* Remove AddAllProjectRefsToSolution.ps1
* Put selenium process tracking into artifacts/tmp/
* build/docker => eng/docker
* Move content from korebuild.json to global.json
* Remove cibuild.{sh,cmd} and update ci.yml
* Ensure vswhere detects BuildTools installations
* Remove sign check exclusions (Arcade does not support)
* list VS required components in global.json
* Update CodeCheck.ps1
* Update arcade (#11143)
* Fixups to broken build
* capture test results in xunit form
* attempt to fix code check
* restore before linux build
* remove duplicate signinfos
* More build fixes
* exclude node_modules from unique project check
* fixup signing props
* More build fixes
* Remove unused NoWarns
* Skip building site extension
* Suppress xunit color in console output
* Install x86 runtime
* Run KillProcesses.ps1 at end of build on CI
* activate env vars in codecheck.ps1
* Fix signing and generate build manifests
* Fixup code check and linux installers build
* Remove custom restore targets
* Import flaky test props
* Restore original manifest generation code and set warnAsErrors=false
* Alwasy restore the toolset
* More attempts at build fixes
* run restore before building deb/rpm packages
* pre-install Yarn.MSBuild
* Remove WIX workaround
* Attempt to fix RPM installers and x86 build
* Fix helix tests, java auto-detection, and code gen steps
* Globally disable MSBuild node resuse and capture MSbuild crash logs
* Cleanup build invocation for installers
* Set max time on xunit tests to 15 min
* Update helix-test.yml
* Fix Hosting tests
* Fix TestHost
* Fix Negotiate.Test
* Fix Mvc tests
* Capture results
* Fix Hosting tests
* Fix Kestrel tests
* Fix ServerComparison tests
* Fix DefaultBuilder functional tests
* Skip template tests
Need to run pack before running tests on mac/linux now
* Fix Identity test
* Fix ServerComparison tests
* Verify nginx installation on ubuntu
* Verify nginx installation?
* Try adding to path
* Try symlink
* Sudo?
* Woot it works
* Disable timeout
Template tests require more than 15 mins
* list env vars
* Use correct dotnet_home
* Prepent path instead
* Fix npmproj tests
* Fix CORS tests
* Debug signalr tests
* Debug signalr
* Capture bin
* Try building templates sequentially
* Debug signalr
* Cleanup
* Disable signalr npm tests
* Debug template test
* Try to fix templates again and make steps more reliable
* Cleanup
* Feedback
- aspnet/AspNetCore-Internal#2231
- use internal pools for all internal builds
- do not sign in builds for internal pull requests
nits:
- VSTS -> Azure DevOps
- restore higher `maxParallel` values for E2E tests on Linux and Windows
Doug Bunting
William Godbe
Pranav K
John Luo
dotnet-maestro[bot]
N. Taylor Mullen
Brennan Conroy
Stephen Halter
Kevin Pilch
DotNet Bot
Javier Calvarro Nelson
Ryan Nowak
Nate McMaster
Doug Bunting
Doug Bunting
Mikael Mengistu