Commit Graph

66 Commits

Author SHA1 Message Date
wtgodbe f20a4143de Update to stable SDK in source build 2019-09-25 15:55:47 -07:00
Doug Bunting 06ca2c4ee3 Re-enable ProjectTemplates tests
- should be mandatory for changes in this branch
2019-09-23 11:44:23 -07:00
Doug Bunting aee5e40803 Stabilize package versions (#14003)
* Mark all blobs as shipping
- available (though not discoverable) in public dotnetcli feed

* Stabilize package versions

* Remove assumption that Microsoft.AspNetCore.AzureAppServices.SiteExtension packages have same version
- Microsoft.AspNetCore.AzureAppServices.SiteExtension.3.0 ships
- Microsoft.AspNetCore.AzureAppServices.SiteExtension.3.0.x?? do not ship

* Make installer versions consistent
- VS.Redist.Common.AspNetCore.SharedFramework and ...TargetingPack packages are non-shipping
- everything else ships

nit: remove extra whitespace in .nuspec files for the packages

* Correct assumptions in framework unit tests
- tests sometimes do not calculate version properties as product projects do
- Microsoft.AspNetCore.App.Ref and ...Runtime packages may rev versions separately

* Fix last 2 `SharedFxTests` failures

* Correct Microsoft.AspNetCore.App* versions used in ProjectTemplates tests
- `$(SharedFxVersion)` is not useful in test projects due to stable versioning

* Add continue on error for test templates
2019-09-15 13:34:08 -07:00
Doug Bunting 04705ee4f1
Use stages pipeline (#13040)
- #11924
- change ci.yml to use stages and post-build.yml
  - add use of publish-build-assets.yml and post-build.yml
  - create manifests and push to artifacts in last build step of each job
    - pass more MSBuild properties into those builds
    - use Arcade to publish installers
  - use distinct `$(AssetManifestFileName)` values per job
    - set global property to override what's hard-coded in Publish.proj
- change codesign-xplat.yml to use empty.proj and normal Arcade signing and publication process
  - remove XPlatPackageSigner.proj
- change default-build.yml to use job.yml
  - remove unused parameters e.g. `matrix`, `poolName`, `variables`
  - use `enableMicrobuild` and `enablePublishTestResults` to eliminate duplicate build steps
- add .dll's and .exe's as files to sign w/ Microsoft400
  - add signcheck exclusions
- remove custom manifest generation i.e. the `GenerateBuildAssetManifest` target and related artifacts
- update docker infrastructure to use same paths in and out of the container
  - avoids problems adding to artifacts from within the builds
- correct typo in build.sh
- use `$env:DOTNET_INSTALL_DIR` in `DotNetCommands`
  - relax expectations that an arch-specific folder exists under (say) `$env:DOTNET_HOME`
  - avoids need to define `$env:DOTNET_HOME` in all jobs on CI
- update dependencies from dotnet/arcade build '20190908.2'
  - upgrade to eg. Arcade SDK '1.0.0-beta.19458.2' package version
  - pick up dotnet/arcade@dd593acc8b fix
    - enable use of `%(PublishFlatContainer)` metadata and correct signing validation issues
- use `$(DotNetFinalVersionKind)` in preparation for servicing builds
  - set `$(IsStableBuild)` for use in Arcade infrastructure
- disable signing validation for now (see #13864)

nits:
- upload logs in first artifact
- remove attempts to package non-existent VSIX
  - follow-up to 29cf7ecb80
- respect verbosity setting in build.sh
- add more information to Artifacts.md
- enable test signing in internal PRs
2019-09-10 20:43:24 -07:00
John Luo a37a2b03f6
Pin SDK to preview 8 in source build (#13707)
- There are issues with NU5048 and NU5128. We need to handle ProjectIconUrl deprecation and issues with NuspecProperty not being set separately.
2019-09-05 17:43:46 -07:00
John Luo 43350b57b9
Revert "Revert "Improve components infrastructure (#12145)" (#12679)" (#12744)
* Revert "Revert "Improve components infrastructure (#12145)" (#12679)"

This reverts commit e2d57e2806. The improvement to components infrastructure is now reinstated with the following changes:

* Check in release JS artifacts and use them as a fallback when it's not possible to build npmproj.
* Dont' build nodejs in source build.
2019-08-05 15:17:55 -07:00
John Luo e2d57e2806
Revert "Improve components infrastructure (#12145)" (#12679)
* Revert "Improve components infrastructure (#12145)"

This reverts commit e149f9c0aa.

* Update JS file
2019-07-29 09:03:57 -07:00
John Luo e149f9c0aa
Improve components infrastructure (#12145)
* Remove generated Web.JS javascript files files

* Build NodeJS project referenced by managed projects by default

* Build Web.JS before other CI jobs

- We can't build Web.JS on Linux Musl for example

* Make BuildAll not override explicit BuildJava/NodeJS/Native/Managed settings

* Capture CodeCheck logs for debugging

* Skip NodeJS projects when generation ref assemblies

* Do not build dependent projects if NodeJS is not built

* Make build fail immediately if BuildNodeJS is true but NodeJS is not installed

* Do not build nodejs in installer builds
2019-07-26 13:31:01 -07:00
John Luo e31813b9cd
Add support for source-build (#11717) 2019-07-08 11:02:30 -07:00
John Luo 94f84c4776
Build SiteExtensions (#11409) 2019-06-24 19:37:51 -07:00
John Luo 0058a58372
Fix Template test runs on macOS and Linux (#11407) 2019-06-19 23:03:39 -07:00
Nate McMaster 4fde84a7ad Replace KoreBuild with Arcade (#11122)
* Replace usages of $(CI) with $(ContinuousIntegrationBuild)

* Move RepoTasks to eng/tools/

* Remove dead code

* Update build.sh/ps1 to invoke Arcade instead of KoreBuild

* build/repo.props => eng/Build.props

* build/CodeSign.{props,targets} => eng/Signing.props

* build/repo.targets => eng/Build.props, eng/CodeGen.proj, and eng/AfterSolutionBuild.targets

* Remove AddAllProjectRefsToSolution.ps1

* Put selenium process tracking into artifacts/tmp/

* build/docker => eng/docker

* Move content from korebuild.json to global.json

* Remove cibuild.{sh,cmd} and update ci.yml

* Ensure vswhere detects BuildTools installations

* Remove sign check exclusions (Arcade does not support)

* list VS required components in global.json

* Update CodeCheck.ps1

* Update arcade (#11143)

* Fixups to broken build

* capture test results in xunit form
* attempt to fix code check
* restore before linux build
* remove duplicate signinfos

* More build fixes

* exclude node_modules from unique project check
* fixup signing props

* More build fixes

* Remove unused NoWarns
* Skip building site extension
* Suppress xunit color in console output
* Install x86 runtime

* Run KillProcesses.ps1 at end of build on CI

* activate env vars in codecheck.ps1

* Fix signing and generate build manifests

* Fixup code check and linux installers build

* Remove custom restore targets

* Import flaky test props

* Restore original manifest generation code and set warnAsErrors=false

* Alwasy restore the toolset

* More attempts at build fixes

* run restore before building deb/rpm packages
* pre-install Yarn.MSBuild
* Remove WIX workaround

* Attempt to fix RPM installers and x86 build

* Fix helix tests, java auto-detection, and code gen steps

* Globally disable MSBuild node resuse and capture MSbuild crash logs

* Cleanup build invocation for installers

* Set max time on xunit tests to 15 min

* Update helix-test.yml

* Fix Hosting tests

* Fix TestHost

* Fix Negotiate.Test

* Fix Mvc tests

* Capture results

* Fix Hosting tests

* Fix Kestrel tests

* Fix ServerComparison tests

* Fix DefaultBuilder functional tests

* Skip template tests

Need to run pack before running tests on mac/linux now

* Fix Identity test

* Fix ServerComparison tests

* Verify nginx installation on ubuntu

* Verify nginx installation?

* Try adding to path

* Try symlink

* Sudo?

* Woot it works

* Disable timeout

Template tests require more than 15 mins

* list env vars

* Use correct dotnet_home

* Prepent path instead

* Fix npmproj tests

* Fix CORS tests

* Debug signalr tests

* Debug signalr

* Capture bin

* Try building templates sequentially

* Debug signalr

* Cleanup

* Disable signalr npm tests

* Debug template test

* Try to fix templates again and make steps more reliable

* Cleanup

* Feedback
2019-06-19 10:03:38 -07:00
Doug Bunting 4f67ff9216 Deploy Microsoft.Web.Xdt.Extensions.pdb
- #10495
- add a general mechanism to publish "loose" PDBs

also, correct a typo from af2f647d635d: always publish the log artifacts
2019-06-17 11:13:12 -07:00
John Luo af2f647d63
Run signcheck if previous steps succeeded an try publishing build artifacts before uploading test results (#11237) 2019-06-14 15:38:11 -07:00
Nate McMaster 6c806f91b1
Add option for only packing runtime-specific packages (#11116)
* Add option for only packing runtime-specific packages
* Remove Microsoft.AspNetCore.Blazor.Templates.dll from the Blazor VSIX
* Cleanup implementation of ANCMSymbols packaging
2019-06-11 20:47:10 -07:00
Nate McMaster 42f79ed842
Set official build ID (#11077)
* Trim trailing slash from SignCheckDirectory
* Set OfficialBuildId
2019-06-10 23:15:11 -07:00
Nate McMaster 41ce223c1c
Replace Internal.AspNetCore.Sdk with Microsoft.DotNet.Arcade.Sdk (#10674)
* Update build.cmd to install .NET Core into $repoRoot/.dotnet instead of $repoRoot/.dotnet/x64
* Move restore sources from build/sources.props into eng/Versions.props (following arcade conventions)
* Remove usages of RuntimeFrameworkVersion in tests and build
* Update Blazor VSIX to use Arcade VSIX tools
* Rename Common.Tests to IIS.Common.TestLib and make it a test asset
* Remove custom versions props for ANCM installer code
* Remove duplicate references to xunit and remove usages of IsTestProject
* Remove duplicate references to Internal.AspNetCore.Analyzers
* Import Arcade.Sdk props and targets and remove custom versioning props
* Remove references to Internal.AspNetCore.Sdk
* Rename PackageLicenseType => PackageLicenseExpression
* Remove dependency on tasks in Internal.AspNetCore.Sdk, add ref to Internal.AspNetCore.BuildTasks as a temporary workaround
* Use Arcade's nuspec support
* Rename SignalR.Client.FunctionalTests to SignalR.Client.FunctionalTestApp
* Fixes for changes to property evaluation order
* Update BaseLineGenerator to netcoreapp3.0
* React to changes in evaluation order in RPM files and quirks in using <Exec> instead of <Run>
* Update Microsoft.Extensions.ApiDescription.Server to react to changes in Arcade packaging
* Workaround aspnet/AspNetCore#11009
2019-06-07 17:19:41 -07:00
Justin Kotalik d3e90a7c07
Guarantee that ANCM package dlls are used for back compat tests (#10858) 2019-06-05 20:11:47 -07:00
Justin Kotalik 760df198c8
Add inproc handler to shared framework (#10586) 2019-05-29 09:58:20 -07:00
Justin Kotalik a20ddd8374
Make Native components a non-required part for build.cmd, fix IIS native code check (#10435) 2019-05-25 15:20:26 -07:00
Doug Bunting 3bb9cda189
Perform symbol publication using Microsoft.SymbolUploader.Build.Task (#10358)
- aspnet/AspNetCore-Internal#2126
- allow release pipeline to publish symbols on all builds: #10113
- add new project that publishes symbols to MSDL and SymWeb
  - release pipeline prepares then restores and builds the new project
  - expire symbols after 10 years (or so)

nits:
- add `$env:DOTNET_SKIP_FIRST_TIME_EXPERIENCE` to ci.yml
2019-05-23 22:14:40 -07:00
John Luo 818dc3b74a
Update log directory (#10301)
* Update log directory

This will now include functional test logs

* There were a few more places logs was used instead of log
2019-05-17 09:02:34 -07:00
Nate McMaster fa73edb9bf
Improve detection of JDK on Windows (#10294) 2019-05-16 12:41:50 -07:00
John Luo 4946c72dab
Add build for linux-musl-arm64 (#10155)
* Add build for linux-musl-arm64

* Remove redundant apk installs
2019-05-14 15:44:52 -07:00
Justin Kotalik 3797bffd9c
Only build site extensions on main build (#9722) 2019-04-30 08:50:44 -07:00
Nate McMaster 52f3f79263 Use VS2019 Enterprise queues for testing (#9571) 2019-04-19 14:54:52 -07:00
Pranav K 56e2080ef7
Always fully build Components.Server.csproj (#9421)
* Always fully build Components.Server.csproj

* Use pre-built js files instead of building on the fly
* Use a sourcemap file for Debug. .gitignore sourcemap files
* Install nodejs on CodeCheck agents

Fixes https://github.com/aspnet/AspNetCore/issues/9402
2019-04-17 09:17:42 -07:00
BrennanConroy 7c76144865
Remove Cpp Client from this repo (#9050) 2019-04-10 09:51:33 -07:00
Justin Kotalik e863da19d0
Run Project templates in separate test check (#8966) 2019-04-04 14:32:45 -07:00
Justin Kotalik df3a702190 Update the SDK (#8670) 2019-04-01 15:18:18 -07:00
Doug Bunting b068c9a62c
Move to BYOC queues (#8694)
* Move to BYOC queues
- aspnet/AspNetCore-Internal#2033
- use conditions matching aspnet/Extensions and dotnet/Arcade
  - e.g. 05cb24592a/azure-pipelines.yml (L49-L54)
- install necessary Build Tools for Visual Studio 2019 components and workloads on CI
- revert part of 85ae18c723 because multiple editions of VS 2019 are now publicly available
  - that is, restore support for multiple editions in InstallVisualStudio.ps1
- update InstallVisualStudio.ps1 to
  - support BuildTools edition of VS 2019

nit: merge a couple of Windows build steps
  - support automatically updating latest version of a VS edition  on the machine
  - include `-Quiet` option for a completely non-interactive installation

* Install SQL Server 2017 Express LocalDB in Windows test job

* Drop back to SQL Server 2016 Express LocalDB

* Instead: Install SQL Server 2017 Express LocalDB and its cumulative update

* Don't move Microsoft.AspNetCore.Blazor.Templates.dll into output directory
- assembly is not of interest and should not be signed

* Revert most of "Don't move Microsoft.AspNetCore.Blazor.Templates.dll into output directory"
- didn't help odd attempt to sign this file and it was only a warning

This reverts commit b55d69c370.

* Back to using SQL Server 2016 Express LocalDB again
- installing cumulative update was taking extra time and not reliably updating the LocalDB install
- reverts part of 70d8d125f9, leaving direct download and some reordering

* Do not assume vsjitdebuger.exe exists
- component is not available in BuildTools edition of VS
2019-03-27 08:31:56 -07:00
Justin Kotalik 47bee4e06c
Get Binlogs from other linux machines. (#8841) 2019-03-27 08:25:02 -07:00
Doug Bunting 45480b10ae
Back to using SQL Server 2016 Express LocalDB again
- installing cumulative update was taking extra time and not reliably updating the LocalDB install
- reverts part of 70d8d125f9, leaving direct download and some reordering
2019-03-26 19:51:18 -07:00
Doug Bunting b55d69c370
Don't move Microsoft.AspNetCore.Blazor.Templates.dll into output directory
- assembly is not of interest and should not be signed
2019-03-26 18:47:11 -07:00
Doug Bunting 70d8d125f9
Instead: Install SQL Server 2017 Express LocalDB and its cumulative update 2019-03-26 18:47:11 -07:00
Doug Bunting 7a5981131c
Drop back to SQL Server 2016 Express LocalDB 2019-03-26 18:47:10 -07:00
Doug Bunting 7fd65f52cf
Install SQL Server 2017 Express LocalDB in Windows test job 2019-03-26 18:47:10 -07:00
Javier Calvarro Nelson 9f1a978230
[Infrastructure improvements] (#8275)
* Improved selenium start and tear down
  * Selenium is set up and torn down in an assembly fixture.
  * Selenium is initialized lazily and in a non-blocking way.
  * Selenium processes are tracked as part of the build and their pids
    written to a file on disk for cleanup in the event of unexpected
    termination of the test process.
  * Browser fixture retries with linear backoff to create a remote
    driver. Under heavy load (like when we are doing a simultaneous NPM
    restore) the selenium server can become unresponsive so we retry
    three times, with a longer comand timeout allowance each time up to
    a max of 3 minutes.
* Moved test project setup to build time instead of runtime.
  * Added target PrepareForTest to create the required files for testing
    * The template creation folder.
    * The template props file to use our built packages.
    * The folder for the custom hive.
  * Added assembly metadata attributes to find all the data we need to
    run the tests.
    * Path to the artifacts shipping packages folder.
    * Path to the artifacts non-shipping packages folder.
    * Path to the test templates creation folder.
    * Path to use for the custom templating hive used in tests.
  * Proper cleanup as part of the build
    * Remove the test templates creation folder.
    * Remove the test packages restore path.
    * Recreate the test templates creation folder.
    * Recreate the test packages restore path.
  * Generated Directory.Build.Props and Directory.Build.Targets in the
    test templates creation folder.
  * Cleaned up potentially stale templatetestsprops.
* Improved test flows
  * Initialization is done lazily and asynchronously.
    * Selenium
    * Browser fixture
    * Template initialization.
  * Flattened test flows to avoid assertions inside deep callstacks.
    * All assertions happen at the test level with improved error messages.
      * With the exception of the migrations assertions.
    * Assertions contain information about which step failed, for what
      project and what failure details.
  * Broke down tests to perform individual steps instead of mixing build
    and publish.
    * Publish project.
    * Build project. (Debug)
    * Run built project.
    * Run published project.
  * Concentrated build logic into the Project class.
    * Context between the different steps of a test is maintained in
      this class.
    * All operations that require coordination are performed within this
      class.
      * There is a lock for dotnet and a lock for nodejs. When building
        SPAs we acquire the nodejs lock to correctly prevent multiple
        runs of nodejs in parallel.

[ApiAuthorization template cleanups]
  * Fix preview3 issues with breaking changes on Entity framework by
    manually configuring the model in ApiAuthorizationDbContext.
  * Add app.db to the project file when using local db.
  * Fix linting errors on angular template.
  * Fix react tests
  * Add tests to cover new auth options in the SPA templates.
2019-03-20 08:44:20 +01:00
Andrew Stanton-Nurse 706778d4de
Add a separate test pass to run flaky tests that doesn't fail the build (#8486)
Similar to https://github.com/aspnet/Extensions/pull/1239.
2019-03-18 18:21:14 -07:00
BrennanConroy a936d77845
Automatically capture crashdumps for C++ client tests (#7940) 2019-02-27 14:20:14 -08:00
Nate McMaster d2d60000eb
Create Linux installers for the aspnetcore targeting pack (#7776)
This creates 3 new build outputs:

* aspnetcore-targeting-pack-$(version).tar.gz
* aspnetcore-targeting-pack-$(version)-linux-x64.deb
* aspnetcore-targeting-pack-$(version)-linux-x64.rpm

Other changes:
* Make RPM packaging consistent with other installers. Vendor == Microsoft Corporation
* Add shared properties for building the targeting pack (or not building it in a servicing build)
2019-02-25 12:00:36 -08:00
Nate McMaster 1b27c9905c
Update dependencies and react to changes (#7844)
* Add workaround for dotnet/sdk#2976
* Add new restore sources
* Update BAR manifest to point to  https://dotnetfeed.blob.core.windows.net/aspnet-aspnetcore/index.json
* Obsolete WithCulture
* Add retries to dockerbuild.sh
* Ensure each test run gets a unique in-memory database
2019-02-22 19:09:08 -08:00
Nate McMaster fbaa3e8b77
Always kill processes and remove direct reference to component governance task (#7793) 2019-02-21 09:29:42 -08:00
Nate McMaster 85ae18c723
Make Visual Studio 2019 a prerequisite to building this repo (#7005)
Changes:

* Make Visual Studio 2019 a prerequisite for building this repo
* Update .sln files
* Update Windows SDK to 17134
* Update developer docs
* Disable ANCM tests
* Update to .NET Core SDK 3.0 Preview 2
* Use Microsoft.NET.Sdk.Razor as a package consistently accross the repo
* React to changes in metadata from Microsoft.NETCore.App
* React to changes in .NET Core SDK
* Attempt to workaround CodeCheck.ps1 failure which doesn't repro locally or on different agents. Possibly due to differences in the version of the PowerShell task?
* Remove dead YML file
* Rename usages of win7-{x64,x86} to win-{x64,x86}
* Update KoreBuild to 3.0.0-build-20190219.1
2019-02-20 09:53:53 -08:00
Nate McMaster 0ec25d3167
Create new 'runtime pack' nuget packages for Microsoft.AspNetCore.App (#7475)
* Create a new package  - Microsoft.AspNetCore.App.Runtime.${rid} - which contains crossgen-ed versions of shared framework assemblies. This will be used by the SDK to replace rid-split packages
* Add a task which can generate the .deps.json file for the shared framework. Unlike the existing task we have, this does not rely on consuming a .deps.json file generated by first running a restore on packages.
* Move the reference to xunit.runner.console into the normal targets and files for managing dependencies
* Produce binlogs during build
2019-02-14 07:25:13 -08:00
Doug Bunting 758ba235fa Touch up component detection
- run only on success of previous steps
2019-02-13 11:32:17 -08:00
Nate McMaster 2ac4619635
Replace custom targets for generating assembly attributes and source control info by using SDK features (#7504) 2019-02-13 10:22:03 -08:00
Doug Bunting d46990856a Add component detection step to builds
- aspnet/AspNetCore-Internal#1254
2019-02-04 16:00:16 -08:00
Justin Kotalik 1d2838ea2e
Upload artifact logs for CI Test runs (#7251) 2019-02-04 13:55:23 -08:00
Nate McMaster 8564093627
CI config: Specify RepositoryBranch when building inside a docker container 2019-02-02 12:14:17 -08:00