Commit Graph

163 Commits

Author SHA1 Message Date
John Luo f0029227cf
Skip site extensions builds in PRs (#13352) 2019-08-23 16:03:04 -07:00
John Luo 43350b57b9
Revert "Revert "Improve components infrastructure (#12145)" (#12679)" (#12744)
* Revert "Revert "Improve components infrastructure (#12145)" (#12679)"

This reverts commit e2d57e2806. The improvement to components infrastructure is now reinstated with the following changes:

* Check in release JS artifacts and use them as a fallback when it's not possible to build npmproj.
* Dont' build nodejs in source build.
2019-08-05 15:17:55 -07:00
John Luo e2d57e2806
Revert "Improve components infrastructure (#12145)" (#12679)
* Revert "Improve components infrastructure (#12145)"

This reverts commit e149f9c0aa.

* Update JS file
2019-07-29 09:03:57 -07:00
John Luo e149f9c0aa
Improve components infrastructure (#12145)
* Remove generated Web.JS javascript files files

* Build NodeJS project referenced by managed projects by default

* Build Web.JS before other CI jobs

- We can't build Web.JS on Linux Musl for example

* Make BuildAll not override explicit BuildJava/NodeJS/Native/Managed settings

* Capture CodeCheck logs for debugging

* Skip NodeJS projects when generation ref assemblies

* Do not build dependent projects if NodeJS is not built

* Make build fail immediately if BuildNodeJS is true but NodeJS is not installed

* Do not build nodejs in installer builds
2019-07-26 13:31:01 -07:00
John Luo e31813b9cd
Add support for source-build (#11717) 2019-07-08 11:02:30 -07:00
John Luo 94f84c4776
Build SiteExtensions (#11409) 2019-06-24 19:37:51 -07:00
John Luo 0058a58372
Fix Template test runs on macOS and Linux (#11407) 2019-06-19 23:03:39 -07:00
Nate McMaster 4fde84a7ad Replace KoreBuild with Arcade (#11122)
* Replace usages of $(CI) with $(ContinuousIntegrationBuild)

* Move RepoTasks to eng/tools/

* Remove dead code

* Update build.sh/ps1 to invoke Arcade instead of KoreBuild

* build/repo.props => eng/Build.props

* build/CodeSign.{props,targets} => eng/Signing.props

* build/repo.targets => eng/Build.props, eng/CodeGen.proj, and eng/AfterSolutionBuild.targets

* Remove AddAllProjectRefsToSolution.ps1

* Put selenium process tracking into artifacts/tmp/

* build/docker => eng/docker

* Move content from korebuild.json to global.json

* Remove cibuild.{sh,cmd} and update ci.yml

* Ensure vswhere detects BuildTools installations

* Remove sign check exclusions (Arcade does not support)

* list VS required components in global.json

* Update CodeCheck.ps1

* Update arcade (#11143)

* Fixups to broken build

* capture test results in xunit form
* attempt to fix code check
* restore before linux build
* remove duplicate signinfos

* More build fixes

* exclude node_modules from unique project check
* fixup signing props

* More build fixes

* Remove unused NoWarns
* Skip building site extension
* Suppress xunit color in console output
* Install x86 runtime

* Run KillProcesses.ps1 at end of build on CI

* activate env vars in codecheck.ps1

* Fix signing and generate build manifests

* Fixup code check and linux installers build

* Remove custom restore targets

* Import flaky test props

* Restore original manifest generation code and set warnAsErrors=false

* Alwasy restore the toolset

* More attempts at build fixes

* run restore before building deb/rpm packages
* pre-install Yarn.MSBuild
* Remove WIX workaround

* Attempt to fix RPM installers and x86 build

* Fix helix tests, java auto-detection, and code gen steps

* Globally disable MSBuild node resuse and capture MSbuild crash logs

* Cleanup build invocation for installers

* Set max time on xunit tests to 15 min

* Update helix-test.yml

* Fix Hosting tests

* Fix TestHost

* Fix Negotiate.Test

* Fix Mvc tests

* Capture results

* Fix Hosting tests

* Fix Kestrel tests

* Fix ServerComparison tests

* Fix DefaultBuilder functional tests

* Skip template tests

Need to run pack before running tests on mac/linux now

* Fix Identity test

* Fix ServerComparison tests

* Verify nginx installation on ubuntu

* Verify nginx installation?

* Try adding to path

* Try symlink

* Sudo?

* Woot it works

* Disable timeout

Template tests require more than 15 mins

* list env vars

* Use correct dotnet_home

* Prepent path instead

* Fix npmproj tests

* Fix CORS tests

* Debug signalr tests

* Debug signalr

* Capture bin

* Try building templates sequentially

* Debug signalr

* Cleanup

* Disable signalr npm tests

* Debug template test

* Try to fix templates again and make steps more reliable

* Cleanup

* Feedback
2019-06-19 10:03:38 -07:00
Doug Bunting 4f67ff9216 Deploy Microsoft.Web.Xdt.Extensions.pdb
- #10495
- add a general mechanism to publish "loose" PDBs

also, correct a typo from af2f647d635d: always publish the log artifacts
2019-06-17 11:13:12 -07:00
John Luo af2f647d63
Run signcheck if previous steps succeeded an try publishing build artifacts before uploading test results (#11237) 2019-06-14 15:38:11 -07:00
Nate McMaster 6c806f91b1
Add option for only packing runtime-specific packages (#11116)
* Add option for only packing runtime-specific packages
* Remove Microsoft.AspNetCore.Blazor.Templates.dll from the Blazor VSIX
* Cleanup implementation of ANCMSymbols packaging
2019-06-11 20:47:10 -07:00
Nate McMaster 42f79ed842
Set official build ID (#11077)
* Trim trailing slash from SignCheckDirectory
* Set OfficialBuildId
2019-06-10 23:15:11 -07:00
Nate McMaster 41ce223c1c
Replace Internal.AspNetCore.Sdk with Microsoft.DotNet.Arcade.Sdk (#10674)
* Update build.cmd to install .NET Core into $repoRoot/.dotnet instead of $repoRoot/.dotnet/x64
* Move restore sources from build/sources.props into eng/Versions.props (following arcade conventions)
* Remove usages of RuntimeFrameworkVersion in tests and build
* Update Blazor VSIX to use Arcade VSIX tools
* Rename Common.Tests to IIS.Common.TestLib and make it a test asset
* Remove custom versions props for ANCM installer code
* Remove duplicate references to xunit and remove usages of IsTestProject
* Remove duplicate references to Internal.AspNetCore.Analyzers
* Import Arcade.Sdk props and targets and remove custom versioning props
* Remove references to Internal.AspNetCore.Sdk
* Rename PackageLicenseType => PackageLicenseExpression
* Remove dependency on tasks in Internal.AspNetCore.Sdk, add ref to Internal.AspNetCore.BuildTasks as a temporary workaround
* Use Arcade's nuspec support
* Rename SignalR.Client.FunctionalTests to SignalR.Client.FunctionalTestApp
* Fixes for changes to property evaluation order
* Update BaseLineGenerator to netcoreapp3.0
* React to changes in evaluation order in RPM files and quirks in using <Exec> instead of <Run>
* Update Microsoft.Extensions.ApiDescription.Server to react to changes in Arcade packaging
* Workaround aspnet/AspNetCore#11009
2019-06-07 17:19:41 -07:00
Justin Kotalik d3e90a7c07
Guarantee that ANCM package dlls are used for back compat tests (#10858) 2019-06-05 20:11:47 -07:00
Justin Kotalik 760df198c8
Add inproc handler to shared framework (#10586) 2019-05-29 09:58:20 -07:00
Justin Kotalik a20ddd8374
Make Native components a non-required part for build.cmd, fix IIS native code check (#10435) 2019-05-25 15:20:26 -07:00
Doug Bunting 3bb9cda189
Perform symbol publication using Microsoft.SymbolUploader.Build.Task (#10358)
- aspnet/AspNetCore-Internal#2126
- allow release pipeline to publish symbols on all builds: #10113
- add new project that publishes symbols to MSDL and SymWeb
  - release pipeline prepares then restores and builds the new project
  - expire symbols after 10 years (or so)

nits:
- add `$env:DOTNET_SKIP_FIRST_TIME_EXPERIENCE` to ci.yml
2019-05-23 22:14:40 -07:00
John Luo 818dc3b74a
Update log directory (#10301)
* Update log directory

This will now include functional test logs

* There were a few more places logs was used instead of log
2019-05-17 09:02:34 -07:00
Nate McMaster fa73edb9bf
Improve detection of JDK on Windows (#10294) 2019-05-16 12:41:50 -07:00
John Luo 4946c72dab
Add build for linux-musl-arm64 (#10155)
* Add build for linux-musl-arm64

* Remove redundant apk installs
2019-05-14 15:44:52 -07:00
Justin Kotalik 3797bffd9c
Only build site extensions on main build (#9722) 2019-04-30 08:50:44 -07:00
Nate McMaster 52f3f79263 Use VS2019 Enterprise queues for testing (#9571) 2019-04-19 14:54:52 -07:00
Pranav K 56e2080ef7
Always fully build Components.Server.csproj (#9421)
* Always fully build Components.Server.csproj

* Use pre-built js files instead of building on the fly
* Use a sourcemap file for Debug. .gitignore sourcemap files
* Install nodejs on CodeCheck agents

Fixes https://github.com/aspnet/AspNetCore/issues/9402
2019-04-17 09:17:42 -07:00
BrennanConroy 7c76144865
Remove Cpp Client from this repo (#9050) 2019-04-10 09:51:33 -07:00
Justin Kotalik e863da19d0
Run Project templates in separate test check (#8966) 2019-04-04 14:32:45 -07:00
Justin Kotalik df3a702190 Update the SDK (#8670) 2019-04-01 15:18:18 -07:00
Doug Bunting b068c9a62c
Move to BYOC queues (#8694)
* Move to BYOC queues
- aspnet/AspNetCore-Internal#2033
- use conditions matching aspnet/Extensions and dotnet/Arcade
  - e.g. 05cb24592a/azure-pipelines.yml (L49-L54)
- install necessary Build Tools for Visual Studio 2019 components and workloads on CI
- revert part of 85ae18c723 because multiple editions of VS 2019 are now publicly available
  - that is, restore support for multiple editions in InstallVisualStudio.ps1
- update InstallVisualStudio.ps1 to
  - support BuildTools edition of VS 2019

nit: merge a couple of Windows build steps
  - support automatically updating latest version of a VS edition  on the machine
  - include `-Quiet` option for a completely non-interactive installation

* Install SQL Server 2017 Express LocalDB in Windows test job

* Drop back to SQL Server 2016 Express LocalDB

* Instead: Install SQL Server 2017 Express LocalDB and its cumulative update

* Don't move Microsoft.AspNetCore.Blazor.Templates.dll into output directory
- assembly is not of interest and should not be signed

* Revert most of "Don't move Microsoft.AspNetCore.Blazor.Templates.dll into output directory"
- didn't help odd attempt to sign this file and it was only a warning

This reverts commit b55d69c370.

* Back to using SQL Server 2016 Express LocalDB again
- installing cumulative update was taking extra time and not reliably updating the LocalDB install
- reverts part of 70d8d125f9, leaving direct download and some reordering

* Do not assume vsjitdebuger.exe exists
- component is not available in BuildTools edition of VS
2019-03-27 08:31:56 -07:00
Justin Kotalik 47bee4e06c
Get Binlogs from other linux machines. (#8841) 2019-03-27 08:25:02 -07:00
Doug Bunting 45480b10ae
Back to using SQL Server 2016 Express LocalDB again
- installing cumulative update was taking extra time and not reliably updating the LocalDB install
- reverts part of 70d8d125f9, leaving direct download and some reordering
2019-03-26 19:51:18 -07:00
Doug Bunting b55d69c370
Don't move Microsoft.AspNetCore.Blazor.Templates.dll into output directory
- assembly is not of interest and should not be signed
2019-03-26 18:47:11 -07:00
Doug Bunting 70d8d125f9
Instead: Install SQL Server 2017 Express LocalDB and its cumulative update 2019-03-26 18:47:11 -07:00
Doug Bunting 7a5981131c
Drop back to SQL Server 2016 Express LocalDB 2019-03-26 18:47:10 -07:00
Doug Bunting 7fd65f52cf
Install SQL Server 2017 Express LocalDB in Windows test job 2019-03-26 18:47:10 -07:00
Javier Calvarro Nelson 9f1a978230
[Infrastructure improvements] (#8275)
* Improved selenium start and tear down
  * Selenium is set up and torn down in an assembly fixture.
  * Selenium is initialized lazily and in a non-blocking way.
  * Selenium processes are tracked as part of the build and their pids
    written to a file on disk for cleanup in the event of unexpected
    termination of the test process.
  * Browser fixture retries with linear backoff to create a remote
    driver. Under heavy load (like when we are doing a simultaneous NPM
    restore) the selenium server can become unresponsive so we retry
    three times, with a longer comand timeout allowance each time up to
    a max of 3 minutes.
* Moved test project setup to build time instead of runtime.
  * Added target PrepareForTest to create the required files for testing
    * The template creation folder.
    * The template props file to use our built packages.
    * The folder for the custom hive.
  * Added assembly metadata attributes to find all the data we need to
    run the tests.
    * Path to the artifacts shipping packages folder.
    * Path to the artifacts non-shipping packages folder.
    * Path to the test templates creation folder.
    * Path to use for the custom templating hive used in tests.
  * Proper cleanup as part of the build
    * Remove the test templates creation folder.
    * Remove the test packages restore path.
    * Recreate the test templates creation folder.
    * Recreate the test packages restore path.
  * Generated Directory.Build.Props and Directory.Build.Targets in the
    test templates creation folder.
  * Cleaned up potentially stale templatetestsprops.
* Improved test flows
  * Initialization is done lazily and asynchronously.
    * Selenium
    * Browser fixture
    * Template initialization.
  * Flattened test flows to avoid assertions inside deep callstacks.
    * All assertions happen at the test level with improved error messages.
      * With the exception of the migrations assertions.
    * Assertions contain information about which step failed, for what
      project and what failure details.
  * Broke down tests to perform individual steps instead of mixing build
    and publish.
    * Publish project.
    * Build project. (Debug)
    * Run built project.
    * Run published project.
  * Concentrated build logic into the Project class.
    * Context between the different steps of a test is maintained in
      this class.
    * All operations that require coordination are performed within this
      class.
      * There is a lock for dotnet and a lock for nodejs. When building
        SPAs we acquire the nodejs lock to correctly prevent multiple
        runs of nodejs in parallel.

[ApiAuthorization template cleanups]
  * Fix preview3 issues with breaking changes on Entity framework by
    manually configuring the model in ApiAuthorizationDbContext.
  * Add app.db to the project file when using local db.
  * Fix linting errors on angular template.
  * Fix react tests
  * Add tests to cover new auth options in the SPA templates.
2019-03-20 08:44:20 +01:00
Andrew Stanton-Nurse 706778d4de
Add a separate test pass to run flaky tests that doesn't fail the build (#8486)
Similar to https://github.com/aspnet/Extensions/pull/1239.
2019-03-18 18:21:14 -07:00
BrennanConroy a936d77845
Automatically capture crashdumps for C++ client tests (#7940) 2019-02-27 14:20:14 -08:00
Nate McMaster d2d60000eb
Create Linux installers for the aspnetcore targeting pack (#7776)
This creates 3 new build outputs:

* aspnetcore-targeting-pack-$(version).tar.gz
* aspnetcore-targeting-pack-$(version)-linux-x64.deb
* aspnetcore-targeting-pack-$(version)-linux-x64.rpm

Other changes:
* Make RPM packaging consistent with other installers. Vendor == Microsoft Corporation
* Add shared properties for building the targeting pack (or not building it in a servicing build)
2019-02-25 12:00:36 -08:00
Nate McMaster 1b27c9905c
Update dependencies and react to changes (#7844)
* Add workaround for dotnet/sdk#2976
* Add new restore sources
* Update BAR manifest to point to  https://dotnetfeed.blob.core.windows.net/aspnet-aspnetcore/index.json
* Obsolete WithCulture
* Add retries to dockerbuild.sh
* Ensure each test run gets a unique in-memory database
2019-02-22 19:09:08 -08:00
Nate McMaster fbaa3e8b77
Always kill processes and remove direct reference to component governance task (#7793) 2019-02-21 09:29:42 -08:00
Nate McMaster 85ae18c723
Make Visual Studio 2019 a prerequisite to building this repo (#7005)
Changes:

* Make Visual Studio 2019 a prerequisite for building this repo
* Update .sln files
* Update Windows SDK to 17134
* Update developer docs
* Disable ANCM tests
* Update to .NET Core SDK 3.0 Preview 2
* Use Microsoft.NET.Sdk.Razor as a package consistently accross the repo
* React to changes in metadata from Microsoft.NETCore.App
* React to changes in .NET Core SDK
* Attempt to workaround CodeCheck.ps1 failure which doesn't repro locally or on different agents. Possibly due to differences in the version of the PowerShell task?
* Remove dead YML file
* Rename usages of win7-{x64,x86} to win-{x64,x86}
* Update KoreBuild to 3.0.0-build-20190219.1
2019-02-20 09:53:53 -08:00
Nate McMaster 0ec25d3167
Create new 'runtime pack' nuget packages for Microsoft.AspNetCore.App (#7475)
* Create a new package  - Microsoft.AspNetCore.App.Runtime.${rid} - which contains crossgen-ed versions of shared framework assemblies. This will be used by the SDK to replace rid-split packages
* Add a task which can generate the .deps.json file for the shared framework. Unlike the existing task we have, this does not rely on consuming a .deps.json file generated by first running a restore on packages.
* Move the reference to xunit.runner.console into the normal targets and files for managing dependencies
* Produce binlogs during build
2019-02-14 07:25:13 -08:00
Doug Bunting 758ba235fa Touch up component detection
- run only on success of previous steps
2019-02-13 11:32:17 -08:00
Nate McMaster 2ac4619635
Replace custom targets for generating assembly attributes and source control info by using SDK features (#7504) 2019-02-13 10:22:03 -08:00
Doug Bunting d46990856a Add component detection step to builds
- aspnet/AspNetCore-Internal#1254
2019-02-04 16:00:16 -08:00
Justin Kotalik 1d2838ea2e
Upload artifact logs for CI Test runs (#7251) 2019-02-04 13:55:23 -08:00
Nate McMaster 8564093627
CI config: Specify RepositoryBranch when building inside a docker container 2019-02-02 12:14:17 -08:00
Nate McMaster 2558166b19
Run code sign of packages before Windows installers are built, and run SignCheck in a separate step 2019-02-02 08:28:30 -08:00
Nate McMaster 4c4a1effcd
Only publish artifacts on successful builds and fix code-signing for the SiteExtension package (#7199) 2019-02-01 19:17:01 -08:00
Pavel Krymets d4c55df8f4
Fix MAX_PATH issue in SiteExtensions and move them to main build (#7177) 2019-02-01 12:37:00 -08:00
Nate McMaster 7ee3399ce2
Fix code-signing for xplat packages and Windows installers (#7191) 2019-02-01 10:52:00 -08:00
Nate McMaster 6081fec894
Add new build.cmd/sh options for suppressing restore, NodeJS, setting target arch and OS name, and more (#7137)
Add new command line parameters for working with the project:

* `-NoBuild`, `-NoRestore` - these already existed, but users found it hard to discover this powershell syntax: '-build:$false'
* `-Arch`/`--arch` - set the target CPU architecture to build. Defaults to x64
* `--os-name` - on non-Windows builds, manually specify if the build should target Alpine. generic Linux, or MacOS
* Rename flags used to build specific project types. The pattern now is `--build-$(group)` or `--no-build-$(group)` (In PowerShell its `-Build$(Group)` or `-NoBuild$(Group). Example: -NoBuildJava

Changes to build definitions:
* Update the ci build definition to build all supported architectures
* Support publishing multiple artifacts per job

Other changes:
* `-NoBuild` implies `-NoRestore`
* Add new properties, `TargetArchitecture`, `TargetOsName`, and `TargetRuntimeIdentifier`
* Replace usages of `SharedFxRid` with these new properties
* To make `--no-build-nodejs` actually work, replaced Components.Browser.JS.csproj with Components.Browser.JS.npmproj
* Fix errors when building for win-arm on a clean machine
* Fix a few other project errors, like using the wrong syntax for DefaultItemExcludes, or using the wrong Platform value for x86
2019-01-31 15:47:47 -08:00
Pavel Krymets 78fabbdb40
Build 3.0 AggregateSite extension (#6849) 2019-01-28 20:09:51 -08:00
Nate McMaster ac04019e42
Remove obsolete build infrastructure (#6995)
Changes:

* Remove obsolete targets which are unnecessary now that this repo no longer builds git submodules in a separate build process
* Remove the need for static analysis of 'ArtifactInfo' items
* Simplify how the code signing task is configured
* Remove unused repo tasks
* Remove duplicate lists of external dependencies and packages to be produced
* Remove obsolete build definition
* Remove obsolete build script parameters
* Add VisualStudioSetupOutputPath
2019-01-28 09:57:59 -08:00
Chris Ross 2b7c26a123
Dynamically install NGinx Internal/#1525 (#6963) 2019-01-25 14:32:32 -08:00
Nate McMaster 436076fd55
Fix broken tests and VS solutions caused by source code reorganization and other build infra cleanup
Changes:
* Fix broken tests and VS solutions caused by source code reorganization
* Add a check to validate generated code and solutions on PRs
* backport some source code reorg to src/Identity
* Fix startvs.cmd if you've already run build.ps1
* Add PR checks for tests on Linux/macOS
* Skip broken Nginx tests
* Add conditions to skip IIS tests on non-Windows platforms
2019-01-04 10:00:26 -08:00
Pavel Krymets a7b783724e
Fix ANCM environment variables bugs (#6083) 2018-12-28 15:48:20 -08:00
Pavel Krymets 7371d90222
Re-enable IIS tests (#6067) 2018-12-21 19:23:48 -08:00
Nate McMaster c3beb1f9b1
build.cmd/sh changes (#5958)
Update the build scripts to support building subfolders or subgroups of projects

* Add build scripts for ci
* Remove obsolete scripts
* Add flags like --test and --pack to control running just test or packaging
* Add flags like --managed and --native to control building sub-types of projects
* Remove KoreBuild bootstrapper flags
* Update to extensions 3.0.0-preview.18619.1 (needed to get a fix for aspnet/Extensions#815 to make this change work on MSBuild.exe)
2018-12-20 16:52:15 -08:00
Nate McMaster f0990ec531
Fix build issues with IIS integration projects
Changes:
* Ensure IIS managed and pkg projects build after the native projects
* Update projects to build test
* Update CI checks to build on macOS and Linux
* Use package baselines to manage ANCM packages
2018-12-13 22:53:31 -08:00
Justin Kotalik 6adfa32402
Merge branch 'release/2.1' into release/2.2 2018-12-13 22:12:19 -08:00
Justin Kotalik 429719b91d
Make IISIntegration use Reference instead of PackageReference (and reactionary work) (#4311) 2018-12-12 16:59:44 -05:00
Nate McMaster 569a14ecb8
Merge branch 'release/2.1' into release/2.2 2018-12-06 17:09:03 -08:00
Nate McMaster de994e0a8b
Create temporary PR validation which just compiles, without tests (#4478) 2018-12-06 12:41:37 -08:00