Commit Graph

136 Commits

Author SHA1 Message Date
Javier Calvarro Nelson 1480b99866
Enforce E2E test prerequisites when building individual solutions only (#11642)
* Fixes multiple issues that happened due to the E2E tests not running on the CI
* Enforces prerequisites for E2E tests on the CI and when building projects with E2E tests.
* Re-enables the E2E tests for templates.
2019-06-29 01:04:36 +02:00
Dylan Dmitri Gray 3cb414afdc
Dylan/event source (#11516)
* Queuing Middleware now supports EventSources. Visibility for: items in queue, duration spent in queue, number of rejected requests.
2019-06-28 15:20:50 -07:00
Hao Kung f35564ba06
Move DataProtection and Extensions.Identity to netstandard2.0/2.1 (#11008) 2019-06-21 14:40:42 -07:00
Nate McMaster 4fde84a7ad Replace KoreBuild with Arcade (#11122)
* Replace usages of $(CI) with $(ContinuousIntegrationBuild)

* Move RepoTasks to eng/tools/

* Remove dead code

* Update build.sh/ps1 to invoke Arcade instead of KoreBuild

* build/repo.props => eng/Build.props

* build/CodeSign.{props,targets} => eng/Signing.props

* build/repo.targets => eng/Build.props, eng/CodeGen.proj, and eng/AfterSolutionBuild.targets

* Remove AddAllProjectRefsToSolution.ps1

* Put selenium process tracking into artifacts/tmp/

* build/docker => eng/docker

* Move content from korebuild.json to global.json

* Remove cibuild.{sh,cmd} and update ci.yml

* Ensure vswhere detects BuildTools installations

* Remove sign check exclusions (Arcade does not support)

* list VS required components in global.json

* Update CodeCheck.ps1

* Update arcade (#11143)

* Fixups to broken build

* capture test results in xunit form
* attempt to fix code check
* restore before linux build
* remove duplicate signinfos

* More build fixes

* exclude node_modules from unique project check
* fixup signing props

* More build fixes

* Remove unused NoWarns
* Skip building site extension
* Suppress xunit color in console output
* Install x86 runtime

* Run KillProcesses.ps1 at end of build on CI

* activate env vars in codecheck.ps1

* Fix signing and generate build manifests

* Fixup code check and linux installers build

* Remove custom restore targets

* Import flaky test props

* Restore original manifest generation code and set warnAsErrors=false

* Alwasy restore the toolset

* More attempts at build fixes

* run restore before building deb/rpm packages
* pre-install Yarn.MSBuild
* Remove WIX workaround

* Attempt to fix RPM installers and x86 build

* Fix helix tests, java auto-detection, and code gen steps

* Globally disable MSBuild node resuse and capture MSbuild crash logs

* Cleanup build invocation for installers

* Set max time on xunit tests to 15 min

* Update helix-test.yml

* Fix Hosting tests

* Fix TestHost

* Fix Negotiate.Test

* Fix Mvc tests

* Capture results

* Fix Hosting tests

* Fix Kestrel tests

* Fix ServerComparison tests

* Fix DefaultBuilder functional tests

* Skip template tests

Need to run pack before running tests on mac/linux now

* Fix Identity test

* Fix ServerComparison tests

* Verify nginx installation on ubuntu

* Verify nginx installation?

* Try adding to path

* Try symlink

* Sudo?

* Woot it works

* Disable timeout

Template tests require more than 15 mins

* list env vars

* Use correct dotnet_home

* Prepent path instead

* Fix npmproj tests

* Fix CORS tests

* Debug signalr tests

* Debug signalr

* Capture bin

* Try building templates sequentially

* Debug signalr

* Cleanup

* Disable signalr npm tests

* Debug template test

* Try to fix templates again and make steps more reliable

* Cleanup

* Feedback
2019-06-19 10:03:38 -07:00
Stephen Halter e727101957
Work around potential race in PipeWriter Redux (#11065)
- Make sure we always await the last flush task before calling FlushAsync
  again instead of preemptively calling FlushAsync and checking to see
  if the ValueTask is incomplete before bothering to acquire the _flushLock
- This now acquires the _flushLock fore every call to Response.Body.Write
  whereas this only happened for truly async writes before.
- I don't think this is a big concern since this should normally be uncontested,
  and DefaultPipeWriter.FlushAsync/GetResult already acquire a lock.
2019-06-13 20:03:29 -07:00
Brennan 902369610a Convert SignalR EventSource usage to new pattern (#11130) 2019-06-13 18:40:00 -07:00
Günther Foidl 90ab2cb965 Spanified Webencoders.Base64UrlEncode (#11047) 2019-06-10 18:18:31 -07:00
John Luo f76be64e5d
Only run selenium tests when build java is specified (#11018) 2019-06-10 14:35:27 -07:00
Chris Ross (ASP.NET) b9c8f34919 Merged PR 1508: [2.2] HttpSys & IIS path normalization
This takes PathNormalizer from Kestrel to normalize the request path and prevent traversals. (e.g. "/./" and "/../").

In 2.1 only HttpSys was affected (https://dev.azure.com/dnceng/internal/_git/aspnet-AspNetCore/pullrequest/1480). In 2.2 HttpSys and IIS-in-proc share this code (with additional tests). In 3.0 we'll refactor it to use more shared source across all three servers.
2019-06-08 06:37:36 +00:00
Chris Ross (ASP.NET) c2bb354675 Merged PR 1480: [2.1] HttpSys path normalization
This takes PathNormalizer from Kestrel to normalize the request path and prevent traversals. (e.g. "/./" and "/../").

In 2.1 only HttpSys was affected. In 2.2 HttpSys and IIS-in-proc will share this code (with additional tests). In 3.0 we'll refactor it to use more shared source across all three servers.
2019-06-07 21:20:28 +00:00
YPTopMan c18fc3d7ca Use throws expressions and expression bodied members (#10764) 2019-06-06 09:56:12 -07:00
Justin Kotalik fdba8a91f9
Add versioning to dotnet-dev-certs (#10908) 2019-06-05 22:04:27 -07:00
Michal Strehovský 9cd2ad15ff Remove use of GetRuntimeInterfaceMap in IsTypeAwaitable (#10572)
`GetRuntimeInterfaceMap` is a rather expensive API and it doesn't look like the use here is warranted.

The replacement should have equivalent behavior.
2019-06-05 10:58:54 -07:00
Hao Kung b75b892eac
Add CertificateAuthentication (#9756) 2019-05-31 22:49:40 -07:00
David Fowler 04bf1bf32e
Implement new bedrock listener abstraction and re-plat Kestrel on top (#10321)
This is a massive set of changes to Kestrel to remove the existing pubternal transport layer and implement a public facing API for listeners and clients, see the details here #10308.

This change only has the server side pieces of the story as I don't want to add the client APIs without having ported SignalR to use them. Here are the highlights:

- Transport.Abstractions is empty (will be removed in a separate PR as it requires removing it from a ton of places)
- TransportConnection has been moved to Connection.Abstractions (we can decide if we need to consolidate with DefaultConnectionContext in a later PR)
- Added FileHandleEndPoint which allows binding to a file handle (could be a pipe or tcp handle)
ListenOptions has been gutted for most pubternal API and returns various types of binding information . The source of truth is the EndPoint instance.
- Cleaned up a bunch of libuv tests decoupling them from Kestrel.Core

## Breaking changes

- Removing pubternal API is itself a breaking change but one that we already planned to do.
- We've removed the ability to set the scheduling mode on Kestrel
- DisposeAsync was added to ConnectionContext 
- NoDelay was removed from ListenOptions. This has been moved to each of the transports. One major difference though is that it's no longer localized per endpoint but is global. We'd need a derived EndPoint type (or maybe extend IPEndPoint) to store both the socket options and the binding information.
2019-05-30 20:34:26 -07:00
Dylan Dmitri Gray 9969e99ef4
Dylan/request throttle (#10413)
* request throttling -- initial implementation

* prevented semaphore leak; added xml docs

* small doc fixes

* reference document

* Added internals folder, added structured logging,

* removed typo'd dependency

* no default MaxConcurrentRequests; other polishing

* renamed SemaphoreWrapper->RequestQueue; cleanup

* moved SyncPoint; prevented possible semaphore leak

* adjusting feedback

* regen refs

* Final changes!
2019-05-27 14:46:14 -07:00
Justin Kotalik a20ddd8374
Make Native components a non-required part for build.cmd, fix IIS native code check (#10435) 2019-05-25 15:20:26 -07:00
Nate McMaster 01a8798936
Update links and disable warnings related to tests that don't work in Helix (#10427) 2019-05-21 16:39:06 -07:00
Nate McMaster 77e08c26d5
Replace all usages of RepositoryRoot with RepoRoot (#10267)
Preparing to adopt the Arcade SDK, which uses the shorter variable name instead
2019-05-15 12:00:19 -07:00
Justin Kotalik 47575a08a5
Handle a few other potentially null properties (#10053) 2019-05-14 18:08:05 -07:00
Justin Kotalik aaaaf572fd
Implement MaxRequestBodySize feature for IIS inprocess (#9475) 2019-05-05 19:30:12 -07:00
Nate McMaster 8863ebfda1
Add --mutex option to yarn commands (#9919)
Attempt to improve reliability of running yarn commands. Our project structure might cause yarn to be launched multiple times. According to yarn docs, this option should avoid conflicts between multiple instances of yarn

https://yarnpkg.com/en/docs/cli/#toc-concurrency-and-mutex
2019-05-02 17:45:55 -07:00
Ben Adams 86728ecc17 Seal some internal derived classes (#9880) 2019-05-01 10:29:05 -07:00
Justin Kotalik 801198c7a9
Fix bad assert (#9835) 2019-04-29 15:40:22 -07:00
Javier Calvarro Nelson 6fa9398781
[MVC][Fixes #8426]TempData does not clear when using __Host- as cookiename prefix (#9741) 2019-04-25 17:32:12 +02:00
Justin Kotalik 87229e315e
Lock during finalizer to not race with dispose in MemoryPoolBlock (#9555) 2019-04-22 15:55:53 -07:00
David Fowler 5f3bdba584
Spanify path -> string logic (#9628)
- Use spans instead of byte[] and array segments when turning the URL into a string
2019-04-22 15:53:56 -07:00
Pavel Krymets 56c064bd53 Display startup errors in ANCM (#8518) 2019-04-18 13:48:48 -07:00
Javier Calvarro Nelson 5889aea06c
Enable custom restore of NPM dependencies for csproj files
Enables custom restore of NPM dependencies for csproj files that require it.
2019-04-14 15:02:37 +02:00
Javier Calvarro Nelson f934bfaa7e
[Https][Tooling] Add digital signature to the KeyUsage of the HTTPS dev-cert (#9293)
* [Https][Tooling] Add digital signature to the KeyUsage of the HTTPS dev-cert
2019-04-12 09:27:54 +02:00
Javier Calvarro Nelson 6f13e7b77a
E2E infrastructure cleanup (#9163) 2019-04-10 17:19:52 +02:00
Javier Calvarro Nelson 4abf766c3c
Update Identity server dependency for preview4 (#9104)
* Updates Identity server dependency for preview4
* Infrastructure improvements.
* Updates react versions.
* Updates migrations.
* Makes templates show up in Visual Studio
2019-04-08 17:06:58 +02:00
Javier Calvarro Nelson 437134f1da
[Infrastructure] Standarize E2E test asserts and increase wait time (#9080)
* Standarize E2E test asserts and increase wait time
2019-04-05 09:00:11 +02:00
Javier Calvarro Nelson 98d8b9ab4a
[Infrastructure] Retry yarn install on failure (#9068)
* [Infrastructure] Retry yarn install on failure
2019-04-04 20:39:52 +02:00
Javier Calvarro Nelson 9f1a978230
[Infrastructure improvements] (#8275)
* Improved selenium start and tear down
  * Selenium is set up and torn down in an assembly fixture.
  * Selenium is initialized lazily and in a non-blocking way.
  * Selenium processes are tracked as part of the build and their pids
    written to a file on disk for cleanup in the event of unexpected
    termination of the test process.
  * Browser fixture retries with linear backoff to create a remote
    driver. Under heavy load (like when we are doing a simultaneous NPM
    restore) the selenium server can become unresponsive so we retry
    three times, with a longer comand timeout allowance each time up to
    a max of 3 minutes.
* Moved test project setup to build time instead of runtime.
  * Added target PrepareForTest to create the required files for testing
    * The template creation folder.
    * The template props file to use our built packages.
    * The folder for the custom hive.
  * Added assembly metadata attributes to find all the data we need to
    run the tests.
    * Path to the artifacts shipping packages folder.
    * Path to the artifacts non-shipping packages folder.
    * Path to the test templates creation folder.
    * Path to use for the custom templating hive used in tests.
  * Proper cleanup as part of the build
    * Remove the test templates creation folder.
    * Remove the test packages restore path.
    * Recreate the test templates creation folder.
    * Recreate the test packages restore path.
  * Generated Directory.Build.Props and Directory.Build.Targets in the
    test templates creation folder.
  * Cleaned up potentially stale templatetestsprops.
* Improved test flows
  * Initialization is done lazily and asynchronously.
    * Selenium
    * Browser fixture
    * Template initialization.
  * Flattened test flows to avoid assertions inside deep callstacks.
    * All assertions happen at the test level with improved error messages.
      * With the exception of the migrations assertions.
    * Assertions contain information about which step failed, for what
      project and what failure details.
  * Broke down tests to perform individual steps instead of mixing build
    and publish.
    * Publish project.
    * Build project. (Debug)
    * Run built project.
    * Run published project.
  * Concentrated build logic into the Project class.
    * Context between the different steps of a test is maintained in
      this class.
    * All operations that require coordination are performed within this
      class.
      * There is a lock for dotnet and a lock for nodejs. When building
        SPAs we acquire the nodejs lock to correctly prevent multiple
        runs of nodejs in parallel.

[ApiAuthorization template cleanups]
  * Fix preview3 issues with breaking changes on Entity framework by
    manually configuring the model in ApiAuthorizationDbContext.
  * Add app.db to the project file when using local db.
  * Fix linting errors on angular template.
  * Fix react tests
  * Add tests to cover new auth options in the SPA templates.
2019-03-20 08:44:20 +01:00
Ryan Nowak d329cca17d Change APIs to internal in Diagnostics
These types are still used, but we don't want them to be public.
2019-03-13 13:15:06 -07:00
Javier Calvarro Nelson d7a9606040
[Templating][Components] Unify and improve E2E testing infrastructure (#8188)
* Unify the Templating and Components testing infrastructure.
* Enable test project discovery in the components E2E tests.
* Enable selectively disabling Selenium tests through build properties.
2019-03-11 18:40:18 +01:00
Hao Kung 435867e8c5
Add ability to skip tests on specific helix queues (#8231) 2019-03-06 23:53:18 -08:00
Justin Kotalik fc9e48877c
Add FormPipeReader (#7964) 2019-03-01 14:41:05 -08:00
Justin Kotalik 3509323ad1
Remove extra await/state machine in FlushAsync (#7646) 2019-02-28 22:42:42 -08:00
Hao Kung a2c8a34556
Reenable some helix tests (#7763)
* Fix identity UI script tests on helix
* Add OnHelix static helper method
2019-02-25 23:39:46 -08:00
Chris Ross cc62bcebfc
Skip HttpSys.Https tests on Helix Internal/#1816 (#7578) 2019-02-15 12:33:23 -08:00
Nate McMaster 338de4710a
Automatically set 'PrivateAssets=All' for .Sources references (#7539)
When a `<Reference>` is named `*.Sources`, set PrivateAssets=All (exclude from generated nuspec) and IncludeAssets=ContentFiles (only consume content files, not .dll's)
2019-02-13 14:31:15 -08:00
Chris Ross 06d7fe73a9
Implement ITlsHandshakeFeature for HttpSys (#7284) 2019-02-13 08:39:45 -08:00
Hao Kung 26acbc3861
Reorganize helix files (#7353) 2019-02-07 16:27:48 -08:00
James Newton-King 574be0d22c
Add readonly modifier to readonly structs (#7169) 2019-02-01 15:04:25 +13:00
Nate McMaster ac04019e42
Remove obsolete build infrastructure (#6995)
Changes:

* Remove obsolete targets which are unnecessary now that this repo no longer builds git submodules in a separate build process
* Remove the need for static analysis of 'ArtifactInfo' items
* Simplify how the code signing task is configured
* Remove unused repo tasks
* Remove duplicate lists of external dependencies and packages to be produced
* Remove obsolete build definition
* Remove obsolete build script parameters
* Add VisualStudioSetupOutputPath
2019-01-28 09:57:59 -08:00
Pavel Krymets 141f3d8781
Reenable application initialization module tests (#7024) 2019-01-28 09:24:42 -08:00
Hao Kung 2791dbf892 Update shared tests to target netcoreapp3.0 on master (#6788) 2019-01-17 09:50:41 -08:00
Chris Ross 1f892d798d
Add AllowSynchronousIO to TestServer and IIS, fix tests (#6404) 2019-01-15 11:48:17 -08:00
Chris Ross 6a44aca6a2
React to WebHostBuilderFactory changes (#6460) 2019-01-15 08:09:27 -08:00
Nate McMaster 3cd5054eb5
Merge branch 'release/2.2' 2019-01-11 07:20:00 -08:00
Nate McMaster 22f3166fa6
Merge branch 'release/2.1' into release/2.2 2019-01-10 01:22:10 -08:00
Nate McMaster d3838620d6
Convert SignalR projects to build with ProjectReference (#6457)
Part of #4246 

Changes:
* Update source code layout to follow the new conventions for this repo
* Update project files to use `<Reference>`
* Update targets to build NPM packages
* Update BuildTools to support custom 'restore' and 'test' targets
2019-01-09 13:12:35 -08:00
David Fowler b12c33dbc3
Replace ManualResetEvents with a TCS (#6173)
* Added TaskCreationOptions.RunContinuationsAsynchronously in a few places
2018-12-29 08:24:57 -04:00
Justin Kotalik e8d898dcd9 Merge branch 'release/2.2' 2018-12-26 14:21:28 -08:00
Nate McMaster f4bb260444
Merge branch release/2.1 into release/2.2 2018-12-21 13:24:33 -08:00
Ajay Bhargav Baaskaran 2a696f6a1c Merge branch 'release/2.2' 2018-12-18 12:50:01 -08:00
Ajay Bhargav Baaskaran 647fbbac7b Merge branch 'release/2.1' into ajbaaska/merge-22 2018-12-17 18:40:08 -08:00
John Luo b8db81898a Merge remote-tracking branch 'origin/master' 2018-12-17 15:16:21 -08:00
John Luo 98636fed6d Merge branch 'release/2.2' 2018-12-17 14:24:30 -08:00
Doug Bunting e717a8443e
Update last few projects to target `netcoreapp3.0` (#5128)
- #3754
- remove `#if`'s for multi-targeting where source is never multi-targeted
- left `StreamPipeReader`, `StreamPipeWriter` and their test classes alone because they're moving to CoreFx
2018-12-17 12:30:28 -08:00
Ajay Bhargav Baaskaran b079041488
Convert Razor to use Reference (#4954) 2018-12-17 12:10:40 -08:00
Nate McMaster 02780bac26
Merge branch 'release/2.2' 2018-12-14 16:03:16 -08:00
John Luo f659f0acce Merge branch 'release/2.1' into release/2.2 2018-12-14 15:48:40 -08:00
Justin Kotalik 68f0aff144 security to project ref (#4618) 2018-12-14 14:49:23 -08:00
Nate McMaster feb641ddda
Merge branch 'release/2.1' into release/2.2 2018-12-14 09:44:15 -08:00
Nate McMaster 13ae0057fb
Replace .Sources package references with local code (#4801)
Replaces package references the following packages with local code:

Microsoft.AspNetCore.Certificates.Generation.Sources
Microsoft.Extensions.CopyOnWriteDictionary.Sources 
Microsoft.Extensions.ClosedGenericMatcher.Sources
Microsoft.Extensions.ObjectMethodExecutor.Sources
Microsoft.Extensions.PropertyActivator.Sources
Microsoft.Extensions.PropertyHelper.Sources
Microsoft.Extensions.RazorViews.Sources
Microsoft.Extensions.SecurityHelper.Sources
Microsoft.Extensions.StackTrace.Sources
Microsoft.Extensions.WebEncoders.Sources
2018-12-14 08:36:36 -08:00
Nate McMaster 790ae47055
Merge branch 'release/2.2' 2018-12-10 17:41:14 -08:00
Nate McMaster 2c4d586809
Merge 'release/2.1' into release/2.2
Co-authored-by: Justin Kotalik <jukotali@microsoft.com>
2018-12-10 15:56:42 -08:00
Justin Kotalik 5f843f223e
[2.1] Make Antiforgery, ResponseCaching, StaticFiles, and Cors use project ref (#4484) 2018-12-07 14:30:50 -08:00
Nate McMaster df3e4b98a2
Backport infrastructure cleanup to share sources and project references (#4513) 2018-12-07 12:20:13 -08:00
Nate McMaster 85e2147ff0
Merge branch 'release/2.2' 2018-12-07 11:11:44 -08:00
Nate McMaster 569a14ecb8
Merge branch 'release/2.1' into release/2.2 2018-12-06 17:09:03 -08:00
Justin Kotalik 736083c2f2
Make HttpSys use ProjectReference and move HttpSys into Servers folder (#4335) 2018-12-04 16:44:07 -08:00
Nate McMaster 006558c412
Merge branch 'release/2.2' 2018-11-21 16:28:33 -08:00
Nate McMaster 71b1bb8887
Merge branch 'release/2.1' into release/2.2 2018-11-21 14:57:45 -08:00
Nate McMaster 6a6c88d80b
Merge source code from aspnet/Diagnostics to this repo 2018-11-21 14:23:55 -08:00
Nate McMaster ed0b421695
Reorganize source code in preparation to move into aspnet/AspNetCore
Prior to reorganization, this source code was found in 8893337fb0
2018-11-21 13:54:33 -08:00
Nate McMaster 7ffbb6ed3b
Continue to produce .Sources packages from src/Shared/
This is temporary until we finish replacing internal PackageReference with project ref
2018-11-20 11:01:36 -08:00
Nate McMaster 0af21077c3
Update .NET Core 3.0 dependencies (#4091)
* Update Extensions to 3.0.0-preview.18569.14
* Update EFCore to 3.0.0-preview.18569.2
* Update corefx to preview.18566.8
* Update core-setup to preview-27117-01
2018-11-20 10:37:13 -08:00
Nate McMaster feb68c1382
Merge branch 'release/2.1' into release/2.2 2018-11-20 09:13:55 -08:00
Nate McMaster dc4c18e9ac
Merge source code from aspnet/Hosting into this repo 2018-11-19 22:02:46 -08:00
Nate McMaster 87ab2744ce
Reorganize source code in preparation to move into aspnet/AspNetCore
Prior to reorganization, this source code was found in 3f7ee338d4
2018-11-19 22:00:32 -08:00
Nate McMaster 406c80bfb0
Reorganize source code in preparation to move into aspnet/AspNetCore
Prior to reorganization, this source code was found in 4598850c20
2018-11-14 16:03:24 -08:00
Nate McMaster 9d0c69f277
Reorganize source code in preparation to move into aspnet/AspNetCore
Prior to reorganization, this source code was found in 15ba5065a9
2018-11-14 11:18:51 -08:00