Commit Graph

8 Commits

Author SHA1 Message Date
Ajay Bhargav B 24be2992de
Run Blazor E2E tests on SauceLabs (#18456)
* Run Blazor E2E tests on SauceLabs

* Added azure pipeline

* update yml

* Update meta

* More changes
2020-01-27 17:54:18 -08:00
Javier Calvarro Nelson 73f969852b
[Templating] Infrastructure improvements (#13672)
* Automatically capture a screenshot when an assertion fails.
* Enable configurable options for different environments (CI|Dev).
* Include log errors in all exceptions.
* Add default timeout configurations and failure timeout configurations for CI and Dev environments.
2019-09-06 16:02:49 +02:00
Javier Calvarro Nelson 1480b99866
Enforce E2E test prerequisites when building individual solutions only (#11642)
* Fixes multiple issues that happened due to the E2E tests not running on the CI
* Enforces prerequisites for E2E tests on the CI and when building projects with E2E tests.
* Re-enables the E2E tests for templates.
2019-06-29 01:04:36 +02:00
Nate McMaster 4fde84a7ad Replace KoreBuild with Arcade (#11122)
* Replace usages of $(CI) with $(ContinuousIntegrationBuild)

* Move RepoTasks to eng/tools/

* Remove dead code

* Update build.sh/ps1 to invoke Arcade instead of KoreBuild

* build/repo.props => eng/Build.props

* build/CodeSign.{props,targets} => eng/Signing.props

* build/repo.targets => eng/Build.props, eng/CodeGen.proj, and eng/AfterSolutionBuild.targets

* Remove AddAllProjectRefsToSolution.ps1

* Put selenium process tracking into artifacts/tmp/

* build/docker => eng/docker

* Move content from korebuild.json to global.json

* Remove cibuild.{sh,cmd} and update ci.yml

* Ensure vswhere detects BuildTools installations

* Remove sign check exclusions (Arcade does not support)

* list VS required components in global.json

* Update CodeCheck.ps1

* Update arcade (#11143)

* Fixups to broken build

* capture test results in xunit form
* attempt to fix code check
* restore before linux build
* remove duplicate signinfos

* More build fixes

* exclude node_modules from unique project check
* fixup signing props

* More build fixes

* Remove unused NoWarns
* Skip building site extension
* Suppress xunit color in console output
* Install x86 runtime

* Run KillProcesses.ps1 at end of build on CI

* activate env vars in codecheck.ps1

* Fix signing and generate build manifests

* Fixup code check and linux installers build

* Remove custom restore targets

* Import flaky test props

* Restore original manifest generation code and set warnAsErrors=false

* Alwasy restore the toolset

* More attempts at build fixes

* run restore before building deb/rpm packages
* pre-install Yarn.MSBuild
* Remove WIX workaround

* Attempt to fix RPM installers and x86 build

* Fix helix tests, java auto-detection, and code gen steps

* Globally disable MSBuild node resuse and capture MSbuild crash logs

* Cleanup build invocation for installers

* Set max time on xunit tests to 15 min

* Update helix-test.yml

* Fix Hosting tests

* Fix TestHost

* Fix Negotiate.Test

* Fix Mvc tests

* Capture results

* Fix Hosting tests

* Fix Kestrel tests

* Fix ServerComparison tests

* Fix DefaultBuilder functional tests

* Skip template tests

Need to run pack before running tests on mac/linux now

* Fix Identity test

* Fix ServerComparison tests

* Verify nginx installation on ubuntu

* Verify nginx installation?

* Try adding to path

* Try symlink

* Sudo?

* Woot it works

* Disable timeout

Template tests require more than 15 mins

* list env vars

* Use correct dotnet_home

* Prepent path instead

* Fix npmproj tests

* Fix CORS tests

* Debug signalr tests

* Debug signalr

* Capture bin

* Try building templates sequentially

* Debug signalr

* Cleanup

* Disable signalr npm tests

* Debug template test

* Try to fix templates again and make steps more reliable

* Cleanup

* Feedback
2019-06-19 10:03:38 -07:00
John Luo f76be64e5d
Only run selenium tests when build java is specified (#11018) 2019-06-10 14:35:27 -07:00
Nate McMaster 77e08c26d5
Replace all usages of RepositoryRoot with RepoRoot (#10267)
Preparing to adopt the Arcade SDK, which uses the shorter variable name instead
2019-05-15 12:00:19 -07:00
Javier Calvarro Nelson 9f1a978230
[Infrastructure improvements] (#8275)
* Improved selenium start and tear down
  * Selenium is set up and torn down in an assembly fixture.
  * Selenium is initialized lazily and in a non-blocking way.
  * Selenium processes are tracked as part of the build and their pids
    written to a file on disk for cleanup in the event of unexpected
    termination of the test process.
  * Browser fixture retries with linear backoff to create a remote
    driver. Under heavy load (like when we are doing a simultaneous NPM
    restore) the selenium server can become unresponsive so we retry
    three times, with a longer comand timeout allowance each time up to
    a max of 3 minutes.
* Moved test project setup to build time instead of runtime.
  * Added target PrepareForTest to create the required files for testing
    * The template creation folder.
    * The template props file to use our built packages.
    * The folder for the custom hive.
  * Added assembly metadata attributes to find all the data we need to
    run the tests.
    * Path to the artifacts shipping packages folder.
    * Path to the artifacts non-shipping packages folder.
    * Path to the test templates creation folder.
    * Path to use for the custom templating hive used in tests.
  * Proper cleanup as part of the build
    * Remove the test templates creation folder.
    * Remove the test packages restore path.
    * Recreate the test templates creation folder.
    * Recreate the test packages restore path.
  * Generated Directory.Build.Props and Directory.Build.Targets in the
    test templates creation folder.
  * Cleaned up potentially stale templatetestsprops.
* Improved test flows
  * Initialization is done lazily and asynchronously.
    * Selenium
    * Browser fixture
    * Template initialization.
  * Flattened test flows to avoid assertions inside deep callstacks.
    * All assertions happen at the test level with improved error messages.
      * With the exception of the migrations assertions.
    * Assertions contain information about which step failed, for what
      project and what failure details.
  * Broke down tests to perform individual steps instead of mixing build
    and publish.
    * Publish project.
    * Build project. (Debug)
    * Run built project.
    * Run published project.
  * Concentrated build logic into the Project class.
    * Context between the different steps of a test is maintained in
      this class.
    * All operations that require coordination are performed within this
      class.
      * There is a lock for dotnet and a lock for nodejs. When building
        SPAs we acquire the nodejs lock to correctly prevent multiple
        runs of nodejs in parallel.

[ApiAuthorization template cleanups]
  * Fix preview3 issues with breaking changes on Entity framework by
    manually configuring the model in ApiAuthorizationDbContext.
  * Add app.db to the project file when using local db.
  * Fix linting errors on angular template.
  * Fix react tests
  * Add tests to cover new auth options in the SPA templates.
2019-03-20 08:44:20 +01:00
Javier Calvarro Nelson d7a9606040
[Templating][Components] Unify and improve E2E testing infrastructure (#8188)
* Unify the Templating and Components testing infrastructure.
* Enable test project discovery in the components E2E tests.
* Enable selectively disabling Selenium tests through build properties.
2019-03-11 18:40:18 +01:00