Follow-up on #32451
Follow-up on #80074
In #43651, the "Windows" OS name was lowercased to achieve target frameworks with RIDs in them. At that time, the original plan was to lowercase all TargetOS values but that had to be cut because of resources. This PR finishes that and updates build systems, YML, markdowns and managed components.
To not break existing developer workflows, the passed-in -os value is automatically lower-cased in the script entry points and validated in the msbuild entry point.
These fixes were built for PR #74886; however, as that PR is so utterly large and unreviewable, I've pulled out the test infra changes for separate review
Changes
- Increase the number of trampolines in the llvm aot compilation process to 20,000 from 10,000 (This avoids running out of them in some of the hardware intrinsics tests
- Add a concept of striping tests when running under GC stress
- To use this new feature, specify <NumberOfStripesToUseInStress>N</NumberOfStripesToUseInStress> within the merged test assembly. If this value is set, then the tests within that merged test assembly will be run across N different work items instead of 1 when running under any form of GC stress based scenario. At this moment the largest supported value of N is 99
- Emit the testresults.xml file as a file which is exported from the tests. This is useful for debugging testresult.xml parsing failures
- Fix the testresults summary generator to never emit an empty CDATA string. If one is present the parser may fail the parse.
- In the XUnitWrapperGenerator fix the implementation of the Outerloop and ActiveIssue when used with a conditional member.
- Add PlatformDetection.IsMonoLLVMAOT, PlatformDetection.IsMonoLLVMFULLAOT, and PlatformDetection.IsMonoInterpreter boolean properties to the PlatformDetection type for use with the ActiveIssue attribute
- Add some documentation about project files for coreclr tests, as well as some documentation on the command line parameters for merged test runner assemblies
* Quality Week 2022: The First Step to a Better Documentation.
* Removed the Markdown Extension's additional development tags.
* Removed redundant stuff.
* Removed redundant stuff.
* Removed guide testing versions to avoid confusion that they would be the only supported versions.
* Addressed some Powershell comments, added a pointer to a yaml with links to other .NET sibling repos, and did a couple mini-corrections. Still got a few comments to address...
* Fixed a ';:' with Powershell's syntax.
* Addressed review comments: Redaction improvements, localization url's, improved code snippets, added info on individual subsets building configurations, changed COMPlus for DOTNET, and other misc improvements.
* Addressed remaining comments.
* Addressed some last comments regarding corerun, and the subset flags in the build scripts.
Co-authored-by: Ivan Diaz <bluehorizon186@gmail.com>
* Add test
* Add CHECKs
* Build scripts and test settings
* COMPlus -> DOTNET
* Fix filenames
* Missing >
* Missing quotes
* use %scriptPath%
* Rework properties, start two examples
* Arch demo
* Cleanup
* Remove tab
* Easier environment variables. Undo precommand changes.
* undo blank line
* clean CHECKs
* Draft of bash
* Bash, conditionals
* More variables, start on run.cmd/sh/py
* another output
* Support in cmd/bash for RunningDisasmChecks
* copy, factor, formatting
* Initial work to include FileCheck. Added SuperFileCheck.
* Able to build SuperFileCheck
* Do not DisasmCheck TypeEquality_r for now. Update some FileChecks to follow SuperFileCheck rules.
* Partially wiring up SuperFileCheck to tests
* Piping list of method names from SuperFileCheck to JitDisasm
* Handling bash a little bit
* Moving SuperFileCheck to tests/Common
* Few tweaks
* Building SuperFileCheck as part of the test build
* Tweaking a few things
* Fixed a bug
* Moving SuperFileCheck back to src\coreclr\tools. Removed checks from TypeEquality_r.
* Restore original logic in Runtime_73681
* Trying to add CI leg for disasmchecks
* Use x64 package if x86 platform detected for JIT tools package
* Remove innerloop for disasmchecks
* Trying to fix build. Only run in Windows for now.
* Update Runtime_73681.cs
Trying to fail test
* Trying to fix build
* Update Runtime_73681.cs
* Update Runtime_73681.cs
* Fixing a few issues
* Trying to run disasmchecks as part of CI
* Trying to run disasmchecks
* Trying to run disasmchecks
* Trying to run disasmchecks
* Revert a change
* Trying to run disasmchecks
* Trying to run disasmchecks
* build SuperFileCheck on non-windows
* few tweaks
* Trying to fix CI
* Including SuperFileCheck for tests
* Cleanup
* More cleanup
* Cleanup
* Changed SuperFileCheck to not publish everything. Changed SuperFileCheck's lookup for FileCheck.
* Invoking SuperFileCheck using dotnet
* Making the test pass
* Only run disasm checks for coreclr and not mono
* Using HasBatchDisasmCheck and HasBashDisasmCheck to determine to run the check
* Enabling filecheck on linux and osx
* Added more comments
* Added ARM64 specific test. Do not run SuperFileCheck if no methods were found.
* Added documentation. Changed disasm-output.
* Minor doc tweak
* Minor doc tweak
* Minor doc tweak
* Minor doc tweak
* Minor doc tweak
* Cleanup. Trying to fix linux
* Fixing test
* Add information on additional functionality
* cleanup
* Add FileCheck snippet
* Undo environment variable changes
* Feedback from Mark
* Cleanup
* Trying to fix linux test run
* Trying to fix linux test run
* A few missing changes from the original branch
* Enable OSX for disasm checks
* cleanup / comment
* Force test failure
* Update Runtime_73681.cs
* Set env vars after errorlevel check
* Reverting back on setting environment variables in test. Added new FileCheck test for mod optimization
* Force a failure by changing the register
* Ignore native binaries for superpmi collect
* Update Runtime_34937.cs
* Force the correct failure
* Update Runtime_34937.cs
* Update Runtime_34937.cs
* Adding specific OS check prefixes. Changed dump-input context amount
* Added getting fully qualified method names with wildcards for SuperFileCheck
* More tests. Fixed a few issues with generics.
* Disabling generic support
* Error if it cannot find enclosing type declaration
* Fixing build
* Remove namespac
* Bring generics back, but in a limited form
Co-authored-by: Mark Plesko <markples@microsoft.com>
* Use CMakeProjectReference instead of ProjectReference
Changing this permits usage of "\runtime\dotnet.cmd build" as opposed
to forcing users to use "\runtime\dotnet.cmd msbuild".
This change adds four new options to the runtime test build command:
1) test <test project path relative to src\tests> - build just that
one test.
2) dir <directory relative to src\tests> - build all tests in the
directory.
3) tree <directory relative to src\tests> - build all tests in the
given subtree.
4) all - use clean rebuild (i.e. don't apply incrementalism)
when building the tests.
Thanks
Tomas
* Build IJW assemblies against LKG SDK ref pack.
* Allow building against the live ref assemblies by passing `-cmakeargs -DCPP_CLI_LIVE_REF_ASSEMBLIES=1` to the test build script.
* Add test instructions
* Error handling
* Add support for the stdcall-mangled _CorDllMain entrypoint on x86 since MSVC uses it if it exists.
* Write out ref-pack path.
* Fix various typos
* `occurence`, `occurance` -> `occurrence`
* `accross` -> `across`
* `adddress`, `addresss` -> `address`
* `alignement`, `aligment` -> `alignment`
* Remove `c` from triple c.
* Remove `s` from triple s.
* In palsuite, `Successs` (with triple s) was in commented out
`Trace()` message, removed all commented out `Trace()` calls
from such files.
* Fix invalid markdown in `unix-test-instructions.md`.
* Revert event source change
* Move the script runtesttilstable.sh under src/tests
* Update most references to build-test / runtest in the runtime repo
With this change I can't find any remaining orphaned references to
the old script names except for a few superpmi-related scripts
I need to follow up on separately as I'm not too familiar with
their functionality.
Thanks
Tomas
Refactor the PAL test framework to be useable in our CI system, and enable it for testing
The existing PAL tests are problematic in a few ways for our CI.
1. They are large in terms of disk space used per test, this interferes with building them properly, especially as they are part of the product build, and not of the test build
2. While part of the product build, the option to build them was well hidden.
3. The tests are not related to our existing tests, and in particular are not driven from managed xunit wrappers.
The change here has 4 components
1. Reduce the volume of the binaries such that they can be passed from a product build to a test job effectively. This is done by merging all tests which are not strictly dependent on secondary libraries.
- This was done via a tool which parsed and replaced all of the CMakeFiles.txt
- And then manual editing to remove all duplicate global symbols
2. The build flag for building the tests has been exposed as a first class option to build-runtime, as well as through the subset mechanism. As of this change, developers who wish to build the tests should use the clr.paltests subset to build the tests.
3. As the tests do not have normal xunit wrappers, but they *do* have a script which will generate xunit output, they cannot quite be run in the existing helix pathway. Add a separate path for launch the appropriate helix workitem as part of outerloop test runs.
4. A new issue exclusion mechanism has been built. Add exclusions by modifying src/coreclr/src/pal/tests/palsuite/issues.targets
- There are a number of failures in the current test suite that should probably be investigated, but making meaningful changes to the tests should not be part of this change.
* Trim trailing whitespaces
* Match raw with rendered
* Delete extra asterisks and |
* Update ELT Hooks - tail calls.md
Co-authored-by: Jan Kotas <jkotas@microsoft.com>
* Build all managed coreclr tests on OSX in CI (#36253)
Remove concept of targetGeneric and targetSpecific
Add OSX corelcr managed tests builds to all pipes
Remove all other platform coreclr managed test build
Remove property `TestUnsupportedOutsideWindows`
Rename conditional `DisableProjectBuild` to `CLRTestTargetUnsupported`
Refactor tests to allow all managed tests to be built on OSX.
Split managed tests which based on target properties conditionally:
+ `<Compile Include/>`
+ `<DefineConstant/>`
This creates a separate test for each target configuration permutation.
Add `<CLRTestTargetUnsupported/>` conditions to select these at build and/or
run time
Append split tests with `_Target*` to identify intended test target
For tests which depend on target specific internal details od System.Private.Corlib,
expose a dummy implementation for unsupported targets.
Clean up
Remove <TraitTags/>
Remove <CLRTestNeedTarget/>
Remove managedOSXBuild
Remove managedTestBuildOsGroup
Remove managedTestBuildOsSubGroup
Use issues.target to disable expected failures
Some tests are configured to run on a targets with
specific charecteristics.
Use issues.targets to conditionally disable tests
not intended to run on all targets.
* Setup pr and batch runs for dev/infrsatructure (#36218) (#36299)
Co-authored-by: Jarret Shook <jashoo@microsoft.com>
* Add alias CoreClrTestBuildHost (#36256)
* Add alias CoreClrTestBuildHost
Intended to allow easy switching of test build host between OSX_x64 & Linux_x64
depending on availablility and reliability.
* Add missing issues.targets from #36253 (#36351)
* Add missing issues.targets from #36253
PR #36253 enabled building all tests on OSX. It was intended to include this list of
tests to disable on unsupported platforms. This was dropped as part of rebasing for
dev/infra branch.
Lack of CI on dev/infra allowed this to be missed.
* Revise issues.targets
* Add src/coreclr/tests/src/baseservices/typeequivalence/simple
* Remove #if Windows from DllImportPathTest
* Build scripts while copying native binaries (#36482)
* Build scripts while copying native binaries
Stop building test scripts during the managed build phase
Build after copying the native targets
Fixes issues relative to building scripts on OSX
* Fix superpmi script generation
* Cleanup issues.targets
* Keep 6 failing IJW tests
* Fix IJW when managed tests are built on OSX (#36711)
* Fix IJW when managed tests are built on OSX
- Refactor Interop.setting.targets to Directory.Build.{targets,props}
- Remove include Interop.setting.targets from interop projects rely on SDK including
Directory.Build.* automatically
- Add new CopyInteropNativeRuntimeDependencies target for copying IJW dependencies
* Add <Copy/> task
- Delete obsolete instruction in Interop/ReadMe.md
* Cleanup Readme.md
* Optional stress dependencies & don't populate CORE_ROOT twice (#36851)
Bruce noticed that we're downloading and installing stress
runtime dependencies several times during build. I have added a new
option to suppress this behavior in build-test and I fixed the
invocations of build-test based on Bruce's analysis. I have also
patched the build scripts to avoid populating CORE_ROOT in
"copynativeonly" mode.
Thanks
Tomas
Fixes: #36797
* Build test scripts whenever CopyNativeProjectBinaries builds (#37028)
* Fix pri0 test build.
* Clean pri1 test build (#37266)
* Clean pri1 test build
* Undo testing comment
* Make sure weakreferencetest does not run on unix
* Fixup WinRT proj
* Fix path
Co-authored-by: Steve MacLean <Steve.MacLean@microsoft.com>
Co-authored-by: Tomáš Rylek <trylek@microsoft.com>
Co-authored-by: Santiago Fernandez Madero <safern@microsoft.com>
Co-authored-by: Alexander Köplinger <alex.koeplinger@outlook.com>
Co-authored-by: dotnet-maestro[bot] <42748379+dotnet-maestro[bot]@users.noreply.github.com>
Co-authored-by: dotnet-maestro[bot] <dotnet-maestro[bot]@users.noreply.github.com>
Co-authored-by: Andy Ayers <andya@microsoft.com>
Co-authored-by: Adeel Mujahid <adeelbm@outlook.com>
Co-authored-by: Ganbarukamo41 <ganbarukamo@gmail.com>
Co-authored-by: monojenkins <jo.shields+jenkins@xamarin.com>
Co-authored-by: thaystg <thaystg@users.noreply.github.com>
Co-authored-by: Viktor Hofer <viktor.hofer@microsoft.com>
Co-authored-by: Aaron Robinson <arobins@microsoft.com>
Co-authored-by: Kevin Jones <kevin@vcsjones.com>
Co-authored-by: Jeremy Barton <jbarton@microsoft.com>
Co-authored-by: Egor Bogatov <egorbo@gmail.com>
Co-authored-by: Simon Nattress <nattress@gmail.com>
Co-authored-by: Aleksey Kliger (λgeek) <alklig@microsoft.com>
Co-authored-by: Krzysztof Wicher <kwicher@microsoft.com>
Co-authored-by: Kunal Pathak <Kunal.Pathak@microsoft.com>
Co-authored-by: Carol Eidt <carol.eidt@microsoft.com>
Co-authored-by: Shimmy <2716316+weitzhandler@users.noreply.github.com>
Co-authored-by: Stephen Toub <stoub@microsoft.com>
Co-authored-by: Layomi Akinrinade <laakinri@microsoft.com>
Co-authored-by: Dan Moseley <danmose@microsoft.com>
Co-authored-by: Eirik Tsarpalis <eirik.tsarpalis@gmail.com>
Co-authored-by: Jeremy Koritzinsky <jekoritz@microsoft.com>
Co-authored-by: Jose Perez Rodriguez <joperezr@microsoft.com>
Co-authored-by: Alexander Chermyanin <flamencist@mail.ru>
Co-authored-by: Miha Zupan <mihazupan.zupan1@gmail.com>
Co-authored-by: Sergey Andreenko <seandree@microsoft.com>
Co-authored-by: Nikola Milosavljevic <nikolam@microsoft.com>
Co-authored-by: Steve Pfister <steveisok@users.noreply.github.com>
Co-authored-by: David Wrighton <davidwr@microsoft.com>
Co-authored-by: buyaa-n <bunamnan@microsoft.com>
Co-authored-by: Krzysztof Wicher <mordotymoja@gmail.com>
Co-authored-by: Vladimir Sadov <vsadov@microsoft.com>
Co-authored-by: Leandro Pereira <leandro.pereira@microsoft.com>
Co-authored-by: Swaroop Sridhar <swaroop.sridhar@microsoft.com>
Co-authored-by: Marie Píchová <11718369+ManickaP@users.noreply.github.com>
Co-authored-by: Michal Strehovský <MichalStrehovsky@users.noreply.github.com>
Co-authored-by: Manish Godse <61718172+mangod9@users.noreply.github.com>
Co-authored-by: Bruce Forstall <brucefo@microsoft.com>
Co-authored-by: Alexis Christoforides <alexis@thenull.net>
Co-authored-by: BrzVlad <BrzVlad@users.noreply.github.com>
Co-authored-by: Alfred Myers <git@alfredmyers.com>
Co-authored-by: Nick Craver <nrcraver@gmail.com>
Co-authored-by: Nathan Ricci <naricc@microsoft.com>
Co-authored-by: Thays Grazia <thaystg@gmail.com>
Co-authored-by: Yoh Deadfall <yoh.deadfall@hotmail.com>
Co-authored-by: Eric Erhardt <eric.erhardt@microsoft.com>
Co-authored-by: Jo Shields <directhex@apebox.org>
Co-authored-by: Jan Vorlicek <janvorli@microsoft.com>
Co-authored-by: Anirudh Agnihotry <anirudhagnihotry098@gmail.com>
Co-authored-by: Roman Marusyk <Marusyk@users.noreply.github.com>
Co-authored-by: Zoltan Varga <vargaz@gmail.com>
Co-authored-by: Youssef Victor <31348972+Youssef1313@users.noreply.github.com>
Co-authored-by: Ben Adams <thundercat@illyriad.co.uk>
Co-authored-by: Marek Safar <marek.safar@gmail.com>
Co-authored-by: Anton Lapounov <antonl@microsoft.com>
Co-authored-by: AraHaan <15173749+AraHaan@users.noreply.github.com>
Co-authored-by: Tom Deseyn <tom.deseyn@gmail.com>
Co-authored-by: Tomas Weinfurt <tweinfurt@yahoo.com>
Co-authored-by: Tomas Weinfurt <furt@Shining.local>
Co-authored-by: David Cantu <dacantu@microsoft.com>
Co-authored-by: Sung Yoon Whang <suwhang@microsoft.com>
Co-authored-by: Levi Broderick <GrabYourPitchforks@users.noreply.github.com>
Co-authored-by: vargaz <vargaz@users.noreply.github.com>
Co-authored-by: David Mason <davmason@microsoft.com>
Co-authored-by: UnityAlex <UnityAlex@users.noreply.github.com>
Co-authored-by: Fan Yang <52458914+fanyang-mono@users.noreply.github.com>
Co-authored-by: Tanner Gooding <tagoo@outlook.com>
Co-authored-by: Egor Chesakov <Egor.Chesakov@microsoft.com>
Co-authored-by: radical <radical@users.noreply.github.com>
Co-authored-by: Ivan Diaz Sanchez <ivdiazsa@microsoft.com>
Co-authored-by: Gleb Balykov <g.balykov@samsung.com>
Co-authored-by: Honza Rameš <ramejan@gmail.com>
Co-authored-by: Eric StJohn <ericstj@microsoft.com>
Co-authored-by: Ivan <ivanbuha@outlook.com>
Co-authored-by: Ryan Lucia <rylucia@microsoft.com>
Co-authored-by: SRV <roman.sakno@gmail.com>
Co-authored-by: Clinton Ingram <clinton.ingram@outlook.com>
Co-authored-by: Mikhail Pilin <ww898@users.noreply.github.com>
Co-authored-by: Jan Kotas <jkotas@microsoft.com>
Co-authored-by: w-flo <w-flo@users.noreply.github.com>
Co-authored-by: Peter Sollich <petersol@microsoft.com>
Co-authored-by: grendello <grendello@users.noreply.github.com>
Co-authored-by: Johan Lorensson <lateralusx.github@gmail.com>
Co-authored-by: Erhan Atesoglu <47518605+eanova@users.noreply.github.com>
Co-authored-by: Matt Mitchell <mmitche@microsoft.com>
Co-authored-by: Josh Schreuder <joshschreuder@gmail.com>
Co-authored-by: Josh Schreuder <josh.schreuder@gmail.com>
Co-authored-by: Andrew Au <andrewau@microsoft.com>
Co-authored-by: Eugene Rozenfeld <erozen@microsoft.com>
Co-authored-by: Vlad Brezae <brezaevlad@gmail.com>
Co-authored-by: Kenneth Pouncey <kjpou@pt.lu>
Co-authored-by: Kirill Frolov <k.frolov@samsung.com>
Co-authored-by: Tomas Weinfurt <furt@DESKTOP-0J9757E.corp.microsoft.com>
* Implicitly build when invoking the test target
To get closer to the VSTest behavior this changes the Test target to
implicitly invoke the Build target unless /p:TestNoBuild is passed in.
(VSTest uses VSTestNoBuild which is controlled by the
dotnet test --no-build flag)
* Fix error logging for test outputs
* Update dotnet msbuild to build docs
* Update ReportGenerator global tool version
Updating ReportGenerator's version to 4.5.0 which removes unnecessary
logging (args passed in).
* Enable Mono in default build
* Integrate Mono into the build system
* Fix System.Private.CoreLib build in Mono
Broken by 9c82a36c23
* Disable Windows test runs
Fails due to https://github.com/dotnet/runtime/issues/1933
Co-authored-by: Viktor Hofer <viktor.hofer@microsoft.com>
* Override the configuration of the CoreCLR and Libraries builds during the root build when CoreCLRConfiguration and LibrariesConfiguration are specified respectively.
* Add coreclrConfiguration and librariesConfiguration parameters to build scripts.
* Update docs.
* Make arguments lowercase on bash.
* Fix typos.
* Move files into hierarchy
* fix more links
* Fix botr and features links
* Remove bad apostrophe
* spelling
* Apply suggestions from code review
Co-Authored-By: Youssef Victor <31348972+Youssef1313@users.noreply.github.com>
* Update docs/coding-guidelines/package-projects.md
Co-Authored-By: Jan Kotas <jkotas@microsoft.com>
Co-authored-by: Youssef Victor <31348972+Youssef1313@users.noreply.github.com>
Co-authored-by: Jan Kotas <jkotas@microsoft.com>
