* Enable NuGet Audit and fix issues (#107639)
* Enable NuGet Audit and fix issues
Microsoft.NET.HostModel can reference the live builds of the packages
it depends on. These will be deployed by the SDK.
Most other audit alerts were due to tasks pulling in old dependencies
that aren't even used by the task. Avoid these by cherry-picking
just the assemblies needed by the tasks and provided by MSBuild / SDK.
This prevents NuGet from downloading the package closure with the
vulnerable packages. We don't need those packages since the tasks
aren't responsible for deploying them. A better solution in the future
would be a targeting pack for MSBuild and the .NET SDK - so that
components that contribute to these hosts have a surface area they can
target without taking on responsibility for servicing.
There is once case where we have a test that references NuGet.* packages
which also bring in stale dependencies that overlap with framework
assemblies. Avoid these by cherry-picking the NuGet packages in the
same way.
* Fix package path on linux
* Only use live JSON from HostModel
SDK pins S.R.M and a few others, so don't make them upgrade yet.
* Add a couple missing assembly references
* Refactor tasks dependencies
Consolidate representation of msbuild-provided task dependencies
* Fix audit warnings in tests
* Remove MetadataLoadContext from WasmAppBuilder package
* Update Analyzer.Testing packages
* Reduce exposure of Microsoft.Build.Tasks.Core
* Fix audit warnings that only occur on browser
* Update Asn1 used by linker analyzer tests
* React to breaking change in analyzer test SDK
* Enable working DryIoc tests
* Fix double-write when LibrariesConfiguration differs from Configuration
* Fix LibrariesConfiguration update target
* Clean up references and add comments.
* Make HostModel references private
This ensures projects referenced will not be rebuilt by tests.
This also means the HostModel package will not list these as references,
but that's OK since the SDK provides them and this is not a shipping
package.
* Use ProjectReferenceExclusion to avoid framework project references
On .NETCore we want to use the targeting pack and avoid rebuilding libs.
* Update src/libraries/System.Runtime.InteropServices.JavaScript/tests/JSImportGenerator.UnitTest/JSImportGenerator.Unit.Tests.csproj
Co-authored-by: Jeremy Koritzinsky <jkoritzinsky@gmail.com>
---------
Co-authored-by: Jeremy Koritzinsky <jkoritzinsky@gmail.com>
* Remove live System.Text.Json reference from HostModel (#108263)
* Reduce changes to src/installer
Since we're no longer trying to reference live S.T.J we don't need these.
* Update JSON toolset version
* Don't error for NuGet audit on non-official builds (#108718)
* Reference live S.T.JSON from DI.ExternalContainers.Tests
* Update STJ in Wasm.Build.Tests
* Make SystemTextJsonToolsetVersion 8.0.4
We cannot count on VS and MSBuild updating by the time 9.0 ships GA.
Fix WASM projects which only target .NET by referencing the LKG and dropping all assets.
For Microsoft.NET.HostModel and other build tasks, keep them on the version we can garuntee is present in VS. NoWarn the Audit warnings here. This is safe because we can ensure one of two things.
1. The package is non-shipping and customers won't see the warning and the referencing repo in the product will ensure an update or exclusion of the dependency. (HostModel)
2. The project excludes the reference entirely as making it PrivateAssets (not in package) and ExcludeAssets=runtime (no possibility of using runtime).
* Fix STJ audit warning in installer tests
---------
Co-authored-by: Jeremy Koritzinsky <jkoritzinsky@gmail.com>
* Remove package references from library tests (#106737)
* Remove package references from library tests
These tests should be referencing the product assemblies so that they
test latest and not old bits.
* Reference the OOB version of SRSF and make sure it's copied
* BinaryFormatter tests should be skipped only on AOT, WASM and Mobile (#106858)
* respect AppContext switch (which is currently enabled for all projects in the root Directory.Build.props file)
* add project reference to all test projects that need working BF (and were being skipped for a while)
* adjust to changes from #104202: EqualityComparer<string>.Default is mapped to StringEqualityComparer, but serialized as GenericEqualityComparer<string>
* Don't use WeakReferences in the round trip test, as the target may get freed in the meantime, fixes#104905 (#106967)
* Enable more BinaryFormatter tests (#107408)
* enable the BinaryFormatter tests in System.Runtime.Serialization.Formatters.Tests
* add new test project, where the flag is disabled and it runs only 3 tests in total that ensure that
* The SerializationGuard is no longer activated since BF was moved to the OOB package, the tests need to reflect that.
* Disable binary formatter tests when DotNetBuildSourceOnly. (#107549)
* [mono][tvos] Do not treat assembly.pdb/xml files as native files to bundle when AOTing on Helix (#107079)
* Do not treat assembly.pdb/xml files as native files to bundle
* Bundle satellite assemblies as well
* [mono][ci] Include PDBs from runtime pack when building on Helix if required (#107348)
---------
Co-authored-by: Eric StJohn <ericstj@microsoft.com>
Co-authored-by: Tom Deseyn <tom.deseyn@gmail.com>
Co-authored-by: Ivan Povazan <55002338+ivanpovazan@users.noreply.github.com>
* Fix wrong default for UseCompilerGeneratedDocXmlFile property
* Suppress undocumented public APIs that went in since the switch got a wrong default
* PR feedback (add warning)
* React to System.Speech not having an intellisense xml file
* Update Directory.Build.props
* Update intellisense.targets
* Remove property from private assembly
* Suppress PNSE doc error for System.Speech
This replaces our dependency of madler/zlib and zlib-intel with a dependency to an in-tree copy of zlib-ng, which will be used across coreclr, mono and nativeaot.
Mobile platforms (android, tvos, ios, maccatalyst) and armv6 are excluded from using the in-tree zlib-ng copy, and they will find and use the system-installed zlib instead.
---------
Co-authored-by: Jeremy Koritzinsky <jekoritz@microsoft.com>
* Delete DebugSymbols
This property does not do what its name says. The symbols are generated regardless of whether this property is true or false. What this property actually does is that it disables C# peephole IL optimizations.
This change results in ~0.5% IL binary size improvement thanks to the Roslyn IL peephole optimizations that it enables.
* Update eng/illink.targets
Co-authored-by: Michal Strehovský <MichalStrehovsky@users.noreply.github.com>
---------
Co-authored-by: Michal Strehovský <MichalStrehovsky@users.noreply.github.com>
Not everything is passing, so I baselined this. Some we'll probably exclude permanently, others are more concerning and we need to determine if it's test issues or product issues.
* Allow using VS to run host tests when libraries use different config
When building the host tests from VS, there are dependencies on projects from the libraries subset. If the libraries are prebuilt with different configuration (typically `Release`) the build in VS breaks because it can't find the project's output.
This change modifies the import of generators (which are the problematic projects) to let them override the configuration of the referenced projects.
It then adds the ability to set environment based on the `-lc` parameter to the build script when starting VS.
Also updates the doc to describe how to use this.
* Update eng/build.ps1
Co-authored-by: Elinor Fung <elfung@microsoft.com>
* Update eng/generators.targets
* Move ShouldUnsetParentConfigurationAndPlatform to repo's Directory.Build.props
---------
Co-authored-by: Elinor Fung <elfung@microsoft.com>
* [wasm/wasi] Consolidate build targets
Current state of build files:
```
wasm: WasmApp.props, WasmApp.targets, WasmApp.Native.targets
wasi: WasiApp.props, WasiApp.targets, WasiApp.Native.targets
``
The wasm, and wasi build have lot of shared code but that is not
representative in the actual files, since the wasi targets started life
as a quick-copy-comment-out-bits of the wasm targets.
This commit consolidates these into:
```
common: WasmApp.Common.props, WasmApp.Common.targets
wasm : WasmApp.props, WasmApp.targets
wasi : WasiApp.props, WasiApp.targets
```
## `WasmApp.Common.{props,targets}`
This has all the common parts of the build for browser-wasm, and wasi,
and includes bits from `WasmApp.{props,targets}`, and
`WasmApp.Native.{props,targets}`.
- The top level target remains the same - `WasmBuildApp`.
- There are a few "public" targets that can be hooked into:
- `PrepareInputsForWasmBuild`
- `WasmGenerateAppBundle`
- `PrepareForWasmBuildNative`
- `WasmLinkDotNet`
- all these public targets have corresponding `*DependsOn` properties
which can be used for extending the build
note: this commit does not add a public target for AOT, but it might be
added in future.
## WasmApp.{props,targets}
This is for `browser-wasm` projects. The file might be renamed in
future.
## WasiApp.{props,targets}
This is for `wasi-wasm` projects. `ILStrip` becomes usable as a feature
for `wasi-wasm` because of this consolidation.
* [wasi] WBT: Use TestOutputWrapper
* [wasm] WBT: Track target name changes
* [wasi] WBT: Add new tests
* cleanup
* cleanup
* address feedback
* address feedback
* Add XUnitLogChecker to log libraries dumps (#93906)
Libraries CI failures that generate dumps will now show the dump output in the console log itself.
* Change NativeAOT condition
* Pass the global property set to false in the post build step.
* Address suggestion:
- Avoid using yml extraHelixArguments.
- Add the new embedded SetCommands optional section in runner scripts.
- Set __IsXUnitLogCheckerSupported in tests.targets as SetScriptCommand items instead of directly in sendtohelixhelp.proj.
* Only add XUnitLogChecker as HelixPayload if its directory is found. Whether it was built or not has already been decided before, and the runner scripts know when to execute it.
* Condition the use of NetCoreAppPrevious TFM
NuGet doesn't support duplicate TFMs in the TargetFrameworks string.
Condition the use of NetCoreAppPrevious TFMs until NuGet supports that
(which is planned afaik).
* Fix ODBC project TFM
* Move portable RID graph into runtime and clean-up
1. Move portable RID graph into runtime
2. Allow updates to both the non-portable and portable RID graphs under
source build.
3. Clean-up project and remove hacks
* Update README and delete test
* Fix RID graph update when the key already exists
* Update src/libraries/Microsoft.NETCore.Platforms/readme.md
Co-authored-by: Jan Kotas <jkotas@microsoft.com>
* Update src/libraries/Microsoft.NETCore.Platforms/readme.md
Co-authored-by: Andy Gocke <angocke@microsoft.com>
---------
Co-authored-by: Jan Kotas <jkotas@microsoft.com>
Co-authored-by: Andy Gocke <angocke@microsoft.com>
* Use live illink
* Include ILLink.Tasks for subsets that depend on it
Also prevent build-native.proj from importing illink.targets.
* Address feedback
- Don't reference ILLink.Tasks from subsets
- Use PackageReference from illink.tasks
- Import props from source directory
Add extensibility to shipping targets to make this possible.
- Set IsSourceProject false for .proj files
* Fix TFM, Configuration, PrivateAssets
* Address feedback
Avoid needing to reorganize SDK imports
* Fix typo
* Fix configuration in ToolsILLinkDir
* Don't change IsSourceProject
* Update eng/illink.targets
Co-authored-by: Viktor Hofer <viktor.hofer@microsoft.com>
* Fix tools configuration
* Normalize directory separator chars
We agreed on (at least under src/libraries) to use the Windows directory separator char `\` instead of a forward slash. This helps when looking at binlogs as those then correctly display a path.
---------
Co-authored-by: Viktor Hofer <viktor.hofer@microsoft.com>
* Propagate TasksConfiguration when building tests
* Mark tasks projects as non-source projects and default to Release for CI/official builds(#84488)
* Update paths when copying to CORE_ROOT
* Flow TasksConfiguration in build & specify generator config
1. Flow the TasksConfiguration in the root build (Subsets.props)
2. Specify the LibrariesConfiguration explicitly when building the
LibraryImport source generators.
3. Set tasks configuration to debug by default.
* Include the Subsets.props change
* Update tasks.proj
---------
Co-authored-by: Elinor Fung <elfung@microsoft.com>
* Fixes https://github.com/dotnet/runtime/issues/42961
Enable strict mode comparison during servicing when
`ApiCompatNetCoreAppBaselineVersion` is the current .NET version. When
the GA targeting pack is released, that property should be updated which
will turn on strict mode comparison and prohibits the addition of new
API.
Suppressing one error in the baseline file that showed-up because
out-of-band assemblies now being passed in as assembly references.
While at it I also cleaned-up how private assemblies are handled during
the build.
* PR feedback
Since we bumped to NDK 23, having the aot compiler itself generate shared libraries stopped working. This is due to NDK 23 moving most of the toolchain into a common bin directory. AS was left over in each of these directories as an alias to bin/<triple>-as.
This change adds a task to collect all of the important NDK toolchain paths. It also fixes up the aot build when AOTWithLibraryFiles is set to true and we want the aot compiler to produce shared libraries.
Fixes https://github.com/dotnet/runtime/issues/87585
When looking at the 8.0 Preview 5 SDK, xml files are missing for certain
assemblies. This is likely because of the TargetFramework condition in
intellisense.targets. Instead of conditioning on the rid-less net8.0
TFM, use the sfx.proj which already assemblies the shared framework and
targeting pack layout to copy the documentation files.
Additionally, remove the XmlDocDir and copy the files into the targeting
pack folder which was a difference between the "real" targeting pack and
the "fake" one that libraries generates.
Doesn't actually work E2E because the inline threadstatics work broke our Android support (the one added in https://github.com/dotnet/corert/pull/8323).
```
C:\Users\michals\.nuget\packages\microsoft.netcore.app.runtime.nativeaot.linux-bionic-arm64\8.0.0-preview.6.23307.4/r
untimes/linux-bionic-arm64/\native\libRuntime.WorkstationGC.a(MiscStubs.S.o): In function `RhpGetInlinedThreadStaticB
ase':
/__w/1/s/src/coreclr/nativeaot/Runtime/arm64/MiscStubs.S:12: undefined reference to `tls_InlinedThreadStatics'
/__w/1/s/src/coreclr/nativeaot/Runtime/arm64/MiscStubs.S:12: undefined reference to `tls_InlinedThreadStatics'
/__w/1/s/src/coreclr/nativeaot/Runtime/arm64/MiscStubs.S:12: undefined reference to `tls_InlinedThreadStatics'
/__w/1/s/src/coreclr/nativeaot/Runtime/arm64/MiscStubs.S:12: undefined reference to `tls_InlinedThreadStatics'
```
* Don't enable ILLink analyzers for unsupported TFMs
This adds a condition to EnableSingleFileAnalyzer to prevent it from requiring
the ILLink pack that is only supported on netcoreapp3.0 and above.
Adding a condition on the TFM requires moving the setting into a targets file.
Also consolidates more of the analyzer logic into this file.
---------
Co-authored-by: Viktor Hofer <viktor.hofer@microsoft.com>
* Haiku: Initial configuration support
Initially recognize Haiku as a supported platform in
Directory.Build.props and all scripts in eng/**.
* Drop some unused configure values
Drop some unused configure values from tryrun.cmake,
configure.cmake and config.h.in.
* Simplify error message in build.sh
Simplified an error message for the `--os` option to avoid maintaining a duplicate list of OSes.
Co-authored-by: Adeel Mujahid <3840695+am11@users.noreply.github.com>
* Try to untangle the rid calculation.
* Ignore __portableOS.
* Add back musl detection.
* Try fix AnyOS tests.
* Try fix coreclr linux_musl_arm64 job.
* src/tests/build.proj: use PackageRID for restore.
* Use RuntimeOS as an override for _packageOS.
* Remove (probably) out-dated override for non-portable ToolsRID.
* CI cross-builds for linux-musl use linux.
* Add missing arch to rid.
* Try simplified ToolsRID.
* Update comments.
* Rename OutputRid to OutputRID.
* Tweak comment.
* Move TargetsLinux{Bionic,Musl} with the other Targets properties.
* Don't use RuntimeOS for PackageRID for CrossBuilds.
* Move RuntimeOS condition to ToolsRID assignment.
* Clean up a few things.
* Fix broken Condition.
* Try fix the linux-bionic build.
* Try fix the linux-bionic build, part II.
* Try fix the linux-bionic build, part III.
* Fix PackageRID.
* Use RuntimeOS as _packageOS override.
* Remove RuntimeOS.
* Fix '-os linux-musl' mapping.
* Clean up .
* Rename __PortableOS to __PortableTargetOS.
* Extend comments.
* Extend OutputRID comment too.
* build.sh: include linux-bionic, linux-musl in usage.
* SourceBuild.props: no need to pass RuntimeOS. Only set ToolsOS, PackageOS when RuntimeOS is not empty.
* Fix comment.
* linux-bionic doesn't get built on linux-bionic.
* Unconditionally assign TargetsLinuxBionic/TargetsLinuxMusl.
* Remove SkipInferTargetOSName.
- Move BinaryFormatter obsoletions to type-level
- Introduce SYSLIB0050 and SYSLIB0051 obsoletion waves
- Remove some dead code
Ref: https://github.com/dotnet/designs/pull/293
