mirror of
https://github.com/VSadov/Satori.git
synced 2025-06-08 03:27:04 +09:00
Fix trailing whitespaces (#40891)
* Trim trailing whitespaces * Match raw with rendered * Delete extra asterisks and | * Update ELT Hooks - tail calls.md Co-authored-by: Jan Kotas <jkotas@microsoft.com>
This commit is contained in:
parent
d9f0ba704d
commit
d14b50ae21
141 changed files with 1128 additions and 1219 deletions
|
@ -6,8 +6,8 @@ The .NET Core and ASP.NET Core support policy, including supported versions can
|
||||||
|
|
||||||
## Reporting a Vulnerability
|
## Reporting a Vulnerability
|
||||||
|
|
||||||
Security issues and bugs should be reported privately to the Microsoft Security Response Center (MSRC), either by emailing secure@microsoft.com or via the portal at https://msrc.microsoft.com.
|
Security issues and bugs should be reported privately to the Microsoft Security Response Center (MSRC), either by emailing secure@microsoft.com or via the portal at https://msrc.microsoft.com.
|
||||||
You should receive a response within 24 hours. If for some reason you do not, please follow up via email to ensure we received your
|
You should receive a response within 24 hours. If for some reason you do not, please follow up via email to ensure we received your
|
||||||
original message. Further information, including the MSRC PGP key, can be found in the [MSRC Report an Issue FAQ](https://www.microsoft.com/en-us/msrc/faqs-report-an-issue).
|
original message. Further information, including the MSRC PGP key, can be found in the [MSRC Report an Issue FAQ](https://www.microsoft.com/en-us/msrc/faqs-report-an-issue).
|
||||||
|
|
||||||
Reports via MSRC may qualify for the .NET Core Bug Bounty. Details of the .NET Core Bug Bounty including terms and conditions are at [https://aka.ms/corebounty](https://aka.ms/corebounty).
|
Reports via MSRC may qualify for the .NET Core Bug Bounty. Details of the .NET Core Bug Bounty including terms and conditions are at [https://aka.ms/corebounty](https://aka.ms/corebounty).
|
||||||
|
|
|
@ -51,4 +51,4 @@ specs can be found here:
|
||||||
can be implicitly converted to `ReadOnlySpan<T>`.
|
can be implicitly converted to `ReadOnlySpan<T>`.
|
||||||
* **AVOID** providing overloads for both `ReadOnlySpan<T>`/`Span<T>` as well as
|
* **AVOID** providing overloads for both `ReadOnlySpan<T>`/`Span<T>` as well as
|
||||||
pointers and arrays as those can be implicitly converted to
|
pointers and arrays as those can be implicitly converted to
|
||||||
`ReadOnlySpan<T>`/`Span<T>`.
|
`ReadOnlySpan<T>`/`Span<T>`.
|
||||||
|
|
|
@ -114,7 +114,7 @@ A code review for enabling nullability generally involves three passes:
|
||||||
- Adding `!` to reference type usage. These essentially suppress the null warning, telling the compiler to treat the expression as if it's non-null. These evaporate at compile-time.
|
- Adding `!` to reference type usage. These essentially suppress the null warning, telling the compiler to treat the expression as if it's non-null. These evaporate at compile-time.
|
||||||
|
|
||||||
- Adding `Debug.Assert(reference != null);` statements. These inform the compiler that the mentioned reference is non-`null`, which will cause the compiler to factor that in and have the effect of suppressing subsequent warnings on that reference (until the flow analysis suggests that could change). As with any `Debug.Assert`, these evaporate at compile-time in release builds (where `DEBUG` isn't defined).
|
- Adding `Debug.Assert(reference != null);` statements. These inform the compiler that the mentioned reference is non-`null`, which will cause the compiler to factor that in and have the effect of suppressing subsequent warnings on that reference (until the flow analysis suggests that could change). As with any `Debug.Assert`, these evaporate at compile-time in release builds (where `DEBUG` isn't defined).
|
||||||
|
|
||||||
- Most any other changes have the potential to change the IL, which should not be necessary for the feature. In particular, it's common for `?`s on dereferences to sneak in, e.g. changing `someVar.SomeMethod()` to `someVar?.SomeMethod()`; that is a change to the IL, and should only be employed when there's an actual known bug that's important to fix, as otherwise we're incurring unnecessary cost. Similarly, it's easy to accidentally add `?` to value types, which has a significant impact, changing the `T` to a `Nullable<T>` and should be avoided.
|
- Most any other changes have the potential to change the IL, which should not be necessary for the feature. In particular, it's common for `?`s on dereferences to sneak in, e.g. changing `someVar.SomeMethod()` to `someVar?.SomeMethod()`; that is a change to the IL, and should only be employed when there's an actual known bug that's important to fix, as otherwise we're incurring unnecessary cost. Similarly, it's easy to accidentally add `?` to value types, which has a significant impact, changing the `T` to a `Nullable<T>` and should be avoided.
|
||||||
|
|
||||||
- Any `!`s added that should have been unnecessary and are required due to either a compiler issue or due to lack of expressibility about annotations should have a `// TODO-NULLABLE: http://link/to/relevant/issue` comment added on the same line.
|
- Any `!`s added that should have been unnecessary and are required due to either a compiler issue or due to lack of expressibility about annotations should have a `// TODO-NULLABLE: http://link/to/relevant/issue` comment added on the same line.
|
||||||
|
|
|
@ -6,7 +6,7 @@ Behavioral Change
|
||||||
|
|
||||||
A behavioral change represents changes to the behavior of a member. A behavioral change may including throwing a new exception, adding or removing internal method calls, or alternating the way in which a return value is calculated. Behavioral changes can be the hardest type of change to categorize as acceptable or not - they can be severe in impact, or relatively innocuous.
|
A behavioral change represents changes to the behavior of a member. A behavioral change may including throwing a new exception, adding or removing internal method calls, or alternating the way in which a return value is calculated. Behavioral changes can be the hardest type of change to categorize as acceptable or not - they can be severe in impact, or relatively innocuous.
|
||||||
|
|
||||||
Binary Compatibility
|
Binary Compatibility
|
||||||
--------------------
|
--------------------
|
||||||
|
|
||||||
Refers to the ability of existing consumers of an API to be able to use a newer version without recompilation. By definition, if an assembly's public signatures have been removed, or altered so that consumers can no longer access the same interface exposed by the assembly, the change is said to be a _binary incompatible change_.
|
Refers to the ability of existing consumers of an API to be able to use a newer version without recompilation. By definition, if an assembly's public signatures have been removed, or altered so that consumers can no longer access the same interface exposed by the assembly, the change is said to be a _binary incompatible change_.
|
||||||
|
@ -16,19 +16,19 @@ Source Compatibility
|
||||||
|
|
||||||
Refers to the ability of existing consumers of an API to recompile against a newer version without any source changes. By definition, if a consumer needs to make changes to its code in order for it to build successfully against a newer version of an API, the change is said to be a _source incompatible change_.
|
Refers to the ability of existing consumers of an API to recompile against a newer version without any source changes. By definition, if a consumer needs to make changes to its code in order for it to build successfully against a newer version of an API, the change is said to be a _source incompatible change_.
|
||||||
|
|
||||||
Design-Time Compatibility
|
Design-Time Compatibility
|
||||||
-------------------------
|
-------------------------
|
||||||
|
|
||||||
_Design-time compatibility_ refers to preserving the design-time experience across versions of Visual Studio and other design-time environments. This can involve details around the UI of the designer, but by far the most interesting design-time compatibility is project compatibility. A potential project (or solution), must be able to be opened, and used on a newer version of a designer.
|
_Design-time compatibility_ refers to preserving the design-time experience across versions of Visual Studio and other design-time environments. This can involve details around the UI of the designer, but by far the most interesting design-time compatibility is project compatibility. A potential project (or solution), must be able to be opened, and used on a newer version of a designer.
|
||||||
|
|
||||||
Backwards Compatibility
|
Backwards Compatibility
|
||||||
-----------------------
|
-----------------------
|
||||||
|
|
||||||
_Backwards compatibility_ refers to the ability of an existing consumer of an API to run against, and behave in the same way against a newer version. By definition, if a consumer is not able to run, or behaves differently against the newer version of the API, then the API is said to be _backwards incompatible_.
|
_Backwards compatibility_ refers to the ability of an existing consumer of an API to run against, and behave in the same way against a newer version. By definition, if a consumer is not able to run, or behaves differently against the newer version of the API, then the API is said to be _backwards incompatible_.
|
||||||
|
|
||||||
Changes that affect backwards compatibility are strongly discouraged. All alternates should be actively considered, since developers will, by default, expect backwards compatibility in newer versions of an API.
|
Changes that affect backwards compatibility are strongly discouraged. All alternates should be actively considered, since developers will, by default, expect backwards compatibility in newer versions of an API.
|
||||||
|
|
||||||
Forwards Compatibility
|
Forwards Compatibility
|
||||||
----------------------
|
----------------------
|
||||||
|
|
||||||
_Forwards compatibility_ is the exact reverse of backwards compatibility; it refers to the ability of an existing consumer of an API to run against, and behave in the way against a _older_ version. By definition, if a consumer is not able to run, or behaves differently against an older version of the API, then the API is said to be _forwards incompatible_.
|
_Forwards compatibility_ is the exact reverse of backwards compatibility; it refers to the ability of an existing consumer of an API to run against, and behave in the way against a _older_ version. By definition, if a consumer is not able to run, or behaves differently against an older version of the API, then the API is said to be _forwards incompatible_.
|
||||||
|
|
|
@ -19,14 +19,14 @@ Breaking Change Rules
|
||||||
### Property, Field, Parameter and Return Values
|
### Property, Field, Parameter and Return Values
|
||||||
✓ **Allowed**
|
✓ **Allowed**
|
||||||
* Increasing the range of accepted values for a property or parameter if the member _is not_ `virtual`
|
* Increasing the range of accepted values for a property or parameter if the member _is not_ `virtual`
|
||||||
|
|
||||||
Note that the range can only increase to the extent that it does not impact the static type. e.g. it is OK to remove `if (x > 10) throw new ArgumentOutOfRangeException("x")`, but it is not OK to change the type of `x` from `int` to `long` or `int?`.
|
Note that the range can only increase to the extent that it does not impact the static type. e.g. it is OK to remove `if (x > 10) throw new ArgumentOutOfRangeException("x")`, but it is not OK to change the type of `x` from `int` to `long` or `int?`.
|
||||||
|
|
||||||
* Returning a value of a more derived type for a property, field, return or `out` value
|
* Returning a value of a more derived type for a property, field, return or `out` value
|
||||||
|
|
||||||
Note, again, that the static type cannot change. e.g. it is OK to return a `string` instance where an `object` was returned previously, but it is not OK to change the return type from `object` to `string`.
|
Note, again, that the static type cannot change. e.g. it is OK to return a `string` instance where an `object` was returned previously, but it is not OK to change the return type from `object` to `string`.
|
||||||
|
|
||||||
✗ **Disallowed**
|
✗ **Disallowed**
|
||||||
* Increasing the range of accepted values for a property or parameter if the member _is_ `virtual`
|
* Increasing the range of accepted values for a property or parameter if the member _is_ `virtual`
|
||||||
|
|
||||||
This is breaking because any existing overridden members will now not function correctly for the extended range of values.
|
This is breaking because any existing overridden members will now not function correctly for the extended range of values.
|
||||||
|
@ -135,7 +135,7 @@ Breaking Change Rules
|
||||||
So long as it does not introduce any new abstract members or change the semantics or behavior of existing members, a type can be introduced into a hierarchy between two existing types. For example, between .NET Framework 1.1 and .NET Framework 2.0, we introduced `DbConnection` as a new base class for `SqlConnection` which previously derived from `Component`.
|
So long as it does not introduce any new abstract members or change the semantics or behavior of existing members, a type can be introduced into a hierarchy between two existing types. For example, between .NET Framework 1.1 and .NET Framework 2.0, we introduced `DbConnection` as a new base class for `SqlConnection` which previously derived from `Component`.
|
||||||
|
|
||||||
* Adding an interface implementation to a type
|
* Adding an interface implementation to a type
|
||||||
|
|
||||||
This is acceptable because it will not adversely affect existing clients. Any changes which could be made to the type being changed in this situation, will have to work within the boundaries of acceptable changes defined here, in order for the new implementation to remain acceptable.
|
This is acceptable because it will not adversely affect existing clients. Any changes which could be made to the type being changed in this situation, will have to work within the boundaries of acceptable changes defined here, in order for the new implementation to remain acceptable.
|
||||||
Extreme caution is urged when adding interfaces that directly affect the ability of the designer or serializer to generate code or data, that cannot be consumed down-level. An example is the `ISerializable` interface.
|
Extreme caution is urged when adding interfaces that directly affect the ability of the designer or serializer to generate code or data, that cannot be consumed down-level. An example is the `ISerializable` interface.
|
||||||
Care should be taken when the interface (or one of the interfaces that this interface requires) has default interface implementations for other interface methods. The default implementation could conflict with other default implementations in a derived class.
|
Care should be taken when the interface (or one of the interfaces that this interface requires) has default interface implementations for other interface methods. The default implementation could conflict with other default implementations in a derived class.
|
||||||
|
@ -205,7 +205,7 @@ Breaking Change Rules
|
||||||
|
|
||||||
* Adding an overload that precludes an existing overload, and defines different behavior
|
* Adding an overload that precludes an existing overload, and defines different behavior
|
||||||
|
|
||||||
This will break existing clients that were bound to the previous overload. For example, if you have a class that has a single version of a method that accepts a `uint`, an existing consumer will
|
This will break existing clients that were bound to the previous overload. For example, if you have a class that has a single version of a method that accepts a `uint`, an existing consumer will
|
||||||
successfully bind to that overload, if simply passing an `int` value. However, if you add an overload that accepts an `int`, recompiling or via late-binding the application will now bind to the new overload. If different behavior results, then this is a breaking change.
|
successfully bind to that overload, if simply passing an `int` value. However, if you add an overload that accepts an `int`, recompiling or via late-binding the application will now bind to the new overload. If different behavior results, then this is a breaking change.
|
||||||
|
|
||||||
* Moving an exposed field onto a class higher in the hierarchy tree of the type from which it was removed
|
* Moving an exposed field onto a class higher in the hierarchy tree of the type from which it was removed
|
||||||
|
|
|
@ -97,7 +97,7 @@ more latitude here in .NET Core.
|
||||||
|
|
||||||
For buckets #2 and #3 we apply a risk-benefit analysis. It doesn't matter if the
|
For buckets #2 and #3 we apply a risk-benefit analysis. It doesn't matter if the
|
||||||
old behavior is "wrong", we still need to think through the implications. This
|
old behavior is "wrong", we still need to think through the implications. This
|
||||||
can result in one of the following outcomes:
|
can result in one of the following outcomes:
|
||||||
|
|
||||||
* **Accepted with compat switch**. Depending on the estimated customer impact,
|
* **Accepted with compat switch**. Depending on the estimated customer impact,
|
||||||
we may decide to add a compat switch that allows consumers to bring back the
|
we may decide to add a compat switch that allows consumers to bring back the
|
||||||
|
|
|
@ -58,7 +58,7 @@ internal static partial class Interop
|
||||||
```
|
```
|
||||||
|
|
||||||
As shown above, platforms may be additive, in that an assembly may use functionality from multiple folders, e.g. System.IO.FileSystem's Linux build will use functionality both from Unix (common across all Unix systems) and from Linux (specific to Linux and not available across non-Linux Unix systems).
|
As shown above, platforms may be additive, in that an assembly may use functionality from multiple folders, e.g. System.IO.FileSystem's Linux build will use functionality both from Unix (common across all Unix systems) and from Linux (specific to Linux and not available across non-Linux Unix systems).
|
||||||
|
|
||||||
- Interop.*.cs files are created in a way such that every assembly consuming the file will need every DllImport it contains.
|
- Interop.*.cs files are created in a way such that every assembly consuming the file will need every DllImport it contains.
|
||||||
- If multiple related DllImports will all be needed by every consumer, they may be declared in the same file, named for the functionality grouping, e.g. Interop.IOErrors.cs.
|
- If multiple related DllImports will all be needed by every consumer, they may be declared in the same file, named for the functionality grouping, e.g. Interop.IOErrors.cs.
|
||||||
- Otherwise, in the limit (and the expected case for most situations) each Interop.*.cs file will contain a single DllImport and associated interop types (e.g. the structs used with that signature) and helper wrappers, e.g. Interop.strerror.cs.
|
- Otherwise, in the limit (and the expected case for most situations) each Interop.*.cs file will contain a single DllImport and associated interop types (e.g. the structs used with that signature) and helper wrappers, e.g. Interop.strerror.cs.
|
||||||
|
@ -104,7 +104,7 @@ internal static partial class Interop // contents of Common\src\Interop\Windows\
|
||||||
|
|
||||||
```
|
```
|
||||||
(Note that this will likely result in some extra constants defined in each assembly that uses interop, which minimally violates one of the goals, but it's very minimal.)
|
(Note that this will likely result in some extra constants defined in each assembly that uses interop, which minimally violates one of the goals, but it's very minimal.)
|
||||||
|
|
||||||
- .csproj project files then include the interop code they need, e.g.
|
- .csproj project files then include the interop code they need, e.g.
|
||||||
```XML
|
```XML
|
||||||
<ItemGroup Condition=" '$(TargetsUnix)' == 'true' ">
|
<ItemGroup Condition=" '$(TargetsUnix)' == 'true' ">
|
||||||
|
@ -170,10 +170,10 @@ To address this, we're moving to a model where all UNIX interop from dotnet/runt
|
||||||
|
|
||||||
Guidelines for shim C++ API:
|
Guidelines for shim C++ API:
|
||||||
|
|
||||||
- Keep them as "thin"/1:1 as possible.
|
- Keep them as "thin"/1:1 as possible.
|
||||||
- We want to write the majority of code in C#.
|
- We want to write the majority of code in C#.
|
||||||
- Never skip the shim and P/Invoke directly to the underlying platform API. It's easy to assume something is safe/guaranteed when it isn't.
|
- Never skip the shim and P/Invoke directly to the underlying platform API. It's easy to assume something is safe/guaranteed when it isn't.
|
||||||
- Don't cheat and take advantage of coincidental agreement between one flavor's ABI and the shim's ABI.
|
- Don't cheat and take advantage of coincidental agreement between one flavor's ABI and the shim's ABI.
|
||||||
- Use PascalCase in a style closer to Win32 than libc.
|
- Use PascalCase in a style closer to Win32 than libc.
|
||||||
- If an export point has a 1:1 correspondence to the platform API, then name it after the platform API in PascalCase (e.g. stat -> Stat, fstat -> FStat).
|
- If an export point has a 1:1 correspondence to the platform API, then name it after the platform API in PascalCase (e.g. stat -> Stat, fstat -> FStat).
|
||||||
- If an export is not 1:1, then spell things out as we typically would in dotnet/runtime code (i.e. don't use abbreviations unless they come from the underlying API.
|
- If an export is not 1:1, then spell things out as we typically would in dotnet/runtime code (i.e. don't use abbreviations unless they come from the underlying API.
|
||||||
|
|
|
@ -201,7 +201,7 @@ Here's a real-world example from the `String` class:
|
||||||
|
|
||||||
```CSharp
|
```CSharp
|
||||||
public partial sealed class String
|
public partial sealed class String
|
||||||
{
|
{
|
||||||
[MethodImpl(MethodImplOptions.InternalCall)]
|
[MethodImpl(MethodImplOptions.InternalCall)]
|
||||||
private extern string? IsInterned();
|
private extern string? IsInterned();
|
||||||
|
|
||||||
|
|
|
@ -1,4 +1,4 @@
|
||||||
Method Descriptor
|
Method Descriptor
|
||||||
=================
|
=================
|
||||||
|
|
||||||
Author: Jan Kotas ([@jkotas](https://github.com/jkotas)) - 2006
|
Author: Jan Kotas ([@jkotas](https://github.com/jkotas)) - 2006
|
||||||
|
|
|
@ -9,7 +9,7 @@ Revisions:
|
||||||
# Introduction
|
# Introduction
|
||||||
|
|
||||||
This document describes ReadyToRun format 3.1 implemented in CoreCLR as of June 2019 and not yet
|
This document describes ReadyToRun format 3.1 implemented in CoreCLR as of June 2019 and not yet
|
||||||
implemented proposed extensions 4.1 for the support of composite R2R file format.
|
implemented proposed extensions 4.1 for the support of composite R2R file format.
|
||||||
**Composite R2R file format** has basically the same structure as the traditional R2R file format
|
**Composite R2R file format** has basically the same structure as the traditional R2R file format
|
||||||
defined in earlier revisions except that the output file represents a larger number of input MSIL
|
defined in earlier revisions except that the output file represents a larger number of input MSIL
|
||||||
assemblies compiled together as a logical unit.
|
assemblies compiled together as a logical unit.
|
||||||
|
@ -320,8 +320,8 @@ basic encoding, with extended encoding for large values).
|
||||||
|
|
||||||
## ReadyToRunSectionType.RuntimeFunctions
|
## ReadyToRunSectionType.RuntimeFunctions
|
||||||
|
|
||||||
This section contains sorted array of `RUNTIME_FUNCTION` entries that describe all code blocks in the image with pointers to their unwind info.
|
This section contains sorted array of `RUNTIME_FUNCTION` entries that describe all code blocks in the image with pointers to their unwind info.
|
||||||
Despite the name, these code block might represent a method body, or it could be just a part of it (e.g. a funclet) that requires its own unwind data.
|
Despite the name, these code block might represent a method body, or it could be just a part of it (e.g. a funclet) that requires its own unwind data.
|
||||||
The standard Windows xdata/pdata format is used.
|
The standard Windows xdata/pdata format is used.
|
||||||
ARM format is used for x86 to compensate for the lack of x86 unwind info standard.
|
ARM format is used for x86 to compensate for the lack of x86 unwind info standard.
|
||||||
The unwind info blob is immediately followed by the GC info blob. The encoding slightly differs for amd64
|
The unwind info blob is immediately followed by the GC info blob. The encoding slightly differs for amd64
|
||||||
|
|
|
@ -47,7 +47,7 @@ This feature is currently only supported for instantiations over reference types
|
||||||
|
|
||||||
The dictionary used by any given generic method is pointed at by the `m_pPerInstInfo` field on the `InstantiatedMethodDesc` structure of that method. It's a direct pointer to the contents of the generic dictionary data.
|
The dictionary used by any given generic method is pointed at by the `m_pPerInstInfo` field on the `InstantiatedMethodDesc` structure of that method. It's a direct pointer to the contents of the generic dictionary data.
|
||||||
|
|
||||||
On generic types, there's an extra level of indirection: the `m_pPerInstInfo` field on the `MethodTable` structure is a pointer to a table of dictionaries, and each entry in that table is a pointer to the actual generic dictionary data. This is because types have inheritance, and derived generic types inherit the dictionaries of their base types.
|
On generic types, there's an extra level of indirection: the `m_pPerInstInfo` field on the `MethodTable` structure is a pointer to a table of dictionaries, and each entry in that table is a pointer to the actual generic dictionary data. This is because types have inheritance, and derived generic types inherit the dictionaries of their base types.
|
||||||
|
|
||||||
Here's an example:
|
Here's an example:
|
||||||
```c#
|
```c#
|
||||||
|
|
|
@ -1,4 +1,4 @@
|
||||||
Type System Overview
|
Type System Overview
|
||||||
====================
|
====================
|
||||||
|
|
||||||
Author: David Wrighton ([@davidwrighton](https://github.com/davidwrighton)) - 2010
|
Author: David Wrighton ([@davidwrighton](https://github.com/davidwrighton)) - 2010
|
||||||
|
|
|
@ -3,7 +3,7 @@ Vectors and Hardware Intrinsics Support
|
||||||
---
|
---
|
||||||
|
|
||||||
# Introduction
|
# Introduction
|
||||||
The CoreCLR runtime has support for several varieties of hardware intrinsics, and various ways to compile code which uses them. This support varies by target processor, and the code produced depends on how the jit compiler is invoked. This document describes the various behaviors of intrinsics in the runtime, and concludes with implications for developers working on the runtime and libraries portions of the runtime.
|
The CoreCLR runtime has support for several varieties of hardware intrinsics, and various ways to compile code which uses them. This support varies by target processor, and the code produced depends on how the jit compiler is invoked. This document describes the various behaviors of intrinsics in the runtime, and concludes with implications for developers working on the runtime and libraries portions of the runtime.
|
||||||
|
|
||||||
# Acronyms and definitions
|
# Acronyms and definitions
|
||||||
| Acronym | Definition
|
| Acronym | Definition
|
||||||
|
@ -44,8 +44,8 @@ There are 2 different implementations of AOT compilation under development at th
|
||||||
|
|
||||||
###Code written in System.Private.CoreLib.dll
|
###Code written in System.Private.CoreLib.dll
|
||||||
#### Crossgen implementation rules
|
#### Crossgen implementation rules
|
||||||
- Any code which uses `Vector<T>` will not be compiled AOT. (See code which throws a TypeLoadException using `IDS_EE_SIMD_NGEN_DISALLOWED`)
|
- Any code which uses `Vector<T>` will not be compiled AOT. (See code which throws a TypeLoadException using `IDS_EE_SIMD_NGEN_DISALLOWED`)
|
||||||
- Code which uses Sse and Sse2 platform hardware intrinsics is always generated as it would be at jit time.
|
- Code which uses Sse and Sse2 platform hardware intrinsics is always generated as it would be at jit time.
|
||||||
- Code which uses Sse3, Ssse3, Sse41, Sse42, Popcnt, Pclmulqdq, and Lzcnt instruction sets will be generated, but the associated IsSupported check will be a runtime check. See `FilterNamedIntrinsicMethodAttribs` for details on how this is done.
|
- Code which uses Sse3, Ssse3, Sse41, Sse42, Popcnt, Pclmulqdq, and Lzcnt instruction sets will be generated, but the associated IsSupported check will be a runtime check. See `FilterNamedIntrinsicMethodAttribs` for details on how this is done.
|
||||||
- Code which uses other instruction sets will be generated as if the processor does not support that instruction set. (For instance, a usage of Avx2.IsSupported in CoreLib will generate native code where it unconditionally returns false, and then if and when tiered compilation occurs, the function may be rejitted and have code where the property returns true.)
|
- Code which uses other instruction sets will be generated as if the processor does not support that instruction set. (For instance, a usage of Avx2.IsSupported in CoreLib will generate native code where it unconditionally returns false, and then if and when tiered compilation occurs, the function may be rejitted and have code where the property returns true.)
|
||||||
- Non-platform intrinsics which require more hardware support than the minimum supported hardware capability will not take advantage of that capability. In particular the code generated for `Vector2/3/4.Dot`, and `Math.Round`, and `MathF.Round`. See `FilterNamedIntrinsicMethodAttribs` for details. MethodImplOptions.AggressiveOptimization may be used to disable precompilation compilation of this sub-par code.
|
- Non-platform intrinsics which require more hardware support than the minimum supported hardware capability will not take advantage of that capability. In particular the code generated for `Vector2/3/4.Dot`, and `Math.Round`, and `MathF.Round`. See `FilterNamedIntrinsicMethodAttribs` for details. MethodImplOptions.AggressiveOptimization may be used to disable precompilation compilation of this sub-par code.
|
||||||
|
@ -58,8 +58,8 @@ The rules here provide the following characteristics.
|
||||||
- AOT generated code which could take advantage of more advanced hardware support experiences a performance penalty until rejitted. (If a customer chooses to disable tiered compilation, then customer code may always run slowly).
|
- AOT generated code which could take advantage of more advanced hardware support experiences a performance penalty until rejitted. (If a customer chooses to disable tiered compilation, then customer code may always run slowly).
|
||||||
|
|
||||||
#### Code review rules for code written in System.Private.CoreLib.dll
|
#### Code review rules for code written in System.Private.CoreLib.dll
|
||||||
- Any use of a platform intrinsic in the codebase MUST be wrapped with a call to the associated IsSupported property. This wrapping MUST be done within the same function that uses the hardware intrinsic, and MUST NOT be in a wrapper function unless it is one of the intrinsics that are enabled by default for crossgen compilation of System.Private.CoreLib (See list above in the implementation rules section).
|
- Any use of a platform intrinsic in the codebase MUST be wrapped with a call to the associated IsSupported property. This wrapping MUST be done within the same function that uses the hardware intrinsic, and MUST NOT be in a wrapper function unless it is one of the intrinsics that are enabled by default for crossgen compilation of System.Private.CoreLib (See list above in the implementation rules section).
|
||||||
- Within a single function that uses platform intrinsics, it must behave identically regardless of whether IsSupported returns true or not. This rule is required as code inside of an IsSupported check that calls a helper function cannot assume that the helper function will itself see its use of the same IsSupported check return true. This is due to the impact of tiered compilation on code execution within the process.
|
- Within a single function that uses platform intrinsics, it must behave identically regardless of whether IsSupported returns true or not. This rule is required as code inside of an IsSupported check that calls a helper function cannot assume that the helper function will itself see its use of the same IsSupported check return true. This is due to the impact of tiered compilation on code execution within the process.
|
||||||
- Excessive use of intrinsics may cause startup performance problems due to additional jitting, or may not achieve desired performance characteristics due to suboptimal codegen.
|
- Excessive use of intrinsics may cause startup performance problems due to additional jitting, or may not achieve desired performance characteristics due to suboptimal codegen.
|
||||||
|
|
||||||
ACCEPTABLE Code
|
ACCEPTABLE Code
|
||||||
|
@ -130,7 +130,7 @@ public class BitOperations
|
||||||
of this method may be compiled as if the Avx2 feature is not available, and is not reliably rejitted
|
of this method may be compiled as if the Avx2 feature is not available, and is not reliably rejitted
|
||||||
at the same time as the PopCount function.
|
at the same time as the PopCount function.
|
||||||
|
|
||||||
As a special note, on the x86 and x64 platforms, this generally unsafe pattern may be used
|
As a special note, on the x86 and x64 platforms, this generally unsafe pattern may be used
|
||||||
with the Sse, Sse2, Sse3, Sssse3, Ssse41 and Sse42 instruction sets as those instruction sets
|
with the Sse, Sse2, Sse3, Sssse3, Ssse41 and Sse42 instruction sets as those instruction sets
|
||||||
are treated specially by both crossgen1 and crossgen2 when compiling System.Private.CoreLib.dll.
|
are treated specially by both crossgen1 and crossgen2 when compiling System.Private.CoreLib.dll.
|
||||||
}
|
}
|
||||||
|
@ -140,16 +140,16 @@ public class BitOperations
|
||||||
### Code written in other assemblies (both first and third party)
|
### Code written in other assemblies (both first and third party)
|
||||||
|
|
||||||
#### Crossgen implementation rules
|
#### Crossgen implementation rules
|
||||||
- Any code which uses an intrinsic from the `System.Runtime.Intrinsics.Arm` or `System.Runtime.Intrinsics.X86` namespace will not be compiled AOT. (See code which throws a TypeLoadException using `IDS_EE_HWINTRINSIC_NGEN_DISALLOWED`)
|
- Any code which uses an intrinsic from the `System.Runtime.Intrinsics.Arm` or `System.Runtime.Intrinsics.X86` namespace will not be compiled AOT. (See code which throws a TypeLoadException using `IDS_EE_HWINTRINSIC_NGEN_DISALLOWED`)
|
||||||
- Any code which uses `Vector<T>` will not be compiled AOT. (See code which throws a TypeLoadException using `IDS_EE_SIMD_NGEN_DISALLOWED`)
|
- Any code which uses `Vector<T>` will not be compiled AOT. (See code which throws a TypeLoadException using `IDS_EE_SIMD_NGEN_DISALLOWED`)
|
||||||
- Any code which uses `Vector64<T>`, `Vector128<T>` or `Vector256<T>` will not be compiled AOT. (See code which throws a TypeLoadException using `IDS_EE_HWINTRINSIC_NGEN_DISALLOWED`)
|
- Any code which uses `Vector64<T>`, `Vector128<T>` or `Vector256<T>` will not be compiled AOT. (See code which throws a TypeLoadException using `IDS_EE_HWINTRINSIC_NGEN_DISALLOWED`)
|
||||||
- Non-platform intrinsics which require more hardware support than the minimum supported hardware capability will not take advantage of that capability. In particular the code generated for Vector2/3/4 is sub-optimal. MethodImplOptions.AggressiveOptimization may be used to disable compilation of this sub-par code.
|
- Non-platform intrinsics which require more hardware support than the minimum supported hardware capability will not take advantage of that capability. In particular the code generated for Vector2/3/4 is sub-optimal. MethodImplOptions.AggressiveOptimization may be used to disable compilation of this sub-par code.
|
||||||
|
|
||||||
#### Characteristics which result from rules
|
#### Characteristics which result from rules
|
||||||
The rules here provide the following characteristics.
|
The rules here provide the following characteristics.
|
||||||
- Use of platform specific hardware intrinsics causes runtime jit and startup time concerns.
|
- Use of platform specific hardware intrinsics causes runtime jit and startup time concerns.
|
||||||
- Use of `Vector<T>` causes runtime jit and startup time concerns
|
- Use of `Vector<T>` causes runtime jit and startup time concerns
|
||||||
- AOT generated code which could take advantage of more advanced hardware support experiences a performance penalty until rejitted. (If a customer chooses to disable tiered compilation, then customer code may always run slowly).
|
- AOT generated code which could take advantage of more advanced hardware support experiences a performance penalty until rejitted. (If a customer chooses to disable tiered compilation, then customer code may always run slowly).
|
||||||
|
|
||||||
#### Code review rules for use of platform intrinsics
|
#### Code review rules for use of platform intrinsics
|
||||||
- Any use of a platform intrinsic in the codebase SHOULD be wrapped with a call to the associated IsSupported property. This wrapping may be done within the same function that uses the hardware intrinsic, but this is not required as long as the programmer can control all entrypoints to a function that uses the hardware intrinsic.
|
- Any use of a platform intrinsic in the codebase SHOULD be wrapped with a call to the associated IsSupported property. This wrapping may be done within the same function that uses the hardware intrinsic, but this is not required as long as the programmer can control all entrypoints to a function that uses the hardware intrinsic.
|
||||||
|
@ -183,7 +183,7 @@ Since System.Private.CoreLib.dll is known to be code reviewed with the code revi
|
||||||
|
|
||||||
# Mechanisms in the JIT to generate correct code to handle varied instruction set support
|
# Mechanisms in the JIT to generate correct code to handle varied instruction set support
|
||||||
|
|
||||||
The JIT receives flags which instruct it on what instruction sets are valid to use, and has access to a new jit interface api `notifyInstructionSetUsage(isa, bool supportBehaviorRequired)`.
|
The JIT receives flags which instruct it on what instruction sets are valid to use, and has access to a new jit interface api `notifyInstructionSetUsage(isa, bool supportBehaviorRequired)`.
|
||||||
|
|
||||||
The notifyInstructionSetUsage api is used to notify the AOT compiler infrastructure that the code may only execute if the runtime environment of the code is exactly the same as the boolean parameter indicates it should be. For instance, if `notifyInstructionSetUsage(Avx, false)` is used, then the code generated must not be used if the `Avx` instruction set is useable. Similarly `notifyInstructionSetUsage(Avx, true)` will indicate that the code may only be used if the `Avx` instruction set is available.
|
The notifyInstructionSetUsage api is used to notify the AOT compiler infrastructure that the code may only execute if the runtime environment of the code is exactly the same as the boolean parameter indicates it should be. For instance, if `notifyInstructionSetUsage(Avx, false)` is used, then the code generated must not be used if the `Avx` instruction set is useable. Similarly `notifyInstructionSetUsage(Avx, true)` will indicate that the code may only be used if the `Avx` instruction set is available.
|
||||||
|
|
||||||
|
|
|
@ -46,7 +46,7 @@ As of .NET 5.0, createdump is supported on MacOS but instead of the MachO dump f
|
||||||
|
|
||||||
### Windows ###
|
### Windows ###
|
||||||
|
|
||||||
As of .NET 5.0, createdump and the below configuration environment variables are supported on Windows. It is implemented using the Windows MiniDumpWriteDump API. This allows consistent crash/unhandled exception dumps across all of our platforms.
|
As of .NET 5.0, createdump and the below configuration environment variables are supported on Windows. It is implemented using the Windows MiniDumpWriteDump API. This allows consistent crash/unhandled exception dumps across all of our platforms.
|
||||||
|
|
||||||
# Configuration/Policy #
|
# Configuration/Policy #
|
||||||
|
|
||||||
|
|
|
@ -17,7 +17,6 @@ post/pre increment, perhaps like this: `Foo(j, a[j++])`. Here `j` is updated vi
|
||||||
when the second arg is evaluated, so the earlier uses of `j` would need to be evaluated and
|
when the second arg is evaluated, so the earlier uses of `j` would need to be evaluated and
|
||||||
saved in a new LclVar.
|
saved in a new LclVar.
|
||||||
|
|
||||||
|
|
||||||
One simple approach would be to create new single definition, single use LclVars for every argument
|
One simple approach would be to create new single definition, single use LclVars for every argument
|
||||||
that is passed. This would preserve the evaluation order. However, it would potentially create
|
that is passed. This would preserve the evaluation order. However, it would potentially create
|
||||||
hundreds of LclVar for moderately sized methods and that would overflow the limited number of
|
hundreds of LclVar for moderately sized methods and that would overflow the limited number of
|
||||||
|
@ -25,7 +24,6 @@ tracked local variables in the JIT. One observation is that many arguments to m
|
||||||
either constants or LclVars and can be set up anytime we want. They usually will not need a
|
either constants or LclVars and can be set up anytime we want. They usually will not need a
|
||||||
new LclVar to preserve the order of evaluation rule.
|
new LclVar to preserve the order of evaluation rule.
|
||||||
|
|
||||||
|
|
||||||
Each argument is an arbitrary expression tree. The JIT tracks a summary of observable side-effects
|
Each argument is an arbitrary expression tree. The JIT tracks a summary of observable side-effects
|
||||||
using a set of five bit flags in every GenTree node: `GTF_ASG`, `GTF_CALL`, `GTF_EXCEPT`, `GTF_GLOB_REF`,
|
using a set of five bit flags in every GenTree node: `GTF_ASG`, `GTF_CALL`, `GTF_EXCEPT`, `GTF_GLOB_REF`,
|
||||||
and `GTF_ORDER_SIDEEFF`. These flags are propagated up the tree so that the top node has a particular
|
and `GTF_ORDER_SIDEEFF`. These flags are propagated up the tree so that the top node has a particular
|
||||||
|
|
|
@ -309,7 +309,7 @@ After LSRA, the graph has the following properties:
|
||||||
|
|
||||||
- However, if such a node is constrained to a set of registers,
|
- However, if such a node is constrained to a set of registers,
|
||||||
and its current location does not satisfy that requirement, LSRA
|
and its current location does not satisfy that requirement, LSRA
|
||||||
must insert a `GT_COPY` node between the node and its parent.
|
must insert a `GT_COPY` node between the node and its parent.
|
||||||
The `_gtRegNum` on the `GT_COPY` node must satisfy the register
|
The `_gtRegNum` on the `GT_COPY` node must satisfy the register
|
||||||
requirement of the parent.
|
requirement of the parent.
|
||||||
|
|
||||||
|
@ -1088,11 +1088,11 @@ term "EH Var" means a `lclVar` marked `lvLiveInOutOfHndlr`):
|
||||||
|
|
||||||
- Adjust the heuristics:
|
- Adjust the heuristics:
|
||||||
|
|
||||||
1. For determining whether an EH var should be a candidate for register allocation,
|
1. For determining whether an EH var should be a candidate for register allocation,
|
||||||
e.g. if the defs outweight the uses.
|
e.g. if the defs outweight the uses.
|
||||||
|
|
||||||
2. For determining when a definition of an EH var should be only stored to the stack,
|
2. For determining when a definition of an EH var should be only stored to the stack,
|
||||||
rather than also remaining live in the register.
|
rather than also remaining live in the register.
|
||||||
|
|
||||||
- If the weight of the defs exceeds the weight of the blocks with successors in exception
|
- If the weight of the defs exceeds the weight of the blocks with successors in exception
|
||||||
regions, consider spilling the `lclVar` to the stack only at those boundaries.
|
regions, consider spilling the `lclVar` to the stack only at those boundaries.
|
||||||
|
@ -1241,7 +1241,7 @@ kill site.
|
||||||
Issue [\#9767](https://github.com/dotnet/runtime/issues/9767) captures the issue that the
|
Issue [\#9767](https://github.com/dotnet/runtime/issues/9767) captures the issue that the
|
||||||
"spill always" stress mode, `LSRA_SPILL_ALWAYS`, `COMPlus_JitStressRegs=0x800` doesn't work properly.
|
"spill always" stress mode, `LSRA_SPILL_ALWAYS`, `COMPlus_JitStressRegs=0x800` doesn't work properly.
|
||||||
|
|
||||||
Issue [\#6261](https://github.com/dotnet/runtime/issues/6261) has to do with `RegOptional`
|
Issue [\#6261](https://github.com/dotnet/runtime/issues/6261) has to do with `RegOptional`
|
||||||
`RefPositions` that are marked as `copyReg` or `moveReg`. See the notes on this issue;
|
`RefPositions` that are marked as `copyReg` or `moveReg`. See the notes on this issue;
|
||||||
I don't think such cases should arise, but there may be some cleanup needed here.
|
I don't think such cases should arise, but there may be some cleanup needed here.
|
||||||
|
|
||||||
|
@ -1249,7 +1249,7 @@ Issue [\#5793](https://github.com/dotnet/runtime/issues/5793) suggests adding a
|
||||||
allocates registers forr mullti-reg nodes in the reverse of the ABI requirements.
|
allocates registers forr mullti-reg nodes in the reverse of the ABI requirements.
|
||||||
|
|
||||||
Issue [#10691](https://github.com/dotnet/runtime/issues/10691) suggests adding a stress mode that
|
Issue [#10691](https://github.com/dotnet/runtime/issues/10691) suggests adding a stress mode that
|
||||||
deliberately trashes registers that are not currently occupied (e.g. at block boundaries).
|
deliberately trashes registers that are not currently occupied (e.g. at block boundaries).
|
||||||
|
|
||||||
References
|
References
|
||||||
----------
|
----------
|
||||||
|
@ -1284,4 +1284,4 @@ References
|
||||||
|
|
||||||
7. <a id="7"/></a> Yatsina, M. "LLVM Greedy Register Allocator," LLVM Dev Meeting, April 2018.
|
7. <a id="7"/></a> Yatsina, M. "LLVM Greedy Register Allocator," LLVM Dev Meeting, April 2018.
|
||||||
<https://llvm.org/devmtg/2018-04/slides/Yatsina-LLVM%20Greedy%20Register%20Allocator.pdf>
|
<https://llvm.org/devmtg/2018-04/slides/Yatsina-LLVM%20Greedy%20Register%20Allocator.pdf>
|
||||||
(Last retrieved July 2020)
|
(Last retrieved July 2020)
|
||||||
|
|
|
@ -606,7 +606,7 @@ public static int PopCount(ulong bitVectorArg)
|
||||||
|
|
||||||
#### Notes
|
#### Notes
|
||||||
The sample I'm going to walk through implements support for pop count (counting the number of '1' bits in a 64-bit value).
|
The sample I'm going to walk through implements support for pop count (counting the number of '1' bits in a 64-bit value).
|
||||||
|
|
||||||
We're going to start by assuming that we have a method with a known signature that implements PopCount.
|
We're going to start by assuming that we have a method with a known signature that implements PopCount.
|
||||||
Here's the implementation we're going to use. It simply takes the input value, and keeps anding with one, and then shifting right.
|
Here's the implementation we're going to use. It simply takes the input value, and keeps anding with one, and then shifting right.
|
||||||
We're first going to simply recognize the name and signature, and replace the method call with a simple PopCnt IR node.
|
We're first going to simply recognize the name and signature, and replace the method call with a simple PopCnt IR node.
|
||||||
|
|
|
@ -13,7 +13,7 @@ Attaching a profiler to a running CoreCLR process involves sending a message fro
|
||||||
2) `uint attachTimeout` - (Required) A timeout that informs the runtime how long to wait while attempting to attach. This does not impact the timeout of trying to send the attach message.
|
2) `uint attachTimeout` - (Required) A timeout that informs the runtime how long to wait while attempting to attach. This does not impact the timeout of trying to send the attach message.
|
||||||
3) `Guid profilerGuid` - (Required) The profiler's GUID to use when initializing.
|
3) `Guid profilerGuid` - (Required) The profiler's GUID to use when initializing.
|
||||||
4) `string profilerPath` - (Required) The path to the profiler on disk.
|
4) `string profilerPath` - (Required) The path to the profiler on disk.
|
||||||
5) `byte[] additionalData` - (Optional) A data blob that will be passed to `ICorProfilerCallback3::InitializeForAttach` as `pvClientData`.
|
5) `byte[] additionalData` - (Optional) A data blob that will be passed to `ICorProfilerCallback3::InitializeForAttach` as `pvClientData`.
|
||||||
|
|
||||||
This method returns a status HR following the usual convention, 0 (S_OK) means a profiler was successfully attached and any other value is an error indicating what went wrong.
|
This method returns a status HR following the usual convention, 0 (S_OK) means a profiler was successfully attached and any other value is an error indicating what went wrong.
|
||||||
|
|
||||||
|
|
|
@ -4,4 +4,4 @@ Over time we will need to modify the Profiler APIs, this document will serve as
|
||||||
|
|
||||||
1. Code Versioning introduced changes documented [here](../../features/code-versioning-profiler-breaking-changes.md)
|
1. Code Versioning introduced changes documented [here](../../features/code-versioning-profiler-breaking-changes.md)
|
||||||
2. The work to allow adding new types and methods after module load means ICorProfilerInfo7::ApplyMetadata will now potentially trigger a GC, and will not be callable in situations where a GC can not happen (for example ICorProfilerCallback::RootReferences).
|
2. The work to allow adding new types and methods after module load means ICorProfilerInfo7::ApplyMetadata will now potentially trigger a GC, and will not be callable in situations where a GC can not happen (for example ICorProfilerCallback::RootReferences).
|
||||||
3. As part of the work to allow ReJIT on attach ReJITted methods will no longer be inlined (ever). Since the inlining is blocked there won't be a `ICorProfilerCallback::JITInlining` callback.
|
3. As part of the work to allow ReJIT on attach ReJITted methods will no longer be inlined (ever). Since the inlining is blocked there won't be a `ICorProfilerCallback::JITInlining` callback.
|
||||||
|
|
|
@ -7,8 +7,6 @@ Profiler attach is a feature that allows you to attach a profiler to an already
|
||||||
|
|
||||||
Please note! You can't just take any profiler you bought and suddenly be able to attach it to a running application. The profiler must be built with "attachability" in mind. So if you're a profiler developer looking to pump some attachability into your product, read on--this article is for you. Everyone else, this article will probably be less useful--but just as riveting.
|
Please note! You can't just take any profiler you bought and suddenly be able to attach it to a running application. The profiler must be built with "attachability" in mind. So if you're a profiler developer looking to pump some attachability into your product, read on--this article is for you. Everyone else, this article will probably be less useful--but just as riveting.
|
||||||
|
|
||||||
#
|
|
||||||
|
|
||||||
# The Players
|
# The Players
|
||||||
|
|
||||||
So how do you get your profiler attached to a running process? The process has already started, and the CLR code which interrogates the environment to determine whether to load a profiler has already run. So how do you kick the process into loading your profiler? The answer: Another process!
|
So how do you get your profiler attached to a running process? The process has already started, and the CLR code which interrogates the environment to determine whether to load a profiler has already run. So how do you kick the process into loading your profiler? The answer: Another process!
|
||||||
|
@ -19,17 +17,17 @@ In order to force your profiler DLL to load into the target profilee process, yo
|
||||||
|
|
||||||
# Inside the Trigger Process
|
# Inside the Trigger Process
|
||||||
|
|
||||||
Your trigger uses a simple API method, AttachProfiler, to request the target process to load your profiler. Where is this method defined? Well, it doesn't make much sense to put it on ICorProfilerInfo, since that interface is only available to a profiler after it's been loaded. You could imagine a C export from mscoree.dll. But because of in-process side-by-side CLR instances, we're moving away from mscoree.dll exports to a COM-based interface model called "metahost".
|
Your trigger uses a simple API method, AttachProfiler, to request the target process to load your profiler. Where is this method defined? Well, it doesn't make much sense to put it on ICorProfilerInfo, since that interface is only available to a profiler after it's been loaded. You could imagine a C export from mscoree.dll. But because of in-process side-by-side CLR instances, we're moving away from mscoree.dll exports to a COM-based interface model called "metahost".
|
||||||
|
|
||||||
## Meta-whos-its?
|
## Meta-whos-its?
|
||||||
|
|
||||||
Whereas the "hosting" interfaces enable one to host and manage a CLR in a process, the "metahost" interfaces allow one to manage multiple CLRs that may be installed onto a machine or loaded into a single process. Here's a high-level view of how you navigate your way through metahost to find AttachProfiler() (there’s a pointer to actual sample code below).
|
Whereas the "hosting" interfaces enable one to host and manage a CLR in a process, the "metahost" interfaces allow one to manage multiple CLRs that may be installed onto a machine or loaded into a single process. Here's a high-level view of how you navigate your way through metahost to find AttachProfiler() (there’s a pointer to actual sample code below).
|
||||||
|
|
||||||
- Get ICLRMetaHost
|
- Get ICLRMetaHost
|
||||||
- Enumerate the CLRs loaded into the target process
|
- Enumerate the CLRs loaded into the target process
|
||||||
- Get ICLRRuntimeInfo for the particular CLR in the target process you want to profile
|
- Get ICLRRuntimeInfo for the particular CLR in the target process you want to profile
|
||||||
- Get the corresponding ICLRProfiling
|
- Get the corresponding ICLRProfiling
|
||||||
- Call ICLRProfiling::AttachProfiler
|
- Call ICLRProfiling::AttachProfiler
|
||||||
|
|
||||||
## Users and Integrity
|
## Users and Integrity
|
||||||
|
|
||||||
|
@ -57,8 +55,6 @@ From your InitializeForAttach implementation, your profiler will call SetEventMa
|
||||||
|
|
||||||
It was impossible to enable all profiling scenarios for attach in the time we had for the V4 release. So only profilers that do **sampling** and **memory** analysis will function properly after attaching to a live process. Attempts to use other profiling APIs after attach will be met with CORPROF\_E\_UNSUPPORTED\_FOR\_ATTACHING\_PROFILER.
|
It was impossible to enable all profiling scenarios for attach in the time we had for the V4 release. So only profilers that do **sampling** and **memory** analysis will function properly after attaching to a live process. Attempts to use other profiling APIs after attach will be met with CORPROF\_E\_UNSUPPORTED\_FOR\_ATTACHING\_PROFILER.
|
||||||
|
|
||||||
###
|
|
||||||
|
|
||||||
## Specific Callback Limitations
|
## Specific Callback Limitations
|
||||||
|
|
||||||
When your attaching profiler calls SetEventMask, you will be limited to only those event mask flags present in the COR\_PRF\_ALLOWABLE\_AFTER\_ATTACH bitmask (you'll find it in corprof.idl). Any other flags, and SetEventMask will return CORPROF\_E\_UNSUPPORTED\_FOR\_ATTACHING\_PROFILER.
|
When your attaching profiler calls SetEventMask, you will be limited to only those event mask flags present in the COR\_PRF\_ALLOWABLE\_AFTER\_ATTACH bitmask (you'll find it in corprof.idl). Any other flags, and SetEventMask will return CORPROF\_E\_UNSUPPORTED\_FOR\_ATTACHING\_PROFILER.
|
||||||
|
@ -67,14 +63,14 @@ When your attaching profiler calls SetEventMask, you will be limited to only tho
|
||||||
|
|
||||||
Most of the ICorProfilerInfo\* methods are available to your attaching profiler, however some are not--particularly those involved in **IL rewriting**. Here's a list of all ICorProfilerInfo\* methods NOT supported for attaching profilers:
|
Most of the ICorProfilerInfo\* methods are available to your attaching profiler, however some are not--particularly those involved in **IL rewriting**. Here's a list of all ICorProfilerInfo\* methods NOT supported for attaching profilers:
|
||||||
|
|
||||||
- GetILFunctionBody
|
- GetILFunctionBody
|
||||||
- GetILFunctionBodyAllocator
|
- GetILFunctionBodyAllocator
|
||||||
- SetILFunctionBody
|
- SetILFunctionBody
|
||||||
- SetILInstrumentedCodeMap
|
- SetILInstrumentedCodeMap
|
||||||
- SetEnterLeaveFunctionHooks\*
|
- SetEnterLeaveFunctionHooks\*
|
||||||
- SetFunctionIDMapper\*
|
- SetFunctionIDMapper\*
|
||||||
- GetNotifiedExceptionClauseInfo
|
- GetNotifiedExceptionClauseInfo
|
||||||
- All methods related to Enter/Leave/Tailcall
|
- All methods related to Enter/Leave/Tailcall
|
||||||
|
|
||||||
It's expected that future releases of the CLR will enable more API methods for use by attaching profilers.
|
It's expected that future releases of the CLR will enable more API methods for use by attaching profilers.
|
||||||
|
|
||||||
|
@ -84,9 +80,9 @@ It's expected that future releases of the CLR will enable more API methods for u
|
||||||
|
|
||||||
To understand limitations around the GC modes, here's a quick review of the GC modes an app can run under:
|
To understand limitations around the GC modes, here's a quick review of the GC modes an app can run under:
|
||||||
|
|
||||||
- **Workstation Blocking mode**. The thread that triggered the GC performs the GC while all other threads executing managed code must wait.
|
- **Workstation Blocking mode**. The thread that triggered the GC performs the GC while all other threads executing managed code must wait.
|
||||||
- **Workstation Concurrent / Background mode (the default)**. Concurrent GC (V1 & V2) allows portions of a full GC to execute while other threads are allowed to run. Background GC (its replacement in V4) takes it one step further, and also allows an ephemeral GC (i.e., gen 0 or gen 1) to execute while a gen 2 GC is executing.
|
- **Workstation Concurrent / Background mode (the default)**. Concurrent GC (V1 & V2) allows portions of a full GC to execute while other threads are allowed to run. Background GC (its replacement in V4) takes it one step further, and also allows an ephemeral GC (i.e., gen 0 or gen 1) to execute while a gen 2 GC is executing.
|
||||||
- **Server mode**. Hosts like ASP.NET may choose to enable server mode which creates a heap + dedicated GC thread per CPU. This allows GCs to be fanned out to multiple threads.
|
- **Server mode**. Hosts like ASP.NET may choose to enable server mode which creates a heap + dedicated GC thread per CPU. This allows GCs to be fanned out to multiple threads.
|
||||||
|
|
||||||
Of course, [Maoni's blog](https://devblogs.microsoft.com/dotnet/author/maoni/) is required reading for anyone who wants to understand how the GC works.
|
Of course, [Maoni's blog](https://devblogs.microsoft.com/dotnet/author/maoni/) is required reading for anyone who wants to understand how the GC works.
|
||||||
|
|
||||||
|
@ -96,15 +92,13 @@ So here's the catch. What if a V4 app starts up in background GC mode _without_
|
||||||
|
|
||||||
Of course, you could forcibly turn off concurrent / background mode every time the app starts up via a config file:
|
Of course, you could forcibly turn off concurrent / background mode every time the app starts up via a config file:
|
||||||
|
|
||||||
|
|
```xml
|
||||||
|
<configuration>
|
||||||
\<configuration\>
|
<runtime>
|
||||||
\<runtime\>
|
<gcConcurrent enabled="false"/>
|
||||||
\<gcConcurrent enabled="false"/\>
|
</runtime>
|
||||||
\</runtime\>
|
</configuration>
|
||||||
\</configuration\>
|
```
|
||||||
|
|
||||||
|
|
|
||||||
|
|
||||||
But you don't really want to be running your apps with a sub-optimal GC mode all the time, just on the off-chance you might need to attach a memory profiler to it. If you suspect you might need to do some memory profiling of a client app, you should just start up your app with the memory profiler to begin with.
|
But you don't really want to be running your apps with a sub-optimal GC mode all the time, just on the off-chance you might need to attach a memory profiler to it. If you suspect you might need to do some memory profiling of a client app, you should just start up your app with the memory profiler to begin with.
|
||||||
|
|
||||||
|
|
|
@ -11,12 +11,12 @@ A profiler that loads on startup of an application has the option to know the en
|
||||||
[NoBirthAnnouncement](media/NoBirthAnnouncement.JPG)
|
[NoBirthAnnouncement](media/NoBirthAnnouncement.JPG)
|
||||||
|
|
||||||
Drawing by Magdalena Hermawan
|
Drawing by Magdalena Hermawan
|
||||||
|
|
||||||
|
|
||||||
There are two fundamental ways your profiler can catch up on the current state of an application:
|
There are two fundamental ways your profiler can catch up on the current state of an application:
|
||||||
|
|
||||||
- Lazy catch-up—as the profiler encounters new IDs, the profiler queries information about those IDs as it needs them, rather than assuming it has a full cache that’s always built up as the IDs are first created. This is analogous to Dorothy meeting a new grown-up, and gracefully accepting the fact that that person exists.
|
- Lazy catch-up—as the profiler encounters new IDs, the profiler queries information about those IDs as it needs them, rather than assuming it has a full cache that’s always built up as the IDs are first created. This is analogous to Dorothy meeting a new grown-up, and gracefully accepting the fact that that person exists.
|
||||||
- Enumeration—for certain kinds of IDs, the profiler can (at attach time) request a complete list of the currently active IDs and query information about them at that time. Sort of like Dorothy first going to the Oz City Hall and looking up the birth records for everyone.
|
- Enumeration—for certain kinds of IDs, the profiler can (at attach time) request a complete list of the currently active IDs and query information about them at that time. Sort of like Dorothy first going to the Oz City Hall and looking up the birth records for everyone.
|
||||||
|
|
||||||
Lazy catch-up is fairly self-explanatory. For example, if your sampling profiler encounters an IP in a FunctionID you’ve never seen before, just look up whatever info you need about that FunctionID the first time you encounter it, rather than assuming you’d already built up a cache when the function was first JITted. And if you discover that FunctionID resides in a module you’ve never seen before, then just look up whatever info you need about that ModuleID at that point, rather than assuming you already have a complete cache of all modules. Many of you are already doing something like this today if you support sampling against regular NGENd images (since you don’t get JIT notifications of those functions anyway).
|
Lazy catch-up is fairly self-explanatory. For example, if your sampling profiler encounters an IP in a FunctionID you’ve never seen before, just look up whatever info you need about that FunctionID the first time you encounter it, rather than assuming you’d already built up a cache when the function was first JITted. And if you discover that FunctionID resides in a module you’ve never seen before, then just look up whatever info you need about that ModuleID at that point, rather than assuming you already have a complete cache of all modules. Many of you are already doing something like this today if you support sampling against regular NGENd images (since you don’t get JIT notifications of those functions anyway).
|
||||||
|
|
||||||
|
@ -26,8 +26,8 @@ Enumeration, on the other hand, has some caveats and is worthwhile to describe i
|
||||||
|
|
||||||
Some kinds of IDs have new enumerator methods as part of the profiling API. In particular:
|
Some kinds of IDs have new enumerator methods as part of the profiling API. In particular:
|
||||||
|
|
||||||
- ICorProfilerInfo3::EnumModules
|
- ICorProfilerInfo3::EnumModules
|
||||||
- ICorProfilerInfo3::EnumJITedFunctions
|
- ICorProfilerInfo3::EnumJITedFunctions
|
||||||
|
|
||||||
Your profiler calls these methods, and they return a standard enumerator you use to iterate through all of the currently-loaded IDs of that type. It’s worth noting that EnumJITedFunctions only enumerates FunctionIDs for which you would receive JITCompilationStarted/Finished events, and will not include FunctionIDs from NGENd modules.
|
Your profiler calls these methods, and they return a standard enumerator you use to iterate through all of the currently-loaded IDs of that type. It’s worth noting that EnumJITedFunctions only enumerates FunctionIDs for which you would receive JITCompilationStarted/Finished events, and will not include FunctionIDs from NGENd modules.
|
||||||
|
|
||||||
|
@ -39,24 +39,24 @@ As you may recall, once your profiler is attached to the process, the CLR calls
|
||||||
|
|
||||||
Bad timeline (loading; enumerating too soon):
|
Bad timeline (loading; enumerating too soon):
|
||||||
|
|
||||||
1. Profiler attaches
|
1. Profiler attaches
|
||||||
2. Profiler calls EnumModules
|
2. Profiler calls EnumModules
|
||||||
3. Module starts to load
|
3. Module starts to load
|
||||||
4. ModuleID is now enumerable
|
4. ModuleID is now enumerable
|
||||||
5. ModuleLoadFinished event would fire here if events were enabled (but they’re not yet!)
|
5. ModuleLoadFinished event would fire here if events were enabled (but they’re not yet!)
|
||||||
6. CLR enables events
|
6. CLR enables events
|
||||||
|
|
||||||
The problem is that the profiler calls EnumModules too early. If your profiler only calls EnumModules after CLR enables events, then you’re assured of either seeing a ModuleID via EnumModules or via a ModuleLoad event. In the above scenario, your profiler might as well have never done enumeration at all, since it will still not be notified of the ModuleID before it comes across that ModuleID in action later on. It gets even worse for modules that unload:
|
The problem is that the profiler calls EnumModules too early. If your profiler only calls EnumModules after CLR enables events, then you’re assured of either seeing a ModuleID via EnumModules or via a ModuleLoad event. In the above scenario, your profiler might as well have never done enumeration at all, since it will still not be notified of the ModuleID before it comes across that ModuleID in action later on. It gets even worse for modules that unload:
|
||||||
|
|
||||||
Bad timeline (unloading; enumerating too soon):
|
Bad timeline (unloading; enumerating too soon):
|
||||||
|
|
||||||
1. Module loads
|
1. Module loads
|
||||||
2. ModuleID is now enumerable
|
2. ModuleID is now enumerable
|
||||||
3. Profiler attaches
|
3. Profiler attaches
|
||||||
4. Profiler calls EnumModules (includes the ModuleID)
|
4. Profiler calls EnumModules (includes the ModuleID)
|
||||||
5. Module starts to unload
|
5. Module starts to unload
|
||||||
6. ModuleUnloadStarted event would fire here if events were enabled (but they’re not yet!)
|
6. ModuleUnloadStarted event would fire here if events were enabled (but they’re not yet!)
|
||||||
7. CLR enables events
|
7. CLR enables events
|
||||||
|
|
||||||
In the above case, the profiler discovers a ModuleID via EnumModules, but has no idea that the module is now in the process of unloading. So the profiler might query information about the stale ModuleID, potentially causing an AV. Again, this is caused because the profiler called the enumeration API too soon (i.e., before the CLR enabled event callbacks).
|
In the above case, the profiler discovers a ModuleID via EnumModules, but has no idea that the module is now in the process of unloading. So the profiler might query information about the stale ModuleID, potentially causing an AV. Again, this is caused because the profiler called the enumeration API too soon (i.e., before the CLR enabled event callbacks).
|
||||||
|
|
||||||
|
@ -68,24 +68,24 @@ When your profiler calls the Enum\* methods, the CLR creates a snapshot of all
|
||||||
|
|
||||||
Bad timeline (loading):
|
Bad timeline (loading):
|
||||||
|
|
||||||
1. Module starts to load
|
1. Module starts to load
|
||||||
2. ModuleLoadFinished event would fire here if events were enabled (but they’re not yet—no profiler is attached!)
|
2. ModuleLoadFinished event would fire here if events were enabled (but they’re not yet—no profiler is attached!)
|
||||||
3. Profiler attaches
|
3. Profiler attaches
|
||||||
4. CLR enables events, calls ProfilerAttachComplete()
|
4. CLR enables events, calls ProfilerAttachComplete()
|
||||||
5. Profiler calls EnumModules
|
5. Profiler calls EnumModules
|
||||||
6. ModuleID is now enumerable
|
6. ModuleID is now enumerable
|
||||||
|
|
||||||
Because 2 comes before 6, it’s possible for a profiler to attach and grab an enumeration in the middle, and thus never hear about a ModuleID (even though the profiler avoided Race #1 from the previous section). Again, an even worse problem occurs for module unloading. Suppose the CLR were to change an ID’s enumerable status to false after sending the unload event. That would also lead to holes:
|
Because 2 comes before 6, it’s possible for a profiler to attach and grab an enumeration in the middle, and thus never hear about a ModuleID (even though the profiler avoided Race #1 from the previous section). Again, an even worse problem occurs for module unloading. Suppose the CLR were to change an ID’s enumerable status to false after sending the unload event. That would also lead to holes:
|
||||||
|
|
||||||
Bad timeline (unloading):
|
Bad timeline (unloading):
|
||||||
|
|
||||||
1. Module loads, event would fire if profiler were attached (but it’s not), then ModuleID becomes enumerable
|
1. Module loads, event would fire if profiler were attached (but it’s not), then ModuleID becomes enumerable
|
||||||
2. Module starts to unload
|
2. Module starts to unload
|
||||||
3. ModuleUnloadStarted event would fire here if events were enabled (but they’re not yet—no profiler is attached!)
|
3. ModuleUnloadStarted event would fire here if events were enabled (but they’re not yet—no profiler is attached!)
|
||||||
4. Profiler attaches
|
4. Profiler attaches
|
||||||
5. CLR enables events, calls ProfilerAttachComplete()
|
5. CLR enables events, calls ProfilerAttachComplete()
|
||||||
6. Profiler calls EnumModules (ModuleID is still enumerable, so profiler discovers ModuleID at this point)
|
6. Profiler calls EnumModules (ModuleID is still enumerable, so profiler discovers ModuleID at this point)
|
||||||
7. ModuleID is no longer enumerable
|
7. ModuleID is no longer enumerable
|
||||||
|
|
||||||
Because 3 comes before 7, a profiler could attach in the middle, grab an enumeration, discover the ModuleID via the enumeration, and have no idea that module was in the process of unloading. If the profiler were to use that ModuleID later on, an AV could result. The above led to the following golden rule:
|
Because 3 comes before 7, a profiler could attach in the middle, grab an enumeration, discover the ModuleID via the enumeration, and have no idea that module was in the process of unloading. If the profiler were to use that ModuleID later on, an AV could result. The above led to the following golden rule:
|
||||||
|
|
||||||
|
@ -93,10 +93,10 @@ Because 3 comes before 7, a profiler could attach in the middle, grab an enumera
|
||||||
|
|
||||||
In other words, an ID becomes enumerable _before_ the LoadFinished (or JITCompilationFinished) event. And an ID ceases to be enumerable _before_ the UnloadStarted event. Or you can think of it as, “The event is always last”. This eliminates any potential holes. So to be even more explicit, here’s the enumerability vs. event ordering:
|
In other words, an ID becomes enumerable _before_ the LoadFinished (or JITCompilationFinished) event. And an ID ceases to be enumerable _before_ the UnloadStarted event. Or you can think of it as, “The event is always last”. This eliminates any potential holes. So to be even more explicit, here’s the enumerability vs. event ordering:
|
||||||
|
|
||||||
1. ID available in enumerations snapped now
|
1. ID available in enumerations snapped now
|
||||||
2. LoadFinished
|
2. LoadFinished
|
||||||
3. ID no longer in enumerations snapped now
|
3. ID no longer in enumerations snapped now
|
||||||
4. UnloadStarted
|
4. UnloadStarted
|
||||||
|
|
||||||
If an ID is present, the profiler will discover the ID via the enumerator or a LoadFinished event (or both). If an ID is not present, the profiler will either not see the ID via the enumerator or will see an UnloadStarted event (or both). In all cases, the event is more recent, and so the profiler should always trust an event over an enumeration that was generated prior. (More on that last point later.)
|
If an ID is present, the profiler will discover the ID via the enumerator or a LoadFinished event (or both). If an ID is not present, the profiler will either not see the ID via the enumerator or will see an UnloadStarted event (or both). In all cases, the event is more recent, and so the profiler should always trust an event over an enumeration that was generated prior. (More on that last point later.)
|
||||||
|
|
||||||
|
@ -104,36 +104,36 @@ The astute reader will notice that what we’ve done here is trade one race for
|
||||||
|
|
||||||
Good timeline (loading with duplicate):
|
Good timeline (loading with duplicate):
|
||||||
|
|
||||||
1. Module starts to load
|
1. Module starts to load
|
||||||
2. ModuleID is now enumerable
|
2. ModuleID is now enumerable
|
||||||
3. Profiler attaches
|
3. Profiler attaches
|
||||||
4. CLR enables events, calls ProfilerAttachComplete()
|
4. CLR enables events, calls ProfilerAttachComplete()
|
||||||
5. Profiler calls EnumModules
|
5. Profiler calls EnumModules
|
||||||
6. Profiler receives ModuleLoadFinished
|
6. Profiler receives ModuleLoadFinished
|
||||||
|
|
||||||
At first it might seem a little strange. The enumerator contains the ModuleID, so the profiler sees that the module is loaded. But then the profiler receives a ModuleLoadFinished event, which might seem odd, since the enumerator implied the module was already loaded. This is what I mean by “duplicate”—the profiler is notified of a ModuleID twice (once via the enumeration, and once via the event). The profiler will need to be resilient to this. Although it’s a bit awkward, it’s better than the alternative of a hole, since the profiler would have no way to know the hole occurred. Unloading has a similar situation:
|
At first it might seem a little strange. The enumerator contains the ModuleID, so the profiler sees that the module is loaded. But then the profiler receives a ModuleLoadFinished event, which might seem odd, since the enumerator implied the module was already loaded. This is what I mean by “duplicate”—the profiler is notified of a ModuleID twice (once via the enumeration, and once via the event). The profiler will need to be resilient to this. Although it’s a bit awkward, it’s better than the alternative of a hole, since the profiler would have no way to know the hole occurred. Unloading has a similar situation:
|
||||||
|
|
||||||
Good timeline (unloading with duplicate):
|
Good timeline (unloading with duplicate):
|
||||||
|
|
||||||
1. Module loads, event would have fired if profiler were attached (but it’s not), ModuleID becomes enumerable
|
1. Module loads, event would have fired if profiler were attached (but it’s not), ModuleID becomes enumerable
|
||||||
2. Module starts to unload
|
2. Module starts to unload
|
||||||
3. ModuleID is no longer enumerable
|
3. ModuleID is no longer enumerable
|
||||||
4. Profiler attaches
|
4. Profiler attaches
|
||||||
5. CLR enables events, calls ProfilerAttachComplete()
|
5. CLR enables events, calls ProfilerAttachComplete()
|
||||||
6. Profiler calls EnumModules
|
6. Profiler calls EnumModules
|
||||||
7. Profiler receives ModuleUnloadStarted event
|
7. Profiler receives ModuleUnloadStarted event
|
||||||
|
|
||||||
In step 6, the profiler does not see the unloading ModuleID (since it’s no longer enumerable). But in step 7 the profiler is notified that the ModuleID is unloading. Perhaps it’s a bit awkward that the profiler would be told that a seemingly nonexistent ModuleID is unloading. But again, this is better than the alternative, where a profiler finds an unloading ID in the enumeration, and is never told that the ModuleID got unloaded. One more case that’s worthwhile to bring out occurs when we move the profiler attach a bit earlier in the sequence.
|
In step 6, the profiler does not see the unloading ModuleID (since it’s no longer enumerable). But in step 7 the profiler is notified that the ModuleID is unloading. Perhaps it’s a bit awkward that the profiler would be told that a seemingly nonexistent ModuleID is unloading. But again, this is better than the alternative, where a profiler finds an unloading ID in the enumeration, and is never told that the ModuleID got unloaded. One more case that’s worthwhile to bring out occurs when we move the profiler attach a bit earlier in the sequence.
|
||||||
|
|
||||||
Good timeline (unloading without duplicate):
|
Good timeline (unloading without duplicate):
|
||||||
|
|
||||||
1. Module loads, event would fire if profiler were attached, ModuleID becomes enumerable
|
1. Module loads, event would fire if profiler were attached, ModuleID becomes enumerable
|
||||||
2. Module starts to unload
|
2. Module starts to unload
|
||||||
3. Profiler attaches
|
3. Profiler attaches
|
||||||
4. CLR enables events, calls ProfilerAttachComplete()
|
4. CLR enables events, calls ProfilerAttachComplete()
|
||||||
5. Profiler calls EnumModules (ModuleID is still present in the enumeration)
|
5. Profiler calls EnumModules (ModuleID is still present in the enumeration)
|
||||||
6. ModuleID is no longer enumerable
|
6. ModuleID is no longer enumerable
|
||||||
7. Profiler receives ModuleUnloadStarted event
|
7. Profiler receives ModuleUnloadStarted event
|
||||||
|
|
||||||
Here the profiler discovers the ModuleID exists in step 5 (as the ModuleID is still enumerable at that point), but the profiler almost immediately after discovers that the module is unloading in step 7. As stated above, events are more recent, and should always take precedence over enumerations that were generated prior. This could get a bit tricky, though, as the profiler generates an enumeration before it iterates over the enumeration. In the above sequence, the enumeration is generated in step 5. However, the profiler could be iterating though the generated enumeration for quite some time, and might not come across the unloading ModuleID until after step 7 (multiple threads means fun for everyone!). For this reason, it’s important for the profiler to give precedence to events that occur after the enumeration was _generated_, even though iteration over that enumeration might occur later.
|
Here the profiler discovers the ModuleID exists in step 5 (as the ModuleID is still enumerable at that point), but the profiler almost immediately after discovers that the module is unloading in step 7. As stated above, events are more recent, and should always take precedence over enumerations that were generated prior. This could get a bit tricky, though, as the profiler generates an enumeration before it iterates over the enumeration. In the above sequence, the enumeration is generated in step 5. However, the profiler could be iterating though the generated enumeration for quite some time, and might not come across the unloading ModuleID until after step 7 (multiple threads means fun for everyone!). For this reason, it’s important for the profiler to give precedence to events that occur after the enumeration was _generated_, even though iteration over that enumeration might occur later.
|
||||||
|
|
||||||
|
@ -151,7 +151,5 @@ It may be beneficial to program your profiler such that, upon attaching to the p
|
||||||
|
|
||||||
It’s worth reiterating a limitation I stated in the first attach post (linked above): the ObjectAllocated() callback is unavailable to profilers that attach to running processes. Therefore, any logic your profiler has that assumes it gets all the ObjectAllocated() callbacks will need to be addressed. Any objects newly allocated since the last GC may still be unknown to your profiler until it comes across their references via GC callbacks during the next GC (unless your profiler comes across those objects in other ways—example: as parameters to methods you hook with the Enter/Leave/Tailcall probes).
|
It’s worth reiterating a limitation I stated in the first attach post (linked above): the ObjectAllocated() callback is unavailable to profilers that attach to running processes. Therefore, any logic your profiler has that assumes it gets all the ObjectAllocated() callbacks will need to be addressed. Any objects newly allocated since the last GC may still be unknown to your profiler until it comes across their references via GC callbacks during the next GC (unless your profiler comes across those objects in other ways—example: as parameters to methods you hook with the Enter/Leave/Tailcall probes).
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
OK, that about covers the first steps your profiler should take once it attaches to a running process. It will either need to use lazy catch-up or the catch-up enumerations (or, quite likely, a combination of both). When using the enumerations, be careful to avoid holes (by calling the enumeration methods from inside ProfilerAttachComplete()), and be resilient to receiving information duplicated across the enumeration and the load / unload events. For memory profilers, be wary of GCs already in progress at the time your profiler attaches, and consider inducing your own GC at attach-time to build your initial cache of GC objects.
|
OK, that about covers the first steps your profiler should take once it attaches to a running process. It will either need to use lazy catch-up or the catch-up enumerations (or, quite likely, a combination of both). When using the enumerations, be careful to avoid holes (by calling the enumeration methods from inside ProfilerAttachComplete()), and be resilient to receiving information duplicated across the enumeration and the load / unload events. For memory profilers, be wary of GCs already in progress at the time your profiler attaches, and consider inducing your own GC at attach-time to build your initial cache of GC objects.
|
||||||
|
|
||||||
|
|
|
@ -11,24 +11,24 @@ On the other hand, if you're hijacking or otherwise calling ICorProfilerInfo fun
|
||||||
|
|
||||||
In 2.0 we've added some simple checks to help you avoid this problem. If you call an unsafe ICorProfilerInfo function asynchronously, instead of crossing its fingers and trying, it will fail with CORPROF\_E\_UNSUPPORTED\_CALL\_SEQUENCE. The general rule of thumb is, nothing is safe to call asynchronously. But here are the exceptions that are safe, and that we specifically allow to be called asynchronously:
|
In 2.0 we've added some simple checks to help you avoid this problem. If you call an unsafe ICorProfilerInfo function asynchronously, instead of crossing its fingers and trying, it will fail with CORPROF\_E\_UNSUPPORTED\_CALL\_SEQUENCE. The general rule of thumb is, nothing is safe to call asynchronously. But here are the exceptions that are safe, and that we specifically allow to be called asynchronously:
|
||||||
|
|
||||||
- GetEventMask/SetEventMask
|
- GetEventMask/SetEventMask
|
||||||
- GetCurrentThreadID
|
- GetCurrentThreadID
|
||||||
- GetThreadContext
|
- GetThreadContext
|
||||||
- GetThreadAppDomain
|
- GetThreadAppDomain
|
||||||
- GetFunctionFromIP
|
- GetFunctionFromIP
|
||||||
- GetFunctionInfo/GetFunctionInfo2
|
- GetFunctionInfo/GetFunctionInfo2
|
||||||
- GetCodeInfo/GetCodeInfo2
|
- GetCodeInfo/GetCodeInfo2
|
||||||
- GetModuleInfo
|
- GetModuleInfo
|
||||||
- GetClassIDInfo/GetClassIDInfo2
|
- GetClassIDInfo/GetClassIDInfo2
|
||||||
- IsArrayClass
|
- IsArrayClass
|
||||||
- SetFunctionIDMapper
|
- SetFunctionIDMapper
|
||||||
- DoStackSnapshot
|
- DoStackSnapshot
|
||||||
|
|
||||||
There are also a few things to keep in mind:
|
There are also a few things to keep in mind:
|
||||||
|
|
||||||
1. ICorProfilerInfo calls made from within the fast-path Enter/Leave callbacks are considered asynchronous. (Though ICorProfilerInfo calls made from within the _slow_-path Enter/Leave callbacks are considered synchronous.) See the blog entries [here](ELT - The Basics.md) and [here](http://blogs.msdn.com/jkeljo/archive/2005/08/11/450506.aspx) for more info on fast / slow path.
|
1. ICorProfilerInfo calls made from within the fast-path Enter/Leave callbacks are considered asynchronous. (Though ICorProfilerInfo calls made from within the _slow_-path Enter/Leave callbacks are considered synchronous.) See the blog entries [here](ELT - The Basics.md) and [here](http://blogs.msdn.com/jkeljo/archive/2005/08/11/450506.aspx) for more info on fast / slow path.
|
||||||
2. ICorProfilerInfo calls made from within instrumented code (i.e., IL you've rewritten to call into your profiler and then into ICorProfilerInfo) are considered asynchronous.
|
2. ICorProfilerInfo calls made from within instrumented code (i.e., IL you've rewritten to call into your profiler and then into ICorProfilerInfo) are considered asynchronous.
|
||||||
3. Calls made inside your FunctionIDMapper hook are considered to be synchronous.
|
3. Calls made inside your FunctionIDMapper hook are considered to be synchronous.
|
||||||
4. Calls made on threads created by your profiler, are always considered to be synchronous. (This is because there's no danger of conflicts resulting from interrupting and then re-entering the CLR on that thread, since a profiler-created thread was not in the CLR to begin with.)
|
4. Calls made on threads created by your profiler, are always considered to be synchronous. (This is because there's no danger of conflicts resulting from interrupting and then re-entering the CLR on that thread, since a profiler-created thread was not in the CLR to begin with.)
|
||||||
5. Calls made inside a StackSnapshotCallback are considered to be synchronous iff the call to DoStackSnapshot was synchronous.
|
5. Calls made inside a StackSnapshotCallback are considered to be synchronous iff the call to DoStackSnapshot was synchronous.
|
||||||
|
|
||||||
|
|
|
@ -15,9 +15,8 @@ Environment variables --\> Registry --\> Profiler DLL on File system.
|
||||||
|
|
||||||
The first link in this chain is to check the environment variables inside the process that should be profiled. If you're running the process from a command-prompt, you can just try a "set co" from the command prompt:
|
The first link in this chain is to check the environment variables inside the process that should be profiled. If you're running the process from a command-prompt, you can just try a "set co" from the command prompt:
|
||||||
|
|
||||||
|
|
|
||||||
```
|
```
|
||||||
**C:\>** set co
|
C:\> set co
|
||||||
(blah blah, other vars beginning with "co")
|
(blah blah, other vars beginning with "co")
|
||||||
```
|
```
|
||||||
|
|
||||||
|
@ -25,7 +24,6 @@ The first link in this chain is to check the environment variables inside the pr
|
||||||
Cor_Enable_Profiling=0x1
|
Cor_Enable_Profiling=0x1
|
||||||
COR_PROFILER={C5F90153-B93E-4138-9DB7-EB7156B07C4C}
|
COR_PROFILER={C5F90153-B93E-4138-9DB7-EB7156B07C4C}
|
||||||
```
|
```
|
||||||
|
|
|
||||||
|
|
||||||
If your scenario doesn't allow you to just run the process from a command prompt, like say an asp.net scenario, you may want to attach a debugger to the process that's supposed to be profiled, or use IFEO (HKEY\_LOCAL\_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Image File Execution Options) to force a debugger to start when the worker process starts. In the debugger, you can then use "!peb" to view the environment block, which will include the environment variables.
|
If your scenario doesn't allow you to just run the process from a command prompt, like say an asp.net scenario, you may want to attach a debugger to the process that's supposed to be profiled, or use IFEO (HKEY\_LOCAL\_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Image File Execution Options) to force a debugger to start when the worker process starts. In the debugger, you can then use "!peb" to view the environment block, which will include the environment variables.
|
||||||
|
|
||||||
|
@ -62,4 +60,3 @@ or even set a breakpoint inside your Profiler DLL's **DllMain.** Now go, and s
|
||||||
If you're still going strong, set a breakpoint in your profiler's **Initialize** () callback. Failures here are actually a popular cause for activation problems. Inside your Initialize() callback, your profiler is likely calling QueryInterface for the ICorProfilerInfoX interface of your choice, and then calling SetEventMask, and doing other initialization-related tasks, like calling SetEnterLeaveFunctionHooks(2). Do any of these fail? Is your Initialize() callback returning a failure HRESULT?
|
If you're still going strong, set a breakpoint in your profiler's **Initialize** () callback. Failures here are actually a popular cause for activation problems. Inside your Initialize() callback, your profiler is likely calling QueryInterface for the ICorProfilerInfoX interface of your choice, and then calling SetEventMask, and doing other initialization-related tasks, like calling SetEnterLeaveFunctionHooks(2). Do any of these fail? Is your Initialize() callback returning a failure HRESULT?
|
||||||
|
|
||||||
Hopefully by now you've isolated the failure point. If not, and your Initialize() is happily returning S\_OK, then your profiler is apparently loading just fine. At least it is when you're debugging it. :-)
|
Hopefully by now you've isolated the failure point. If not, and your Initialize() is happily returning S\_OK, then your profiler is apparently loading just fine. At least it is when you're debugging it. :-)
|
||||||
|
|
||||||
|
|
|
@ -9,9 +9,8 @@ SOS.DLL is a debugger extension DLL that ships with the CLR. You'll find it sit
|
||||||
|
|
||||||
In windbg, you'll need mscorwks.dll to load first, and then you can load SOS. Often, I don't need SOS until well into my debugging session, at which point mscorwks.dll has already been loaded anyway. However, there are some cases where you'd like SOS loaded at the first possible moment, so you can use some of its commands early (like !bpmd to set a breakpoint on a managed method). So a surefire way to get SOS loaded ASAP is to have the debugger break when mscorwks gets loaded (e.g., "sxe ld mscorwks"). Once mscorwks is loaded, you can load SOS using the .loadby command:
|
In windbg, you'll need mscorwks.dll to load first, and then you can load SOS. Often, I don't need SOS until well into my debugging session, at which point mscorwks.dll has already been loaded anyway. However, there are some cases where you'd like SOS loaded at the first possible moment, so you can use some of its commands early (like !bpmd to set a breakpoint on a managed method). So a surefire way to get SOS loaded ASAP is to have the debugger break when mscorwks gets loaded (e.g., "sxe ld mscorwks"). Once mscorwks is loaded, you can load SOS using the .loadby command:
|
||||||
|
|
||||||
|
|
|
||||||
```
|
```
|
||||||
0:000\> **sxe ld mscorwks**
|
0:000\> sxe ld mscorwks
|
||||||
0:000\> g
|
0:000\> g
|
||||||
ModLoad: 79e70000 7a3ff000 C:\Windows\Microsoft.NET\Framework\v2.0.50727\mscorwks.dll
|
ModLoad: 79e70000 7a3ff000 C:\Windows\Microsoft.NET\Framework\v2.0.50727\mscorwks.dll
|
||||||
eax=00000000 ebx=00000000 ecx=00000000 edx=00000000 esi=7efdd000 edi=20000000
|
eax=00000000 ebx=00000000 ecx=00000000 edx=00000000 esi=7efdd000 edi=20000000
|
||||||
|
@ -19,9 +18,8 @@ In windbg, you'll need mscorwks.dll to load first, and then you can load SOS. O
|
||||||
cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00000202
|
cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00000202
|
||||||
ntdll!NtMapViewOfSection+0x12:
|
ntdll!NtMapViewOfSection+0x12:
|
||||||
77a1a9fa c22800 ret 28h
|
77a1a9fa c22800 ret 28h
|
||||||
0:000\> **.loadby sos mscorwks**
|
0:000\> .loadby sos mscorwks
|
||||||
```
|
```
|
||||||
|
|
|
||||||
|
|
||||||
With SOS loaded, you can now use its commands to inspect the various IDs that the profiling API passes to your profiler.
|
With SOS loaded, you can now use its commands to inspect the various IDs that the profiling API passes to your profiler.
|
||||||
|
|
||||||
|
@ -35,7 +33,6 @@ As far as your profiler is concerned, a FunctionID is just an opaque number. It
|
||||||
|
|
||||||
Ok, so FunctionID = (MethodDesc \*). How does that help you? SOS just so happens to have a command to inspect MethodDescs: !dumpmd. So if you're in a debugger looking at your profiler code that's operating on a FunctionID, it can beneficial to you to find out which function that FunctionID actually refers to. In the example below, the debugger will break in my proifler's JITCompilationStarted callback and look at the FunctionID. It's assumed that you've already loaded SOS as per above.
|
Ok, so FunctionID = (MethodDesc \*). How does that help you? SOS just so happens to have a command to inspect MethodDescs: !dumpmd. So if you're in a debugger looking at your profiler code that's operating on a FunctionID, it can beneficial to you to find out which function that FunctionID actually refers to. In the example below, the debugger will break in my proifler's JITCompilationStarted callback and look at the FunctionID. It's assumed that you've already loaded SOS as per above.
|
||||||
|
|
||||||
|
|
|
||||||
```
|
```
|
||||||
0:000\> bu UnitTestSampleProfiler!SampleCallbackImpl::JITCompilationStarted
|
0:000\> bu UnitTestSampleProfiler!SampleCallbackImpl::JITCompilationStarted
|
||||||
0:000\> g
|
0:000\> g
|
||||||
|
@ -50,36 +47,30 @@ Breakpoint 0 hit
|
||||||
UnitTestSampleProfiler!SampleCallbackImpl::JITCompilationStarted:
|
UnitTestSampleProfiler!SampleCallbackImpl::JITCompilationStarted:
|
||||||
10003fc0 55 push ebp
|
10003fc0 55 push ebp
|
||||||
```
|
```
|
||||||
|
|
|
||||||
|
|
||||||
The debugger is now sitting at the beginning of my profiler's JITCompilationStarted callback. Let's take a look at the parameters.
|
The debugger is now sitting at the beginning of my profiler's JITCompilationStarted callback. Let's take a look at the parameters.
|
||||||
|
|
||||||
|
|
|
||||||
```
|
```
|
||||||
0:000\> dv
|
0:000\> dv
|
||||||
this = 0x00c133f8
|
this = 0x00c133f8
|
||||||
**functionID = 0x1e3170**
|
functionID = 0x1e3170
|
||||||
fIsSafeToBlock = 1
|
fIsSafeToBlock = 1
|
||||||
```
|
```
|
||||||
|
|
|
||||||
|
|
||||||
Aha, that's the FunctionID about to get JITted. Now use SOS to see what that function really is.
|
Aha, that's the FunctionID about to get JITted. Now use SOS to see what that function really is.
|
||||||
|
|
||||||
|
|
|
||||||
```
|
```
|
||||||
0:000\> !dumpmd 0x1e3170
|
0:000\> !dumpmd 0x1e3170
|
||||||
Method Name: test.Class1.Main(System.String[])
|
Method Name: test.Class1.Main(System.String[])
|
||||||
Class: 001e1288
|
Class: 001e1288
|
||||||
**MethodTable: 001e3180** mdToken: 06000001
|
MethodTable: 001e3180 mdToken: 06000001
|
||||||
Module: 001e2d8c
|
Module: 001e2d8c
|
||||||
IsJitted: no
|
IsJitted: no
|
||||||
m\_CodeOrIL: ffffffff
|
m\_CodeOrIL: ffffffff
|
||||||
```
|
```
|
||||||
|
|
|
||||||
|
|
||||||
Lots of juicy info here, though the Method Name typically is what helps me the most in my debugging sessions. mdToken tells us the metadata token for this method. MethodTable tells us where another internal CLR data structure is stored that contains information about the class containing the function. In fact, the profiing API's ClassID is simply a MethodTable \*. [Note: the "Class: 001e1288" in the output above is very different from the MethodTable, and thus different from the profiling API's ClassID. Don't let the name fool you!] So we could go and inspect a bit further by dumping information about the MethodTable:
|
Lots of juicy info here, though the Method Name typically is what helps me the most in my debugging sessions. mdToken tells us the metadata token for this method. MethodTable tells us where another internal CLR data structure is stored that contains information about the class containing the function. In fact, the profiing API's ClassID is simply a MethodTable \*. [Note: the "Class: 001e1288" in the output above is very different from the MethodTable, and thus different from the profiling API's ClassID. Don't let the name fool you!] So we could go and inspect a bit further by dumping information about the MethodTable:
|
||||||
|
|
||||||
|
|
|
||||||
```
|
```
|
||||||
0:000\> !dumpmt 0x001e3180
|
0:000\> !dumpmt 0x001e3180
|
||||||
EEClass: 001e1288
|
EEClass: 001e1288
|
||||||
|
@ -91,7 +82,6 @@ Lots of juicy info here, though the Method Name typically is what helps me the m
|
||||||
Number of IFaces in IFaceMap: 0
|
Number of IFaces in IFaceMap: 0
|
||||||
Slots in VTable: 6
|
Slots in VTable: 6
|
||||||
```
|
```
|
||||||
|
|
|
||||||
|
|
||||||
And of course, !dumpmt can be used anytime you come across a ClassID and want more info on it.
|
And of course, !dumpmt can be used anytime you come across a ClassID and want more info on it.
|
||||||
|
|
||||||
|
@ -126,17 +116,15 @@ It would probably be quicker to list what _isn't_ useful! I encourage you to do
|
||||||
|
|
||||||
!bpmd lets you place a breakpoint on a managed method. Just specify the module name and the fully-qualified method name. For example:
|
!bpmd lets you place a breakpoint on a managed method. Just specify the module name and the fully-qualified method name. For example:
|
||||||
|
|
||||||
|
|
|
||||||
```
|
```
|
||||||
!bpmd MyModule.exe MyNamespace.MyClass.Foo
|
!bpmd MyModule.exe MyNamespace.MyClass.Foo
|
||||||
```
|
```
|
||||||
|
|
|
||||||
|
|
||||||
If the method hasn't jitted yet, no worries. A "pending" breakpoint is placed. If your profiler performs IL rewriting, then using !bpmd on startup to set a managed breakpoint can be a handy way to break into the debugger just before your instrumented code will run (which, in turn, is typically just after your instrumented code has been jitted). This can help you in reproducing and diagnosing issues your profiler may run into when instrumenting particular functions (due to something interesting about the signature, generics, etc.).
|
If the method hasn't jitted yet, no worries. A "pending" breakpoint is placed. If your profiler performs IL rewriting, then using !bpmd on startup to set a managed breakpoint can be a handy way to break into the debugger just before your instrumented code will run (which, in turn, is typically just after your instrumented code has been jitted). This can help you in reproducing and diagnosing issues your profiler may run into when instrumenting particular functions (due to something interesting about the signature, generics, etc.).
|
||||||
|
|
||||||
!PrintException: If you use this without arguments you get to see a pretty-printing of the last outstanding managed exception on the thread; or specify a particular Exception object's address.
|
!PrintException: If you use this without arguments you get to see a pretty-printing of the last outstanding managed exception on the thread; or specify a particular Exception object's address.
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
Ok, that about does it for SOS. Hopefully this info can help you track down problems a little faster, or better yet, perhaps this can help you step through and verify your code before problems arise.
|
Ok, that about does it for SOS. Hopefully this info can help you track down problems a little faster, or better yet, perhaps this can help you step through and verify your code before problems arise.
|
||||||
|
|
||||||
|
|
|
@ -5,9 +5,6 @@ In my initial [post](DoStackSnapshot - Exception Filters.md) about DoStackSnapsh
|
||||||
|
|
||||||
The quick answer is that **nonvolatile (i.e., preserved), integer registers** should be valid. You don't really need many registers to walk the stack anyway. Obviously, you want a good stack pointer and instruction pointer. And hey, a frame pointer is handy when you come across an EBP-based frame in x86 (RBP on x64). These are all included in the set, of course. Specifically by architecture, you can trust these fields in your context:
|
The quick answer is that **nonvolatile (i.e., preserved), integer registers** should be valid. You don't really need many registers to walk the stack anyway. Obviously, you want a good stack pointer and instruction pointer. And hey, a frame pointer is handy when you come across an EBP-based frame in x86 (RBP on x64). These are all included in the set, of course. Specifically by architecture, you can trust these fields in your context:
|
||||||
|
|
||||||
x86: Edi, Esi, Ebx, Ebp, Esp, Eip
|
- x86: Edi, Esi, Ebx, Ebp, Esp, Eip
|
||||||
x64: Rdi, Rsi, Rbx, Rbp, Rsp, Rip, R12:R15
|
- x64: Rdi, Rsi, Rbx, Rbp, Rsp, Rip, R12:R15
|
||||||
ia64: IntS0:IntS3, RsBSP, StIFS, RsPFS, IntSp, StIIP, StIPSR
|
- ia64: IntS0:IntS3, RsBSP, StIFS, RsPFS, IntSp, StIIP, StIPSR
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
|
@ -7,38 +7,38 @@ For those of you diehard C# fans, you might be unaware of the existence of excep
|
||||||
|
|
||||||
First, a little background. For the full deal, check out the MSDN Library topic on VB.NET's [try/catch/finally statements](http://msdn.microsoft.com/library/default.asp?url=/library/en-us/vblr7/html/vastmTryCatchFinally.asp). But here's an appetizer. In VB.NET you can do this:
|
First, a little background. For the full deal, check out the MSDN Library topic on VB.NET's [try/catch/finally statements](http://msdn.microsoft.com/library/default.asp?url=/library/en-us/vblr7/html/vastmTryCatchFinally.asp). But here's an appetizer. In VB.NET you can do this:
|
||||||
|
|
||||||
```
|
```
|
||||||
Function Negative() As Boolean
|
Function Negative() As Boolean
|
||||||
Return False
|
Return False
|
||||||
End Function
|
End Function
|
||||||
|
|
||||||
Function Positive() As Boolean
|
Function Positive() As Boolean
|
||||||
Return True
|
Return True
|
||||||
End Function
|
End Function
|
||||||
|
|
||||||
Sub Thrower
|
Sub Thrower
|
||||||
Throw New Exception
|
Throw New Exception
|
||||||
End Sub
|
End Sub
|
||||||
|
|
||||||
Sub Main()
|
Sub Main()
|
||||||
Try
|
Try
|
||||||
Thrower()
|
Thrower()
|
||||||
Catch ex As Exception When Negative()
|
Catch ex As Exception When Negative()
|
||||||
MsgBox("Negative")
|
MsgBox("Negative")
|
||||||
Catch ex As Exception When Positive()
|
Catch ex As Exception When Positive()
|
||||||
MsgBox("Positive")
|
MsgBox("Positive")
|
||||||
End Try
|
End Try
|
||||||
End Sub
|
End Sub
|
||||||
```
|
```
|
||||||
|
|
||||||
The filters are the things that come after "When". We all know that, when an exception is thrown, its type must match the type specified in a Catch clause in order for that Catch clause to be executed. "When" is a way to further restrict whether a Catch clause will be executed. Now, not only must the exception's type match, but also the When clause must evaluate to True for that Catch clause to be chosen. In the example above, when we run, we'll skip the first Catch clause (because its filter returned False), and execute the second, thus showing a message box with "Positive" in it.
|
The filters are the things that come after "When". We all know that, when an exception is thrown, its type must match the type specified in a Catch clause in order for that Catch clause to be executed. "When" is a way to further restrict whether a Catch clause will be executed. Now, not only must the exception's type match, but also the When clause must evaluate to True for that Catch clause to be chosen. In the example above, when we run, we'll skip the first Catch clause (because its filter returned False), and execute the second, thus showing a message box with "Positive" in it.
|
||||||
|
|
||||||
The thing you need to realize about DoStackSnapshot's behavior (indeed, CLR in general) is that the execution of a When clause is really a separate function call. In the above example, imagine we take a stack snapshot while inside Positive(). Our managed-only stack trace, as reported by DoStackSnapshot, would then look like this (stack grows up):
|
The thing you need to realize about DoStackSnapshot's behavior (indeed, CLR in general) is that the execution of a When clause is really a separate function call. In the above example, imagine we take a stack snapshot while inside Positive(). Our managed-only stack trace, as reported by DoStackSnapshot, would then look like this (stack grows up):
|
||||||
|
|
||||||
Positive
|
Positive\
|
||||||
Main
|
Main\
|
||||||
Thrower
|
Thrower\
|
||||||
Main
|
Main
|
||||||
|
|
||||||
It's that highlighted Main that seems odd at first. While the exception is thrown inside Thrower(), the CLR needs to execute the filter clauses to figure out which Catch wins. These filter executions are actually _function calls_. Since filter clauses don't have their own names, we just use the name of the function containing the filter clause for stack reporting purposes. Thus, the highlighted Main above is the execution of a filter clause located inside Main (in this case, "When Positive()"). When each filter clause completes, we "return" back to Thrower() to continue our search for the filter that returns True. Since this is how the call stack is built up, that's what DoStackSnapshot will report.
|
It's that highlighted Main that seems odd at first. While the exception is thrown inside Thrower(), the CLR needs to execute the filter clauses to figure out which Catch wins. These filter executions are actually _function calls_. Since filter clauses don't have their own names, we just use the name of the function containing the filter clause for stack reporting purposes. Thus, the highlighted Main above is the execution of a filter clause located inside Main (in this case, "When Positive()"). When each filter clause completes, we "return" back to Thrower() to continue our search for the filter that returns True. Since this is how the call stack is built up, that's what DoStackSnapshot will report.
|
||||||
|
|
||||||
|
|
|
@ -5,14 +5,14 @@ Generally, corerror.h tells you all you need to know about what kinds of HRESULT
|
||||||
|
|
||||||
### E\_FAIL
|
### E\_FAIL
|
||||||
|
|
||||||
I don't much like E\_FAIL. If DoStackSnapshot fails, you will typically see a more descriptive, custom HRESULT. However, there are regrettably a few ways DoStackSnapshot can fail where you'll see the dreaded E\_FAIL instead. From your code's point of view, you shouldn't assume E\_FAIL will always imply one of the cases below (or conversely that each of these cases will always result in E\_FAIL). But this is just good stuff to know as you develop and debug your profiler, so you don't get blindsided.
|
I don't much like E\_FAIL. If DoStackSnapshot fails, you will typically see a more descriptive, custom HRESULT. However, there are regrettably a few ways DoStackSnapshot can fail where you'll see the dreaded E\_FAIL instead. From your code's point of view, you shouldn't assume E\_FAIL will always imply one of the cases below (or conversely that each of these cases will always result in E\_FAIL). But this is just good stuff to know as you develop and debug your profiler, so you don't get blindsided.
|
||||||
|
|
||||||
1) No managed frames on stack
|
1) No managed frames on stack
|
||||||
|
|
||||||
If you call DoStackSnapshot when there are no managed functions on your target thread's stack, you can get E\_FAIL. For example, if you try to walk the stack of a target thread very early on in its execution, there simply might not be any managed frames there yet. Or, if you try to walk the stack of the finalizer thread while it's waiting to do work, there will certainly be no managed frames on its stack. It's also possible that walking a stack with no managed frames on it will yield S\_OK instead of E\_FAIL (e.g., if the target thread is jit-compiling the first managed function to be called on that thread). Again, your code probably doesn't need to worry about all these cases. If we call your StackSnapshotCallback for a managed frame, you can trust that frame is there. If we don't call your StackSnapshotCallback, you can assume there are no managed frames on the stack.
|
If you call DoStackSnapshot when there are no managed functions on your target thread's stack, you can get E\_FAIL. For example, if you try to walk the stack of a target thread very early on in its execution, there simply might not be any managed frames there yet. Or, if you try to walk the stack of the finalizer thread while it's waiting to do work, there will certainly be no managed frames on its stack. It's also possible that walking a stack with no managed frames on it will yield S\_OK instead of E\_FAIL (e.g., if the target thread is jit-compiling the first managed function to be called on that thread). Again, your code probably doesn't need to worry about all these cases. If we call your StackSnapshotCallback for a managed frame, you can trust that frame is there. If we don't call your StackSnapshotCallback, you can assume there are no managed frames on the stack.
|
||||||
|
|
||||||
2) OS kernel handling a hardware exception
|
2) OS kernel handling a hardware exception
|
||||||
|
|
||||||
This one is less likely to happen, but it certainly can. When an app throws a hardware exception (e.g., divide by 0), the offending thread enters the Windows kernel. The kernel spends some time recording the thread's current user-mode register context, modifying some registers, and moving the instruction pointer to the user-mode exception dispatch routine. At this point the thread is ready to reenter user-mode. But if you are unlucky enough to call DoStackSnapshot while the target thread is still in the kernel doing this stuff, you will get E\_FAIL.
|
This one is less likely to happen, but it certainly can. When an app throws a hardware exception (e.g., divide by 0), the offending thread enters the Windows kernel. The kernel spends some time recording the thread's current user-mode register context, modifying some registers, and moving the instruction pointer to the user-mode exception dispatch routine. At this point the thread is ready to reenter user-mode. But if you are unlucky enough to call DoStackSnapshot while the target thread is still in the kernel doing this stuff, you will get E\_FAIL.
|
||||||
|
|
||||||
3) Detectably bad seed
|
3) Detectably bad seed
|
||||||
|
@ -25,11 +25,11 @@ Generally, this HRESULT means that your profiler requested to abort the stack wa
|
||||||
|
|
||||||
One of the beautiful things about running 64-bit Windows is that you can get the Windows OS to perform (native) stack walks for you. Read up on [RtlVirtualUnwind](http://msdn.microsoft.com/library/default.asp?url=/library/en-us/debug/base/rtlvirtualunwind.asp) if you're unfamiliar with this. The Windows OS has a critical section to protect a block of memory used to help perform this stack walk. So what would happen if:
|
One of the beautiful things about running 64-bit Windows is that you can get the Windows OS to perform (native) stack walks for you. Read up on [RtlVirtualUnwind](http://msdn.microsoft.com/library/default.asp?url=/library/en-us/debug/base/rtlvirtualunwind.asp) if you're unfamiliar with this. The Windows OS has a critical section to protect a block of memory used to help perform this stack walk. So what would happen if:
|
||||||
|
|
||||||
- The OS's exception handling code causes a thread to walk its own stack
|
- The OS's exception handling code causes a thread to walk its own stack
|
||||||
- The thread therefore enters this critical section
|
- The thread therefore enters this critical section
|
||||||
- Your profiler (via DoStackSnapshot) suspends this thread while the thread is still inside the critical section
|
- Your profiler (via DoStackSnapshot) suspends this thread while the thread is still inside the critical section
|
||||||
- DoStackSnapshot uses RtlVirtualUnwind to help walk this suspended thread
|
- DoStackSnapshot uses RtlVirtualUnwind to help walk this suspended thread
|
||||||
- RtlVirtualUnwind (executing on the current thread) tries to enter the critical section (already owned by suspended target thread)
|
- RtlVirtualUnwind (executing on the current thread) tries to enter the critical section (already owned by suspended target thread)
|
||||||
|
|
||||||
If your answer was "deadlock", congratulations! DoStackSnapshot has some code that tries to avoid this scenario, by aborting the stack walk before the deadlock can occur. When this happens, DoStackSnapshot will return CORPROF\_E\_STACKSNAPSHOT\_ABORTED. Note that this whole scenario is pretty rare, and only happens on WIN64.
|
If your answer was "deadlock", congratulations! DoStackSnapshot has some code that tries to avoid this scenario, by aborting the stack walk before the deadlock can occur. When this happens, DoStackSnapshot will return CORPROF\_E\_STACKSNAPSHOT\_ABORTED. Note that this whole scenario is pretty rare, and only happens on WIN64.
|
||||||
|
|
||||||
|
|
|
@ -5,79 +5,84 @@ The CLR Profiling API allows you to hook managed functions so that your profiler
|
||||||
|
|
||||||
### Setting up the hooks
|
### Setting up the hooks
|
||||||
|
|
||||||
1. On initialization, your profiler must call SetEnterLeaveFunctionHooks(2) to specify which functions inside your profiler should be called whenever a managed function is entered, returns, or exits via tail call, respectively.
|
1. On initialization, your profiler must call SetEnterLeaveFunctionHooks(2) to specify which functions inside your profiler should be called whenever a managed function is entered, returns, or exits via tail call, respectively.
|
||||||
_(Profiler calls this…)_
|
|
||||||
|
_(Profiler calls this…)_
|
||||||
|
|
||||||
```
|
```
|
||||||
HRESULT SetEnterLeaveFunctionHooks(
|
HRESULT SetEnterLeaveFunctionHooks(
|
||||||
[in] FunctionEnter \*pFuncEnter,
|
[in] FunctionEnter *pFuncEnter,
|
||||||
[in] FunctionLeave \*pFuncLeave,
|
[in] FunctionLeave *pFuncLeave,
|
||||||
[in] FunctionTailcall \*pFuncTailcall);
|
[in] FunctionTailcall *pFuncTailcall);
|
||||||
```
|
|
||||||
|
|
||||||
_(Profiler implements these…)_
|
|
||||||
```
|
|
||||||
typedef void FunctionEnter(FunctionID funcID);
|
|
||||||
typedef void FunctionLeave(FunctionID funcID);
|
|
||||||
typedef void FunctionTailcall(FunctionID funcID);
|
|
||||||
```
|
```
|
||||||
|
|
||||||
**OR**
|
_(Profiler implements these…)_
|
||||||
|
|
||||||
_(Profiler calls this…)_
|
```
|
||||||
```
|
typedef void FunctionEnter(FunctionID funcID);
|
||||||
HRESULT SetEnterLeaveFunctionHooks2(
|
typedef void FunctionLeave(FunctionID funcID);
|
||||||
[in] FunctionEnter2 *pFuncEnter,
|
typedef void FunctionTailcall(FunctionID funcID);
|
||||||
[in] FunctionLeave2 *pFuncLeave,
|
```
|
||||||
[in] FunctionTailcall2 *pFuncTailcall);
|
|
||||||
```
|
|
||||||
|
|
||||||
|
|
||||||
_(Profiler implements these…)_
|
**OR**
|
||||||
```
|
|
||||||
typedef void FunctionEnter2(
|
|
||||||
FunctionID funcId,
|
|
||||||
UINT_PTR clientData,
|
|
||||||
COR_PRF_FRAME_INFO func,
|
|
||||||
COR_PRF_FUNCTION_ARGUMENT_INFO *argumentInfo);
|
|
||||||
|
|
||||||
typedef void FunctionLeave2(
|
_(Profiler calls this…)_
|
||||||
FunctionID funcId,
|
|
||||||
UINT_PTR clientData,
|
|
||||||
COR_PRF_FRAME_INFO func,
|
|
||||||
COR_PRF_FUNCTION_ARGUMENT_RANGE *retvalRange);
|
|
||||||
|
|
||||||
typedef void FunctionTailcall2(
|
```
|
||||||
FunctionID funcId,
|
HRESULT SetEnterLeaveFunctionHooks2(
|
||||||
UINT_PTR clientData,
|
[in] FunctionEnter2 *pFuncEnter,
|
||||||
COR_PRF_FRAME_INFO func);
|
[in] FunctionLeave2 *pFuncLeave,
|
||||||
```
|
[in] FunctionTailcall2 *pFuncTailcall);
|
||||||
|
```
|
||||||
|
|
||||||
This step alone does not cause the enter/leave/tailcall (ELT) hooks to be called. But you must do this on startup to get things rolling.
|
_(Profiler implements these…)_
|
||||||
|
|
||||||
2. At any time during the run, your profiler calls SetEventMask specifying COR\_PRF\_MONITOR\_ENTERLEAVE in the bitmask. Your profiler may set or reset this flag at any time to cause ELT hooks to be called or ignored, respectively.
|
```
|
||||||
|
typedef void FunctionEnter2(
|
||||||
|
FunctionID funcId,
|
||||||
|
UINT_PTR clientData,
|
||||||
|
COR_PRF_FRAME_INFO func,
|
||||||
|
COR_PRF_FUNCTION_ARGUMENT_INFO *argumentInfo);
|
||||||
|
|
||||||
|
typedef void FunctionLeave2(
|
||||||
|
FunctionID funcId,
|
||||||
|
UINT_PTR clientData,
|
||||||
|
COR_PRF_FRAME_INFO func,
|
||||||
|
COR_PRF_FUNCTION_ARGUMENT_RANGE *retvalRange);
|
||||||
|
|
||||||
|
typedef void FunctionTailcall2(
|
||||||
|
FunctionID funcId,
|
||||||
|
UINT_PTR clientData,
|
||||||
|
COR_PRF_FRAME_INFO func);
|
||||||
|
```
|
||||||
|
|
||||||
|
This step alone does not cause the enter/leave/tailcall (ELT) hooks to be called. But you must do this on startup to get things rolling.
|
||||||
|
|
||||||
|
2. At any time during the run, your profiler calls SetEventMask specifying COR\_PRF\_MONITOR\_ENTERLEAVE in the bitmask. Your profiler may set or reset this flag at any time to cause ELT hooks to be called or ignored, respectively.
|
||||||
|
|
||||||
### FunctionIDMapper
|
### FunctionIDMapper
|
||||||
|
|
||||||
In addition to the above two steps, your profiler may specify more granularly which managed functions should have ELT hooks compiled into them:
|
In addition to the above two steps, your profiler may specify more granularly which managed functions should have ELT hooks compiled into them:
|
||||||
|
|
||||||
1. At any time, your profiler may call ICorProfilerInfo2::SetFunctionIDMapper to specify a special hook to be called when a function is JITted.
|
1. At any time, your profiler may call ICorProfilerInfo2::SetFunctionIDMapper to specify a special hook to be called when a function is JITted.
|
||||||
|
|
||||||
_(Profiler calls this…)_
|
_(Profiler calls this…)_
|
||||||
```
|
|
||||||
HRESULT SetFunctionIDMapper([in] FunctionIDMapper \*pFunc);
|
```
|
||||||
```
|
HRESULT SetFunctionIDMapper([in] FunctionIDMapper \*pFunc);
|
||||||
|
```
|
||||||
|
|
||||||
|
_(Profiler implements this…)_
|
||||||
|
|
||||||
|
```
|
||||||
|
typedef UINT_PTR __stdcall FunctionIDMapper(
|
||||||
|
FunctionID funcId,
|
||||||
|
BOOL *pbHookFunction);
|
||||||
|
```
|
||||||
|
|
||||||
_(Profiler implements this…)_
|
|
||||||
```
|
|
||||||
typedef UINT_PTR __stdcall FunctionIDMapper(
|
|
||||||
FunctionID funcId,
|
|
||||||
BOOL *pbHookFunction);
|
|
||||||
```
|
|
||||||
|
|
||||||
|
|
||||||
2. When FunctionIDMapper is called:
|
2. When FunctionIDMapper is called:
|
||||||
a. Your profiler sets the pbHookFunction [out] parameter appropriately to determine whether the function identified by funcId should have ELT hooks compiled into it.
|
a. Your profiler sets the pbHookFunction \[out] parameter appropriately to determine whether the function identified by funcId should have ELT hooks compiled into it.
|
||||||
b. Of course, the primary purpose of FunctionIDMapper is to allow your profiler to specify an alternate ID for that function. Your profiler does this by returning that ID from FunctionIDMapper . The CLR will pass this alternate ID to your ELT hooks (as funcID if you're using the 1.x ELT, and as clientData if you're using the 2.x ELT).
|
b. Of course, the primary purpose of FunctionIDMapper is to allow your profiler to specify an alternate ID for that function. Your profiler does this by returning that ID from FunctionIDMapper . The CLR will pass this alternate ID to your ELT hooks (as funcID if you're using the 1.x ELT, and as clientData if you're using the 2.x ELT).
|
||||||
|
|
||||||
### Writing your ELT hooks
|
### Writing your ELT hooks
|
||||||
|
@ -92,8 +97,6 @@ The solution is “NGEN /Profile”. For example, if you run this command agains
|
||||||
|
|
||||||
`ngen install MyAssembly.dll /Profile`
|
`ngen install MyAssembly.dll /Profile`
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
it will NGEN MyAssembly.dll with the “Profile” flavor (also called “profiler-enhanced”). This flavor causes extra hooks to be baked in to enable features like ELT hooks, loader callbacks, managed/unmanaged code transition callbacks, and the JITCachedFunctionSearchStarted/Finished callbacks.
|
it will NGEN MyAssembly.dll with the “Profile” flavor (also called “profiler-enhanced”). This flavor causes extra hooks to be baked in to enable features like ELT hooks, loader callbacks, managed/unmanaged code transition callbacks, and the JITCachedFunctionSearchStarted/Finished callbacks.
|
||||||
|
|
||||||
The original NGENd versions of all your assemblies still stay around in your NGEN cache. NGEN /Profile simply causes a new set of NGENd assemblies to be generated as well, marked as the “profiler-enhanced” set of NGENd assemblies. At run-time, the CLR determines which flavor should be loaded. If a profiler is attached and enables certain features that only work with profiler-enhanced (not regular) NGENd assemblies (such as ELT via a call to SetEnterLeaveFunctionHooks(2), or any of several other features that are requested by setting particular event flags via SetEventMask), then the CLR will only load profiler-enhanced NGENd images--and if none exist then the CLR degrades to JIT in order to support the features requested by the profiler. In contrast, if the profiler does not specify such event flags, or there is no profiler to begin with, then the CLR loads the regular-flavored NGENd assemblies.
|
The original NGENd versions of all your assemblies still stay around in your NGEN cache. NGEN /Profile simply causes a new set of NGENd assemblies to be generated as well, marked as the “profiler-enhanced” set of NGENd assemblies. At run-time, the CLR determines which flavor should be loaded. If a profiler is attached and enables certain features that only work with profiler-enhanced (not regular) NGENd assemblies (such as ELT via a call to SetEnterLeaveFunctionHooks(2), or any of several other features that are requested by setting particular event flags via SetEventMask), then the CLR will only load profiler-enhanced NGENd images--and if none exist then the CLR degrades to JIT in order to support the features requested by the profiler. In contrast, if the profiler does not specify such event flags, or there is no profiler to begin with, then the CLR loads the regular-flavored NGENd assemblies.
|
||||||
|
@ -128,4 +131,3 @@ Why do you care? Well, it's always good to know what price you're paying. If you
|
||||||
### Next time...
|
### Next time...
|
||||||
|
|
||||||
That about covers it for the ELT basics. Next installment of this riveting series will talk about that enigma known as tailcall.
|
That about covers it for the ELT basics. Next installment of this riveting series will talk about that enigma known as tailcall.
|
||||||
|
|
||||||
|
|
|
@ -79,7 +79,7 @@ typedef void FunctionTailcall2(
|
||||||
COR_PRF_FRAME_INFO func);
|
COR_PRF_FRAME_INFO func);
|
||||||
```
|
```
|
||||||
|
|
||||||
**Tip** : More than once I've seen profiler writers make the following mistake. They will take their naked assembly-language wrapper for their Enter2 and Leave2 hooks, and paste it again to use as the Tailcall2 assembly-language wrapper. The problem is they forget that the Tailcall2 hook takes a different number of parameters than the Enter2 / Leave2 hooks (or, more to the point, a different number of _bytes_ is passed on the stack to invoke the Tailcall2 hook). So, they'll take the "ret 16" at the end of their Enter2/Leave2 hook wrappers and stick that into their Tailcall2 hook wrapper, forgetting to change it to a "ret 12". Don't make the same mistake!
|
**Tip** : More than once I've seen profiler writers make the following mistake. They will take their naked assembly-language wrapper for their Enter2 and Leave2 hooks, and paste it again to use as the Tailcall2 assembly-language wrapper. The problem is they forget that the Tailcall2 hook takes a different number of parameters than the Enter2 / Leave2 hooks (or, more to the point, a different number of _bytes_ is passed on the stack to invoke the Tailcall2 hook). So, they'll take the "ret 16" at the end of their Enter2/Leave2 hook wrappers and stick that into their Tailcall2 hook wrapper, forgetting to change it to a "ret 12". Don't make the same mistake!
|
||||||
|
|
||||||
It's worth noting what these parameters mean. With the Enter and Leave hooks it's pretty obvious that the parameters your hook is given (e.g., funcId) apply to the function being Entered or Left. But what about the Tailcall hook? Do the Tailcall hook's parameters describe the caller (function making the tail call) or the callee (function being tail called into)?
|
It's worth noting what these parameters mean. With the Enter and Leave hooks it's pretty obvious that the parameters your hook is given (e.g., funcId) apply to the function being Entered or Left. But what about the Tailcall hook? Do the Tailcall hook's parameters describe the caller (function making the tail call) or the callee (function being tail called into)?
|
||||||
|
|
||||||
|
@ -95,7 +95,7 @@ Ok, enough dilly-dallying. What should your profiler do in its Tailcall hook? Tw
|
||||||
|
|
||||||
The [CLRProfiler](http://www.microsoft.com/downloads/details.aspx?FamilyID=a362781c-3870-43be-8926-862b40aa0cd0&DisplayLang=en) is a great example of using Enter/Leave/Tailcall hooks to maintain shadow stacks. A shadow stack is your profiler's own copy of the current stack of function calls on a given thread at any given time. Upon Enter of a function, you push that FunctionID (and whatever other info interests you, such as arguments) onto your data structure that represents that thread's stack. Upon Leave of a function, you pop that FunctionID. This gives you a live list of managed calls in play on the thread. The CLRProfiler uses shadow stacks so that whenever the managed app being profiled chooses to allocate a new object, the CLRProfiler can know the managed call stack that led to the allocation. (Note that an alternate way of accomplishing this would be to call DoStackSnapshot at every allocation point instead of maintaining a shadow stack. Since objects are allocated so frequently, however, you'd end up calling DoStackSnapshot extremely frequently and will often see worse performance than if you had been maintaining shadow stacks in the first place.)
|
The [CLRProfiler](http://www.microsoft.com/downloads/details.aspx?FamilyID=a362781c-3870-43be-8926-862b40aa0cd0&DisplayLang=en) is a great example of using Enter/Leave/Tailcall hooks to maintain shadow stacks. A shadow stack is your profiler's own copy of the current stack of function calls on a given thread at any given time. Upon Enter of a function, you push that FunctionID (and whatever other info interests you, such as arguments) onto your data structure that represents that thread's stack. Upon Leave of a function, you pop that FunctionID. This gives you a live list of managed calls in play on the thread. The CLRProfiler uses shadow stacks so that whenever the managed app being profiled chooses to allocate a new object, the CLRProfiler can know the managed call stack that led to the allocation. (Note that an alternate way of accomplishing this would be to call DoStackSnapshot at every allocation point instead of maintaining a shadow stack. Since objects are allocated so frequently, however, you'd end up calling DoStackSnapshot extremely frequently and will often see worse performance than if you had been maintaining shadow stacks in the first place.)
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
OK, so when your profiler maintains a shadow stack, it's clear what your profiler should do on Enter or Leave, but what should it do on Tailcall? There are a couple ways one could imagine answering that question, but only one of them will work! Taking the example from the top of this post, imagine the stack looks like this:
|
OK, so when your profiler maintains a shadow stack, it's clear what your profiler should do on Enter or Leave, but what should it do on Tailcall? There are a couple ways one could imagine answering that question, but only one of them will work! Taking the example from the top of this post, imagine the stack looks like this:
|
||||||
|
|
||||||
|
@ -121,9 +121,11 @@ Method 2: On tailcall, "mark" the FunctionID at the top of your stack as needing
|
||||||
|
|
||||||
With this strategy, for the duration of the call to Three(), the shadow stack will look like this:
|
With this strategy, for the duration of the call to Three(), the shadow stack will look like this:
|
||||||
|
|
||||||
Three
|
```
|
||||||
Helper (marked for deferred pop)
|
Three
|
||||||
Main
|
Helper (marked for deferred pop)
|
||||||
|
Main
|
||||||
|
```
|
||||||
|
|
||||||
which some might consider more user-friendly. And as soon as Three() returns, your profiler will sneakily do a double-pop leaving just this:
|
which some might consider more user-friendly. And as soon as Three() returns, your profiler will sneakily do a double-pop leaving just this:
|
||||||
|
|
||||||
|
@ -163,9 +165,11 @@ Method 2: Shadow stack fails
|
||||||
|
|
||||||
At stage (4), the shadow stack looks like this:
|
At stage (4), the shadow stack looks like this:
|
||||||
|
|
||||||
Helper
|
```
|
||||||
Thread.Sleep (marked for "deferred pop")
|
Helper
|
||||||
|
Thread.Sleep (marked for "deferred pop")
|
||||||
Main
|
Main
|
||||||
|
```
|
||||||
|
|
||||||
If you think it might be complicated to explain tail calls to your users so they can understand the Method 1 form of shadow stack presentation, just try explaining why it makes sense to present to them that Thread.Sleep() is calling Helper()!
|
If you think it might be complicated to explain tail calls to your users so they can understand the Method 1 form of shadow stack presentation, just try explaining why it makes sense to present to them that Thread.Sleep() is calling Helper()!
|
||||||
|
|
||||||
|
@ -184,11 +188,11 @@ static public void Main()
|
||||||
|
|
||||||
would yield:
|
would yield:
|
||||||
```
|
```
|
||||||
Helper
|
Helper
|
||||||
Thread.Sleep (marked for "deferred pop")
|
Thread.Sleep (marked for "deferred pop")
|
||||||
Thread.Sleep (marked for "deferred pop")
|
Thread.Sleep (marked for "deferred pop")
|
||||||
Thread.Sleep (marked for "deferred pop")
|
Thread.Sleep (marked for "deferred pop")
|
||||||
Thread.Sleep (marked for "deferred pop")
|
Thread.Sleep (marked for "deferred pop")
|
||||||
Main
|
Main
|
||||||
```
|
```
|
||||||
|
|
||||||
|
@ -211,11 +215,11 @@ static public void Helper()
|
||||||
|
|
||||||
would yield:
|
would yield:
|
||||||
```
|
```
|
||||||
Thread.Sleep (marked for "deferred pop")
|
Thread.Sleep (marked for "deferred pop")
|
||||||
Thread.Sleep (marked for "deferred pop")
|
Thread.Sleep (marked for "deferred pop")
|
||||||
Thread.Sleep (marked for "deferred pop")
|
Thread.Sleep (marked for "deferred pop")
|
||||||
Thread.Sleep (marked for "deferred pop")
|
Thread.Sleep (marked for "deferred pop")
|
||||||
Helper
|
Helper
|
||||||
Main
|
Main
|
||||||
```
|
```
|
||||||
|
|
||||||
|
@ -337,7 +341,7 @@ ildasm Class1.exe
|
||||||
Inside ildasm, use File.Dump to generate a text file that contains a textual representation of the IL from Class1.exe. Call it Class1WithTail.il. Open up that file and add the tail. prefix just before the call you want optimized into a tail call (see highlighted yellow for changes):
|
Inside ildasm, use File.Dump to generate a text file that contains a textual representation of the IL from Class1.exe. Call it Class1WithTail.il. Open up that file and add the tail. prefix just before the call you want optimized into a tail call (see highlighted yellow for changes):
|
||||||
|
|
||||||
```
|
```
|
||||||
.method private hidebysig static int32
|
.method private hidebysig static int32
|
||||||
Helper(int32 i) cil managed
|
Helper(int32 i) cil managed
|
||||||
{
|
{
|
||||||
~~// Code size 45 (0x2d)
|
~~// Code size 45 (0x2d)
|
||||||
|
@ -386,5 +390,5 @@ If you didn't learn anything, I hope you at least got some refreshing sleep than
|
||||||
- Since some managed functions may tail call into native helper functions inside the CLR (for which you won't get an Enter hook notification), your Tailcall hook should treat the tail call as if it were a Leave, and not depend on the next Enter hook correlating to the target of the last tail call. With shadow stacks, for example, this means you should simply pop the calling function off your shadow stack in your Tailcall hook.
|
- Since some managed functions may tail call into native helper functions inside the CLR (for which you won't get an Enter hook notification), your Tailcall hook should treat the tail call as if it were a Leave, and not depend on the next Enter hook correlating to the target of the last tail call. With shadow stacks, for example, this means you should simply pop the calling function off your shadow stack in your Tailcall hook.
|
||||||
- Since tail calls can be elusive to find in practice, it's well worth your while to use ildasm/ilasm to manufacture explicit tail calls so you can step through your Tailcall hook and test its logic.
|
- Since tail calls can be elusive to find in practice, it's well worth your while to use ildasm/ilasm to manufacture explicit tail calls so you can step through your Tailcall hook and test its logic.
|
||||||
|
|
||||||
_David has been a developer at Microsoft for over 70 years (allowing for his upcoming time-displacement correction). He joined Microsoft in 2079, first starting in the experimental time-travel group. His current assignment is to apply his knowledge of the future to eliminate the "Wait for V3" effect customers commonly experience in his source universe. By using Retroactive Hindsight-ellisenseTM his goal is to "get it right the first time, this time" in a variety of product groups._
|
_David has been a developer at Microsoft for over 70 years (allowing for his upcoming time-displacement correction). He joined Microsoft in 2079, first starting in the experimental time-travel group. His current assignment is to apply his knowledge of the future to eliminate the "Wait for V3" effect customers commonly experience in his source universe. By using Retroactive Hindsight-ellisenseTM his goal is to "get it right the first time, this time" in a variety of product groups._
|
||||||
|
|
||||||
|
|
|
@ -7,7 +7,7 @@ If you’re writing a profiler that you expect to run against CLR 2.0 or greater
|
||||||
|
|
||||||
Let's say a C# developer writes code like this:
|
Let's say a C# developer writes code like this:
|
||||||
|
|
||||||
|
|
||||||
```
|
```
|
||||||
class MyClass<S>
|
class MyClass<S>
|
||||||
{
|
{
|
||||||
|
@ -57,8 +57,6 @@ HRESULT GetFunctionInfo2([in] FunctionID funcId,
|
||||||
|
|
||||||
typeArgs[]: This is the array of **type arguments** to MyClass\<int\>.Foo\<float\>. So this will be an array of only one element: the ClassID for float. (The int in MyClass\<int\> is a type argument to MyClass, not to Foo, and you would only see that when you call GetClassIDInfo2 with MyClass\<int\>.)
|
typeArgs[]: This is the array of **type arguments** to MyClass\<int\>.Foo\<float\>. So this will be an array of only one element: the ClassID for float. (The int in MyClass\<int\> is a type argument to MyClass, not to Foo, and you would only see that when you call GetClassIDInfo2 with MyClass\<int\>.)
|
||||||
|
|
||||||
##
|
|
||||||
|
|
||||||
## GetClassIDInfo2
|
## GetClassIDInfo2
|
||||||
|
|
||||||
OK, someone in parentheses said something about calling GetClassIDInfo2, so let’s do that. Since we got the ClassID for MyClass\<int\> above, let’s pass it to GetClassIDInfo2 to see what we get:
|
OK, someone in parentheses said something about calling GetClassIDInfo2, so let’s do that. Since we got the ClassID for MyClass\<int\> above, let’s pass it to GetClassIDInfo2 to see what we get:
|
||||||
|
@ -89,21 +87,19 @@ To understand why, it’s necessary to understand an internal optimization the C
|
||||||
|
|
||||||
For now, the important point is that, once we’re inside JITted code that is shared across different generic instantiations, how can one know which instantiation is the actual one that caused the current invocation? Well, in many cases, the CLR may not have that data readily lying around. However, as a profiler, you can capture this information and pass it back to the CLR when it needs it. This is done through a COR\_PRF\_FRAME\_INFO. There are two ways your profiler can get a COR\_PRF\_FRAME\_INFO:
|
For now, the important point is that, once we’re inside JITted code that is shared across different generic instantiations, how can one know which instantiation is the actual one that caused the current invocation? Well, in many cases, the CLR may not have that data readily lying around. However, as a profiler, you can capture this information and pass it back to the CLR when it needs it. This is done through a COR\_PRF\_FRAME\_INFO. There are two ways your profiler can get a COR\_PRF\_FRAME\_INFO:
|
||||||
|
|
||||||
1. Via slow-path Enter/Leave/Tailcall probes
|
1. Via slow-path Enter/Leave/Tailcall probes
|
||||||
2. Via your DoStackSnapshot callback
|
2. Via your DoStackSnapshot callback
|
||||||
|
|
||||||
I lied. #1 is really the only way for your profiler to get a COR\_PRF\_FRAME\_INFO. #2 may seem like a way—at least the profiling API suggests that the CLR gives your profiler a COR\_PRF\_FRAME\_INFO in the DSS callback—but unfortunately the COR\_PRF\_FRAME\_INFO you get there is pretty useless. I suspect the COR\_PRF\_FRAME\_INFO parameter was added to the signature of the profiler’s DSS callback function so that it could “light up” at some point in the future when we could work on finding out how to create a sufficiently helpful COR\_PRF\_FRAME\_INFO during stack walks. However, that day has not yet arrived. So if you want a COR\_PRF\_FRAME\_INFO, you’ll need to grab it—and use it from—your slow-path Enter/Leave/Tailcall probe.
|
I lied. #1 is really the only way for your profiler to get a COR\_PRF\_FRAME\_INFO. #2 may seem like a way—at least the profiling API suggests that the CLR gives your profiler a COR\_PRF\_FRAME\_INFO in the DSS callback—but unfortunately the COR\_PRF\_FRAME\_INFO you get there is pretty useless. I suspect the COR\_PRF\_FRAME\_INFO parameter was added to the signature of the profiler’s DSS callback function so that it could “light up” at some point in the future when we could work on finding out how to create a sufficiently helpful COR\_PRF\_FRAME\_INFO during stack walks. However, that day has not yet arrived. So if you want a COR\_PRF\_FRAME\_INFO, you’ll need to grab it—and use it from—your slow-path Enter/Leave/Tailcall probe.
|
||||||
|
|
||||||
With a valid COR\_PRF\_FRAME\_INFO, GetFunctionInfo2 will give you helpful, specific ClassIDs in the typeArgs [out] array and pClassId [out] parameter. If the profiler passes NULL for COR\_PRF\_FRAME\_INFO, here’s what you can expect:
|
With a valid COR\_PRF\_FRAME\_INFO, GetFunctionInfo2 will give you helpful, specific ClassIDs in the typeArgs [out] array and pClassId [out] parameter. If the profiler passes NULL for COR\_PRF\_FRAME\_INFO, here’s what you can expect:
|
||||||
|
|
||||||
- If you’re using CLR V2, pClassId will point to NULL if the function sits on _any_ generic class (shared or not). In CLR V4 this got a little better, and you’ll generally only see pClassId point to NULL if the function sits on a “shared” generic class (instantiated with reference types).
|
- If you’re using CLR V2, pClassId will point to NULL if the function sits on _any_ generic class (shared or not). In CLR V4 this got a little better, and you’ll generally only see pClassId point to NULL if the function sits on a “shared” generic class (instantiated with reference types).
|
||||||
- Note: If it’s impossible for the profiler to have a COR\_PRF\_FRAME\_INFO handy to pass to GetFunctionInfo2, and that results in a NULL \*pClassID, the profiler can always use the metadata interfaces to find the mdTypeDef token of the class on which the function resides for the purposes of pretty-printing the class name to the user. Of course, the profiler will not know the specific instantiating type arguments that were used on the class in that case.
|
- Note: If it’s impossible for the profiler to have a COR\_PRF\_FRAME\_INFO handy to pass to GetFunctionInfo2, and that results in a NULL \*pClassID, the profiler can always use the metadata interfaces to find the mdTypeDef token of the class on which the function resides for the purposes of pretty-printing the class name to the user. Of course, the profiler will not know the specific instantiating type arguments that were used on the class in that case.
|
||||||
- the typeArgs [out] array will contain the ClassID for **System.\_\_Canon** , rather than the actual instantiating type(s), if the function itself is generic and is instantiated with reference type argument(s).
|
- the typeArgs [out] array will contain the ClassID for **System.\_\_Canon** , rather than the actual instantiating type(s), if the function itself is generic and is instantiated with reference type argument(s).
|
||||||
|
|
||||||
It’s worth noting here that there is a bug in GetFunctionInfo2, in that the [out] pClassId you get for the class containing the function can be wrong with generic virtual functions. Take a look at [this forum post](http://social.msdn.microsoft.com/Forums/en-US/netfxtoolsdev/thread/ed6f972f-712a-48df-8cce-74f8951503fa/) for more information and a workaround.
|
It’s worth noting here that there is a bug in GetFunctionInfo2, in that the [out] pClassId you get for the class containing the function can be wrong with generic virtual functions. Take a look at [this forum post](http://social.msdn.microsoft.com/Forums/en-US/netfxtoolsdev/thread/ed6f972f-712a-48df-8cce-74f8951503fa/) for more information and a workaround.
|
||||||
|
|
||||||
##
|
|
||||||
|
|
||||||
## ClassIDs & FunctionIDs vs. Metadata Tokens
|
## ClassIDs & FunctionIDs vs. Metadata Tokens
|
||||||
|
|
||||||
Although you can infer this from the above, let’s take a breather and review. When you have multiple generic instantiations of a generic type, that type is defined with one mdTypeDef (metadata token), but you’ll see multiple ClassIDs (one per instantiation). When you have multiple generic instantiations of a generic method, it’s defined with one mdMethodDef (metadata token), but you’ll see multiple FunctionIDs (one per instantiation).
|
Although you can infer this from the above, let’s take a breather and review. When you have multiple generic instantiations of a generic type, that type is defined with one mdTypeDef (metadata token), but you’ll see multiple ClassIDs (one per instantiation). When you have multiple generic instantiations of a generic method, it’s defined with one mdMethodDef (metadata token), but you’ll see multiple FunctionIDs (one per instantiation).
|
||||||
|
@ -120,14 +116,14 @@ If you got curious, and ran such a profiler under the debugger, you could use th
|
||||||
|
|
||||||
If your profiler performs IL rewriting, it’s important to understand that it must NOT do instantiation-specific IL rewriting. Huh? Let’s take an example. Suppose you’re profiling code that uses MyClass\<int\>.Foo\<float\> and MyClass\<int\>.Foo\<long\>. Your profiler will see two JITCompilationStarted callbacks, and will have two opportunities to rewrite the IL. Your profiler may call GetFunctionInfo2 on those two FunctionIDs and determine that they’re two different instantiations of the same generic function. You may then be tempted to make use of the fact that one is instantiated with float, and the other with long, and provide different IL for the two different JIT compilations. The problem with this is that the IL stored in metadata, as well as the IL provided to SetILFunctionBody, is always specified relative to the mdMethodDef. (Remember, SetILFunctionBody doesn’t take a FunctionID as input; it takes an mdMethodDef.) And it’s the profiler’s responsibility always to specify the same rewritten IL for any given mdMethodDef no matter how many times it’s JITted. And a given mdMethodDef can be JITted multiple times due to a number of reasons:
|
If your profiler performs IL rewriting, it’s important to understand that it must NOT do instantiation-specific IL rewriting. Huh? Let’s take an example. Suppose you’re profiling code that uses MyClass\<int\>.Foo\<float\> and MyClass\<int\>.Foo\<long\>. Your profiler will see two JITCompilationStarted callbacks, and will have two opportunities to rewrite the IL. Your profiler may call GetFunctionInfo2 on those two FunctionIDs and determine that they’re two different instantiations of the same generic function. You may then be tempted to make use of the fact that one is instantiated with float, and the other with long, and provide different IL for the two different JIT compilations. The problem with this is that the IL stored in metadata, as well as the IL provided to SetILFunctionBody, is always specified relative to the mdMethodDef. (Remember, SetILFunctionBody doesn’t take a FunctionID as input; it takes an mdMethodDef.) And it’s the profiler’s responsibility always to specify the same rewritten IL for any given mdMethodDef no matter how many times it’s JITted. And a given mdMethodDef can be JITted multiple times due to a number of reasons:
|
||||||
|
|
||||||
- Two threads simultaneously trying to call the same function for the first time (and thus both trying to JIT that function)
|
- Two threads simultaneously trying to call the same function for the first time (and thus both trying to JIT that function)
|
||||||
- Strange dependency chains involving class constructors (more on this in the MSDN [reference topic](http://msdn.microsoft.com/en-us/library/ms230586.aspx))
|
- Strange dependency chains involving class constructors (more on this in the MSDN [reference topic](http://msdn.microsoft.com/en-us/library/ms230586.aspx))
|
||||||
- Multiple AppDomains using the same (non-domain-neutral) function
|
- Multiple AppDomains using the same (non-domain-neutral) function
|
||||||
- And of course multiple generic instantiations!
|
- And of course multiple generic instantiations!
|
||||||
|
|
||||||
Regardless of the reason, the profiler must always rewrite with exactly the same IL. Otherwise, an invariant in the CLR will have been broken by the profiler, and you will get strange, undefined behavior as a result. And no one wants that.
|
Regardless of the reason, the profiler must always rewrite with exactly the same IL. Otherwise, an invariant in the CLR will have been broken by the profiler, and you will get strange, undefined behavior as a result. And no one wants that.
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
That’s it! Hopefully this gives you a good idea of how the CLR Profiling API will behave in the face of generic classes and functions, and what is expected of your profiler.
|
That’s it! Hopefully this gives you a good idea of how the CLR Profiling API will behave in the face of generic classes and functions, and what is expected of your profiler.
|
||||||
|
|
||||||
|
|
|
@ -31,16 +31,12 @@ Yes, that is a good example. You are an astute reader. Memory profilers that w
|
||||||
|
|
||||||
# Going from metadata token to run-time ID
|
# Going from metadata token to run-time ID
|
||||||
|
|
||||||
#
|
|
||||||
|
|
||||||
#
|
|
||||||
|
|
||||||
As I mentioned above, the safest way to do this is to build up your own map and do reverse-lookups as necessary. If that scheme meets your needs, then by all means do that, and stop reading! But in the cases where this is insufficient, you may need to resort to using GetFunctionFromToken(AndTypeArgs) and GetClassFromToken(AndTypeArgs). There is no simple, foolproof way to use these APIs safely, but here is your guideline:
|
As I mentioned above, the safest way to do this is to build up your own map and do reverse-lookups as necessary. If that scheme meets your needs, then by all means do that, and stop reading! But in the cases where this is insufficient, you may need to resort to using GetFunctionFromToken(AndTypeArgs) and GetClassFromToken(AndTypeArgs). There is no simple, foolproof way to use these APIs safely, but here is your guideline:
|
||||||
|
|
||||||
**Never call GetFunctionFromToken(AndTypeArgs) and GetClassFromToken(AndTypeArgs) unless you’re certain the relevant types have been loaded.** (“Relevant types” include the ClassID containing the FunctionID whose mdMethodDef you pass to GetFunctionFromToken(AndTypeArgs), and the ClassID whose mdTypeDef you pass to GetClassFromToken(AndTypeArgs).) If these types have not been loaded, _you may cause them to be loaded now_! This is bad because:
|
**Never call GetFunctionFromToken(AndTypeArgs) and GetClassFromToken(AndTypeArgs) unless you’re certain the relevant types have been loaded.** (“Relevant types” include the ClassID containing the FunctionID whose mdMethodDef you pass to GetFunctionFromToken(AndTypeArgs), and the ClassID whose mdTypeDef you pass to GetClassFromToken(AndTypeArgs).) If these types have not been loaded, _you may cause them to be loaded now_! This is bad because:
|
||||||
|
|
||||||
- This is an easy way to crash the app. Trying to load a type at the wrong time could cause cycles, causing infinite loops (depending on what your profiler does in response to class load notifications) or outright crashes. For example, trying to load a type while its containing assembly is still in an early phase of loading is a great and fun way to crash the CLR.
|
- This is an easy way to crash the app. Trying to load a type at the wrong time could cause cycles, causing infinite loops (depending on what your profiler does in response to class load notifications) or outright crashes. For example, trying to load a type while its containing assembly is still in an early phase of loading is a great and fun way to crash the CLR.
|
||||||
- You will impact the behavior of the app. If you’re lucky enough not to crash the app, you’ve still impacted its behavior, by causing types to get loaded in a different order than they normally would. Any impact to app behavior like this makes it difficult for your users to reproduce problems that they are trying to use your tool to diagnose, or may hide problems that they don’t discover until they run their application outside of your tool.
|
- You will impact the behavior of the app. If you’re lucky enough not to crash the app, you’ve still impacted its behavior, by causing types to get loaded in a different order than they normally would. Any impact to app behavior like this makes it difficult for your users to reproduce problems that they are trying to use your tool to diagnose, or may hide problems that they don’t discover until they run their application outside of your tool.
|
||||||
|
|
||||||
## Determining whether a class was loaded
|
## Determining whether a class was loaded
|
||||||
|
|
||||||
|
@ -54,14 +50,14 @@ MyRetType MyClass::MyFunction(MyArgumentType myArgumentType)
|
||||||
|
|
||||||
then you can be reasonably assured that the following are loaded:
|
then you can be reasonably assured that the following are loaded:
|
||||||
|
|
||||||
- MyClass
|
- MyClass
|
||||||
- MyArgumentType (if it’s a value-type)
|
- MyArgumentType (if it’s a value-type)
|
||||||
- MyRetType (if it’s a value-type)
|
- MyRetType (if it’s a value-type)
|
||||||
- For any class you know is loaded, so should be:
|
- For any class you know is loaded, so should be:
|
||||||
- its base class
|
- its base class
|
||||||
- its value-type fields (not necessarily reference-type fields!)
|
- its value-type fields (not necessarily reference-type fields!)
|
||||||
- implemented interfaces
|
- implemented interfaces
|
||||||
- value-type generic type arguments (and even reference-type generic type arguments in the case of MyClass)
|
- value-type generic type arguments (and even reference-type generic type arguments in the case of MyClass)
|
||||||
|
|
||||||
So much for stacks. What if you encounter an instance of a class on the heap? Surely the class is loaded then, right? Well, probably. If you encounter an object on heap just after GC (inside **GarbageCollectionFinished** , before you return), it should be safe to inspect the class’s layout, and then peek through ObjectIDs to see the values of their fields.
|
So much for stacks. What if you encounter an instance of a class on the heap? Surely the class is loaded then, right? Well, probably. If you encounter an object on heap just after GC (inside **GarbageCollectionFinished** , before you return), it should be safe to inspect the class’s layout, and then peek through ObjectIDs to see the values of their fields.
|
||||||
|
|
||||||
|
@ -73,7 +69,7 @@ In general, a lot of the uncertainty above comes from types stored in NGENd modu
|
||||||
|
|
||||||
Now is a good time remind you that, not only is it dangerous to inspect run-time IDs too early (i.e., before they load); it’s also dangerous to inspect run-time IDs too late (i.e., after they **unload** ). For example, if you store ClassIDs and FunctionIDs for later use, and use them “too late”, you can easily crash the CLR. The profiling API does pretty much no validation of anything (in many cases, it’s incapable of doing so without using up significant amounts of memory to maintain lookup tables for everything). So we generally take any run-time ID that you pass to ICorProfilerInfo\* methods, cast it to an internal CLR structure ptr, and go boom if the ID is bad.
|
Now is a good time remind you that, not only is it dangerous to inspect run-time IDs too early (i.e., before they load); it’s also dangerous to inspect run-time IDs too late (i.e., after they **unload** ). For example, if you store ClassIDs and FunctionIDs for later use, and use them “too late”, you can easily crash the CLR. The profiling API does pretty much no validation of anything (in many cases, it’s incapable of doing so without using up significant amounts of memory to maintain lookup tables for everything). So we generally take any run-time ID that you pass to ICorProfilerInfo\* methods, cast it to an internal CLR structure ptr, and go boom if the ID is bad.
|
||||||
|
|
||||||
There is no way to just ask the CLR if a FunctionID or ClassID is valid. Indeed, classes could get unloaded, and new classes loaded, and your ClassID may now refer to a totally different (valid) class.
|
There is no way to just ask the CLR if a FunctionID or ClassID is valid. Indeed, classes could get unloaded, and new classes loaded, and your ClassID may now refer to a totally different (valid) class.
|
||||||
|
|
||||||
You need to keep track of the unloads yourself. You are notified when run-time IDs go out of scope (today, this happens at the level of an AppDomain unloading or a collectible assembly unloading—in both cases all IDs “contained” in the unloading thing are now invalid). Once a run-time ID is out of scope, you are not allowed to pass that run-time ID back to the CLR. In fact, you should consider whether thread synchronization will be necessary in your profiler to maintain this invariant. For example, if a run-time ID gets unloaded on thread A, you’re still not allowed to pass that run-time ID back to the CLR on thread B. So you may need to block on a critical section in thread A during the \*UnloadStarted / AppDomainShutdown\* callbacks, to prevent them from returning to the CLR until any uses of the contained IDs in thread B are finished.
|
You need to keep track of the unloads yourself. You are notified when run-time IDs go out of scope (today, this happens at the level of an AppDomain unloading or a collectible assembly unloading—in both cases all IDs “contained” in the unloading thing are now invalid). Once a run-time ID is out of scope, you are not allowed to pass that run-time ID back to the CLR. In fact, you should consider whether thread synchronization will be necessary in your profiler to maintain this invariant. For example, if a run-time ID gets unloaded on thread A, you’re still not allowed to pass that run-time ID back to the CLR on thread B. So you may need to block on a critical section in thread A during the \*UnloadStarted / AppDomainShutdown\* callbacks, to prevent them from returning to the CLR until any uses of the contained IDs in thread B are finished.
|
||||||
|
|
||||||
|
@ -91,16 +87,16 @@ ResolveTypeRef doesn’t know about any of this—it was never designed to be us
|
||||||
|
|
||||||
If you absolutely need to resolve refs to defs, your best bet may be to use your own algorithm which will be as accurate as you can make it, under the circumstances, and which will never try to locate a module that hasn’t been loaded yet. That means that you shouldn’t try to resolve a ref to a def if that def hasn’t actually been loaded into a type by the CLR. Consider using an algorithm similar to the following:
|
If you absolutely need to resolve refs to defs, your best bet may be to use your own algorithm which will be as accurate as you can make it, under the circumstances, and which will never try to locate a module that hasn’t been loaded yet. That means that you shouldn’t try to resolve a ref to a def if that def hasn’t actually been loaded into a type by the CLR. Consider using an algorithm similar to the following:
|
||||||
|
|
||||||
1. Get the AssemblyRef from the TypeRef to get to the name, public key token and version of the assembly where the type should reside.
|
1. Get the AssemblyRef from the TypeRef to get to the name, public key token and version of the assembly where the type should reside.
|
||||||
2. Enumerate all loaded modules that the Profiling API has notified you of (or via [EnumModules](http://msdn.microsoft.com/en-us/library/dd490890)) (you can filter out a specific AppDomain at this point if you want).
|
2. Enumerate all loaded modules that the Profiling API has notified you of (or via [EnumModules](http://msdn.microsoft.com/en-us/library/dd490890)) (you can filter out a specific AppDomain at this point if you want).
|
||||||
3. In each enumerated module, search for a TypeDef with the same name and namespace as the TypeRef (IMetaDataImport::FindTypeDefByName)
|
3. In each enumerated module, search for a TypeDef with the same name and namespace as the TypeRef (IMetaDataImport::FindTypeDefByName)
|
||||||
4. Pay attention to **type forwarding**! Once you find the TypeDef, it may actually be an “exported” type, in which case you will need to follow the trail to the next module. Read toward the bottom of [this post](Type Forwarding.md) for more info.
|
4. Pay attention to **type forwarding**! Once you find the TypeDef, it may actually be an “exported” type, in which case you will need to follow the trail to the next module. Read toward the bottom of [this post](Type Forwarding.md) for more info.
|
||||||
|
|
||||||
The above can be a little bit smarter by paying attention to what order you choose to search through the modules:
|
The above can be a little bit smarter by paying attention to what order you choose to search through the modules:
|
||||||
|
|
||||||
- First search for the TypeDef in assemblies which exactly match the name, public key token and version for the AssemblyRef.
|
- First search for the TypeDef in assemblies which exactly match the name, public key token and version for the AssemblyRef.
|
||||||
- If that fails, then search through assemblies matching name and public key token (where the version is higher than the one supplied – this can happen for Framework assemblies).
|
- If that fails, then search through assemblies matching name and public key token (where the version is higher than the one supplied – this can happen for Framework assemblies).
|
||||||
- If that fails, then search through all the other assemblies
|
- If that fails, then search through all the other assemblies
|
||||||
|
|
||||||
I must warn you that the above scheme is **not tested and not supported. Use at your own risk!**
|
I must warn you that the above scheme is **not tested and not supported. Use at your own risk!**
|
||||||
|
|
||||||
|
@ -108,7 +104,7 @@ I must warn you that the above scheme is **not tested and not supported. Use at
|
||||||
|
|
||||||
Although I cannot comment on what will or will not be in any particular future version of the CLR, I can tell you that it is clear to us on the CLR team that we have work to do, to make dealing with metadata tokens and their corresponding run-time type information easier from the profiling API. After all, it doesn’t take a rocket scientist to read the above and conclude that it does take a rocket scientist to actually follow all this advice. So for now, enjoy the fact that what you do is really hard, making you difficult to replace, and thus your job all the more secure. You’re welcome.
|
Although I cannot comment on what will or will not be in any particular future version of the CLR, I can tell you that it is clear to us on the CLR team that we have work to do, to make dealing with metadata tokens and their corresponding run-time type information easier from the profiling API. After all, it doesn’t take a rocket scientist to read the above and conclude that it does take a rocket scientist to actually follow all this advice. So for now, enjoy the fact that what you do is really hard, making you difficult to replace, and thus your job all the more secure. You’re welcome.
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
Special thanks to David Wrighton and Karel Zikmund, who have helped considerably with all content in this entry around the type system and metadata.
|
Special thanks to David Wrighton and Karel Zikmund, who have helped considerably with all content in this entry around the type system and metadata.
|
||||||
|
|
||||||
|
|
|
@ -9,9 +9,9 @@ The Detach feature allows a profiler that the user is finished with to be unload
|
||||||
|
|
||||||
Not every V4 profiler is allowed to detach from a running process. The general rule is that a profiler which has caused an irreversible impact in the process it’s profiling should _not_ attempt to detach. The CLR catches the following cases:
|
Not every V4 profiler is allowed to detach from a running process. The general rule is that a profiler which has caused an irreversible impact in the process it’s profiling should _not_ attempt to detach. The CLR catches the following cases:
|
||||||
|
|
||||||
- Profiler set immutable flags (COR\_PRF\_MONITOR\_IMMUTABLE) via SetEventMask.
|
- Profiler set immutable flags (COR\_PRF\_MONITOR\_IMMUTABLE) via SetEventMask.
|
||||||
- Profiler performed IL rewriting via SetILFunctionBody
|
- Profiler performed IL rewriting via SetILFunctionBody
|
||||||
- Profiler used the Enter/Leave/Tailcall methods to add callouts to its probes
|
- Profiler used the Enter/Leave/Tailcall methods to add callouts to its probes
|
||||||
|
|
||||||
If the profiler attempts to detach after doing any of the above, the CLR will disallow the attempt (see below for details).
|
If the profiler attempts to detach after doing any of the above, the CLR will disallow the attempt (see below for details).
|
||||||
|
|
||||||
|
@ -25,20 +25,20 @@ There’s one, deceptively simple-looking method the profiler calls to detach it
|
||||||
|
|
||||||
So, the sequence works like this:
|
So, the sequence works like this:
|
||||||
|
|
||||||
1. The profiler **deactivates all the ways control could enter the profiler** (aside from the CLR Profiling API itself). This means removing any Windows callbacks, timer interrupts, hijacking, disabling any other components that may try to call into the profiler DLL, etc. The profiler must also wait for all threads that it has created (e.g., a sampling thread, inter-process communication threads, a ForceGC thread, etc.) to exit, except for the one thread the profiler will use to call RequestProfilerDetach(). Any threads created by the CLR, of course, should not be tampered with.
|
1. The profiler **deactivates all the ways control could enter the profiler** (aside from the CLR Profiling API itself). This means removing any Windows callbacks, timer interrupts, hijacking, disabling any other components that may try to call into the profiler DLL, etc. The profiler must also wait for all threads that it has created (e.g., a sampling thread, inter-process communication threads, a ForceGC thread, etc.) to exit, except for the one thread the profiler will use to call RequestProfilerDetach(). Any threads created by the CLR, of course, should not be tampered with.
|
||||||
- Your profiler must block here until all those ways control can enter your profiler DLL have truly been deactivated (e.g., just setting a flag to disable sampling may not be enough if your sampling thread is currently performing a sample already in progress). You must coordinate with all components of your profiler so that your profiler DLL knows that everything is verifiably deactivated, and all profiler-created threads have exited (except for the one thread the profiler will use to call RequestProfilerDetach()).
|
- Your profiler must block here until all those ways control can enter your profiler DLL have truly been deactivated (e.g., just setting a flag to disable sampling may not be enough if your sampling thread is currently performing a sample already in progress). You must coordinate with all components of your profiler so that your profiler DLL knows that everything is verifiably deactivated, and all profiler-created threads have exited (except for the one thread the profiler will use to call RequestProfilerDetach()).
|
||||||
2. If the profiler will use a thread of its own creation to call RequestProfilerDetach() (which is the typical way this API will be called), that thread must own a reference onto the profiler’s DLL, via its own **LoadLibrary()** call that it makes on the profiler DLL. This can either be done when the thread starts up, or now, or sometime in between. But that reference must be added at some point before calling RequestProfilerDetach().
|
2. If the profiler will use a thread of its own creation to call RequestProfilerDetach() (which is the typical way this API will be called), that thread must own a reference onto the profiler’s DLL, via its own **LoadLibrary()** call that it makes on the profiler DLL. This can either be done when the thread starts up, or now, or sometime in between. But that reference must be added at some point before calling RequestProfilerDetach().
|
||||||
3. Profiler calls ICorProfilerInfo3:: **RequestProfilerDetach** ().
|
3. Profiler calls ICorProfilerInfo3:: **RequestProfilerDetach** ().
|
||||||
- (A) This causes the CLR to (synchronously) set internal state to avoid making any further calls into the profiler via the ICorProfilerCallback\* interfaces, and to refuse any calls from the profiler into ICorProfilerInfo\* interfaces (such calls will now fail early with CORPROF\_E\_PROFILER\_DETACHING).
|
- (A) This causes the CLR to (synchronously) set internal state to avoid making any further calls into the profiler via the ICorProfilerCallback\* interfaces, and to refuse any calls from the profiler into ICorProfilerInfo\* interfaces (such calls will now fail early with CORPROF\_E\_PROFILER\_DETACHING).
|
||||||
- (B) The CLR also (asynchronously) begins a period safety check on another thread to determine when all pre-existing calls into the profiler via the ICorProfilerCallback\* interfaces have returned.
|
- (B) The CLR also (asynchronously) begins a period safety check on another thread to determine when all pre-existing calls into the profiler via the ICorProfilerCallback\* interfaces have returned.
|
||||||
- Note: It is expected that your profiler will not make any more “unsolicited” calls back into the CLR via any interfaces (ICorProfilerInfo\*, hosting, metahost, metadata, etc.). By “unsolicited”, I’m referring to calls that didn’t originate from the CLR via ICorProfilerCallback\*. In other words, it’s ok for the profiler to continue to do its usual stuff in its implementation of ICorProfilerCallback methods (which may include calling into the CLR via ICorProfilerInfo\*), as the CLR will wait for those outer ICorProfilerCallback methods to return as per 3B. But the profiler must not make any other calls into the CLR (i.e., that are not sandwiched inside an ICorProfilerCallback call). You should already have deactivated any component of your profiler that would make such unsolicited calls in step 1.
|
- Note: It is expected that your profiler will not make any more “unsolicited” calls back into the CLR via any interfaces (ICorProfilerInfo\*, hosting, metahost, metadata, etc.). By “unsolicited”, I’m referring to calls that didn’t originate from the CLR via ICorProfilerCallback\*. In other words, it’s ok for the profiler to continue to do its usual stuff in its implementation of ICorProfilerCallback methods (which may include calling into the CLR via ICorProfilerInfo\*), as the CLR will wait for those outer ICorProfilerCallback methods to return as per 3B. But the profiler must not make any other calls into the CLR (i.e., that are not sandwiched inside an ICorProfilerCallback call). You should already have deactivated any component of your profiler that would make such unsolicited calls in step 1.
|
||||||
4. Assuming the above RequestProfilerDetach call was made on a profiler-created thread, that thread must now call [**FreeLibraryAndExitThread**](http://msdn.microsoft.com/en-us/library/ms683153(VS.85).aspx)**()**. (Note: that’s a specialized Windows API that combines FreeLibrary() and ExitThread() in such a way that races can be avoided—do not call FreeLibrary() and ExitThread() separately.)
|
4. Assuming the above RequestProfilerDetach call was made on a profiler-created thread, that thread must now call [**FreeLibraryAndExitThread**](http://msdn.microsoft.com/en-us/library/ms683153(VS.85).aspx)**()**. (Note: that’s a specialized Windows API that combines FreeLibrary() and ExitThread() in such a way that races can be avoided—do not call FreeLibrary() and ExitThread() separately.)
|
||||||
5. On another thread, the CLR continues its **period safety checks** from 3B above. Eventually the CLR determines that there are no more ICorProfilerCallback\* interface calls currently executing, and it is therefore safe to unload the profiler.
|
5. On another thread, the CLR continues its **period safety checks** from 3B above. Eventually the CLR determines that there are no more ICorProfilerCallback\* interface calls currently executing, and it is therefore safe to unload the profiler.
|
||||||
6. The CLR calls ICorProfilerCallback3:: **ProfilerDetachSucceeded**. The profiler can use this signal to know that it’s about to be unloaded. It’s expected that the profiler will do very little in this callback—probably just notifying the user that the profiler is about to be unloaded. Any cleanup the profiler needs to do should already have been done during step 1.
|
6. The CLR calls ICorProfilerCallback3:: **ProfilerDetachSucceeded**. The profiler can use this signal to know that it’s about to be unloaded. It’s expected that the profiler will do very little in this callback—probably just notifying the user that the profiler is about to be unloaded. Any cleanup the profiler needs to do should already have been done during step 1.
|
||||||
7. CLR makes the necessary number of **Release** () calls on ICorProfilerCallback3. The reference count should go down to 0 at this point, and the profiler may deallocate any memory it had previously allocated to support its callback implementation.
|
7. CLR makes the necessary number of **Release** () calls on ICorProfilerCallback3. The reference count should go down to 0 at this point, and the profiler may deallocate any memory it had previously allocated to support its callback implementation.
|
||||||
8. CLR calls **FreeLibrary** () on the profiler DLL. This should be the last reference to the profiler’s DLL, and your DLL will now be unloaded.
|
8. CLR calls **FreeLibrary** () on the profiler DLL. This should be the last reference to the profiler’s DLL, and your DLL will now be unloaded.
|
||||||
- Note: in some cases, it’s theoretically possible that step 4 doesn’t happen until _after_ this step, in which case the last reference to the profiler’s DLL will actually be released by your profiler’s thread that called RequestProfilerDetach and then FreeLibraryAndExitThread. That’s because steps 1-4 happen on your profiler’s thread, and steps 5-8 happen on a dedicated CLR thread (for detaching profilers) sometime after step 3 is completed. So there’s a race between step 4 and all of steps 5-8. There’s no harm in this, so long as you’re playing nice by doing your own LoadLibrary and FreeLibraryAndExitThread as described above.
|
- Note: in some cases, it’s theoretically possible that step 4 doesn’t happen until _after_ this step, in which case the last reference to the profiler’s DLL will actually be released by your profiler’s thread that called RequestProfilerDetach and then FreeLibraryAndExitThread. That’s because steps 1-4 happen on your profiler’s thread, and steps 5-8 happen on a dedicated CLR thread (for detaching profilers) sometime after step 3 is completed. So there’s a race between step 4 and all of steps 5-8. There’s no harm in this, so long as you’re playing nice by doing your own LoadLibrary and FreeLibraryAndExitThread as described above.
|
||||||
9. The CLR adds an Informational entry to the Application Event Log noting that the profiler has been unloaded. The CLR is now ready to service any profiler attach requests.
|
9. The CLR adds an Informational entry to the Application Event Log noting that the profiler has been unloaded. The CLR is now ready to service any profiler attach requests.
|
||||||
|
|
||||||
## RequestProfilerDetach
|
## RequestProfilerDetach
|
||||||
|
|
||||||
|
@ -46,17 +46,17 @@ Let’s dive a little deeper into the method you call to detach your profiler:
|
||||||
|
|
||||||
`HRESULT RequestProfilerDetach([in] DWORD dwExpectedCompletionMilliseconds);`
|
`HRESULT RequestProfilerDetach([in] DWORD dwExpectedCompletionMilliseconds);`
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
First off, you’ll notice this is on ICorProfilerInfo3, the interface your profiler DLL uses, in the same process as your profilee. Although the AttachProfiler API is called from outside the process, this detach method is called from in-process. Why? Well, the general rule with profilers is that _everything_ is done in-process. Attach is an exception because your profiler isn’t in the process yet. You need to somehow trigger your profiler to load, and you can’t do that from a process in which you have no code executing yet! So Attach is sort of a boot-strapping API that has to be called from a process of your own making.
|
First off, you’ll notice this is on ICorProfilerInfo3, the interface your profiler DLL uses, in the same process as your profilee. Although the AttachProfiler API is called from outside the process, this detach method is called from in-process. Why? Well, the general rule with profilers is that _everything_ is done in-process. Attach is an exception because your profiler isn’t in the process yet. You need to somehow trigger your profiler to load, and you can’t do that from a process in which you have no code executing yet! So Attach is sort of a boot-strapping API that has to be called from a process of your own making.
|
||||||
|
|
||||||
Once your profiler DLL is up and running, it is in charge of everything, from within the same process as the profilee. And detach is no exception. Now with that said, it’s probably typical that your profiler will detach in response to an end user action—probably via some GUI that you ship that runs in its own process. So a case could be made that the CLR team could have made your life easier by providing an out-of-process way to do a detach, so that your GUI could easily trigger a detach, just as it triggered the attach. However, you could make that same argument about all the ways you might want to control a profiler via a GUI, such as these commands:
|
Once your profiler DLL is up and running, it is in charge of everything, from within the same process as the profilee. And detach is no exception. Now with that said, it’s probably typical that your profiler will detach in response to an end user action—probably via some GUI that you ship that runs in its own process. So a case could be made that the CLR team could have made your life easier by providing an out-of-process way to do a detach, so that your GUI could easily trigger a detach, just as it triggered the attach. However, you could make that same argument about all the ways you might want to control a profiler via a GUI, such as these commands:
|
||||||
|
|
||||||
- Do a GC now and show me the heap
|
- Do a GC now and show me the heap
|
||||||
- Dial up or down the sampling frequency
|
- Dial up or down the sampling frequency
|
||||||
- Change which instrumented methods should log their invocations
|
- Change which instrumented methods should log their invocations
|
||||||
- Start / stop monitoring exceptions
|
- Start / stop monitoring exceptions
|
||||||
- etc.
|
- etc.
|
||||||
|
|
||||||
The point is, if you have a GUI to control your profiler, then you probably already have an inter-process mechanism for the GUI to communicate with your profiler DLL. So think of “detach” as yet one more command your GUI will send to your profiler DLL.
|
The point is, if you have a GUI to control your profiler, then you probably already have an inter-process mechanism for the GUI to communicate with your profiler DLL. So think of “detach” as yet one more command your GUI will send to your profiler DLL.
|
||||||
|
|
||||||
|
@ -66,10 +66,10 @@ The CLR uses that value in its Sleep() statement that sits between each periodic
|
||||||
|
|
||||||
Until the profiler can be unloaded, it will be considered “loaded” (though deactivated in the sense that no new callback methods will be called). This prevents any new profiler from attaching.
|
Until the profiler can be unloaded, it will be considered “loaded” (though deactivated in the sense that no new callback methods will be called). This prevents any new profiler from attaching.
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
Ok, that wraps up how detaching works. If you remember only one thing from this post, remember that it’s really easy to cause an application you profile to AV after your profiler unloads if you’re not careful. While the CLR tracks outgoing ICorProfilerCallback\* calls, it does not track any other way that control can enter your profiler DLL. _Before_ your profiler calls RequestProfilerDetach:
|
Ok, that wraps up how detaching works. If you remember only one thing from this post, remember that it’s really easy to cause an application you profile to AV after your profiler unloads if you’re not careful. While the CLR tracks outgoing ICorProfilerCallback\* calls, it does not track any other way that control can enter your profiler DLL. _Before_ your profiler calls RequestProfilerDetach:
|
||||||
|
|
||||||
- You must take care to deactivate all other ways control can enter your profiler DLL
|
- You must take care to deactivate all other ways control can enter your profiler DLL
|
||||||
- Your profiler must block until all those other ways control can enter your profiler DLL have verifiably been deactivated
|
- Your profiler must block until all those other ways control can enter your profiler DLL have verifiably been deactivated
|
||||||
|
|
||||||
|
|
|
@ -17,31 +17,31 @@ It’s nice to be able to get call stacks whenever you want them. But with powe
|
||||||
|
|
||||||
So let’s take a look at the beast. Here’s what your profiler calls (you can find this in ICorProfilerInfo2, in corprof.idl):
|
So let’s take a look at the beast. Here’s what your profiler calls (you can find this in ICorProfilerInfo2, in corprof.idl):
|
||||||
```
|
```
|
||||||
HRESULT DoStackSnapshot(
|
HRESULT DoStackSnapshot(
|
||||||
[in] ThreadID thread,
|
[in] ThreadID thread,
|
||||||
[in] StackSnapshotCallback *callback,
|
[in] StackSnapshotCallback *callback,
|
||||||
[in] ULONG32 infoFlags,
|
[in] ULONG32 infoFlags,
|
||||||
[in] void *clientData,
|
[in] void *clientData,
|
||||||
[in, size_is(contextSize), length_is(contextSize)] BYTE context[],
|
[in, size_is(contextSize), length_is(contextSize)] BYTE context[],
|
||||||
[in] ULONG32 contextSize);
|
[in] ULONG32 contextSize);
|
||||||
```
|
```
|
||||||
And here’s what the CLR calls on your profiler (you can also find this in corprof.idl). You’ll pass a pointer to your implementation of this function in the callback parameter above.
|
And here’s what the CLR calls on your profiler (you can also find this in corprof.idl). You’ll pass a pointer to your implementation of this function in the callback parameter above.
|
||||||
```
|
```
|
||||||
typedef HRESULT \_\_stdcall StackSnapshotCallback(
|
typedef HRESULT __stdcall StackSnapshotCallback(
|
||||||
FunctionID funcId,
|
FunctionID funcId,
|
||||||
UINT_PTR ip,
|
UINT_PTR ip,
|
||||||
COR_PRF_FRAME_INFO frameInfo,
|
COR_PRF_FRAME_INFO frameInfo,
|
||||||
ULONG32 contextSize,
|
ULONG32 contextSize,
|
||||||
BYTE context[],
|
BYTE context[],
|
||||||
void *clientData);
|
void *clientData);
|
||||||
```
|
```
|
||||||
|
|
||||||
It’s like a sandwich. When your profiler wants to walk the stack, you call DoStackSnapshot. Before the CLR returns from that call, it calls your StackSnapshotCallback several times, once for each managed frame (or run of unmanaged frames) on the stack:
|
It’s like a sandwich. When your profiler wants to walk the stack, you call DoStackSnapshot. Before the CLR returns from that call, it calls your StackSnapshotCallback several times, once for each managed frame (or run of unmanaged frames) on the stack:
|
||||||
```
|
```
|
||||||
Profiler calls DoStackSnapshot. Whole wheat bread
|
Profiler calls DoStackSnapshot. Whole wheat bread
|
||||||
CLR calls StackSnapshotCallback. Lettuce frame (“leaf”-most frame, ha)
|
CLR calls StackSnapshotCallback. Lettuce frame (“leaf”-most frame, ha)
|
||||||
CLR calls StackSnapshotCallback. Tomato frame
|
CLR calls StackSnapshotCallback. Tomato frame
|
||||||
CLR calls StackSnapshotCallback. Bacon frame (root or “main” frame)
|
CLR calls StackSnapshotCallback. Bacon frame (root or “main” frame)
|
||||||
CLR returns back to profiler from DoStackSnapshot Whole wheat bread
|
CLR returns back to profiler from DoStackSnapshot Whole wheat bread
|
||||||
```
|
```
|
||||||
|
|
||||||
|
@ -77,50 +77,20 @@ Before I continue from this exciting cliffhanger, a brief interlude. Everyone k
|
||||||
|
|
||||||
Now that we’re speaking the same language. Let’s look at a mixed-mode stack:
|
Now that we’re speaking the same language. Let’s look at a mixed-mode stack:
|
||||||
|
|
||||||
|
|
```
|
||||||
|
|
||||||
Unmanaged
|
Unmanaged
|
||||||
|
|
||||||
|
|
|
||||||
|
|
|
||||||
|
|
||||||
D (Managed)
|
D (Managed)
|
||||||
|
|
||||||
|
|
|
||||||
|
|
|
||||||
|
|
||||||
Unmanaged
|
Unmanaged
|
||||||
|
|
||||||
|
|
|
||||||
|
|
|
||||||
|
|
||||||
C (Managed)
|
C (Managed)
|
||||||
|
|
||||||
|
|
|
||||||
|
|
|
||||||
|
|
||||||
B (Managed)
|
B (Managed)
|
||||||
|
|
||||||
|
|
|
||||||
|
|
|
||||||
|
|
||||||
Unmanaged
|
Unmanaged
|
||||||
|
|
||||||
|
|
|
||||||
|
|
|
||||||
|
|
||||||
A (Managed)
|
A (Managed)
|
||||||
|
|
||||||
|
|
|
||||||
|
|
|
||||||
|
|
||||||
Main (Managed)
|
Main (Managed)
|
||||||
|
```
|
||||||
|
|
|
||||||
|
|
||||||
Stepping back a bit, it’s worthwhile to understand why DoStackSnapshot exists in the first place. It’s there to help you walk _managed_ frames on the stack. If you tried to walk managed frames yourself, you would get unreliable results, particularly on 32 bits, because of some wacky calling conventions used in managed code. The CLR understands these calling conventions, and DoStackSnapshot is therefore in a uniquely suitable position to help you decode them. However, DoStackSnapshot is not a complete solution if you want to be able to walk the entire stack, including unmanaged frames. Here’s where you have a choice:
|
Stepping back a bit, it’s worthwhile to understand why DoStackSnapshot exists in the first place. It’s there to help you walk _managed_ frames on the stack. If you tried to walk managed frames yourself, you would get unreliable results, particularly on 32 bits, because of some wacky calling conventions used in managed code. The CLR understands these calling conventions, and DoStackSnapshot is therefore in a uniquely suitable position to help you decode them. However, DoStackSnapshot is not a complete solution if you want to be able to walk the entire stack, including unmanaged frames. Here’s where you have a choice:
|
||||||
|
|
||||||
1. Do nothing and report stacks with “unmanaged holes” to your users, or
|
1. Do nothing and report stacks with “unmanaged holes” to your users, or
|
||||||
2. Write your own unmanaged stack walker to fill in those holes.
|
2. Write your own unmanaged stack walker to fill in those holes.
|
||||||
|
|
||||||
When DoStackSnapshot comes across a block of unmanaged frames, it calls your StackSnapshotCallback with funcId=0. (I think I mentioned this before, but I’m not sure you were listening.) If you’re going with option #1 above, simply do nothing in your callback when funcId=0. We’ll call you again for the next managed frame and you can wake up at that point.
|
When DoStackSnapshot comes across a block of unmanaged frames, it calls your StackSnapshotCallback with funcId=0. (I think I mentioned this before, but I’m not sure you were listening.) If you’re going with option #1 above, simply do nothing in your callback when funcId=0. We’ll call you again for the next managed frame and you can wake up at that point.
|
||||||
|
@ -145,81 +115,67 @@ But before you get too deep, note that the issue of whether and how to seed a st
|
||||||
|
|
||||||
For the truly adventurous profiler that is doing an asynchronous, cross-thread, seeded stack walk while filling in the unmanaged holes, here’s what it would look like.
|
For the truly adventurous profiler that is doing an asynchronous, cross-thread, seeded stack walk while filling in the unmanaged holes, here’s what it would look like.
|
||||||
|
|
||||||
|
|
Block of Unmanaged Frames
|
||||||
|
|
||||||
Block of
|
1. You suspend the target thread (target thread’s suspend count is now 1)
|
||||||
Unmanaged
|
2. You get the target thread’s current register context
|
||||||
Frames
|
3. You determine if the register context points to unmanaged code (e.g., call ICorProfilerInfo2::GetFunctionFromIP(), and see if you get back a 0 FunctionID)
|
||||||
|
|
||||||
|
|
|
||||||
1. You suspend the target thread (target thread’s suspend count is now 1)
|
|
||||||
2. You get the target thread’s current register context
|
|
||||||
3. You determine if the register context points to unmanaged code (e.g., call ICorProfilerInfo2::GetFunctionFromIP(), and see if you get back a 0 FunctionID)
|
|
||||||
4. In this case the register context does point to unmanaged code, so you perform an unmanaged stack walk until you find the top-most managed frame (D)
|
4. In this case the register context does point to unmanaged code, so you perform an unmanaged stack walk until you find the top-most managed frame (D)
|
||||||
|
|
|
||||||
|
|
|
||||||
|
|
||||||
Function D
|
```
|
||||||
(Managed)
|
Function D
|
||||||
|
(Managed)
|
||||||
|
```
|
||||||
|
|
||||||
|
|
|
||||||
1. You call DoStackSnapshot with your seed context. CLR suspends target thread again: its suspend count is now 2. Our sandwich begins.
|
1. You call DoStackSnapshot with your seed context. CLR suspends target thread again: its suspend count is now 2. Our sandwich begins.
|
||||||
|
|
||||||
1. CLR calls your StackSnapshotCallback with FunctionID for D.
|
1. CLR calls your StackSnapshotCallback with FunctionID for D.
|
||||||
|
|
|
||||||
|
|
|
||||||
|
|
||||||
Block of
|
```
|
||||||
Unmanaged
|
Block of
|
||||||
Frames
|
Unmanaged
|
||||||
|
Frames
|
||||||
|
```
|
||||||
|
|
||||||
|
|
|
||||||
1. CLR calls your StackSnapshotCallback with FunctionID=0. You’ll need to walk this block yourself. You can stop when you hit the first managed frame, or you can cheat: delay your unmanaged walk until sometime after your next callback, as the next callback will tell you exactly where the next managed frame begins (and thus where your unmanaged walk should end).
|
1. CLR calls your StackSnapshotCallback with FunctionID=0. You’ll need to walk this block yourself. You can stop when you hit the first managed frame, or you can cheat: delay your unmanaged walk until sometime after your next callback, as the next callback will tell you exactly where the next managed frame begins (and thus where your unmanaged walk should end).
|
||||||
|
|
|
||||||
|
|
|
||||||
|
|
||||||
Function C
|
```
|
||||||
(Managed)
|
Function C
|
||||||
|
(Managed)
|
||||||
|
|
```
|
||||||
1. CLR calls your StackSnapshotCallback with FunctionID for C.
|
1. CLR calls your StackSnapshotCallback with FunctionID for C.
|
||||||
|
|
|
||||||
|
|
|
||||||
|
|
||||||
Function B
|
```
|
||||||
(Managed)
|
Function B
|
||||||
|
(Managed)
|
||||||
|
```
|
||||||
|
|
||||||
|
|
|
||||||
1. CLR calls your StackSnapshotCallback with FunctionID for B.
|
1. CLR calls your StackSnapshotCallback with FunctionID for B.
|
||||||
|
|
|
||||||
|
|
|
||||||
|
|
||||||
Block of
|
```
|
||||||
Unmanaged
|
Block of
|
||||||
Frames
|
Unmanaged
|
||||||
|
Frames
|
||||||
|
```
|
||||||
|
|
||||||
|
|
|
||||||
1. CLR calls your StackSnapshotCallback with FunctionID=0. Again, you’ll need to walk this block yourself.
|
1. CLR calls your StackSnapshotCallback with FunctionID=0. Again, you’ll need to walk this block yourself.
|
||||||
|
|
|
||||||
|
|
|
||||||
|
|
||||||
Function A
|
```
|
||||||
(Managed)
|
Function A
|
||||||
|
(Managed)
|
||||||
|
```
|
||||||
|
|
||||||
|
|
|
||||||
1. CLR calls your StackSnapshotCallback with FunctionID for A.
|
1. CLR calls your StackSnapshotCallback with FunctionID for A.
|
||||||
|
|
|
||||||
|
|
|
||||||
|
|
||||||
Main
|
```
|
||||||
(Managed)
|
Main
|
||||||
|
(Managed)
|
||||||
|
```
|
||||||
|
|
||||||
|
|
1. CLR calls your StackSnapshotCallback with FunctionID for Main.
|
||||||
1. CLR calls your StackSnapshotCallback with FunctionID for Main.
|
|
||||||
2. DoStackSnapshot “resumes” target thread (its suspend count is now 1) and returns. Our sandwich is complete.
|
2. DoStackSnapshot “resumes” target thread (its suspend count is now 1) and returns. Our sandwich is complete.
|
||||||
|
|
||||||
1. You resume target thread (its suspend count is now 0, so it’s resumed for real).
|
1. You resume target thread (its suspend count is now 0, so it’s resumed for real).
|
||||||
|
|
|
||||||
|
|
||||||
**Triumph over evil**
|
**Triumph over evil**
|
||||||
|
|
||||||
|
@ -253,8 +209,8 @@ Problem 2: _While you suspend the target thread, the target thread tries to susp
|
||||||
|
|
||||||
“Come on! Like that could really happen.” Believe it or not, if:
|
“Come on! Like that could really happen.” Believe it or not, if:
|
||||||
|
|
||||||
- Your app runs on a multiproc box, and
|
- Your app runs on a multiproc box, and
|
||||||
- Thread A runs on one proc and thread B runs on another, and
|
- Thread A runs on one proc and thread B runs on another, and
|
||||||
- A tries to suspend B while B tries to suspend A
|
- A tries to suspend B while B tries to suspend A
|
||||||
|
|
||||||
then it’s possible that both suspensions win, and both threads end up suspended. It’s like the line from that movie: “Multiproc means never having to say, ‘I lose.’”. Since each thread is waiting for the other to wake it up, they stay suspended forever. It is the most romantic of all deadlocks.
|
then it’s possible that both suspensions win, and both threads end up suspended. It’s like the line from that movie: “Multiproc means never having to say, ‘I lose.’”. Since each thread is waiting for the other to wake it up, they stay suspended forever. It is the most romantic of all deadlocks.
|
||||||
|
@ -265,7 +221,7 @@ Ok, so, why is the target thread trying to suspend you anyway? Well, in a hypot
|
||||||
|
|
||||||
A less obvious reason that the target thread might try to suspend your walking thread is due to the inner workings of the CLR. The CLR suspends application threads to help with things like garbage collection. So if your walker tries to walk (and thus suspend) the thread doing the GC at the same time the thread doing the GC tries to suspend your walker, you are hosed.
|
A less obvious reason that the target thread might try to suspend your walking thread is due to the inner workings of the CLR. The CLR suspends application threads to help with things like garbage collection. So if your walker tries to walk (and thus suspend) the thread doing the GC at the same time the thread doing the GC tries to suspend your walker, you are hosed.
|
||||||
|
|
||||||
The way out, fortunately, is quite simple. The CLR is only going to suspend threads it needs to suspend in order to do its work. Let’s label the two threads involved in your stack walk: Thread A = the current thread (the thread performing the walk), and Thread B = the target thread (the thread whose stack is walked). As long as Thread A has _never executed managed code_ (and is therefore of no use to the CLR during a garbage collection), then the CLR will never try to suspend Thread A. This means it’s safe for your profiler to have Thread A suspend Thread B, as the CLR will have no reason for B to suspend A.
|
The way out, fortunately, is quite simple. The CLR is only going to suspend threads it needs to suspend in order to do its work. Let’s label the two threads involved in your stack walk: Thread A = the current thread (the thread performing the walk), and Thread B = the target thread (the thread whose stack is walked). As long as Thread A has _never executed managed code_ (and is therefore of no use to the CLR during a garbage collection), then the CLR will never try to suspend Thread A. This means it’s safe for your profiler to have Thread A suspend Thread B, as the CLR will have no reason for B to suspend A.
|
||||||
|
|
||||||
If you’re writing a sampling profiler, it’s quite natural to ensure all of this. You will typically have a separate thread of your own creation that responds to timer interrupts and walks the stacks of other threads. Call this your sampler thread. Since you create this sampler thread yourself and have control over what it executes, the CLR will have no reason to suspend it. And this also fixes the “poorly-written profiler” example above, since this sampler thread is the only thread of your profiler trying to walk or suspend other threads. So your profiler will never try to directly suspend the sampler thread.
|
If you’re writing a sampling profiler, it’s quite natural to ensure all of this. You will typically have a separate thread of your own creation that responds to timer interrupts and walks the stacks of other threads. Call this your sampler thread. Since you create this sampler thread yourself and have control over what it executes, the CLR will have no reason to suspend it. And this also fixes the “poorly-written profiler” example above, since this sampler thread is the only thread of your profiler trying to walk or suspend other threads. So your profiler will never try to directly suspend the sampler thread.
|
||||||
|
|
||||||
|
@ -281,7 +237,7 @@ Lucky for you, the CLR notifies profilers when a thread is about to be destroyed
|
||||||
|
|
||||||
Rule 2: Block in ThreadDestroyed callback until that thread’s stack walk is complete
|
Rule 2: Block in ThreadDestroyed callback until that thread’s stack walk is complete
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
**_GC helps you make a cycle_**
|
**_GC helps you make a cycle_**
|
||||||
|
|
||||||
|
@ -293,25 +249,25 @@ A while back I mentioned that it is clearly a bad idea for your profiler to hold
|
||||||
|
|
||||||
Example #1:
|
Example #1:
|
||||||
|
|
||||||
- Thread A successfully grabs and now owns one of your profiler locks
|
- Thread A successfully grabs and now owns one of your profiler locks
|
||||||
- Thread B = thread doing the GC
|
- Thread B = thread doing the GC
|
||||||
- Thread B calls profiler’s GarbageCollectionStarted callback
|
- Thread B calls profiler’s GarbageCollectionStarted callback
|
||||||
- Thread B blocks on the same profiler lock
|
- Thread B blocks on the same profiler lock
|
||||||
- Thread A executes GetClassFromTokenAndTypeArgs()
|
- Thread A executes GetClassFromTokenAndTypeArgs()
|
||||||
- GetClassFromTokenAndTypeArgs tries to trigger a GC, but notices a GC is already in progress.
|
- GetClassFromTokenAndTypeArgs tries to trigger a GC, but notices a GC is already in progress.
|
||||||
- Thread A blocks, waiting for GC currently in progress (Thread B) to complete
|
- Thread A blocks, waiting for GC currently in progress (Thread B) to complete
|
||||||
- But B is waiting for A, because of your profiler lock.
|
- But B is waiting for A, because of your profiler lock.
|
||||||
|
|
||||||

|

|
||||||
|
|
||||||
Example #2:
|
Example #2:
|
||||||
|
|
||||||
- Thread A successfully grabs and now owns one of your profiler locks
|
- Thread A successfully grabs and now owns one of your profiler locks
|
||||||
- Thread B calls profiler’s ModuleLoadStarted callback
|
- Thread B calls profiler’s ModuleLoadStarted callback
|
||||||
- Thread B blocks on the same profiler lock
|
- Thread B blocks on the same profiler lock
|
||||||
- Thread A executes GetClassFromTokenAndTypeArgs()
|
- Thread A executes GetClassFromTokenAndTypeArgs()
|
||||||
- GetClassFromTokenAndTypeArgs triggers a GC
|
- GetClassFromTokenAndTypeArgs triggers a GC
|
||||||
- Thread A (now doing the GC) waits for B to be ready to be collected
|
- Thread A (now doing the GC) waits for B to be ready to be collected
|
||||||
- But B is waiting for A, because of your profiler lock.
|
- But B is waiting for A, because of your profiler lock.
|
||||||
|
|
||||||

|

|
||||||
|
@ -332,10 +288,10 @@ Yeah, if you read carefully, you’ll see that this rule never even mentions DoS
|
||||||
|
|
||||||
I’m just about tuckered out, so I’m gonna close this out with a quick summary of the highlights. Here's what's important to remember.
|
I’m just about tuckered out, so I’m gonna close this out with a quick summary of the highlights. Here's what's important to remember.
|
||||||
|
|
||||||
1. Synchronous stack walks involve walking the current thread in response to a profiler callback. These don’t require seeding, suspending, or any special rules. Enjoy!
|
1. Synchronous stack walks involve walking the current thread in response to a profiler callback. These don’t require seeding, suspending, or any special rules. Enjoy!
|
||||||
2. Asynchronous walks require a seed if the top of the stack is unmanaged code not part of a PInvoke or COM call. You supply a seed by directly suspending the target thread and walking it yourself, until you find the top-most managed frame. If you don’t supply a seed in this case, DoStackSnapshot will just return a failure code to you.
|
2. Asynchronous walks require a seed if the top of the stack is unmanaged code not part of a PInvoke or COM call. You supply a seed by directly suspending the target thread and walking it yourself, until you find the top-most managed frame. If you don’t supply a seed in this case, DoStackSnapshot will just return a failure code to you.
|
||||||
3. If you directly suspend threads, remember that only a thread that has never run managed code can suspend another thread
|
3. If you directly suspend threads, remember that only a thread that has never run managed code can suspend another thread
|
||||||
4. When doing asynchronous walks, always block in your ThreadDestroyed callback until that thread’s stack walk is complete
|
4. When doing asynchronous walks, always block in your ThreadDestroyed callback until that thread’s stack walk is complete
|
||||||
5. Do not hold a lock while your profiler calls into a CLR function that can trigger a GC
|
5. Do not hold a lock while your profiler calls into a CLR function that can trigger a GC
|
||||||
|
|
||||||
Finally, a note of thanks to the rest of the CLR Profiling API team, as the writing of these rules is truly a team effort. And special thanks to Sean Selitrennikoff who provided an earlier incarnation of much of this content.
|
Finally, a note of thanks to the rest of the CLR Profiling API team, as the writing of these rules is truly a team effort. And special thanks to Sean Selitrennikoff who provided an earlier incarnation of much of this content.
|
||||||
|
|
|
@ -2,13 +2,13 @@
|
||||||
|
|
||||||
This post is organized in chronological order, telling what your profiler should be doing at the following times in the process:
|
This post is organized in chronological order, telling what your profiler should be doing at the following times in the process:
|
||||||
|
|
||||||
- Startup Time
|
- Startup Time
|
||||||
- ModuleLoadFinished Time
|
- ModuleLoadFinished Time
|
||||||
- RequestReJIT Time
|
- RequestReJIT Time
|
||||||
- Actual ReJIT Time
|
- Actual ReJIT Time
|
||||||
- RequestRevert Time
|
- RequestRevert Time
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
## Startup Time
|
## Startup Time
|
||||||
|
|
||||||
|
@ -22,12 +22,6 @@ Typically, your profiler will also create a new thread at this point, call it yo
|
||||||
|
|
||||||
## ModuleLoadFinished Time
|
## ModuleLoadFinished Time
|
||||||
|
|
||||||
###
|
|
||||||
|
|
||||||
###
|
|
||||||
|
|
||||||
###
|
|
||||||
|
|
||||||
### Metadata Changes
|
### Metadata Changes
|
||||||
|
|
||||||
As each module loads, you will likely need to add metadata so that your future ReJITs will have the tokens they need. What you do here heavily depends on the kind of instrumentation you want to do. I’m assuming you’re doing instrumentation that adds some calls from the user code into brand new profiler helper methods you will add somewhere. If you plan to instrument mscorlib, you will likely want to add those profiler helper methods into mscorlib (remember, mscorlib is not allowed to contain an AssemblyRef that points to any other assembly!). Otherwise, perhaps you plan to ship a managed helper assembly that will sit on your user’s disk, and all your profiler helper methods will reside in this on-disk managed helper assembly.
|
As each module loads, you will likely need to add metadata so that your future ReJITs will have the tokens they need. What you do here heavily depends on the kind of instrumentation you want to do. I’m assuming you’re doing instrumentation that adds some calls from the user code into brand new profiler helper methods you will add somewhere. If you plan to instrument mscorlib, you will likely want to add those profiler helper methods into mscorlib (remember, mscorlib is not allowed to contain an AssemblyRef that points to any other assembly!). Otherwise, perhaps you plan to ship a managed helper assembly that will sit on your user’s disk, and all your profiler helper methods will reside in this on-disk managed helper assembly.
|
||||||
|
@ -46,19 +40,15 @@ This won’t make much sense until you’ve read the next section, but I’m pla
|
||||||
|
|
||||||
Now imagine your user has turned some dial on your out-of-process GUI, to request that some functions get instrumented (or re-instrumented (or re-re-instrumented (or …))). This results in a signal sent to your in-process profiler component. Your ReJIT Thread now knows it must call **RequestReJIT**. You can call this API once in bulk for a list of functions to ReJIT. Note that functions are expressed in terms of ModuleID + mdMethodDef metadata tokens. A few things to note about this:
|
Now imagine your user has turned some dial on your out-of-process GUI, to request that some functions get instrumented (or re-instrumented (or re-re-instrumented (or …))). This results in a signal sent to your in-process profiler component. Your ReJIT Thread now knows it must call **RequestReJIT**. You can call this API once in bulk for a list of functions to ReJIT. Note that functions are expressed in terms of ModuleID + mdMethodDef metadata tokens. A few things to note about this:
|
||||||
|
|
||||||
- You request that all instantiations of a generic function (or function on a generic class) get ReJITted with a single ModuleID + mdMethodDef pair. You cannot request a specific instantiation be ReJITted, or provide instantiation-specific IL. This is nothing new, as classic first-JIT-instrumentation should never be customized per instantiation either. But the ReJIT API is designed with this restriction in mind, as you’ll see later on.
|
- You request that all instantiations of a generic function (or function on a generic class) get ReJITted with a single ModuleID + mdMethodDef pair. You cannot request a specific instantiation be ReJITted, or provide instantiation-specific IL. This is nothing new, as classic first-JIT-instrumentation should never be customized per instantiation either. But the ReJIT API is designed with this restriction in mind, as you’ll see later on.
|
||||||
- ModuleID is specific to one AppDomain for unshared modules, or the SharedDomain for shared modules. Thus:
|
- ModuleID is specific to one AppDomain for unshared modules, or the SharedDomain for shared modules. Thus:
|
||||||
- If ModuleID is shared, then your request will simultaneously apply to all domains using the shared copy of this module (and thus function)
|
- If ModuleID is shared, then your request will simultaneously apply to all domains using the shared copy of this module (and thus function)
|
||||||
- If ModuleID is unshared, then your request will apply only to the single AppDomain using this module (and function)
|
- If ModuleID is unshared, then your request will apply only to the single AppDomain using this module (and function)
|
||||||
- Therefore, if you want this ReJIT request to apply to _all unshared copies_ of this function:
|
- Therefore, if you want this ReJIT request to apply to _all unshared copies_ of this function:
|
||||||
- You’ll need to include all such ModuleIDs in this request.
|
- You’ll need to include all such ModuleIDs in this request.
|
||||||
- And… any _future_ unshared loads of this module will result in new ModuleIDs. So as those loads happen, you’ll need to make further calls to RequestReJIT with the new ModuleIDs to ensure those copies get ReJITted as well.
|
- And… any _future_ unshared loads of this module will result in new ModuleIDs. So as those loads happen, you’ll need to make further calls to RequestReJIT with the new ModuleIDs to ensure those copies get ReJITted as well.
|
||||||
- This is optional, and only need be done if you truly want this ReJIT request to apply to all unshared copies of the function. You’re perfectly welcome to ReJIT only those unshared copies you want (and / or the shared copy).
|
- This is optional, and only need be done if you truly want this ReJIT request to apply to all unshared copies of the function. You’re perfectly welcome to ReJIT only those unshared copies you want (and / or the shared copy).
|
||||||
- Now you can re-read the “Re-Request Prior ReJITs” section above. :-)
|
- Now you can re-read the “Re-Request Prior ReJITs” section above. :-)
|
||||||
|
|
||||||
##
|
|
||||||
|
|
||||||
###
|
|
||||||
|
|
||||||
### More on AppDomains
|
### More on AppDomains
|
||||||
|
|
||||||
|
@ -81,18 +71,18 @@ You may have noticed that you have read a whole lot of words so far, but we have
|
||||||
IF this is the first generic instantiation to ReJIT, for a given RequestReJIT call (or this is not a generic at all), THEN:
|
IF this is the first generic instantiation to ReJIT, for a given RequestReJIT call (or this is not a generic at all), THEN:
|
||||||
|
|
||||||
- CLR calls **GetReJITParameters**
|
- CLR calls **GetReJITParameters**
|
||||||
- This callback passes an ICorProfilerFunctionControl to your profiler. Inside your implementation of GetReJITParameters (and no later!) you may call into ICorProfilerFunctionControl to provide the instrumented IL and codegen flags that the CLR should use during the ReJIT
|
- This callback passes an ICorProfilerFunctionControl to your profiler. Inside your implementation of GetReJITParameters (and no later!) you may call into ICorProfilerFunctionControl to provide the instrumented IL and codegen flags that the CLR should use during the ReJIT
|
||||||
- Therefore it is here where you may:
|
- Therefore it is here where you may:
|
||||||
- Call GetILFunctionBody
|
- Call GetILFunctionBody
|
||||||
- Add any new LocalVarSigTokens to the function’s module’s metadata. (You may not do any other metadata modifications here, though!)
|
- Add any new LocalVarSigTokens to the function’s module’s metadata. (You may not do any other metadata modifications here, though!)
|
||||||
- Rewrite the IL to your specifications, passing it to ICorProfilerFunctionControl::SetILFunctionBody.
|
- Rewrite the IL to your specifications, passing it to ICorProfilerFunctionControl::SetILFunctionBody.
|
||||||
- You may NOT call ICorProfilerInfo::SetILFunctionBody for a ReJIT! This API still exists if you want to do classic first-JIT IL rewriting only.
|
- You may NOT call ICorProfilerInfo::SetILFunctionBody for a ReJIT! This API still exists if you want to do classic first-JIT IL rewriting only.
|
||||||
- Note that GetReJITParameters expresses the function getting compiled in terms of the ModuleID + mdMethodDef pair you previously specified to RequestReJIT, and _not_ in terms of a FunctionID. As mentioned before, you may not provide instantiation-specific IL!
|
- Note that GetReJITParameters expresses the function getting compiled in terms of the ModuleID + mdMethodDef pair you previously specified to RequestReJIT, and _not_ in terms of a FunctionID. As mentioned before, you may not provide instantiation-specific IL!
|
||||||
|
|
||||||
And then, for all ReJITs (regardless of whether they are for the first generic instantiation or not):
|
And then, for all ReJITs (regardless of whether they are for the first generic instantiation or not):
|
||||||
|
|
||||||
- CLR calls **ReJITCompilationStarted**
|
- CLR calls **ReJITCompilationStarted**
|
||||||
- CLR calls **ReJITCompilationFinished**
|
- CLR calls **ReJITCompilationFinished**
|
||||||
|
|
||||||
These callbacks express the function getting compiled in terms of FunctionID + ReJITID. (ReJITID is simply a disambiguating value so that each ReJITted version of a function instantiation can be uniquely identified via FunctionID + ReJITID.) Your profiler doesn’t need to do anything in the above callbacks if it doesn’t want to. They just notify you that the ReJIT is occurring, and get called for each generic instantiation (or non-generic) that gets ReJITted.
|
These callbacks express the function getting compiled in terms of FunctionID + ReJITID. (ReJITID is simply a disambiguating value so that each ReJITted version of a function instantiation can be uniquely identified via FunctionID + ReJITID.) Your profiler doesn’t need to do anything in the above callbacks if it doesn’t want to. They just notify you that the ReJIT is occurring, and get called for each generic instantiation (or non-generic) that gets ReJITted.
|
||||||
|
|
||||||
|
@ -114,12 +104,12 @@ Note that RequestRevert allows you to revert back to the original JITted IL, and
|
||||||
|
|
||||||
If there are any errors with performing the ReJIT, you will be notified by the dedicated callback ICorProfilerCallback4::ReJITError(). Errors can happen at a couple times:
|
If there are any errors with performing the ReJIT, you will be notified by the dedicated callback ICorProfilerCallback4::ReJITError(). Errors can happen at a couple times:
|
||||||
|
|
||||||
- RequestReJIT Time: These are fundamental errors with the request itself. This can include bad parameter values, requesting to ReJIT dynamic (Ref.Emit) code, out of memory, etc. If errors occur here, you’ll get a callback to your implementation of ReJITError(), sandwiched inside your call to RequestReJIT on your ReJIT Thread.
|
- RequestReJIT Time: These are fundamental errors with the request itself. This can include bad parameter values, requesting to ReJIT dynamic (Ref.Emit) code, out of memory, etc. If errors occur here, you’ll get a callback to your implementation of ReJITError(), sandwiched inside your call to RequestReJIT on your ReJIT Thread.
|
||||||
- Actual ReJIT Time: These are errors we don’t encounter until actually trying to ReJIT the function itself. When these later errors occur, your implementation of ReJITError() is called on whatever CLR thread encountered the error.
|
- Actual ReJIT Time: These are errors we don’t encounter until actually trying to ReJIT the function itself. When these later errors occur, your implementation of ReJITError() is called on whatever CLR thread encountered the error.
|
||||||
|
|
||||||
You’ll note that ReJITError can provide you not only the ModuleID + mdMethodDef pair that caused the error, but optionally a FunctionID as well. Depending on the nature of the error occurred, the FunctionID may be available, so that your profiler may know the exact generic instantiation involved with the error. If FunctionID is null, then the error was fundamental to the generic function itself (and thus occurred for all instantiations).
|
You’ll note that ReJITError can provide you not only the ModuleID + mdMethodDef pair that caused the error, but optionally a FunctionID as well. Depending on the nature of the error occurred, the FunctionID may be available, so that your profiler may know the exact generic instantiation involved with the error. If FunctionID is null, then the error was fundamental to the generic function itself (and thus occurred for all instantiations).
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
Ok, that about covers it on how your profiler is expected to use ReJIT. As you can see, there are several different tasks your profiler needs to do at different times to get everything right. But I trust you, you’re smart.
|
Ok, that about covers it on how your profiler is expected to use ReJIT. As you can see, there are several different tasks your profiler needs to do at different times to get everything right. But I trust you, you’re smart.
|
||||||
|
|
||||||
|
|
|
@ -3,43 +3,46 @@
|
||||||
|
|
||||||
If your profiler plays with metadata, you've undoubtedly come across signature blobs. They’re used to encode type information for method definitions & references, local variables, and a whole lot more. They’re wonderfully compact, recursively versatile, and sometimes, well, challenging to parse. Fortunately, [Rico Mariani](https://docs.microsoft.com/en-us/archive/blogs/ricom/) was feeling generous one day, and churned out a simple parser that can read these types of signatures:
|
If your profiler plays with metadata, you've undoubtedly come across signature blobs. They’re used to encode type information for method definitions & references, local variables, and a whole lot more. They’re wonderfully compact, recursively versatile, and sometimes, well, challenging to parse. Fortunately, [Rico Mariani](https://docs.microsoft.com/en-us/archive/blogs/ricom/) was feeling generous one day, and churned out a simple parser that can read these types of signatures:
|
||||||
|
|
||||||
MethodDefSig
|
- MethodDefSig
|
||||||
MethodRefSig
|
- MethodRefSig
|
||||||
StandAloneMethodSig
|
- StandAloneMethodSig
|
||||||
FieldSig
|
- FieldSig
|
||||||
PropertySig
|
- PropertySig
|
||||||
LocalVarSig
|
- LocalVarSig
|
||||||
|
|
||||||
Here are the files:
|
Here are the files:
|
||||||
[sigparse.cpp](samples/sigparse.cpp) (Rico's signature parser)
|
|
||||||
[sigformat.cpp](samples/sigformat.cpp) (An example extension to the parser)
|
- [sigparse.cpp](samples/sigparse.cpp) (Rico's signature parser)
|
||||||
[PlugInToYourProfiler.cpp](samples/PlugInToYourProfiler.cpp) (Example code to plug the extension into your profiler)
|
- [sigformat.cpp](samples/sigformat.cpp) (An example extension to the parser)
|
||||||
|
- [PlugInToYourProfiler.cpp](samples/PlugInToYourProfiler.cpp) (Example code to plug the extension into your profiler)
|
||||||
|
|
||||||
Open up **sigparse.cpp** in your favorite editor and take a look at the grammar at the top. The grammar comes from the ECMA CLI spec. Jonathan Keljo has a [link](http://blogs.msdn.com/jkeljo/archive/2005/08/04/447726.aspx) to it from his blog. This tells you the types of signature blobs the parser can handle.
|
Open up **sigparse.cpp** in your favorite editor and take a look at the grammar at the top. The grammar comes from the ECMA CLI spec. Jonathan Keljo has a [link](http://blogs.msdn.com/jkeljo/archive/2005/08/04/447726.aspx) to it from his blog. This tells you the types of signature blobs the parser can handle.
|
||||||
|
|
||||||
Sigparse.cpp is structured without any dependencies on any headers, so you can easily absorb it into your profiler project. There are two things you will need to do to make use of the code. I provided examples of each of these in the download above to help you out:
|
Sigparse.cpp is structured without any dependencies on any headers, so you can easily absorb it into your profiler project. There are two things you will need to do to make use of the code. I provided examples of each of these in the download above to help you out:
|
||||||
|
|
||||||
1. You will **extend the code** to make use of the parsed components of the signature however you like. Perhaps you’ll build up your own internal structures based on what you find. Or maybe you’ll build a pretty-printer that displays method prototypes in the managed language of your choice.
|
1. You will **extend the code** to make use of the parsed components of the signature however you like. Perhaps you’ll build up your own internal structures based on what you find. Or maybe you’ll build a pretty-printer that displays method prototypes in the managed language of your choice.
|
||||||
2. You will then **call the code** to perform the parse on signature blobs you encounter while profiling.
|
2. You will then **call the code** to perform the parse on signature blobs you encounter while profiling.
|
||||||
|
|
||||||
## Extending the code
|
## Extending the code
|
||||||
|
|
||||||
Simply derive a new class from SigParser, and override the virtual functions. The functions you override are events to be handled as the parser traverses the signature in top-down fashion. For example, when the parser encounters a MethodDef, you might see calls to your overrides of:
|
Simply derive a new class from SigParser, and override the virtual functions. The functions you override are events to be handled as the parser traverses the signature in top-down fashion. For example, when the parser encounters a MethodDef, you might see calls to your overrides of:
|
||||||
|
|
||||||
NotifyBeginMethod()
|
```
|
||||||
NotifyParamCount()
|
NotifyBeginMethod()
|
||||||
NotifyBeginRetType()
|
NotifyParamCount()
|
||||||
NotifyBeginType()
|
NotifyBeginRetType()
|
||||||
NotifyTypeSimple()
|
NotifyBeginType()
|
||||||
NotifyEndType()
|
NotifyTypeSimple()
|
||||||
NotifyEndRetType()
|
NotifyEndType()
|
||||||
NotifyBeginParam()
|
NotifyEndRetType()
|
||||||
NotifyBeginType()
|
NotifyBeginParam()
|
||||||
NotifyTypeSimple()
|
NotifyBeginType()
|
||||||
NotifyEndType()
|
NotifyTypeSimple()
|
||||||
NotifyEndParam()
|
NotifyEndType()
|
||||||
_… (more parameter notifications occur here if more parameters exist)_
|
NotifyEndParam()
|
||||||
|
_… (more parameter notifications occur here if more parameters exist)_
|
||||||
NotifyEndMethod()
|
NotifyEndMethod()
|
||||||
|
```
|
||||||
|
|
||||||
And yes, generics are handled as well.
|
And yes, generics are handled as well.
|
||||||
|
|
||||||
|
@ -60,4 +63,3 @@ Don't worry, it's optional. I mentioned above that only signatures whose grammar
|
||||||
The only gotcha is that TypeSpecs & MethodSpecs don’t have a unique byte that introduces them. For example, GENERICINST could indicate the beginning of a TypeSpec or a MethodSpec. You’ll see that SigParser::Parse() switches on the intro byte to determine what it’s looking at. So to keep things simple, you’ll want to add a couple more top-level functions to SigParser to parse TypeSpecs & MethodSpecs (say, ParseTypeSpec() & ParseMethodSpec()). You’d then call those functions instead of Parse() when you have a TypeSpec or MethodSpec on your hands. Of course, if you don’t care about TypeSpecs and MethodSpecs, you can use the code as is and not worry. But this stuff is so much fun, you’ll probably want to add the capability anyway.
|
The only gotcha is that TypeSpecs & MethodSpecs don’t have a unique byte that introduces them. For example, GENERICINST could indicate the beginning of a TypeSpec or a MethodSpec. You’ll see that SigParser::Parse() switches on the intro byte to determine what it’s looking at. So to keep things simple, you’ll want to add a couple more top-level functions to SigParser to parse TypeSpecs & MethodSpecs (say, ParseTypeSpec() & ParseMethodSpec()). You’d then call those functions instead of Parse() when you have a TypeSpec or MethodSpec on your hands. Of course, if you don’t care about TypeSpecs and MethodSpecs, you can use the code as is and not worry. But this stuff is so much fun, you’ll probably want to add the capability anyway.
|
||||||
|
|
||||||
Hope you find this useful. And thanks again to Rico Mariani for sigparse.cpp!
|
Hope you find this useful. And thanks again to Rico Mariani for sigparse.cpp!
|
||||||
|
|
||||||
|
|
|
@ -8,28 +8,28 @@ _First, Grant talked about the 64-bit JITs (one for x64, one for ia64):_
|
||||||
For the 64-bit JIT, we tail call whenever we’re allowed to. Here’s what prevents us from tail calling (in no particular order):
|
For the 64-bit JIT, we tail call whenever we’re allowed to. Here’s what prevents us from tail calling (in no particular order):
|
||||||
|
|
||||||
- We inline the call instead (we never inline recursive calls to the same method, but we will tail call them)
|
- We inline the call instead (we never inline recursive calls to the same method, but we will tail call them)
|
||||||
- The call/callvirt/calli is followed by something other than nop or ret IL instructions.
|
- The call/callvirt/calli is followed by something other than nop or ret IL instructions.
|
||||||
- The caller or callee return a value type.
|
- The caller or callee return a value type.
|
||||||
- The caller and callee return different types.
|
- The caller and callee return different types.
|
||||||
- The caller is synchronized (MethodImplOptions.Synchronized).
|
- The caller is synchronized (MethodImplOptions.Synchronized).
|
||||||
- The caller is a shared generic method.
|
- The caller is a shared generic method.
|
||||||
- The caller has imperative security (a call to Assert, Demand, Deny, etc.).
|
- The caller has imperative security (a call to Assert, Demand, Deny, etc.).
|
||||||
- The caller has declarative security (custom attributes).
|
- The caller has declarative security (custom attributes).
|
||||||
- The caller is varargs
|
- The caller is varargs
|
||||||
- The callee is varargs.
|
- The callee is varargs.
|
||||||
- The runtime forbids the JIT to tail call. (_There are various reasons the runtime may disallow tail calling, such as caller / callee being in different assemblies, the call going to the application's entrypoint, any conflicts with usage of security features, and other esoteric cases._)
|
- The runtime forbids the JIT to tail call. (_There are various reasons the runtime may disallow tail calling, such as caller / callee being in different assemblies, the call going to the application's entrypoint, any conflicts with usage of security features, and other esoteric cases._)
|
||||||
- The il did not have the tail. prefix and we are not optimizing (the profiler and debugger control this)
|
- The il did not have the tail. prefix and we are not optimizing (the profiler and debugger control this)
|
||||||
- The il did not have the tail. prefix and the caller had a localloc instruction (think alloca or dynamic stack allocation)
|
- The il did not have the tail. prefix and the caller had a localloc instruction (think alloca or dynamic stack allocation)
|
||||||
- The caller is getting some GS security cookie checks
|
- The caller is getting some GS security cookie checks
|
||||||
- The il did not have the tail. prefix and a local or parameter has had its address taken (ldarga, or ldloca)
|
- The il did not have the tail. prefix and a local or parameter has had its address taken (ldarga, or ldloca)
|
||||||
- The caller is the same as the callee and the runtime disallows inlining
|
- The caller is the same as the callee and the runtime disallows inlining
|
||||||
- The callee is invoked via stub dispatch (_i.e., via intermediate code that's generated at runtime to optimize certain types of calls_).
|
- The callee is invoked via stub dispatch (_i.e., via intermediate code that's generated at runtime to optimize certain types of calls_).
|
||||||
- For x64 we have these additional restrictions:
|
- For x64 we have these additional restrictions:
|
||||||
|
|
||||||
- The callee has one or more parameters that are valuetypes of size 3,5,6,7 or \>8 bytes
|
- The callee has one or more parameters that are valuetypes of size 3,5,6,7 or \>8 bytes
|
||||||
- The callee has more than 4 arguments (don’t forget to count the this pointer, generics, etc.) and more than the caller
|
- The callee has more than 4 arguments (don’t forget to count the this pointer, generics, etc.) and more than the caller
|
||||||
- For all of the parameters passed on the stack the GC-ness must match between the caller and callee. (_"GC-ness" means the state of being a pointer to the beginning of an object managed by the GC, or a pointer to the interior of an object managed by the GC (e.g., a byref field), or neither (e.g., an integer or struct)._)
|
- For all of the parameters passed on the stack the GC-ness must match between the caller and callee. (_"GC-ness" means the state of being a pointer to the beginning of an object managed by the GC, or a pointer to the interior of an object managed by the GC (e.g., a byref field), or neither (e.g., an integer or struct)._)
|
||||||
- For ia64 we have this additional restriction:
|
- For ia64 we have this additional restriction:
|
||||||
|
|
||||||
- Any of the callee arguments do not get passed in a register.
|
- Any of the callee arguments do not get passed in a register.
|
||||||
|
|
||||||
|
|
|
@ -9,14 +9,12 @@ Type forwarding is nothing new. However, in CLR V4, we are enabling type forwar
|
||||||
|
|
||||||
The example I’ll use where the .NET Framework uses type forwarding is the TimeZoneInfo class. In CLR V4, TimeZoneInfo is now forwarded from System.Core.dll to mscorlib.dll. If you open the CLR V4 copy of System.Core.dll in ildasm and choose Dump, you'll see the following:
|
The example I’ll use where the .NET Framework uses type forwarding is the TimeZoneInfo class. In CLR V4, TimeZoneInfo is now forwarded from System.Core.dll to mscorlib.dll. If you open the CLR V4 copy of System.Core.dll in ildasm and choose Dump, you'll see the following:
|
||||||
|
|
||||||
|
|
|
||||||
```
|
```
|
||||||
.class extern /*27000004*/ forwarder System.TimeZoneInfo
|
.class extern /*27000004*/ forwarder System.TimeZoneInfo
|
||||||
{
|
{
|
||||||
.assembly extern mscorlib /*23000001*/
|
.assembly extern mscorlib /*23000001*/
|
||||||
}
|
}
|
||||||
```
|
```
|
||||||
|
|
|
||||||
|
|
||||||
In each assembly’s metadata is an exported types table. The above means that System.Core.dll's exported types table includes an entry for System.TimeZoneInfo (indexed by token 27000004). What's significant is that System.Core.dll no longer has a typeDef for System.TimeZoneInfo, only an exported type. The fact that the token begins at the left with 0x27 tells you that it's an mdtExportedType (not a mdtTypeDef, which begins at the left with 0x02).
|
In each assembly’s metadata is an exported types table. The above means that System.Core.dll's exported types table includes an entry for System.TimeZoneInfo (indexed by token 27000004). What's significant is that System.Core.dll no longer has a typeDef for System.TimeZoneInfo, only an exported type. The fact that the token begins at the left with 0x27 tells you that it's an mdtExportedType (not a mdtTypeDef, which begins at the left with 0x02).
|
||||||
|
|
||||||
|
@ -28,15 +26,15 @@ This walkthrough assumes you have .NET 4.0 or later installed **and** an older r
|
||||||
|
|
||||||
Code up a simple C# app that uses System.TimeZoneInfo:
|
Code up a simple C# app that uses System.TimeZoneInfo:
|
||||||
```
|
```
|
||||||
namespace test
|
namespace test
|
||||||
{
|
{
|
||||||
class Class1
|
class Class1
|
||||||
{
|
{
|
||||||
static void Main(string[] args)
|
static void Main(string[] args)
|
||||||
{
|
{
|
||||||
System.TimeZoneInfo ti = null;
|
System.TimeZoneInfo ti = null;
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
```
|
```
|
||||||
|
|
||||||
|
@ -49,7 +47,7 @@ csc /debug+ /o- /r:"C:\Program Files (x86)\Reference Assemblies\Microsoft\Framew
|
||||||
Again, be sure you’re using an old csc.exe from, say, a NET 3.5 installation. To verify, open up Class1.exe in ildasm, and take a look at Main(). It should look something like this:
|
Again, be sure you’re using an old csc.exe from, say, a NET 3.5 installation. To verify, open up Class1.exe in ildasm, and take a look at Main(). It should look something like this:
|
||||||
|
|
||||||
```
|
```
|
||||||
.method /*06000001*/ private hidebysig static
|
.method /*06000001*/ private hidebysig static
|
||||||
void Main(string[] args) cil managed
|
void Main(string[] args) cil managed
|
||||||
{
|
{
|
||||||
.entrypoint
|
.entrypoint
|
||||||
|
@ -70,11 +68,11 @@ Note that, if you were to build the above C# code using the .NET 4.0 C# compiler
|
||||||
Ok, so how do we run this pre-.NET 4.0 executable against .NET 4.0? A config file, of course. Paste the following into a file named Class1.exe.config that sits next to Class1.exe:
|
Ok, so how do we run this pre-.NET 4.0 executable against .NET 4.0? A config file, of course. Paste the following into a file named Class1.exe.config that sits next to Class1.exe:
|
||||||
|
|
||||||
```
|
```
|
||||||
<configuration\>
|
<configuration>
|
||||||
<startup\>
|
<startup>
|
||||||
<supportedRuntime version="v4.0.20506"/>
|
<supportedRuntime version="v4.0.20506"/>
|
||||||
</startup\>
|
</startup>
|
||||||
</configuration\>
|
</configuration>
|
||||||
```
|
```
|
||||||
|
|
||||||
The above will force Class1.exe to bind against .NET 4.0 Beta 1. And when it comes time to look for TimeZoneInfo, the CLR will first look in System.Core.dll, find the exported types table entry, and then hop over to mscorlib.dll to load the type. What does that look like to your profiler? Make your guess and hold that thought. First, another walkthrough…
|
The above will force Class1.exe to bind against .NET 4.0 Beta 1. And when it comes time to look for TimeZoneInfo, the CLR will first look in System.Core.dll, find the exported types table entry, and then hop over to mscorlib.dll to load the type. What does that look like to your profiler? Make your guess and hold that thought. First, another walkthrough…
|
||||||
|
@ -83,14 +81,14 @@ The above will force Class1.exe to bind against .NET 4.0 Beta 1. And when it co
|
||||||
|
|
||||||
To experiment with forwarding your own types, the process is:
|
To experiment with forwarding your own types, the process is:
|
||||||
|
|
||||||
- Create Version 1 of your library
|
- Create Version 1 of your library
|
||||||
|
|
||||||
- Create version 1 of your library assembly that defines your type (MyLibAssemblyA.dll)
|
- Create version 1 of your library assembly that defines your type (MyLibAssemblyA.dll)
|
||||||
- Create an app that references your type in MyLibAssemblyA.dll (MyClient.exe)
|
- Create an app that references your type in MyLibAssemblyA.dll (MyClient.exe)
|
||||||
- Create version 2 of your library
|
- Create version 2 of your library
|
||||||
|
|
||||||
- Recompile MyLibAssemblyA.dll to forward your type elsewhere (MyLibAssemblyB.dll)
|
- Recompile MyLibAssemblyA.dll to forward your type elsewhere (MyLibAssemblyB.dll)
|
||||||
- Don’t recompile MyClient.exe. Let it still think the type is defined in MyLibAssemblyA.dll.
|
- Don’t recompile MyClient.exe. Let it still think the type is defined in MyLibAssemblyA.dll.
|
||||||
|
|
||||||
### Version 1
|
### Version 1
|
||||||
|
|
||||||
|
@ -140,9 +138,9 @@ Ok, time to upgrade!
|
||||||
### Version 2
|
### Version 2
|
||||||
Time goes by, your library is growing, and its time to split it into two DLLs. Gotta move Foo into the new DLL. Save this into MyLibAssemblyB.cs
|
Time goes by, your library is growing, and its time to split it into two DLLs. Gotta move Foo into the new DLL. Save this into MyLibAssemblyB.cs
|
||||||
```
|
```
|
||||||
using System;
|
using System;
|
||||||
public class Foo
|
public class Foo
|
||||||
{
|
{
|
||||||
}
|
}
|
||||||
```
|
```
|
||||||
|
|
||||||
|
@ -170,7 +168,7 @@ Foo, MyLibAssemblyB, Version=0.0.0.0, Culture=neutral, PublicKeyToken=null
|
||||||
|
|
||||||
And this all despite the fact that MyClient.exe still believes that Foo lives in MyLibAssemblyA:
|
And this all despite the fact that MyClient.exe still believes that Foo lives in MyLibAssemblyA:
|
||||||
```
|
```
|
||||||
.method /*06000001*/ public hidebysig static
|
.method /*06000001*/ public hidebysig static
|
||||||
void Main() cil managed
|
void Main() cil managed
|
||||||
{
|
{
|
||||||
.entrypoint
|
.entrypoint
|
||||||
|
@ -188,7 +186,6 @@ And this all despite the fact that MyClient.exe still believes that Foo lives in
|
||||||
IL\_001c: ret
|
IL\_001c: ret
|
||||||
} // end of method Test::Main
|
} // end of method Test::Main
|
||||||
```
|
```
|
||||||
|
|
|
||||||
|
|
||||||
## Profilers
|
## Profilers
|
||||||
|
|
||||||
|
@ -199,5 +196,3 @@ This should make life easy for profilers, since they generally expect to be able
|
||||||
However, type forwarding is important to understand if your profiler needs to follow metadata references directly. More generally, if your profiler is reading through metadata and expects to come across a typeDef (e.g., perhaps a metadata reference points to a type in that module, or perhaps your profiler expects certain known types to be in certain modules), then your profiler should be prepared to find an mdtExportedType instead, and to deal gracefully with it rather than doing something silly like crashing.
|
However, type forwarding is important to understand if your profiler needs to follow metadata references directly. More generally, if your profiler is reading through metadata and expects to come across a typeDef (e.g., perhaps a metadata reference points to a type in that module, or perhaps your profiler expects certain known types to be in certain modules), then your profiler should be prepared to find an mdtExportedType instead, and to deal gracefully with it rather than doing something silly like crashing.
|
||||||
|
|
||||||
In any case, whether you think your profiler will be affected by type forwarding, be sure to test, test, test!
|
In any case, whether you think your profiler will be affected by type forwarding, be sure to test, test, test!
|
||||||
|
|
||||||
|
|
|
@ -24,7 +24,7 @@ The package store can be either a global system-wide folder or a dotnet.exe rela
|
||||||
+ netcoreapp2.1
|
+ netcoreapp2.1
|
||||||
+ refs
|
+ refs
|
||||||
+ netcoreapp2.0
|
+ netcoreapp2.0
|
||||||
+ netcoreapp2.1
|
+ netcoreapp2.1
|
||||||
```
|
```
|
||||||
|
|
||||||
The layout within `netcoreapp*` folders is a NuGet cache layout.
|
The layout within `netcoreapp*` folders is a NuGet cache layout.
|
||||||
|
@ -34,7 +34,7 @@ The layout within `netcoreapp*` folders is a NuGet cache layout.
|
||||||
|
|
||||||
To compose the layout of the shared package store, we will use a dotnet command called `dotnet store`. We expect the *hosting providers* (ex: Antares) to use the command to prime their machines and framework authors who want to provide *pre-optimized package archives* create the compressed archive layouts.
|
To compose the layout of the shared package store, we will use a dotnet command called `dotnet store`. We expect the *hosting providers* (ex: Antares) to use the command to prime their machines and framework authors who want to provide *pre-optimized package archives* create the compressed archive layouts.
|
||||||
|
|
||||||
The layout is composed from a list of package names and versions specified as xml:
|
The layout is composed from a list of package names and versions specified as xml:
|
||||||
|
|
||||||
**Roslyn Example**
|
**Roslyn Example**
|
||||||
```xml
|
```xml
|
||||||
|
@ -72,7 +72,7 @@ The output folder will be consumed by the runtime by adding to the `DOTNET_SHARE
|
||||||
|
|
||||||
# Building apps with shared packages
|
# Building apps with shared packages
|
||||||
|
|
||||||
The current mechanism to build applications that share assemblies is by not specifying a RID in the project file. Then, a portable app model is assumed and assemblies that are part of Microsoft.NETCore.App are found under the `dotnet` install root. With shared package store, applications have the ability to filter any set of packages from their publish output. Thus the decision of a portable or a standalone application is not made at the time of project authoring but is instead done at publish time.
|
The current mechanism to build applications that share assemblies is by not specifying a RID in the project file. Then, a portable app model is assumed and assemblies that are part of Microsoft.NETCore.App are found under the `dotnet` install root. With shared package store, applications have the ability to filter any set of packages from their publish output. Thus the decision of a portable or a standalone application is not made at the time of project authoring but is instead done at publish time.
|
||||||
|
|
||||||
## Project Authoring
|
## Project Authoring
|
||||||
We will by default treat `Microsoft.NETCore.App` as though `type: platform` is always specified, thus requiring no explicit RID specification by the user. It will be an `ERROR` to specify a RID in the csproj file using the `<RuntimeIdentifier/>` tag.
|
We will by default treat `Microsoft.NETCore.App` as though `type: platform` is always specified, thus requiring no explicit RID specification by the user. It will be an `ERROR` to specify a RID in the csproj file using the `<RuntimeIdentifier/>` tag.
|
||||||
|
|
|
@ -8,7 +8,7 @@ To support any C++/CLI users that wish to use .NET Core, the runtime and hosting
|
||||||
* Load the appropriate version of .NET Core for the assembly if a .NET Core instance is not running, or validate that the currently running .NET Core instance can satisfy the assemblies requirements.
|
* Load the appropriate version of .NET Core for the assembly if a .NET Core instance is not running, or validate that the currently running .NET Core instance can satisfy the assemblies requirements.
|
||||||
* Load the (already-in-memory) assembly into the runtime.
|
* Load the (already-in-memory) assembly into the runtime.
|
||||||
* Patch the vtfixup table tokens to point to JIT stubs.
|
* Patch the vtfixup table tokens to point to JIT stubs.
|
||||||
|
|
||||||
## Design
|
## Design
|
||||||
|
|
||||||
IJW activation has a variety of hard problems associated with it, mainly with loading in mixed mode assemblies that are not the application.
|
IJW activation has a variety of hard problems associated with it, mainly with loading in mixed mode assemblies that are not the application.
|
||||||
|
|
|
@ -1,7 +1,7 @@
|
||||||
Configuring Huge Pages for loading composite binaries using CoreCLR on Linux
|
Configuring Huge Pages for loading composite binaries using CoreCLR on Linux
|
||||||
----
|
----
|
||||||
|
|
||||||
Huge pages can provide performance benefits to reduce the cost of TLB cache misses when
|
Huge pages can provide performance benefits to reduce the cost of TLB cache misses when
|
||||||
executing code. In general, the largest available wins may be achieved by enabling huge
|
executing code. In general, the largest available wins may be achieved by enabling huge
|
||||||
pages for use by the GC, which will dominate the memory use in the process, but in some
|
pages for use by the GC, which will dominate the memory use in the process, but in some
|
||||||
circumstances, if the application is sufficiently large, there may be a benefit to using
|
circumstances, if the application is sufficiently large, there may be a benefit to using
|
||||||
|
@ -16,7 +16,7 @@ images using the hugetlbfs. Doing some requires several steps.
|
||||||
2. The composite image must be copied into a hugetlbfs filesystem which is visible to the .NET process instead of the composite image being loaded from the normal path.
|
2. The composite image must be copied into a hugetlbfs filesystem which is visible to the .NET process instead of the composite image being loaded from the normal path.
|
||||||
- IMPORTANT: The composite image must NOT be located in the normal path next to the application binary, or that file will be used instead of the huge page version.
|
- IMPORTANT: The composite image must NOT be located in the normal path next to the application binary, or that file will be used instead of the huge page version.
|
||||||
- The environment variable `COMPlus_NativeImageSearchPaths` must be set to point at the location of the hugetlbfs in use. For instance, `COMPlus_NativeImageSearchPaths` might be set to `/var/lib/hugetlbfs/user/USER/pagesize-2MB`
|
- The environment variable `COMPlus_NativeImageSearchPaths` must be set to point at the location of the hugetlbfs in use. For instance, `COMPlus_NativeImageSearchPaths` might be set to `/var/lib/hugetlbfs/user/USER/pagesize-2MB`
|
||||||
- As the cp command does not support copying into a hugetlbfs due to lack of support for the write syscall in that file system, a custom copy application must be used. A sample application that may be used to perform this task has a source listing in Appendix A.
|
- As the cp command does not support copying into a hugetlbfs due to lack of support for the write syscall in that file system, a custom copy application must be used. A sample application that may be used to perform this task has a source listing in Appendix A.
|
||||||
3. The machine must be configured to have sufficient huge pages available in the appropriate huge page pool. The memory requirements of huge page PE loading are as follows.
|
3. The machine must be configured to have sufficient huge pages available in the appropriate huge page pool. The memory requirements of huge page PE loading are as follows.
|
||||||
- Sufficient pages to hold the unmodified copy of the composite image in the hugetlbfs. These pages will be used by the initial copy which emplaces the composite image into huge pages.
|
- Sufficient pages to hold the unmodified copy of the composite image in the hugetlbfs. These pages will be used by the initial copy which emplaces the composite image into huge pages.
|
||||||
- By default the runtime will map each page of the composite image using a MAP_PRIVATE mapping. This will require that the maximum number of huge pages is large enough to hold a completely separate copy of the image as loaded.
|
- By default the runtime will map each page of the composite image using a MAP_PRIVATE mapping. This will require that the maximum number of huge pages is large enough to hold a completely separate copy of the image as loaded.
|
||||||
|
@ -62,7 +62,7 @@ int main(int argc, char** argv)
|
||||||
printf("fdSrc fstat failed\n");
|
printf("fdSrc fstat failed\n");
|
||||||
return 1;
|
return 1;
|
||||||
}
|
}
|
||||||
|
|
||||||
addrSrc = mmap(0, st.st_size, PROT_READ | PROT_WRITE, MAP_SHARED, fdSrc, 0);
|
addrSrc = mmap(0, st.st_size, PROT_READ | PROT_WRITE, MAP_SHARED, fdSrc, 0);
|
||||||
if (addrSrc == MAP_FAILED)
|
if (addrSrc == MAP_FAILED)
|
||||||
{
|
{
|
||||||
|
|
|
@ -120,7 +120,7 @@ while the old code is active in some stack frames. An implementation
|
||||||
must come up with solutions to several related sub problems, which we
|
must come up with solutions to several related sub problems, which we
|
||||||
describe briefly here, and in more detail below.
|
describe briefly here, and in more detail below.
|
||||||
|
|
||||||
* **Patchpoints** : Identify where in the original method OSR is possible.
|
* **Patchpoints** : Identify where in the original method OSR is possible.
|
||||||
We will use the term _patchpoint_ to describe a particular location in a
|
We will use the term _patchpoint_ to describe a particular location in a
|
||||||
method's code that supports OSR transitions.
|
method's code that supports OSR transitions.
|
||||||
* **Triggers** : Determine what will trigger an OSR transition
|
* **Triggers** : Determine what will trigger an OSR transition
|
||||||
|
@ -258,13 +258,13 @@ PatchpointHelper(int ppID, int* counter)
|
||||||
|
|
||||||
switch (s)
|
switch (s)
|
||||||
{
|
{
|
||||||
case Unknown:
|
case Unknown:
|
||||||
*counter = initialThreshold;
|
*counter = initialThreshold;
|
||||||
SetState(s, Active);
|
SetState(s, Active);
|
||||||
return;
|
return;
|
||||||
|
|
||||||
case Active:
|
case Active:
|
||||||
*counter = checkThreshold;
|
*counter = checkThreshold;
|
||||||
SetState(s, Pending);
|
SetState(s, Pending);
|
||||||
RequestAlternative(ppID);
|
RequestAlternative(ppID);
|
||||||
return;
|
return;
|
||||||
|
@ -273,7 +273,7 @@ PatchpointHelper(int ppID, int* counter)
|
||||||
*counter = checkThreshold;
|
*counter = checkThreshold;
|
||||||
return;
|
return;
|
||||||
|
|
||||||
case Ready:
|
case Ready:
|
||||||
Transition(...); // does not return
|
Transition(...); // does not return
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
@ -483,7 +483,7 @@ this is to just leave the original frame in place, and have the OSR frame
|
||||||
#### 3.4.1 The Prototype
|
#### 3.4.1 The Prototype
|
||||||
|
|
||||||
The original method conditionally calls to the patchpoint helper at
|
The original method conditionally calls to the patchpoint helper at
|
||||||
patchpoints. The helper will return if there is no transition.
|
patchpoints. The helper will return if there is no transition.
|
||||||
|
|
||||||
For a transition, the helper will capture context and virtually unwind itself
|
For a transition, the helper will capture context and virtually unwind itself
|
||||||
and the original method from the stack to recover callee-save register values
|
and the original method from the stack to recover callee-save register values
|
||||||
|
@ -554,7 +554,7 @@ frame pointers.
|
||||||
When control is executing in a funclet there are effectively two activation
|
When control is executing in a funclet there are effectively two activation
|
||||||
records on the stack that share a single frame: the parent frame and the
|
records on the stack that share a single frame: the parent frame and the
|
||||||
funclet frame. The funclet frame is largely a stub frame and most of the frame
|
funclet frame. The funclet frame is largely a stub frame and most of the frame
|
||||||
state is kept in the parent frame.
|
state is kept in the parent frame.
|
||||||
|
|
||||||
These two frames are not adjacent; they are separated by some number of runtime
|
These two frames are not adjacent; they are separated by some number of runtime
|
||||||
frames. This means it is going to be difficult for our system to handle
|
frames. This means it is going to be difficult for our system to handle
|
||||||
|
@ -799,7 +799,7 @@ G_M6138_IG03:
|
||||||
FFC9 dec ecx
|
FFC9 dec ecx
|
||||||
894DF0 mov dword ptr [rbp-10H], ecx
|
894DF0 mov dword ptr [rbp-10H], ecx
|
||||||
837DF000 cmp dword ptr [rbp-10H], 0 // ... > 0 ?
|
837DF000 cmp dword ptr [rbp-10H], 0 // ... > 0 ?
|
||||||
7F0E jg SHORT G_M6138_IG05
|
7F0E jg SHORT G_M6138_IG05
|
||||||
|
|
||||||
G_M6138_IG04: ;; bbWeight=0.01
|
G_M6138_IG04: ;; bbWeight=0.01
|
||||||
488D4DF0 lea rcx, bword ptr [rbp-10H] // &patchpointCounter
|
488D4DF0 lea rcx, bword ptr [rbp-10H] // &patchpointCounter
|
||||||
|
@ -910,7 +910,7 @@ For example:
|
||||||
5F pop rdi
|
5F pop rdi
|
||||||
4883C448 add rsp, 72
|
4883C448 add rsp, 72
|
||||||
5D pop rbp
|
5D pop rbp
|
||||||
C3 ret
|
C3 ret
|
||||||
```
|
```
|
||||||
with unwind info:
|
with unwind info:
|
||||||
```
|
```
|
||||||
|
|
|
@ -14,18 +14,18 @@ Dedup - string deduplication is often shortened to dedup in this document.
|
||||||
|
|
||||||
This is an opt-in feature and should have no performance penalty when it’s off. And by default it’s off.
|
This is an opt-in feature and should have no performance penalty when it’s off. And by default it’s off.
|
||||||
|
|
||||||
When it’s on, we aim to –
|
When it’s on, we aim to –
|
||||||
|
|
||||||
- Only deduplicate strings in old generations of the GC heap.
|
- Only deduplicate strings in old generations of the GC heap.
|
||||||
- Not increase the STW pauses for ephemeral GCs.
|
- Not increase the STW pauses for ephemeral GCs.
|
||||||
- Not regress string allocation speed.
|
- Not regress string allocation speed.
|
||||||
- Provide static analysis and runtime checks to detect patterns incompatible with string deduping. This is required to enable customers to opt-in into this feature with confidence.
|
- Provide static analysis and runtime checks to detect patterns incompatible with string deduping. This is required to enable customers to opt-in into this feature with confidence.
|
||||||
|
|
||||||
## Details
|
## Details
|
||||||
|
|
||||||
#### **History**
|
#### **History**
|
||||||
|
|
||||||
The string deduplication feature has been brought up before. See [runtime issue #9022](https://github.com/dotnet/runtime/issues/9022) for discussion.
|
The string deduplication feature has been brought up before. See [runtime issue #9022](https://github.com/dotnet/runtime/issues/9022) for discussion.
|
||||||
|
|
||||||
And a proof of concept was gracefully [attempted](https://github.com/dotnet/coreclr/pull/15135) by [@Rattenkrieg](https://github.com/Rattenkrieg) before. But it was incomplete and the design didn’t have the kind of perf characteristics desired – it had most of the logic in GC vs outside GC.
|
And a proof of concept was gracefully [attempted](https://github.com/dotnet/coreclr/pull/15135) by [@Rattenkrieg](https://github.com/Rattenkrieg) before. But it was incomplete and the design didn’t have the kind of perf characteristics desired – it had most of the logic in GC vs outside GC.
|
||||||
|
|
||||||
|
@ -33,9 +33,9 @@ An example of a user implemented string deduplication is Roslyn’s [StringTable
|
||||||
|
|
||||||
#### **Customer impact estimation and validation**
|
#### **Customer impact estimation and validation**
|
||||||
|
|
||||||
As a general rule we want to have this for all features we add to the runtime.
|
As a general rule we want to have this for all features we add to the runtime.
|
||||||
|
|
||||||
Issue #[9022](https://github.com/dotnet/runtime/issues/9022) It mentioned some general data:
|
Issue #[9022](https://github.com/dotnet/runtime/issues/9022) It mentioned some general data:
|
||||||
|
|
||||||
“The expectation is that typical apps have 20% of their GC heap be strings. Some measurements we have seen is that for at least some applications, 10-30% of strings all may be duplicated, so this might save 2-3% of the GC heap. Not huge, but the feature is not that difficult either.”
|
“The expectation is that typical apps have 20% of their GC heap be strings. Some measurements we have seen is that for at least some applications, 10-30% of strings all may be duplicated, so this might save 2-3% of the GC heap. Not huge, but the feature is not that difficult either.”
|
||||||
|
|
||||||
|
@ -48,9 +48,9 @@ There are 2 sources of data we could get –
|
||||||
|
|
||||||
#### **Design outline**
|
#### **Design outline**
|
||||||
|
|
||||||
This is an opt in feature. When the runtime detects it’s turned on, it creates a dedup thread to do the work.
|
This is an opt in feature. When the runtime detects it’s turned on, it creates a dedup thread to do the work.
|
||||||
|
|
||||||
Detection of duplicated strings is done by looking into a hash table. The key into this hash table is the hash code of the content of a string. Detailed description of this detection is later in this doc.
|
Detection of duplicated strings is done by looking into a hash table. The key into this hash table is the hash code of the content of a string. Detailed description of this detection is later in this doc.
|
||||||
|
|
||||||
As the dedup thread goes through the old generations linearly, it looks for references to a string object (denoted by the method table) and either calculates or looks up the hash code of that string to see if it already exists in the hash table. If so it will attempt to change the reference to point to that string with a CAS operation. If this fails, which means some other thread changed the reference at the mean time, we simply ignore this and move on. We expect the CAS failure rate to be very low.
|
As the dedup thread goes through the old generations linearly, it looks for references to a string object (denoted by the method table) and either calculates or looks up the hash code of that string to see if it already exists in the hash table. If so it will attempt to change the reference to point to that string with a CAS operation. If this fails, which means some other thread changed the reference at the mean time, we simply ignore this and move on. We expect the CAS failure rate to be very low.
|
||||||
|
|
||||||
|
@ -58,17 +58,17 @@ Since the new string reference we will write to the heap has the exact same type
|
||||||
|
|
||||||
The dedup hash table acts as weak references to the strings. Depending on the scenario we might choose to null out these weak references or not (if it’s more performant to rebuild the hash table). If we we do the former these weak references would be treated as short weak handles so the following will happen before we scan for finalization -
|
The dedup hash table acts as weak references to the strings. Depending on the scenario we might choose to null out these weak references or not (if it’s more performant to rebuild the hash table). If we we do the former these weak references would be treated as short weak handles so the following will happen before we scan for finalization -
|
||||||
|
|
||||||
- During BGC final mark phase we will need to null out the strings that are not marked in the hash table. This can be made concurrent.
|
- During BGC final mark phase we will need to null out the strings that are not marked in the hash table. This can be made concurrent.
|
||||||
|
|
||||||
- During a full blocking GC we will need to null out the strings that are not marked in the hash table, and relocate the ones that got promoted if we are doing a compacting GC.
|
- During a full blocking GC we will need to null out the strings that are not marked in the hash table, and relocate the ones that got promoted if we are doing a compacting GC.
|
||||||
|
|
||||||
**Alternate design points**
|
**Alternate design points**
|
||||||
|
|
||||||
- Should we create multiple threads to do the work?
|
- Should we create multiple threads to do the work?
|
||||||
|
|
||||||
Deduping can be done leisurely, so it doesn’t merit having multiple threads.
|
Deduping can be done leisurely, so it doesn’t merit having multiple threads.
|
||||||
|
|
||||||
- Can we use an existing thread to do the work on?
|
- Can we use an existing thread to do the work on?
|
||||||
|
|
||||||
The finalizer thread is something that’s idling most of the time. However there are already plenty of types of work scheduled to potentially run on the finalizer thread so adding yet another thing, especially an opt in feature, can get messy.
|
The finalizer thread is something that’s idling most of the time. However there are already plenty of types of work scheduled to potentially run on the finalizer thread so adding yet another thing, especially an opt in feature, can get messy.
|
||||||
|
|
||||||
|
@ -80,9 +80,9 @@ Only strings allocated on the managed heap will be considered for deduplication.
|
||||||
|
|
||||||
Currently calling GetHashCode of a string calculates a 32-bit hash code. This is not stored anywhere, unlike the default hash code that’s stored either in the syncblk or a syncblk entry, depending whether the syncblk is also used by something else like locking. As the deduping thread goes through the heap it will calculate the 32-bit hash code and actually install it.
|
Currently calling GetHashCode of a string calculates a 32-bit hash code. This is not stored anywhere, unlike the default hash code that’s stored either in the syncblk or a syncblk entry, depending whether the syncblk is also used by something else like locking. As the deduping thread goes through the heap it will calculate the 32-bit hash code and actually install it.
|
||||||
|
|
||||||
However, a 32-bit hash code means we always need to check for collision by actually comparing the string content if the hash code is the same. And for large strings having to compare the string content could be very costly. For LOH compaction we already allocate a padding object for each large object (which currently takes up at most 0.4% of LOH space on 64-bit). We could make this padding object 1-ptr size larger and store the address of the string it’s deduped too. Likewise we can also use this to store the fact “this is the copy the hash table keeps track of so no need to dedup”. This way we can avoid having to do the detection multiple times for the same string. Below illustrates a scenario where large strings are deduplicated.
|
However, a 32-bit hash code means we always need to check for collision by actually comparing the string content if the hash code is the same. And for large strings having to compare the string content could be very costly. For LOH compaction we already allocate a padding object for each large object (which currently takes up at most 0.4% of LOH space on 64-bit). We could make this padding object 1-ptr size larger and store the address of the string it’s deduped too. Likewise we can also use this to store the fact “this is the copy the hash table keeps track of so no need to dedup”. This way we can avoid having to do the detection multiple times for the same string. Below illustrates a scenario where large strings are deduplicated.
|
||||||
|
|
||||||
`pad | s0 | pad | s1 | pad | s0_1`
|
`pad | s0 | pad | s1 | pad | s0_1`
|
||||||
|
|
||||||
`obj0 (-> s0) | obj1 (-> s0_1) | obj2 (->s1) | obj3 (->s0_1) | obj4 (->s1) `
|
`obj0 (-> s0) | obj1 (-> s0_1) | obj2 (->s1) | obj3 (->s0_1) | obj4 (->s1) `
|
||||||
|
|
||||||
|
@ -90,12 +90,12 @@ Each string obj, ie, s*, is a string on LOH and has a padding object in front of
|
||||||
|
|
||||||
s0_1 has the same content as s0. s1 has the same hash code but not the same content.
|
s0_1 has the same content as s0. s1 has the same hash code but not the same content.
|
||||||
|
|
||||||
"obj->s" means obj points to a string object s, or has a ref to s. So obj0 has a ref to s0, obj1 has a ref to s0_1, obj2 has a ref to s1 and so on.
|
"obj->s" means obj points to a string object s, or has a ref to s. So obj0 has a ref to s0, obj1 has a ref to s0_1, obj2 has a ref to s1 and so on.
|
||||||
|
|
||||||
1. As we go through the heap, we see obj0 which points to s0.
|
1. As we go through the heap, we see obj0 which points to s0.
|
||||||
2. s0’s hash is calculated which we use to look into the hash table.
|
2. s0’s hash is calculated which we use to look into the hash table.
|
||||||
3. We see that no entries exist for that hash so we create an entry for it and in s0’s padding indicates that it’s stored in the hash table, ie, it’s the copy we keep.
|
3. We see that no entries exist for that hash so we create an entry for it and in s0’s padding indicates that it’s stored in the hash table, ie, it’s the copy we keep.
|
||||||
4. Then we see obj1 which points to s0_1 whose hash doesn’t exist yet. We calculate the hash for s0_1, and see that there’s an entry for this hash already in the hash table, now we compare the content and see that it’s the same, now we store s0 in the padding object before s0_1 and change obj1’s ref to point to s0.
|
4. Then we see obj1 which points to s0_1 whose hash doesn’t exist yet. We calculate the hash for s0_1, and see that there’s an entry for this hash already in the hash table, now we compare the content and see that it’s the same, now we store s0 in the padding object before s0_1 and change obj1’s ref to point to s0.
|
||||||
5. Then we see obj2 and calculate s1’s hash. We notice an entry already exists for that hash so we compare the content and the content is not the same as s0’s. So we enter s1 into the hash table and indicate that it’s stored in the hash table.
|
5. Then we see obj2 and calculate s1’s hash. We notice an entry already exists for that hash so we compare the content and the content is not the same as s0’s. So we enter s1 into the hash table and indicate that it’s stored in the hash table.
|
||||||
6. Then we see obj3, and s0_1 indicates that it should be deduped to s0 so we change obj3’s ref to point to s0_1 right away.
|
6. Then we see obj3, and s0_1 indicates that it should be deduped to s0 so we change obj3’s ref to point to s0_1 right away.
|
||||||
7. Then we see obj4 which points to s1 and s1 indicates it’s stored in the hash table so we don’t need to dedup.
|
7. Then we see obj4 which points to s1 and s1 indicates it’s stored in the hash table so we don’t need to dedup.
|
||||||
|
@ -106,11 +106,11 @@ Since we know the size of a string object trivially, we know which strings are o
|
||||||
|
|
||||||
- If `InterlockCompareExchangePointer` fails because the ref was modified while we were finding a copy to dedup to (or insert into the hash table), we skip this ref.
|
- If `InterlockCompareExchangePointer` fails because the ref was modified while we were finding a copy to dedup to (or insert into the hash table), we skip this ref.
|
||||||
- If too many collisions exist for a hash code, we skip deduping for strings with that hash code. This avoids the DoS attack by creating too many strings for the same hash.
|
- If too many collisions exist for a hash code, we skip deduping for strings with that hash code. This avoids the DoS attack by creating too many strings for the same hash.
|
||||||
- If the string is too large. At some point going through a very large string to calculate its hash code will become simply not worth the effort. We'll need to do some perf investigation to figure out a good limit.
|
- If the string is too large. At some point going through a very large string to calculate its hash code will become simply not worth the effort. We'll need to do some perf investigation to figure out a good limit.
|
||||||
|
|
||||||
**Alternate design points**
|
**Alternate design points**
|
||||||
|
|
||||||
- Should we calculate the hash codes for SOH strings as gen1 GCs promote them into gen2?
|
- Should we calculate the hash codes for SOH strings as gen1 GCs promote them into gen2?
|
||||||
|
|
||||||
This would increase gen1 pause.
|
This would increase gen1 pause.
|
||||||
|
|
||||||
|
@ -132,7 +132,7 @@ The following scenarios become problematic or more problematic when deduping is
|
||||||
|
|
||||||
- Mutating the string content
|
- Mutating the string content
|
||||||
|
|
||||||
Strings are supposed to be immutable. However in unsafe code you can change the string content after it’s created. Changing string content already asking for trouble without deduping – you could be changing the interned copy which means you are modifying someone else’s string which could cause completely unpredictable results for them.
|
Strings are supposed to be immutable. However in unsafe code you can change the string content after it’s created. Changing string content already asking for trouble without deduping – you could be changing the interned copy which means you are modifying someone else’s string which could cause completely unpredictable results for them.
|
||||||
|
|
||||||
The most common way is to use the fixed keyword:
|
The most common way is to use the fixed keyword:
|
||||||
|
|
||||||
|
@ -143,7 +143,7 @@ fixed (char* p = str)
|
||||||
}
|
}
|
||||||
```
|
```
|
||||||
|
|
||||||
There are other ways such as
|
There are other ways such as
|
||||||
|
|
||||||
`((char*)(gcHandlePointingToString.AddrOfPinnedObject())) = 'c';`
|
`((char*)(gcHandlePointingToString.AddrOfPinnedObject())) = 'c';`
|
||||||
|
|
||||||
|
@ -153,7 +153,7 @@ Or
|
||||||
|
|
||||||
- Locking on a string
|
- Locking on a string
|
||||||
|
|
||||||
Locking on a string object is already discouraged due to a string can be interned. Having string dedup on can make this problematic more often if the string you called lock on is now deduped to a different string object.
|
Locking on a string object is already discouraged due to a string can be interned. Having string dedup on can make this problematic more often if the string you called lock on is now deduped to a different string object.
|
||||||
|
|
||||||
- Reference equality
|
- Reference equality
|
||||||
|
|
||||||
|
@ -187,7 +187,7 @@ To start with we will provide analysis for the following –
|
||||||
|
|
||||||
I’m seeing that there are almost 600 places in libraries that do `fixed (char*` but hopefully most of them do not actually modify the string content. We should definitely be encouraging folks to switch to using `string.Create` like what [PR#31700](https://github.com/dotnet/runtime/pull/31700) did.
|
I’m seeing that there are almost 600 places in libraries that do `fixed (char*` but hopefully most of them do not actually modify the string content. We should definitely be encouraging folks to switch to using `string.Create` like what [PR#31700](https://github.com/dotnet/runtime/pull/31700) did.
|
||||||
|
|
||||||
2. Using lock on a string object.
|
2. Using lock on a string object.
|
||||||
|
|
||||||
- Reference equality checks on strings
|
- Reference equality checks on strings
|
||||||
|
|
||||||
|
@ -195,13 +195,13 @@ Since `ReferenceEquals` is performance critical API, we cannot do checks in its
|
||||||
|
|
||||||
We do have some libraries that rely on `ReferenceEquals`. We need to figure out what to do about them. See discussion [here](https://github.com/dotnet/runtime/pull/31971#pullrequestreview-355531406).
|
We do have some libraries that rely on `ReferenceEquals`. We need to figure out what to do about them. See discussion [here](https://github.com/dotnet/runtime/pull/31971#pullrequestreview-355531406).
|
||||||
|
|
||||||
- Additional checks in heap verification
|
- Additional checks in heap verification
|
||||||
|
|
||||||
Heap verification will now include checks to verify that no one changes the string content after it’s hash is computed. This can be turned on when a certain level of COMPlus_HeapVerify is specified.
|
Heap verification will now include checks to verify that no one changes the string content after it’s hash is computed. This can be turned on when a certain level of COMPlus_HeapVerify is specified.
|
||||||
|
|
||||||
- Stress mode
|
- Stress mode
|
||||||
|
|
||||||
Instead of waiting till the productive moment to start the next deduping cycle, we can have a stress mode where we dedup randomly to catch problems sooner, same idea as GC stress to detect GC holes sooner.
|
Instead of waiting till the productive moment to start the next deduping cycle, we can have a stress mode where we dedup randomly to catch problems sooner, same idea as GC stress to detect GC holes sooner.
|
||||||
|
|
||||||
We could even artificially create duplicates in this stress mode to find places that depend on object identity.
|
We could even artificially create duplicates in this stress mode to find places that depend on object identity.
|
||||||
|
|
||||||
|
@ -219,8 +219,8 @@ We might see some performance gains using RTM (Restricted Transactional Memory)
|
||||||
|
|
||||||
**Deduping other types of objects**
|
**Deduping other types of objects**
|
||||||
|
|
||||||
We might consider to not limit deduping to just strings. There was a discussion in [runtime issue #12628](https://github.com/dotnet/runtime/issues/12628).
|
We might consider to not limit deduping to just strings. There was a discussion in [runtime issue #12628](https://github.com/dotnet/runtime/issues/12628).
|
||||||
|
|
||||||
**Deduping long lived references on stack**
|
**Deduping long lived references on stack**
|
||||||
|
|
||||||
There might be merit to look into deduping long lived refs on the stack. The amount of work it requires and the return makes it low priority but it may help with some corner cases.
|
There might be merit to look into deduping long lived refs on the stack. The amount of work it requires and the return makes it low priority but it may help with some corner cases.
|
||||||
|
|
|
@ -52,9 +52,7 @@ The proposal for this is to "roll-backwards" starting with the "found" version.
|
||||||
|
|
||||||
#### Roll-forward uses app's TFM
|
#### Roll-forward uses app's TFM
|
||||||
|
|
||||||
A secondary issue with with the store's naming convention for framework. It contains a path such as:
|
A secondary issue with with the store's naming convention for framework. It contains a path such as: `\dotnet\store\x64\netcoreapp2.0\microsoft.applicationinsights\2.4.0` where 'netcoreapp2.0' is a "tfm" (target framework moniker). During roll-forward cases, the tfm is still the value specified in the app's runtimeconfig. The host only includes store folders that match that tfm, so it may not find packages from other deps files that were generated off a different tfm. In addition, with the advent of multiple frameworks, it makes it cumbersome to be forced to install to every tfm because multiple frameworks may use the same package, and because each package is still identified by an exact version.
|
||||||
`\dotnet\store\x64\netcoreapp2.0\microsoft.applicationinsights\2.4.0`
|
|
||||||
where 'netcoreapp2.0' is a "tfm" (target framework moniker). During roll-forward cases, the tfm is still the value specified in the app's runtimeconfig. The host only includes store folders that match that tfm, so it may not find packages from other deps files that were generated off a different tfm. In addition, with the advent of multiple frameworks, it makes it cumbersome to be forced to install to every tfm because multiple frameworks may use the same package, and because each package is still identified by an exact version.
|
|
||||||
|
|
||||||
The proposal for this is to add an "any" tfm.
|
The proposal for this is to add an "any" tfm.
|
||||||
|
|
||||||
|
@ -80,9 +78,8 @@ Where "found" means the version that is being used at run time including roll-fo
|
||||||
## 2.1 proposal (add an "any" tfm to store)
|
## 2.1 proposal (add an "any" tfm to store)
|
||||||
For example,
|
For example,
|
||||||
`\dotnet\store\x64\any\microsoft.applicationinsights\2.4.0`
|
`\dotnet\store\x64\any\microsoft.applicationinsights\2.4.0`
|
||||||
|
|
||||||
The `any` tfm would be used if the specified tfm (e.g. netcoreapp2.0) is not found:
|
The `any` tfm would be used if the specified tfm (e.g. netcoreapp2.0) is not found: `\dotnet\store\x64\netcoreapp2.0\microsoft.applicationinsights\2.4.0`
|
||||||
`\dotnet\store\x64\netcoreapp2.0\microsoft.applicationinsights\2.4.0`
|
|
||||||
|
|
||||||
_Possible risk: doesn't this make "uninstall" more difficult? Because multiple installs may write the same packages and try to remove packages that another installer created?_
|
_Possible risk: doesn't this make "uninstall" more difficult? Because multiple installs may write the same packages and try to remove packages that another installer created?_
|
||||||
|
|
||||||
|
@ -95,7 +92,7 @@ The current ordering for resolving deps files is:
|
||||||
1) The app's deps file
|
1) The app's deps file
|
||||||
2) The additional-deps file(s)
|
2) The additional-deps file(s)
|
||||||
3) The framework(s) deps file(s)
|
3) The framework(s) deps file(s)
|
||||||
|
|
||||||
The order is important because "first-in" wins. Since the additional-deps is before the framework, the additional-deps will "win" in all cases except during a minor\major roll-forward. The reason minor\major roll-forward is different is because the framework has special logic (new in 2.1) to compare assembly and file version numbers from the deps files, and pick the newest.
|
The order is important because "first-in" wins. Since the additional-deps is before the framework, the additional-deps will "win" in all cases except during a minor\major roll-forward. The reason minor\major roll-forward is different is because the framework has special logic (new in 2.1) to compare assembly and file version numbers from the deps files, and pick the newest.
|
||||||
|
|
||||||
The proposed ordering change for 2.1 is:
|
The proposed ordering change for 2.1 is:
|
||||||
|
@ -108,7 +105,7 @@ In addition, the additional-deps will always look for assembly and file version
|
||||||
## 2.1 proposal (add runtimeconfig knob to to disable `%DOTNET_ADDITIONAL_DEPS%`)
|
## 2.1 proposal (add runtimeconfig knob to to disable `%DOTNET_ADDITIONAL_DEPS%`)
|
||||||
<strike>
|
<strike>
|
||||||
Add an `additionalDepsLookup` option to the runtimeconfig with these values:
|
Add an `additionalDepsLookup` option to the runtimeconfig with these values:
|
||||||
|
|
||||||
0) The `%DOTNET_ADDITIONAL_DEPS%` is not used
|
0) The `%DOTNET_ADDITIONAL_DEPS%` is not used
|
||||||
1) `DOTNET_ADDITIONAL_DEPS` is used (the default)
|
1) `DOTNET_ADDITIONAL_DEPS` is used (the default)
|
||||||
</strike>
|
</strike>
|
||||||
|
|
|
@ -26,4 +26,4 @@ Code versioning, and in particular its use for tiered compilation means that the
|
||||||
2. The timing of ReJITCompilationFinished has been adjusted to be slightly earlier (after the new code body is generated, but prior to updating the previous jitted code to modify control flow). This raises a slim possibility for a ReJIT error to be reported after ReJITCompilationFinished in the case of OOM or process memory corruption.
|
2. The timing of ReJITCompilationFinished has been adjusted to be slightly earlier (after the new code body is generated, but prior to updating the previous jitted code to modify control flow). This raises a slim possibility for a ReJIT error to be reported after ReJITCompilationFinished in the case of OOM or process memory corruption.
|
||||||
|
|
||||||
|
|
||||||
There are likely some other variations of the changed behavior I haven't thought of yet, but if further testing, code review, or discussion brings it to the surface I'll add it here. Feel free to get in touch on github (@noahfalk), or if you have anything you want to discuss in private you can email me at noahfalk AT microsoft.com
|
There are likely some other variations of the changed behavior I haven't thought of yet, but if further testing, code review, or discussion brings it to the surface I'll add it here. Feel free to get in touch on github (@noahfalk), or if you have anything you want to discuss in private you can email me at noahfalk AT microsoft.com
|
||||||
|
|
|
@ -2,7 +2,7 @@
|
||||||
|
|
||||||
Covariant return methods is a runtime feature designed to support the [covariant return types](https://github.com/dotnet/csharplang/blob/master/proposals/covariant-returns.md) and [records](https://github.com/dotnet/csharplang/blob/master/proposals/records.md) C# language features posed for C# 9.0.
|
Covariant return methods is a runtime feature designed to support the [covariant return types](https://github.com/dotnet/csharplang/blob/master/proposals/covariant-returns.md) and [records](https://github.com/dotnet/csharplang/blob/master/proposals/records.md) C# language features posed for C# 9.0.
|
||||||
|
|
||||||
This feature allows an overriding method to have a return type that is different than the one on the method it overrides, but compatible with it. The type compability rules are defined in ECMA I.8.7.1. Example: using a more derived return type.
|
This feature allows an overriding method to have a return type that is different than the one on the method it overrides, but compatible with it. The type compability rules are defined in ECMA I.8.7.1. Example: using a more derived return type.
|
||||||
|
|
||||||
Covariant return methods can only be described through MethodImpl records, and as an initial implementation will only be applicable to methods on reference types. Methods on interfaces and value types will not be supported (may be supported later in the future).
|
Covariant return methods can only be described through MethodImpl records, and as an initial implementation will only be applicable to methods on reference types. Methods on interfaces and value types will not be supported (may be supported later in the future).
|
||||||
|
|
||||||
|
@ -24,9 +24,9 @@ During enumeration of MethodImpls on a type (`MethodTableBuilder::EnumerateMetho
|
||||||
+ Load the `TypeHandle` of the return type of the method on base type.
|
+ Load the `TypeHandle` of the return type of the method on base type.
|
||||||
+ Load the `TypeHandle` of the return type of the method on the current type being validated.
|
+ Load the `TypeHandle` of the return type of the method on the current type being validated.
|
||||||
+ Verify that the second `TypeHandle` is compatible with the first `TypeHandle` using the `MethodTable::CanCastTo()` API. If they are not compatible, a TypeLoadException is thrown.
|
+ Verify that the second `TypeHandle` is compatible with the first `TypeHandle` using the `MethodTable::CanCastTo()` API. If they are not compatible, a TypeLoadException is thrown.
|
||||||
|
|
||||||
The only exception where `CanCastTo()` will return true for an incompatible type according to the ECMA rules is for structs implementing interfaces, so we explicitly check for that case and throw a TypeLoadException if we hit it.
|
The only exception where `CanCastTo()` will return true for an incompatible type according to the ECMA rules is for structs implementing interfaces, so we explicitly check for that case and throw a TypeLoadException if we hit it.
|
||||||
|
|
||||||
Once a method is flagged for return type checking, every time the vtable slot containing that method gets overridden on a derived type, the new override will also be checked for compatiblity. This is to ensure that no derived type can implicitly override some virtual method that has already been overridden by some MethodImpl with a covariant return type.
|
Once a method is flagged for return type checking, every time the vtable slot containing that method gets overridden on a derived type, the new override will also be checked for compatiblity. This is to ensure that no derived type can implicitly override some virtual method that has already been overridden by some MethodImpl with a covariant return type.
|
||||||
|
|
||||||
### VTable Slot Unification
|
### VTable Slot Unification
|
||||||
|
@ -64,7 +64,7 @@ This slot unification step will also take place during the last step of type loa
|
||||||
An interface method may be both non-final and have a MethodImpl that declares that it overrides another interface method. If it does, NO other interface method may .override it. Instead further overrides must override the method that it overrode. Also the overriding method may only override 1 method.
|
An interface method may be both non-final and have a MethodImpl that declares that it overrides another interface method. If it does, NO other interface method may .override it. Instead further overrides must override the method that it overrode. Also the overriding method may only override 1 method.
|
||||||
|
|
||||||
The default interface method resolution algorithm shall change from:
|
The default interface method resolution algorithm shall change from:
|
||||||
|
|
||||||
``` console
|
``` console
|
||||||
Given interface method M and type T.
|
Given interface method M and type T.
|
||||||
Let MSearch = M
|
Let MSearch = M
|
||||||
|
|
|
@ -59,7 +59,7 @@ Note, this approach is probably more complete than we will finish in one release
|
||||||
|
|
||||||
For non-generic code this is straightforward. Either compile all the non-generic code in the binary, or compile only that which is specified via a profile guided optimization step. This choice shall be driven by a per "input assembly" switch as in the presence of a composite R2R image we likely will want to have different policy for different assemblies, as has proven valuable in the past. Until proven otherwise, per assembly specification of this behavior shall be considered to be sufficient.
|
For non-generic code this is straightforward. Either compile all the non-generic code in the binary, or compile only that which is specified via a profile guided optimization step. This choice shall be driven by a per "input assembly" switch as in the presence of a composite R2R image we likely will want to have different policy for different assemblies, as has proven valuable in the past. Until proven otherwise, per assembly specification of this behavior shall be considered to be sufficient.
|
||||||
|
|
||||||
We shall set a guideline for how much generic code to generate, and the amount of generic code to generate shall be gated as a multiplier of the amount of non-generic code generated.
|
We shall set a guideline for how much generic code to generate, and the amount of generic code to generate shall be gated as a multiplier of the amount of non-generic code generated.
|
||||||
|
|
||||||
For generic code we also need a per assembly switch to adjust between various behaviors, but the proposal is as follows:
|
For generic code we also need a per assembly switch to adjust between various behaviors, but the proposal is as follows:
|
||||||
|
|
||||||
|
@ -87,7 +87,7 @@ Runtime Layer
|
||||||
|
|
||||||
Each layer in this stack will be compiled as a consistent set of crossgen2 compilations.
|
Each layer in this stack will be compiled as a consistent set of crossgen2 compilations.
|
||||||
|
|
||||||
I propose to reduce the generics duplication problem to allow duplication between layers, but not within a layer. There are two ways to do this. The first of which is to produce composite R2R images for a layer. Within a single composite R2R image generation, running heuristics and generating generics eagerly should be straightforward. This composite R2R image would have all instantiations statically computed that are local to that particular layer of compilation, and also any instantiations from other layers. The duplication problem would be reduced in that a single analysis would trigger these multi-layer dependent compilations, and so which there may be duplication between layers, there wouldn't be duplication within a layer. And given that the count of layers is not expected to exceed 3 or 4, that duplication will not be a major concern.
|
I propose to reduce the generics duplication problem to allow duplication between layers, but not within a layer. There are two ways to do this. The first of which is to produce composite R2R images for a layer. Within a single composite R2R image generation, running heuristics and generating generics eagerly should be straightforward. This composite R2R image would have all instantiations statically computed that are local to that particular layer of compilation, and also any instantiations from other layers. The duplication problem would be reduced in that a single analysis would trigger these multi-layer dependent compilations, and so which there may be duplication between layers, there wouldn't be duplication within a layer. And given that the count of layers is not expected to exceed 3 or 4, that duplication will not be a major concern.
|
||||||
|
|
||||||
The second approach is to split compilation up into assembly level units, run the heuristics per assembly, generate the completely local generics in the individual assemblies, and then nominate a final mop up assembly that consumes a series of data files produced by the individual assembly compilations and holds all of the stuff that didn't make sense in the individual assemblies. In my opinion this second approach would be better for debug builds, but the first approach is strictly better for release builds, and really shouldn't be terribly slow.
|
The second approach is to split compilation up into assembly level units, run the heuristics per assembly, generate the completely local generics in the individual assemblies, and then nominate a final mop up assembly that consumes a series of data files produced by the individual assembly compilations and holds all of the stuff that didn't make sense in the individual assemblies. In my opinion this second approach would be better for debug builds, but the first approach is strictly better for release builds, and really shouldn't be terribly slow.
|
||||||
|
|
||||||
|
|
|
@ -108,4 +108,4 @@ For EventCounter and PollingCounter we expect simple viewers to use the display
|
||||||
|
|
||||||
### Metadata
|
### Metadata
|
||||||
|
|
||||||
To add any optional metadata about the counters that we do not already provide a way of encoding, users can call the `AddMetaData(string key, string value)` API. This API exists on all variants of the Counter APIs, and allows users to add one or many key-value pairs of metadata, which is dumped to the Payload as a comma-separated string value. This API exists so that users can add any metadata about their Counter that is not known to us and is different from the ones we provide by default (i.e. `DisplayName`, `CounterType`, `DisplayRateTimeScale`).
|
To add any optional metadata about the counters that we do not already provide a way of encoding, users can call the `AddMetaData(string key, string value)` API. This API exists on all variants of the Counter APIs, and allows users to add one or many key-value pairs of metadata, which is dumped to the Payload as a comma-separated string value. This API exists so that users can add any metadata about their Counter that is not known to us and is different from the ones we provide by default (i.e. `DisplayName`, `CounterType`, `DisplayRateTimeScale`).
|
||||||
|
|
|
@ -30,8 +30,7 @@ In the `.runtimeconfig.json` these values are defined like this:
|
||||||
```
|
```
|
||||||
|
|
||||||
#### Framework name
|
#### Framework name
|
||||||
Each framework reference identifies the framework by its name.
|
Each framework reference identifies the framework by its name. Framework names are case sensitive (since they're used as folder names even on Linux systems).
|
||||||
Framework names are case sensitive (since they're used as folder names even on Linux systems).
|
|
||||||
|
|
||||||
#### Version
|
#### Version
|
||||||
Framework version must be a [SemVer V2](https://semver.org) valid version.
|
Framework version must be a [SemVer V2](https://semver.org) valid version.
|
||||||
|
@ -146,13 +145,15 @@ Pros
|
||||||
|
|
||||||
Cons
|
Cons
|
||||||
* Testing behavior of new releases with pre-release versions is not fully possible (see below).
|
* Testing behavior of new releases with pre-release versions is not fully possible (see below).
|
||||||
* Some special cases don't work:
|
* Some special cases don't work.
|
||||||
One special case which would not work:
|
|
||||||
*Component A which asks for `2.0.0 LatestMajor` is loaded first on a machine which has `3.0.0` and also `3.1.0-preview` installed. Because it's the first in the process it will resolve the runtime according to the above rules - that is prefer release version - and thus will select `3.0.0`.
|
|
||||||
Later on component B is loaded which asks for `3.1.0-preview LatestMajor` (for example the one in active development). This load will fail since `3.0.0` is not enough to run this component.
|
|
||||||
Loading the components in reverse order (B first and then A) will work since the `3.1.0-preview` runtime will be selected.*
|
|
||||||
|
|
||||||
Modification to automatic roll forward to latest patch:
|
One special case which would not work:
|
||||||
|
*Component A which asks for `2.0.0 LatestMajor` is loaded first on a machine which has `3.0.0` and also `3.1.0-preview` installed. Because it's the first in the process it will resolve the runtime according to the above rules - that is prefer release version - and thus will select `3.0.0`.*
|
||||||
|
|
||||||
|
*Later on component B is loaded which asks for `3.1.0-preview LatestMajor` (for example the one in active development). This load will fail since `3.0.0` is not enough to run this component.*
|
||||||
|
*Loading the components in reverse order (B first and then A) will work since the `3.1.0-preview` runtime will be selected.*
|
||||||
|
|
||||||
|
Modification to automatic roll forward to latest patch:
|
||||||
Existing behavior is to find a matching framework based on the above rules and then apply roll forward to latest patch (except if `Disable` is specified). The new behavior should be:
|
Existing behavior is to find a matching framework based on the above rules and then apply roll forward to latest patch (except if `Disable` is specified). The new behavior should be:
|
||||||
* If the above rules find a matching pre-release version of a framework, then automatic roll forward to latest patch is not applied.
|
* If the above rules find a matching pre-release version of a framework, then automatic roll forward to latest patch is not applied.
|
||||||
* If the above rules find a matching release version of a framework, automatic roll forward to latest patch is applied.
|
* If the above rules find a matching release version of a framework, automatic roll forward to latest patch is applied.
|
||||||
|
@ -218,12 +219,12 @@ Items lower in the list override those higher in the list. At each precedence sc
|
||||||
This setting is also described in [roll-forward-on-no-candidate-fx](roll-forward-on-no-candidate-fx.md). It can be specified as a property either for the entire `.runtimeconfig.json` or per framework reference (it has no environment variable of command line argument). It disables rolling forward to the latest patch.
|
This setting is also described in [roll-forward-on-no-candidate-fx](roll-forward-on-no-candidate-fx.md). It can be specified as a property either for the entire `.runtimeconfig.json` or per framework reference (it has no environment variable of command line argument). It disables rolling forward to the latest patch.
|
||||||
|
|
||||||
The host will compute effective value of `applyPatches` for each framework reference.
|
The host will compute effective value of `applyPatches` for each framework reference.
|
||||||
The `applyPatches` value is only considered if the effective `rollForward` value for a given framework reference is
|
The `applyPatches` value is only considered if the effective `rollForward` value for a given framework reference is
|
||||||
* `LatestPatch`
|
* `LatestPatch`
|
||||||
* `Minor`
|
* `Minor`
|
||||||
* `Major`
|
* `Major`
|
||||||
|
|
||||||
For the other values `applyPatches` is ignored.
|
For the other values `applyPatches` is ignored.
|
||||||
*This is to maintain backward compatibility with `rollForwardOnNoCandidateFx`. `applyPatches` is now considered obsolete.*
|
*This is to maintain backward compatibility with `rollForwardOnNoCandidateFx`. `applyPatches` is now considered obsolete.*
|
||||||
|
|
||||||
If `applyPatches` is set to `true` (the default), then the roll-forward rules described above apply fully.
|
If `applyPatches` is set to `true` (the default), then the roll-forward rules described above apply fully.
|
||||||
|
@ -259,7 +260,7 @@ There's a direct mapping from the `rollForward` setting to the internal represen
|
||||||
| `rollForward` | `version_compatibility_range` | `roll_to_highest_version` |
|
| `rollForward` | `version_compatibility_range` | `roll_to_highest_version` |
|
||||||
| --------------------- | ----------------------------- | ------------------------------------------ |
|
| --------------------- | ----------------------------- | ------------------------------------------ |
|
||||||
| `Disable` | `exact` | `false` |
|
| `Disable` | `exact` | `false` |
|
||||||
| `LatestPatch` | `patch` | `false` (always picks latest patch anyway) |
|
| `LatestPatch` | `patch` | `false` (always picks latest patch anyway) |
|
||||||
| `Minor` | `minor` | `false` |
|
| `Minor` | `minor` | `false` |
|
||||||
| `LatestMinor` | `minor` | `true` |
|
| `LatestMinor` | `minor` | `true` |
|
||||||
| `Major` | `major` | `false` |
|
| `Major` | `major` | `false` |
|
||||||
|
@ -306,7 +307,7 @@ Steps
|
||||||
* By doing this for all `framework references` here, before the next loop, we minimize the number of re-try attempts.
|
* By doing this for all `framework references` here, before the next loop, we minimize the number of re-try attempts.
|
||||||
4. For each `framework reference` in `config fx references`:
|
4. For each `framework reference` in `config fx references`:
|
||||||
5. --> If the framework's `name` is not in `resolved frameworks` Then resolve the `framework reference` to the actual framework on disk:
|
5. --> If the framework's `name` is not in `resolved frameworks` Then resolve the `framework reference` to the actual framework on disk:
|
||||||
* If the framework `name` already exists in the `effective fx references` reconcile the currently processed `framework reference` with the one from the `effective fx references` (see above for the algorithm).
|
* If the framework `name` already exists in the `effective fx references` reconcile the currently processed `framework reference` with the one from the `effective fx references` (see above for the algorithm).
|
||||||
*Term "reconcile framework references" is used for this in the code, this used to be called "soft-roll-forward" as well.*
|
*Term "reconcile framework references" is used for this in the code, this used to be called "soft-roll-forward" as well.*
|
||||||
* The reconciliation will always pick the higher `version` and will merge the `rollForward` and `applyPatches` settings.
|
* The reconciliation will always pick the higher `version` and will merge the `rollForward` and `applyPatches` settings.
|
||||||
* The reconciliation may fail if it's not possible to roll forward from one `framework reference` to the other.
|
* The reconciliation may fail if it's not possible to roll forward from one `framework reference` to the other.
|
||||||
|
@ -368,7 +369,7 @@ This might be more of an issue for components (COM and such), which we will reco
|
||||||
The above proposal will impact behavior of existing apps (because framework resolution is in `hostfxr` which is global on the machine for all frameworks). This is a description of the changes as they apply to apps using either default settings, `rollForwardOnNoCandidateFx` or `applyPatches`.
|
The above proposal will impact behavior of existing apps (because framework resolution is in `hostfxr` which is global on the machine for all frameworks). This is a description of the changes as they apply to apps using either default settings, `rollForwardOnNoCandidateFx` or `applyPatches`.
|
||||||
|
|
||||||
### Fixing ordering issues
|
### Fixing ordering issues
|
||||||
In 2.* the algorithm had a bug in it which caused it to resolve different version depending solely on the order of framework references. Consider this example:
|
In 2.* the algorithm had a bug in it which caused it to resolve different version depending solely on the order of framework references. Consider this example:
|
||||||
|
|
||||||
`Microsoft.NETCore.App` is available on the machine with versions `2.1.1` and `2.1.2`.
|
`Microsoft.NETCore.App` is available on the machine with versions `2.1.1` and `2.1.2`.
|
||||||
|
|
||||||
|
|
|
@ -1,5 +1,5 @@
|
||||||
# .NET Core Globalization Invariant Mode
|
# .NET Core Globalization Invariant Mode
|
||||||
|
|
||||||
Author: [Tarek Mahmoud Sayed](https://github.com/tarekgh)
|
Author: [Tarek Mahmoud Sayed](https://github.com/tarekgh)
|
||||||
|
|
||||||
The globalization invariant mode - new in .NET Core 2.0 - enables you to remove application dependencies on globalization data and [globalization behavior](https://docs.microsoft.com/en-us/dotnet/standard/globalization-localization/). This mode is an opt-in feature that provides more flexibility if you care more about reducing dependencies and the size of distribution than globalization functionality or globalization-correctness.
|
The globalization invariant mode - new in .NET Core 2.0 - enables you to remove application dependencies on globalization data and [globalization behavior](https://docs.microsoft.com/en-us/dotnet/standard/globalization-localization/). This mode is an opt-in feature that provides more flexibility if you care more about reducing dependencies and the size of distribution than globalization functionality or globalization-correctness.
|
||||||
|
@ -17,7 +17,7 @@ The following scenarios are affected when the invariant mode is enabled. Their i
|
||||||
- Time Zone display name on Linux
|
- Time Zone display name on Linux
|
||||||
|
|
||||||
## Background
|
## Background
|
||||||
|
|
||||||
Globalization rules and the data that represents those rules frequently change, often due to country-specific policy changes (for example, changes in currency symbol, sorting behavior or time zones). Developers expect globalization behavior to always be current and for their applications to adapt to new data over time. In order to keep up with those changes, .NET Core (and the .NET Framework, too) depends on the underlying OS to keep up with these changes.
|
Globalization rules and the data that represents those rules frequently change, often due to country-specific policy changes (for example, changes in currency symbol, sorting behavior or time zones). Developers expect globalization behavior to always be current and for their applications to adapt to new data over time. In order to keep up with those changes, .NET Core (and the .NET Framework, too) depends on the underlying OS to keep up with these changes.
|
||||||
|
|
||||||
Relying on the underlying OS for globalization data has the following benefits:
|
Relying on the underlying OS for globalization data has the following benefits:
|
||||||
|
@ -32,11 +32,11 @@ Globalization support has the following potential challenges for applications:
|
||||||
* Installing/carrying the [ICU](http://icu-project.org) package on Linux (~28 MB).
|
* Installing/carrying the [ICU](http://icu-project.org) package on Linux (~28 MB).
|
||||||
|
|
||||||
Note: On Linux, .NET Core relies on globalization data from ICU. For example, [.NET Core Linux Docker images](https://github.com/dotnet/dotnet-docker/blob/master/2.0/runtime-deps/stretch/amd64/Dockerfile) install this component. Globalization data is available on Windows and macOS as part of their base installs.
|
Note: On Linux, .NET Core relies on globalization data from ICU. For example, [.NET Core Linux Docker images](https://github.com/dotnet/dotnet-docker/blob/master/2.0/runtime-deps/stretch/amd64/Dockerfile) install this component. Globalization data is available on Windows and macOS as part of their base installs.
|
||||||
|
|
||||||
## Cultures and culture data
|
## Cultures and culture data
|
||||||
|
|
||||||
When enabling the invariant mode, all cultures behave like the invariant culture. The invariant culture has the following characteristics:
|
When enabling the invariant mode, all cultures behave like the invariant culture. The invariant culture has the following characteristics:
|
||||||
|
|
||||||
* Culture names (English, native display, ISO, language names) will return invariant names. For instance, when requesting culture native name, you will get "Invariant Language (Invariant Country)".
|
* Culture names (English, native display, ISO, language names) will return invariant names. For instance, when requesting culture native name, you will get "Invariant Language (Invariant Country)".
|
||||||
* All cultures LCID will have value 0x1000 (which means Custom Locale ID). The exception is the invariant cultures which will still have 0x7F.
|
* All cultures LCID will have value 0x1000 (which means Custom Locale ID). The exception is the invariant cultures which will still have 0x7F.
|
||||||
* All culture parents will be invariant. In other word, there will not be any neutral cultures by default but the apps can still create a culture like "en".
|
* All culture parents will be invariant. In other word, there will not be any neutral cultures by default but the apps can still create a culture like "en".
|
||||||
|
@ -45,45 +45,45 @@ When enabling the invariant mode, all cultures behave like the invariant culture
|
||||||
* Numbers will always be formatted as the invariant culture. For example, decimal point will always be formatted as ".". Number strings previously formatted with cultures that have different symbols will fail parsing.
|
* Numbers will always be formatted as the invariant culture. For example, decimal point will always be formatted as ".". Number strings previously formatted with cultures that have different symbols will fail parsing.
|
||||||
* All cultures will have currency symbol as "¤"
|
* All cultures will have currency symbol as "¤"
|
||||||
* Culture enumeration will always return a list with one culture which is the invariant culture.
|
* Culture enumeration will always return a list with one culture which is the invariant culture.
|
||||||
|
|
||||||
## String casing
|
## String casing
|
||||||
|
|
||||||
String casing (ToUpper and ToLower) will be performed for the ASCII range only. Requests to case code points outside that range will not be performed, however no exception will be thrown. In other words, casing will only be performed for character range ['a'..'z'].
|
String casing (ToUpper and ToLower) will be performed for the ASCII range only. Requests to case code points outside that range will not be performed, however no exception will be thrown. In other words, casing will only be performed for character range ['a'..'z'].
|
||||||
|
|
||||||
Turkish I casing will not be supported when using Turkish cultures.
|
Turkish I casing will not be supported when using Turkish cultures.
|
||||||
|
|
||||||
## String sorting and searching
|
## String sorting and searching
|
||||||
|
|
||||||
String operations like [Compare](https://docs.microsoft.com/dotnet/api/?term=string.compare), [IndexOf](https://docs.microsoft.com/dotnet/api/?term=string.indexof) and [LastIndexOf](https://docs.microsoft.com/dotnet/api/?term=string.lastindexof) are always performed as [ordinal](https://en.wikipedia.org/wiki/Ordinal_number) and not linguistic operations regardless of the string comparing options passed to the APIs.
|
String operations like [Compare](https://docs.microsoft.com/dotnet/api/?term=string.compare), [IndexOf](https://docs.microsoft.com/dotnet/api/?term=string.indexof) and [LastIndexOf](https://docs.microsoft.com/dotnet/api/?term=string.lastindexof) are always performed as [ordinal](https://en.wikipedia.org/wiki/Ordinal_number) and not linguistic operations regardless of the string comparing options passed to the APIs.
|
||||||
|
|
||||||
The [ignore case](https://docs.microsoft.com/dotnet/api/system.globalization.compareoptions.ignorecase) string sorting option is supported but only for the ASCII range as mentioned previously.
|
The [ignore case](https://docs.microsoft.com/dotnet/api/system.globalization.compareoptions.ignorecase) string sorting option is supported but only for the ASCII range as mentioned previously.
|
||||||
|
|
||||||
For example, the following comparison will resolve to being unequal:
|
For example, the following comparison will resolve to being unequal:
|
||||||
|
|
||||||
* 'i', compared to
|
* 'i', compared to
|
||||||
* Turkish I '\u0130', given
|
* Turkish I '\u0130', given
|
||||||
* Turkish culture, using
|
* Turkish culture, using
|
||||||
* CompareOptions.Ignorecase
|
* CompareOptions.Ignorecase
|
||||||
|
|
||||||
However, the following comparison will resolve to being equal:
|
However, the following comparison will resolve to being equal:
|
||||||
|
|
||||||
* 'i', compared to
|
* 'i', compared to
|
||||||
* 'I', using
|
* 'I', using
|
||||||
* CompareOptions.Ignorecase
|
* CompareOptions.Ignorecase
|
||||||
|
|
||||||
It is worth noticing that all other [sort comparison options](https://docs.microsoft.com/dotnet/api/system.globalization.compareoptions) (for example, ignore symbols, ignore space, Katakana, Hiragana) will have no effect in the invariant mode (they are ignored).
|
|
||||||
|
|
||||||
## Sort keys
|
|
||||||
|
|
||||||
Sort keys are used mostly when indexing some data (for example, database indexing). When generating sort keys of 2 strings and comparing the sort keys the results should hold the exact same results as if comparing the original 2 strings. In the invariant mode, sort keys will be generated according to ordinal comparison while respecting ignore casing options.
|
|
||||||
|
|
||||||
## String normalization
|
|
||||||
|
|
||||||
String normalization normalizes a string into some form (for example, composed, decomposed forms). Normalization data is required to perform these operations, which isn't available in invariant mode. In this mode, all strings are considered as already normalized, per the following behavior:
|
|
||||||
|
|
||||||
* If the app requested to normalize any string, the original string is returned without modification.
|
It is worth noticing that all other [sort comparison options](https://docs.microsoft.com/dotnet/api/system.globalization.compareoptions) (for example, ignore symbols, ignore space, Katakana, Hiragana) will have no effect in the invariant mode (they are ignored).
|
||||||
|
|
||||||
|
## Sort keys
|
||||||
|
|
||||||
|
Sort keys are used mostly when indexing some data (for example, database indexing). When generating sort keys of 2 strings and comparing the sort keys the results should hold the exact same results as if comparing the original 2 strings. In the invariant mode, sort keys will be generated according to ordinal comparison while respecting ignore casing options.
|
||||||
|
|
||||||
|
## String normalization
|
||||||
|
|
||||||
|
String normalization normalizes a string into some form (for example, composed, decomposed forms). Normalization data is required to perform these operations, which isn't available in invariant mode. In this mode, all strings are considered as already normalized, per the following behavior:
|
||||||
|
|
||||||
|
* If the app requested to normalize any string, the original string is returned without modification.
|
||||||
* If the app asked if any string is normalized, the return value will always be `true`.
|
* If the app asked if any string is normalized, the return value will always be `true`.
|
||||||
|
|
||||||
## Internationalized Domain Names (IDN) support
|
## Internationalized Domain Names (IDN) support
|
||||||
|
|
||||||
[Internationalized Domain Names](https://en.wikipedia.org/wiki/Internationalized_domain_name) require globalization data to perform conversion to ASCII or Unicode forms, which isn't available in the invariant mode. In this mode, IDN functionality has the following behavior:
|
[Internationalized Domain Names](https://en.wikipedia.org/wiki/Internationalized_domain_name) require globalization data to perform conversion to ASCII or Unicode forms, which isn't available in the invariant mode. In this mode, IDN functionality has the following behavior:
|
||||||
|
@ -91,13 +91,13 @@ String normalization normalizes a string into some form (for example, composed,
|
||||||
* IDN support doesn't conform to the latest standard.
|
* IDN support doesn't conform to the latest standard.
|
||||||
* IDN support will be incorrect if the input IDN string is not normalized since normalization is not supported in invariant mode.
|
* IDN support will be incorrect if the input IDN string is not normalized since normalization is not supported in invariant mode.
|
||||||
* Some basic IDN strings will still produce correct values.
|
* Some basic IDN strings will still produce correct values.
|
||||||
|
|
||||||
## Time zone display name in Linux
|
## Time zone display name in Linux
|
||||||
|
|
||||||
When running on Linux, ICU is used to get the time zone display name. In invariant mode, the standard time zone names are returned instead.
|
When running on Linux, ICU is used to get the time zone display name. In invariant mode, the standard time zone names are returned instead.
|
||||||
|
|
||||||
## Enabling the invariant mode
|
## Enabling the invariant mode
|
||||||
|
|
||||||
Applications can enable the invariant mode by either of the following:
|
Applications can enable the invariant mode by either of the following:
|
||||||
|
|
||||||
1. in project file:
|
1. in project file:
|
||||||
|
@ -119,13 +119,13 @@ Applications can enable the invariant mode by either of the following:
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
```
|
```
|
||||||
|
|
||||||
3. setting environment variable value `DOTNET_SYSTEM_GLOBALIZATION_INVARIANT` to `true` or `1`.
|
3. setting environment variable value `DOTNET_SYSTEM_GLOBALIZATION_INVARIANT` to `true` or `1`.
|
||||||
|
|
||||||
Note: value set in project file or `runtimeconfig.json` has higher priority than the environment variable.
|
Note: value set in project file or `runtimeconfig.json` has higher priority than the environment variable.
|
||||||
|
|
||||||
## APP behavior with and without the invariant config switch
|
## APP behavior with and without the invariant config switch
|
||||||
|
|
||||||
- If the invariant config switch is not set or it is set false
|
- If the invariant config switch is not set or it is set false
|
||||||
- The framework will depend on the OS for the globalization support.
|
- The framework will depend on the OS for the globalization support.
|
||||||
- On Linux, if the ICU package is not installed, the application will fail to start.
|
- On Linux, if the ICU package is not installed, the application will fail to start.
|
||||||
|
|
|
@ -40,4 +40,4 @@ This feature certainly provides a somewhat duplicate functionality to the existi
|
||||||
* Currently we don't consider frameworks for the app when computing probing paths for resolving assets from the component's `.deps.json`. This is a different behavior from the app startup where these are considered. Is it important - needed?
|
* Currently we don't consider frameworks for the app when computing probing paths for resolving assets from the component's `.deps.json`. This is a different behavior from the app startup where these are considered. Is it important - needed?
|
||||||
* Add ability to corelate tracing with the runtime - probably some kind of activity ID
|
* Add ability to corelate tracing with the runtime - probably some kind of activity ID
|
||||||
* Handling of native assets - currently returning just probing paths. Would be cleaner to return full resolved paths. But we would have to keep some probing paths. In the case of missing `.deps.json` the native library should be looked for in the component directory - thus requires probing - we can't figure out which of the files in the folder are native libraries in the hosts.
|
* Handling of native assets - currently returning just probing paths. Would be cleaner to return full resolved paths. But we would have to keep some probing paths. In the case of missing `.deps.json` the native library should be looked for in the component directory - thus requires probing - we can't figure out which of the files in the folder are native libraries in the hosts.
|
||||||
* Handling of satellite assemblies (resource assets) - currently returning just probing paths which exclude the culture. So from a resolved asset `./foo/en-us/resource.dll` we only take `./foo` as the probing path. Consider using full paths instead - probably would require more parsing as we would have to be able to figure out the culture ID somewhere to build the true map AssemblyName->path in the managed class. Just like for native assets, if there's no `.deps.json` the only possible solution is to use probing, so the probing semantics would have to be supported anyway.
|
* Handling of satellite assemblies (resource assets) - currently returning just probing paths which exclude the culture. So from a resolved asset `./foo/en-us/resource.dll` we only take `./foo` as the probing path. Consider using full paths instead - probably would require more parsing as we would have to be able to figure out the culture ID somewhere to build the true map AssemblyName->path in the managed class. Just like for native assets, if there's no `.deps.json` the only possible solution is to use probing, so the probing semantics would have to be supported anyway.
|
||||||
|
|
|
@ -7,25 +7,23 @@ Note that the exit code returned by running an application via `dotnet.exe` or `
|
||||||
|
|
||||||
* `Success` (`0`) - Operation was successful.
|
* `Success` (`0`) - Operation was successful.
|
||||||
|
|
||||||
* `Success_HostAlreadyInitialized` (`0x00000001`) - Initialization was successful, but another host context is already initialized, so the returned context is "secondary". The requested context was otherwise fully compatible with the already initialized context.
|
* `Success_HostAlreadyInitialized` (`0x00000001`) - Initialization was successful, but another host context is already initialized, so the returned context is "secondary". The requested context was otherwise fully compatible with the already initialized context. This is returned by `hostfxr_initialize_for_runtime_config` if it's called when the host is already initialized in the process. Comes from `corehost_initialize` in `hostpolicy`.
|
||||||
This is returned by `hostfxr_initialize_for_runtime_config` if it's called when the host is already initialized in the process. Comes from `corehost_initialize` in `hostpolicy`.
|
|
||||||
|
|
||||||
* `Success_DifferentRuntimeProperties` (`0x00000002`) - Initialization was successful, but another host context is already initialized and the requested context specified some runtime properties which are not the same (either in value or in presence) to the already initialized context.
|
* `Success_DifferentRuntimeProperties` (`0x00000002`) - Initialization was successful, but another host context is already initialized and the requested context specified some runtime properties which are not the same (either in value or in presence) to the already initialized context. This is returned by `hostfxr_initialize_for_runtime_config` if it's called when the host is already initialized in the process. Comes from `corehost_initialize` in `hostpolicy`.
|
||||||
This is returned by `hostfxr_initialize_for_runtime_config` if it's called when the host is already initialized in the process. Comes from `corehost_initialize` in `hostpolicy`.
|
|
||||||
|
|
||||||
|
|
||||||
### Failure error/exit codes
|
### Failure error/exit codes
|
||||||
|
|
||||||
* `InvalidArgFailure` (`0x80008081`) - One of the specified arguments for the operation is invalid.
|
* `InvalidArgFailure` (`0x80008081`) - One of the specified arguments for the operation is invalid.
|
||||||
|
|
||||||
* `CoreHostLibLoadFailure` (`0x80008082`) - There was a failure loading a dependent library. If any of the hosting components calls `LoadLibrary`/`dlopen` on a dependent library and the call fails, this error code is returned. The most common case for this failure is if the dependent library is missing some of its dependencies (for example the necessary CRT is missing on the machine), likely corrupt or incomplete install.
|
* `CoreHostLibLoadFailure` (`0x80008082`) - There was a failure loading a dependent library. If any of the hosting components calls `LoadLibrary`/`dlopen` on a dependent library and the call fails, this error code is returned. The most common case for this failure is if the dependent library is missing some of its dependencies (for example the necessary CRT is missing on the machine), likely corrupt or incomplete install.
|
||||||
This error code is also returned from `corehost_resolve_component_dependencies` if it's called on a `hostpolicy` which has not been initialized via the hosting layer. This would typically happen if `coreclr` is loaded directly without the hosting layer and then `AssemblyDependencyResolver` is used (which is an unsupported scenario).
|
This error code is also returned from `corehost_resolve_component_dependencies` if it's called on a `hostpolicy` which has not been initialized via the hosting layer. This would typically happen if `coreclr` is loaded directly without the hosting layer and then `AssemblyDependencyResolver` is used (which is an unsupported scenario).
|
||||||
|
|
||||||
* `CoreHostLibMissingFailure` (`0x80008083`) - One of the dependent libraries is missing. Typically when the `hostfxr`, `hostpolicy` or `coreclr` dynamic libraries are not present in the expected locations. Probably means corrupted or incomplete installation.
|
* `CoreHostLibMissingFailure` (`0x80008083`) - One of the dependent libraries is missing. Typically when the `hostfxr`, `hostpolicy` or `coreclr` dynamic libraries are not present in the expected locations. Probably means corrupted or incomplete installation.
|
||||||
|
|
||||||
* `CoreHostEntryPointFailure` (`0x80008084`) - One of the dependent libraries is missing a required entry point.
|
* `CoreHostEntryPointFailure` (`0x80008084`) - One of the dependent libraries is missing a required entry point.
|
||||||
|
|
||||||
* `CoreHostCurHostFindFailure` (`0x80008085`) - If the hosting component is trying to use the path to the current module (the hosting component itself) and from it deduce the location of the installation. Either the location of the current module could not be determined (some weird OS call failure) or the location is not in the right place relative to other expected components.
|
* `CoreHostCurHostFindFailure` (`0x80008085`) - If the hosting component is trying to use the path to the current module (the hosting component itself) and from it deduce the location of the installation. Either the location of the current module could not be determined (some weird OS call failure) or the location is not in the right place relative to other expected components.
|
||||||
For example the `hostfxr` may look at its location and try to deduce the location of the `shared` folder with the framework from it. It assumes the typical install layout on disk. If this doesn't work, this error will be returned.
|
For example the `hostfxr` may look at its location and try to deduce the location of the `shared` folder with the framework from it. It assumes the typical install layout on disk. If this doesn't work, this error will be returned.
|
||||||
|
|
||||||
* `CoreClrResolveFailure` (`0x80008087`) - If the `coreclr` library could not be found. The hosting layer (`hostpolicy`) looks for `coreclr` library either next to the app itself (for self-contained) or in the root framework (for framework-dependent). This search can be done purely by looking at disk or more commonly by looking into the respective `.deps.json`. If the `coreclr` library is missing in `.deps.json` or it's there but doesn't exist on disk, this error is returned.
|
* `CoreClrResolveFailure` (`0x80008087`) - If the `coreclr` library could not be found. The hosting layer (`hostpolicy`) looks for `coreclr` library either next to the app itself (for self-contained) or in the root framework (for framework-dependent). This search can be done purely by looking at disk or more commonly by looking into the respective `.deps.json`. If the `coreclr` library is missing in `.deps.json` or it's there but doesn't exist on disk, this error is returned.
|
||||||
|
@ -61,7 +59,7 @@ For example the `hostfxr` may look at its location and try to deduce the locatio
|
||||||
* Other inconsistencies (for example `rollForward` and `applyPatches` are not allowed to be specified in the same config file)
|
* Other inconsistencies (for example `rollForward` and `applyPatches` are not allowed to be specified in the same config file)
|
||||||
* Any of the above failures reading the `.runtimecofig.dev.json` file
|
* Any of the above failures reading the `.runtimecofig.dev.json` file
|
||||||
* Self-contained `.runtimeconfig.json` used in `hostfxr_initialize_for_runtime_config`
|
* Self-contained `.runtimeconfig.json` used in `hostfxr_initialize_for_runtime_config`
|
||||||
Note that missing `.runtimconfig.json` is not an error (means self-contained app).
|
Note that missing `.runtimconfig.json` is not an error (means self-contained app).
|
||||||
This error code is also used when there is a problem reading the CLSID map file in `comhost`.
|
This error code is also used when there is a problem reading the CLSID map file in `comhost`.
|
||||||
|
|
||||||
* `AppArgNotRunnable` (`0x80008094`) - Used internally when the command line for `dotnet.exe` doesn't contain path to the application to run. In such case the command line is considered to be a CLI/SDK command. This error code should never be returned to external caller.
|
* `AppArgNotRunnable` (`0x80008094`) - Used internally when the command line for `dotnet.exe` doesn't contain path to the application to run. In such case the command line is considered to be a CLI/SDK command. This error code should never be returned to external caller.
|
||||||
|
@ -70,8 +68,8 @@ This error code is also used when there is a problem reading the CLSID map file
|
||||||
* The `apphost` binary has not been imprinted with the path to the app to run (so freshly built `apphost.exe` from the branch will fail to run like this)
|
* The `apphost` binary has not been imprinted with the path to the app to run (so freshly built `apphost.exe` from the branch will fail to run like this)
|
||||||
* The `apphost` is a bundle (single-file exe) and it failed to extract correctly.
|
* The `apphost` is a bundle (single-file exe) and it failed to extract correctly.
|
||||||
|
|
||||||
* `FrameworkMissingFailure` (`0x80008096`) - It was not possible to find a compatible framework version. This originates in `hostfxr` (`resolve_framework_reference`) and means that the app specified a reference to a framework in its `.runtimeconfig.json` which could not be resolved. The failure to resolve can mean that no such framework is available on the disk, or that the available frameworks don't match the minimum version specified or that the roll forward options specified excluded all available frameworks.
|
* `FrameworkMissingFailure` (`0x80008096`) - It was not possible to find a compatible framework version. This originates in `hostfxr` (`resolve_framework_reference`) and means that the app specified a reference to a framework in its `.runtimeconfig.json` which could not be resolved. The failure to resolve can mean that no such framework is available on the disk, or that the available frameworks don't match the minimum version specified or that the roll forward options specified excluded all available frameworks.
|
||||||
Typically this would be used if a 3.0 app is trying to run on a machine which has no 3.0 installed.
|
Typically this would be used if a 3.0 app is trying to run on a machine which has no 3.0 installed.
|
||||||
It would also be used for example if a 32bit 3.0 app is running on a machine which has 3.0 installed but only for 64bit.
|
It would also be used for example if a 32bit 3.0 app is running on a machine which has 3.0 installed but only for 64bit.
|
||||||
|
|
||||||
* `HostApiFailed` (`0x80008097`) - Returned by `hostfxr_get_native_search_directories` if the `hostpolicy` could not calculate the `NATIVE_DLL_SEARCH_DIRECTORIES`.
|
* `HostApiFailed` (`0x80008097`) - Returned by `hostfxr_get_native_search_directories` if the `hostpolicy` could not calculate the `NATIVE_DLL_SEARCH_DIRECTORIES`.
|
||||||
|
@ -87,7 +85,7 @@ It would also be used for example if a 32bit 3.0 app is running on a machine whi
|
||||||
|
|
||||||
* `SdkResolverResolveFailure` (`0x8000809b`) - Returned from `hostfxr_resolve_sdk2` when it fails to find matching SDK. Similar to `LibHostSdkFindFailure` but only used in the `hostfxr_resolve_sdk2`.
|
* `SdkResolverResolveFailure` (`0x8000809b`) - Returned from `hostfxr_resolve_sdk2` when it fails to find matching SDK. Similar to `LibHostSdkFindFailure` but only used in the `hostfxr_resolve_sdk2`.
|
||||||
|
|
||||||
* `FrameworkCompatFailure` (`0x8000809c`) - During processing of `.runtimeconfig.json` there were two framework references to the same framework which were not compatible. This can happen if the app specified a framework reference to a lower-level framework which is also specified by a higher-level framework which is also used by the app.
|
* `FrameworkCompatFailure` (`0x8000809c`) - During processing of `.runtimeconfig.json` there were two framework references to the same framework which were not compatible. This can happen if the app specified a framework reference to a lower-level framework which is also specified by a higher-level framework which is also used by the app.
|
||||||
For example, this would happen if the app referenced `Microsoft.AspNet.App` version 2.0 and `Microsoft.NETCore.App` version 3.0. In such case the `Microsoft.AspNet.App` has `.runtimeconfig.json` which also references `Microsoft.NETCore.App` but it only allows versions 2.0 up to 2.9 (via roll forward options). So the version 3.0 requested by the app is incompatible.
|
For example, this would happen if the app referenced `Microsoft.AspNet.App` version 2.0 and `Microsoft.NETCore.App` version 3.0. In such case the `Microsoft.AspNet.App` has `.runtimeconfig.json` which also references `Microsoft.NETCore.App` but it only allows versions 2.0 up to 2.9 (via roll forward options). So the version 3.0 requested by the app is incompatible.
|
||||||
|
|
||||||
* `FrameworkCompatRetry` (`0x8000809d`) - Error used internally if the processing of framework references from `.runtimeconfig.json` reached a point where it needs to reprocess another already processed framework reference. If this error is returned to the external caller, it would mean there's a bug in the framework resolution algorithm.
|
* `FrameworkCompatRetry` (`0x8000809d`) - Error used internally if the processing of framework references from `.runtimeconfig.json` reached a point where it needs to reprocess another already processed framework reference. If this error is returned to the external caller, it would mean there's a bug in the framework resolution algorithm.
|
||||||
|
@ -100,10 +98,10 @@ For example, this would happen if the app referenced `Microsoft.AspNet.App` vers
|
||||||
|
|
||||||
* `LibHostDuplicateProperty` (`0x800080a1`) - The `.runtimeconfig.json` specified by the app contains a runtime property which is also produced by the hosting layer. For example if the `.runtimeconfig.json` would specify a property `TRUSTED_PLATFORM_ROOTS`, this error code would be returned. It is not allowed to specify properties which are otherwise populated by the hosting layer (`hostpolicy`) as there is not good way to resolve such conflicts.
|
* `LibHostDuplicateProperty` (`0x800080a1`) - The `.runtimeconfig.json` specified by the app contains a runtime property which is also produced by the hosting layer. For example if the `.runtimeconfig.json` would specify a property `TRUSTED_PLATFORM_ROOTS`, this error code would be returned. It is not allowed to specify properties which are otherwise populated by the hosting layer (`hostpolicy`) as there is not good way to resolve such conflicts.
|
||||||
|
|
||||||
* `HostApiUnsupportedVersion` (`0x800080a2`) - Feature which requires certain version of the hosting layer binaries was used on a version which doesn't support it.
|
* `HostApiUnsupportedVersion` (`0x800080a2`) - Feature which requires certain version of the hosting layer binaries was used on a version which doesn't support it.
|
||||||
For example if COM component specified to run on 2.0 `Microsoft.NETCore.App` - as that contains older version of `hostpolicy` which doesn't support the necessary features to provide COM services.
|
For example if COM component specified to run on 2.0 `Microsoft.NETCore.App` - as that contains older version of `hostpolicy` which doesn't support the necessary features to provide COM services.
|
||||||
|
|
||||||
* `HostInvalidState` (`0x800080a3`) - Error code returned by the hosting APIs in `hostfxr` if the current state is incompatible with the requested operation. There are many such cases, please refer to the documentation of the hosting APIs for details.
|
* `HostInvalidState` (`0x800080a3`) - Error code returned by the hosting APIs in `hostfxr` if the current state is incompatible with the requested operation. There are many such cases, please refer to the documentation of the hosting APIs for details.
|
||||||
For example if `hostfxr_get_runtime_property_value` is called with the `host_context_handle` `nullptr` (meaning get property from the active runtime) but there's no active runtime in the process.
|
For example if `hostfxr_get_runtime_property_value` is called with the `host_context_handle` `nullptr` (meaning get property from the active runtime) but there's no active runtime in the process.
|
||||||
|
|
||||||
* `HostPropertyNotFound` (`0x800080a4`) - property requested by `hostfxr_get_runtime_property_value` doesn't exist.
|
* `HostPropertyNotFound` (`0x800080a4`) - property requested by `hostfxr_get_runtime_property_value` doesn't exist.
|
||||||
|
|
|
@ -25,29 +25,29 @@ The dotnet host uses probing when it searches for actual file on disk for a give
|
||||||
The library relative path in this case is `newtonsoft.json/11.0.2` and the asset relative path is `lib/netstandard2.0/Newtonsoft.Json.dll`. So the goal of the probing logic is to find the `Newtonsoft.Json.dll` file using the above relative paths.
|
The library relative path in this case is `newtonsoft.json/11.0.2` and the asset relative path is `lib/netstandard2.0/Newtonsoft.Json.dll`. So the goal of the probing logic is to find the `Newtonsoft.Json.dll` file using the above relative paths.
|
||||||
|
|
||||||
## Probing
|
## Probing
|
||||||
The probing itself is done by going over a list of probing paths, which are ordered according to their priority. For each path, the host will append the relative parts of the path as per above and see if the file actually exists on the disk.
|
The probing itself is done by going over a list of probing paths, which are ordered according to their priority. For each path, the host will append the relative parts of the path as per above and see if the file actually exists on the disk.
|
||||||
If the file is found, the probing is done, and the full path just resolved is stored.
|
If the file is found, the probing is done, and the full path just resolved is stored.
|
||||||
If the file is not found, the probing continues with the next path on the list.
|
If the file is not found, the probing continues with the next path on the list.
|
||||||
If all paths are tried and the asset is still not found this is reported as an error (with the exception of app's `.deps.json` asset, in which case it's ignored).
|
If all paths are tried and the asset is still not found this is reported as an error (with the exception of app's `.deps.json` asset, in which case it's ignored).
|
||||||
|
|
||||||
## Probing paths
|
## Probing paths
|
||||||
The list of probing paths ordered according to their priority. First path in the list below is tried first and so on.
|
The list of probing paths ordered according to their priority. First path in the list below is tried first and so on.
|
||||||
* Servicing paths
|
* Servicing paths
|
||||||
Servicing paths are only used for serviceable assets, that is the corresponding library record must specify `serviceable: true`.
|
Servicing paths are only used for serviceable assets, that is the corresponding library record must specify `serviceable: true`.
|
||||||
The base servicing path is
|
The base servicing path is
|
||||||
* On Windows x64 `%ProgramFiles(x86)%\coreservicing`
|
* On Windows x64 `%ProgramFiles(x86)%\coreservicing`
|
||||||
* On Windows x86 `%ProgramFiles%\coreservicing`
|
* On Windows x86 `%ProgramFiles%\coreservicing`
|
||||||
* Otherwise (Linux/Mac) `$CORE_SERVICING`
|
* Otherwise (Linux/Mac) `$CORE_SERVICING`
|
||||||
|
|
||||||
Given the base servicing path, the probing paths are
|
Given the base servicing path, the probing paths are
|
||||||
* Servicing NI probe path `<servicing base>/|arch|` - this is used only for `runtime` assets
|
* Servicing NI probe path `<servicing base>/|arch|` - this is used only for `runtime` assets
|
||||||
* Servicing normal probe path `<servicing base>/pkgs` - this is used for all assets
|
* Servicing normal probe path `<servicing base>/pkgs` - this is used for all assets
|
||||||
|
|
||||||
* The application (or framework if we're resolving framework assets) directory
|
* The application (or framework if we're resolving framework assets) directory
|
||||||
* Framework directories
|
* Framework directories
|
||||||
If the app (or framework) has dependencies on frameworks, these frameworks are used as probing paths.
|
If the app (or framework) has dependencies on frameworks, these frameworks are used as probing paths.
|
||||||
The order is from the higher level framework to lower level framework. The app is considered the highest level, it direct dependencies are next and so on.
|
The order is from the higher level framework to lower level framework. The app is considered the highest level, it direct dependencies are next and so on.
|
||||||
For assets from frameworks, only that framework and lower level frameworks are considered.
|
For assets from frameworks, only that framework and lower level frameworks are considered.
|
||||||
Note: These directories come directly out of the framework resolution process. Special note on Windows where global locations are always considered even if the app is not executed via the shared `dotnet.exe`. More details can be found in [Multi-level Shared FX Lookup](multilevel-sharedfx-lookup.md).
|
Note: These directories come directly out of the framework resolution process. Special note on Windows where global locations are always considered even if the app is not executed via the shared `dotnet.exe`. More details can be found in [Multi-level Shared FX Lookup](multilevel-sharedfx-lookup.md).
|
||||||
* Shared store paths
|
* Shared store paths
|
||||||
* `$DOTNET_SHARED_STORE/|arch|/|tfm|` - The environment variable `DOTNET_SHARED_STORE` can contain multiple paths, in which case each is appended with `|arch|/|tfm|` and used as a probing path.
|
* `$DOTNET_SHARED_STORE/|arch|/|tfm|` - The environment variable `DOTNET_SHARED_STORE` can contain multiple paths, in which case each is appended with `|arch|/|tfm|` and used as a probing path.
|
||||||
|
@ -56,10 +56,10 @@ The list of probing paths ordered according to their priority. First path in the
|
||||||
* On Windows, the global shared store is used
|
* On Windows, the global shared store is used
|
||||||
* If running in WOW64 mode - `%ProgramFiles(x86)%\dotnet\store\|arch|\|tfm|`
|
* If running in WOW64 mode - `%ProgramFiles(x86)%\dotnet\store\|arch|\|tfm|`
|
||||||
* Otherwise - `%ProgramFiles%\dotnet\store\|arch|\|tfm|`
|
* Otherwise - `%ProgramFiles%\dotnet\store\|arch|\|tfm|`
|
||||||
* Additional probing paths
|
* Additional probing paths
|
||||||
In these paths the `|arch|/|tfm|` string can be used and will be replaced with the actual values before using the path.
|
In these paths the `|arch|/|tfm|` string can be used and will be replaced with the actual values before using the path.
|
||||||
* `--additionalprobingpath` command line arguments
|
* `--additionalprobingpath` command line arguments
|
||||||
* `additionalProbingPaths` specified in `.runtimeconfig.json` and `.runtimeconfig.dev.json` for the app and each framework (highest to lowest)
|
* `additionalProbingPaths` specified in `.runtimeconfig.json` and `.runtimeconfig.dev.json` for the app and each framework (highest to lowest)
|
||||||
|
|
||||||
|
|
||||||
Note about framework-dependent and self-contained apps. With regard to probing the main difference is that self-contained apps don't have any framework dependencies, so all assets (including assemblies which normally come from a framework) are probed for in the app's directory.
|
Note about framework-dependent and self-contained apps. With regard to probing the main difference is that self-contained apps don't have any framework dependencies, so all assets (including assemblies which normally come from a framework) are probed for in the app's directory.
|
||||||
|
|
|
@ -57,8 +57,8 @@ Each part may be either
|
||||||
* the assembly name must be considered a valid assembly name as specified
|
* the assembly name must be considered a valid assembly name as specified
|
||||||
by the `AssemblyName` class.
|
by the `AssemblyName` class.
|
||||||
|
|
||||||
Note that white-spaces are preserved and considered part of the specified
|
Note that white-spaces are preserved and considered part of the specified
|
||||||
path/name. So for example path separator followed by a white-space and
|
path/name. So for example path separator followed by a white-space and
|
||||||
another path separator is invalid, since the white-space only string
|
another path separator is invalid, since the white-space only string
|
||||||
in between the path separators will be considered as assembly name.
|
in between the path separators will be considered as assembly name.
|
||||||
|
|
||||||
|
@ -90,10 +90,10 @@ centralized, while still allowing user code to do its own thing if it
|
||||||
so desires.
|
so desires.
|
||||||
|
|
||||||
The producer of `StartupHook.dll` needs to ensure that
|
The producer of `StartupHook.dll` needs to ensure that
|
||||||
`StartupHook.dll` is compatible with the dependencies specified in the
|
`StartupHook.dll` is compatible with the dependencies specified in the
|
||||||
main application's deps.json, since those dependencies are put on the
|
main application's deps.json, since those dependencies are put on the
|
||||||
Trusted Platform Assemblies (TPA) list during the runtime startup,
|
Trusted Platform Assemblies (TPA) list during the runtime startup,
|
||||||
before `StartupHook.dll` is loaded. This means that `StartupHook.dll`
|
before `StartupHook.dll` is loaded. This means that `StartupHook.dll`
|
||||||
needs to be built against the same or lower version of .NET Core than the app.
|
needs to be built against the same or lower version of .NET Core than the app.
|
||||||
|
|
||||||
## Example
|
## Example
|
||||||
|
|
|
@ -1,6 +1,6 @@
|
||||||
# Host tracing
|
# Host tracing
|
||||||
|
|
||||||
The various .NET Core host components provide detailed tracing of diagnostic information which can help solve issues around runtime, framework and assembly resolution and others.
|
The various .NET Core host components provide detailed tracing of diagnostic information which can help solve issues around runtime, framework and assembly resolution and others.
|
||||||
|
|
||||||
## Existing support
|
## Existing support
|
||||||
Currently (as of .NET Core 2.1) the host tracing is only written to the `stderr` output of the process. It can be turned on by setting `COREHOST_TRACE=1`.
|
Currently (as of .NET Core 2.1) the host tracing is only written to the `stderr` output of the process. It can be turned on by setting `COREHOST_TRACE=1`.
|
||||||
|
@ -47,13 +47,13 @@ The functions behave exactly the same in both components. The `listener` paramet
|
||||||
* a pointer to an implementation of `host_trace_listener` which is then registered the only listener for all tracing.
|
* a pointer to an implementation of `host_trace_listener` which is then registered the only listener for all tracing.
|
||||||
* `NULL` value which unregisters any previously registered listener. After this call tracing is disabled.
|
* `NULL` value which unregisters any previously registered listener. After this call tracing is disabled.
|
||||||
|
|
||||||
Custom host can and should register the trace listener as the first thing it does with the respective host component to ensure that all tracing is routed to it.
|
Custom host can and should register the trace listener as the first thing it does with the respective host component to ensure that all tracing is routed to it.
|
||||||
|
|
||||||
Only one trace listener can be registered at any given time.
|
Only one trace listener can be registered at any given time.
|
||||||
|
|
||||||
Registering custom trace listener or setting it to `NULL` doesn't override the tracing enabled by environment variables. If a trace listener is registered and the `COREHOST_TRACE=1` is set as well, the traces will be routed to both the `stderr` as well as the registered listener.
|
Registering custom trace listener or setting it to `NULL` doesn't override the tracing enabled by environment variables. If a trace listener is registered and the `COREHOST_TRACE=1` is set as well, the traces will be routed to both the `stderr` as well as the registered listener.
|
||||||
|
|
||||||
The `hostfxr` component will propagate the trace listener to the `hostpolicy` component before it calls into it. So custom host only needs to register its trace listener with the `hostfxr` component and not both. The propagation of the trace listener is only done for the duration necessary after which it will be unregistered again. So custom host might need to register its own listener if it makes calls directly to `hostpolicy` on top of the calls to `hostfxr`.
|
The `hostfxr` component will propagate the trace listener to the `hostpolicy` component before it calls into it. So custom host only needs to register its trace listener with the `hostfxr` component and not both. The propagation of the trace listener is only done for the duration necessary after which it will be unregistered again. So custom host might need to register its own listener if it makes calls directly to `hostpolicy` on top of the calls to `hostfxr`.
|
||||||
In case of new (.NET Core 3) `hostfxr` component which would call into an old (.NET Core 2.1) `hostpolicy` component, the `hostfxr` will not perform the propagation in any way since the older `hostpolicy` doesn't support this mechanism.
|
In case of new (.NET Core 3) `hostfxr` component which would call into an old (.NET Core 2.1) `hostpolicy` component, the `hostfxr` will not perform the propagation in any way since the older `hostpolicy` doesn't support this mechanism.
|
||||||
|
|
||||||
The trace listener interface looks like this:
|
The trace listener interface looks like this:
|
||||||
|
@ -68,14 +68,14 @@ struct host_trace_listener
|
||||||
}
|
}
|
||||||
```
|
```
|
||||||
|
|
||||||
The `message` parameter is a standard `NUL` terminated string and it's the message to trace with the respective verbosity level.
|
The `message` parameter is a standard `NUL` terminated string and it's the message to trace with the respective verbosity level.
|
||||||
The `activityId` parameter is a standard `NUL` terminated string. It's used to correlate traces for a given binding event. The content of the string is not yet defined, but the trace listeners should consider it opaque. Trace listeners should include this string in the trace of the message in some form. The parameter may be `NULL` in which case the trace doesn't really belong to any specific binding event.
|
The `activityId` parameter is a standard `NUL` terminated string. It's used to correlate traces for a given binding event. The content of the string is not yet defined, but the trace listeners should consider it opaque. Trace listeners should include this string in the trace of the message in some form. The parameter may be `NULL` in which case the trace doesn't really belong to any specific binding event.
|
||||||
|
|
||||||
Methods on the trace listener interface can be called from any thread in the app, and should be able to handle multiple calls at the same time from different threads.
|
Methods on the trace listener interface can be called from any thread in the app, and should be able to handle multiple calls at the same time from different threads.
|
||||||
|
|
||||||
## Future investments
|
## Future investments
|
||||||
### Trace content
|
### Trace content
|
||||||
Currently the host components tend to trace a lot. The trace contains lot of interesting information but it's done in a very verbose way which is sometimes hard to navigate. Future investment should look at the common scenarios which are using the host tracing and optimize the trace output for those scenarios. This doesn't necessarily mean decrease the amount of tracing, but possibly introduce "summary sections" which would describe the end result decisions for certain scenarios.
|
Currently the host components tend to trace a lot. The trace contains lot of interesting information but it's done in a very verbose way which is sometimes hard to navigate. Future investment should look at the common scenarios which are using the host tracing and optimize the trace output for those scenarios. This doesn't necessarily mean decrease the amount of tracing, but possibly introduce "summary sections" which would describe the end result decisions for certain scenarios.
|
||||||
It would also be good to review the usage of verbose versus info tracing and make it consistent.
|
It would also be good to review the usage of verbose versus info tracing and make it consistent.
|
||||||
|
|
||||||
### Interaction with other diagnostics in the .NET Core
|
### Interaction with other diagnostics in the .NET Core
|
||||||
|
|
|
@ -23,7 +23,7 @@ public enum EventListenerSettings
|
||||||
None,
|
None,
|
||||||
RawEventDispatch
|
RawEventDispatch
|
||||||
}
|
}
|
||||||
```
|
```
|
||||||
|
|
||||||
This parameter is used to specify the desired dispatch behavior (in this case, do not deserialize event payloads).
|
This parameter is used to specify the desired dispatch behavior (in this case, do not deserialize event payloads).
|
||||||
|
|
||||||
|
@ -33,7 +33,7 @@ The new raw dispatch API will be:
|
||||||
|
|
||||||
```
|
```
|
||||||
public void OnEventWrittenRaw(RawEventWrittenEventArgs args);
|
public void OnEventWrittenRaw(RawEventWrittenEventArgs args);
|
||||||
|
|
||||||
public sealed class RawEventWrittenEventArgs
|
public sealed class RawEventWrittenEventArgs
|
||||||
{
|
{
|
||||||
|
|
||||||
|
@ -52,9 +52,9 @@ public sealed class RawEventWrittenEventArgs
|
||||||
public EventLevel Level { get; }
|
public EventLevel Level { get; }
|
||||||
public long OSThreadId { get; }
|
public long OSThreadId { get; }
|
||||||
public DateTime TimeStamp { get; }
|
public DateTime TimeStamp { get; }
|
||||||
|
|
||||||
// Replacement properties for Payload and PayloadNames.
|
// Replacement properties for Payload and PayloadNames.
|
||||||
public ReadOnlySpan<byte> Metadata { get; }
|
public ReadOnlySpan<byte> Metadata { get; }
|
||||||
public ReadOnlySpan<byte> Payload { get; }
|
public ReadOnlySpan<byte> Payload { get; }
|
||||||
}
|
}
|
||||||
```
|
```
|
||||||
|
|
|
@ -50,7 +50,7 @@ we propose using two complementary strategies:
|
||||||
|
|
||||||
* In the composite R2R file with embedded metadata, there must be a new table of COR headers
|
* In the composite R2R file with embedded metadata, there must be a new table of COR headers
|
||||||
and metadata blobs representing the MSIL metadata from all the input assemblies. The table
|
and metadata blobs representing the MSIL metadata from all the input assemblies. The table
|
||||||
must be indexable by simple assembly name for fast lookup.
|
must be indexable by simple assembly name for fast lookup.
|
||||||
|
|
||||||
* in contrast to managed assemblies and single-input R2R executables, composite R2R files
|
* in contrast to managed assemblies and single-input R2R executables, composite R2R files
|
||||||
don't expose any COR header (it's not meaningful as the file potentially contains a larger
|
don't expose any COR header (it's not meaningful as the file potentially contains a larger
|
||||||
|
@ -98,7 +98,7 @@ this encoding are still work in progress and likely to further evolve.
|
||||||
version bubble is represented by an arbitrary mixture of single-input and composite R2R files.
|
version bubble is represented by an arbitrary mixture of single-input and composite R2R files.
|
||||||
If that is the case, manifest metadata would need to be decoupled from the index to
|
If that is the case, manifest metadata would need to be decoupled from the index to
|
||||||
`READYTORUN_SECTION_ASSEMBLIES`.
|
`READYTORUN_SECTION_ASSEMBLIES`.
|
||||||
|
|
||||||
Alternatively we could make it such that `READYTORUN_SECTION_MANIFEST_METADATA` holds all
|
Alternatively we could make it such that `READYTORUN_SECTION_MANIFEST_METADATA` holds all
|
||||||
component assemblies of the current composite image at the beginning of the AssemblyRef table
|
component assemblies of the current composite image at the beginning of the AssemblyRef table
|
||||||
followed by the other needed assemblies *within the version bubble outside of the current
|
followed by the other needed assemblies *within the version bubble outside of the current
|
||||||
|
@ -157,7 +157,7 @@ that could be subsequently opened by ILDASM or ILSpy.
|
||||||
|
|
||||||
Ideally we should patch ILDASM / ILSpy to cleanly handle the composite R2R file format; sadly this may
|
Ideally we should patch ILDASM / ILSpy to cleanly handle the composite R2R file format; sadly this may
|
||||||
end up being a relatively complex change due to the presence of multiple MSIL metadata blocks in the
|
end up being a relatively complex change due to the presence of multiple MSIL metadata blocks in the
|
||||||
file.
|
file.
|
||||||
|
|
||||||
# Required diagnostic changes
|
# Required diagnostic changes
|
||||||
|
|
||||||
|
|
|
@ -11,19 +11,19 @@ If the version specified is a _production_ version, the default behavior is:
|
||||||
|
|
||||||
```
|
```
|
||||||
For instance:
|
For instance:
|
||||||
|
|
||||||
Desired version: 1.0.1
|
Desired version: 1.0.1
|
||||||
Available versions: 1.0.0, 1.0.1, 1.0.2, 1.0.3, 1.1.0, 1.1.1, 2.0.1
|
Available versions: 1.0.0, 1.0.1, 1.0.2, 1.0.3, 1.1.0, 1.1.1, 2.0.1
|
||||||
Chosen version: 1.0.3
|
Chosen version: 1.0.3
|
||||||
|
|
||||||
Desired version: 1.0.1
|
Desired version: 1.0.1
|
||||||
Available versions: 1.0.0, 1.1.0-preview1-x, 1.1.0-preview2-x, 1.2.0-preview1-x
|
Available versions: 1.0.0, 1.1.0-preview1-x, 1.1.0-preview2-x, 1.2.0-preview1-x
|
||||||
Chosen version: 1.1.0-preview2-x
|
Chosen version: 1.1.0-preview2-x
|
||||||
|
|
||||||
Desired version: 1.0.1
|
Desired version: 1.0.1
|
||||||
Available versions: 1.0.0, 1.1.0-preview1-x, 1.2.0, 1.2.1-preview1-x
|
Available versions: 1.0.0, 1.1.0-preview1-x, 1.2.0, 1.2.1-preview1-x
|
||||||
Chosen version: 1.2.0
|
Chosen version: 1.2.0
|
||||||
|
|
||||||
Desired version: 1.0.1
|
Desired version: 1.0.1
|
||||||
Available versions: 1.0.0, 2.0.0
|
Available versions: 1.0.0, 2.0.0
|
||||||
Chosen version: there is no compatible version available
|
Chosen version: there is no compatible version available
|
||||||
|
@ -40,14 +40,14 @@ This means _preview_ is never rolled forward to _production_.
|
||||||
Desired version: 1.0.1-preview2-x
|
Desired version: 1.0.1-preview2-x
|
||||||
Available versions: 1.0.1-preview2-x, 1.0.1-preview3-x
|
Available versions: 1.0.1-preview2-x, 1.0.1-preview3-x
|
||||||
Chosen version: 1.0.1-preview2-x
|
Chosen version: 1.0.1-preview2-x
|
||||||
|
|
||||||
Desired version: 1.0.1-preview2-x
|
Desired version: 1.0.1-preview2-x
|
||||||
Available versions: 1.0.1-preview3-x
|
Available versions: 1.0.1-preview3-x
|
||||||
Chosen version: 1.0.1-preview3-x
|
Chosen version: 1.0.1-preview3-x
|
||||||
|
|
||||||
Desired version: 1.0.1-preview2-x
|
Desired version: 1.0.1-preview2-x
|
||||||
Available versions: 1.0.1, 1.0.2-preview3-x
|
Available versions: 1.0.1, 1.0.2-preview3-x
|
||||||
Chosen version: there is no compatible version available
|
Chosen version: there is no compatible version available
|
||||||
|
|
||||||
## Settings to control behavior
|
## Settings to control behavior
|
||||||
### applyPatches
|
### applyPatches
|
||||||
|
@ -59,7 +59,7 @@ Once a compatible framework version is chosen as explained above, the latest pat
|
||||||
Desired version: 1.0.1
|
Desired version: 1.0.1
|
||||||
Available versions: 1.0.1, 1.0.2
|
Available versions: 1.0.1, 1.0.2
|
||||||
Chosen version: 1.0.2
|
Chosen version: 1.0.2
|
||||||
|
|
||||||
Patch roll forward: disabled
|
Patch roll forward: disabled
|
||||||
Desired version: 1.0.1
|
Desired version: 1.0.1
|
||||||
Available versions: 1.0.1, 1.0.2
|
Available versions: 1.0.1, 1.0.2
|
||||||
|
@ -79,7 +79,7 @@ To specify the exact desired framework version, use the command line argument '-
|
||||||
- Command line argument ('--roll-forward-on-no-candidate-fx' argument)
|
- Command line argument ('--roll-forward-on-no-candidate-fx' argument)
|
||||||
- Runtime configuration file ('rollForwardOnNoCandidateFx' property)
|
- Runtime configuration file ('rollForwardOnNoCandidateFx' property)
|
||||||
- DOTNET_ROLL_FORWARD_ON_NO_CANDIDATE_FX environment variable
|
- DOTNET_ROLL_FORWARD_ON_NO_CANDIDATE_FX environment variable
|
||||||
|
|
||||||
The valid values:
|
The valid values:
|
||||||
|
|
||||||
0) Off (_do not roll forward_)
|
0) Off (_do not roll forward_)
|
||||||
|
@ -101,7 +101,7 @@ If this feature is enabled and no compatible framework version is found, we'll s
|
||||||
Desired Version: 1.0.0
|
Desired Version: 1.0.0
|
||||||
Available versions: 1.1.1, 1.1.3, 1.2.0
|
Available versions: 1.1.1, 1.1.3, 1.2.0
|
||||||
Chosen version: 1.1.1
|
Chosen version: 1.1.1
|
||||||
|
|
||||||
Patch roll forward: enabled
|
Patch roll forward: enabled
|
||||||
Roll Forward On No Candidate Fx: 0 (disabled)
|
Roll Forward On No Candidate Fx: 0 (disabled)
|
||||||
Desired Version: 1.0.0
|
Desired Version: 1.0.0
|
||||||
|
@ -111,7 +111,7 @@ If this feature is enabled and no compatible framework version is found, we'll s
|
||||||
|
|
||||||
It's important to notice that, even if "Roll Forward On No Candidate Fx" is enabled, only the specified framework version will be accepted if the '--fx-version' argument is used.
|
It's important to notice that, even if "Roll Forward On No Candidate Fx" is enabled, only the specified framework version will be accepted if the '--fx-version' argument is used.
|
||||||
|
|
||||||
Since there are three ways to specify the values, conflicts will be resolved by the order listed above (command line has priority over config, which has priority over the environment variable).
|
Since there are three ways to specify the values, conflicts will be resolved by the order listed above (command line has priority over config, which has priority over the environment variable).
|
||||||
```
|
```
|
||||||
For instance:
|
For instance:
|
||||||
|
|
||||||
|
@ -123,8 +123,8 @@ Since there are three ways to specify the values, conflicts will be resolved by
|
||||||
'rollForwardOnNoCandidateFx' property is set to '1'
|
'rollForwardOnNoCandidateFx' property is set to '1'
|
||||||
DOTNET_ROLL_FORWARD_ON_NO_CANDIDATE_FX env var is set to '1'
|
DOTNET_ROLL_FORWARD_ON_NO_CANDIDATE_FX env var is set to '1'
|
||||||
The feature is DISABLED.
|
The feature is DISABLED.
|
||||||
```
|
```
|
||||||
|
|
||||||
There is no inheritance when there are chained framework references. If the app references FX1, and FX1 references FX2, then the resolution of FX2 only takes into account settings from `.runtimeconfig.json` in FX1, CLI and env. variable. The settings in the app's `.runtimeconfig.json` have no effect on resolution of FX2.
|
There is no inheritance when there are chained framework references. If the app references FX1, and FX1 references FX2, then the resolution of FX2 only takes into account settings from `.runtimeconfig.json` in FX1, CLI and env. variable. The settings in the app's `.runtimeconfig.json` have no effect on resolution of FX2.
|
||||||
|
|
||||||
## Multilevel SharedFx Lookup
|
## Multilevel SharedFx Lookup
|
||||||
|
|
|
@ -2,7 +2,7 @@
|
||||||
|
|
||||||
## Purpose
|
## Purpose
|
||||||
|
|
||||||
The CLR possesses a rich built-in marshaling mechanism for interoperability with native code that is handled at runtime. This system was designed to free .NET developers from having to author complex and potentially ABI sensitive [type conversion code][typemarshal_link] from a managed to an unmanaged environment. The built-in system works with both [P/Invoke][pinvoke_link] (i.e. `DllImportAttribute`) and [COM interop](https://docs.microsoft.com/dotnet/standard/native-interop/cominterop). The generated portion is typically called an ["IL Stub"][il_stub_link] since the stub is generated by inserting IL instructions into a stream and then passing that stream to the JIT for compilation.
|
The CLR possesses a rich built-in marshaling mechanism for interoperability with native code that is handled at runtime. This system was designed to free .NET developers from having to author complex and potentially ABI sensitive [type conversion code][typemarshal_link] from a managed to an unmanaged environment. The built-in system works with both [P/Invoke][pinvoke_link] (i.e. `DllImportAttribute`) and [COM interop](https://docs.microsoft.com/dotnet/standard/native-interop/cominterop). The generated portion is typically called an ["IL Stub"][il_stub_link] since the stub is generated by inserting IL instructions into a stream and then passing that stream to the JIT for compilation.
|
||||||
|
|
||||||
A consequence of this approach is that marshaling code is not immediately available post-link for AOT scenarios (e.g. [`crossgen`](../../workflow/building/coreclr/crossgen.md) and [`crossgen2`](crossgen2-compilation-structure-enhancements.md)). The immediate unavailability of this code has been mitigated by a complex mechanism to have marshalling code generated by during AOT compilation. The [IL Linker][ilinker_link] is another tool that struggles with runtime generated code since it is unable to understand all potential used types without seeing what is generated.
|
A consequence of this approach is that marshaling code is not immediately available post-link for AOT scenarios (e.g. [`crossgen`](../../workflow/building/coreclr/crossgen.md) and [`crossgen2`](crossgen2-compilation-structure-enhancements.md)). The immediate unavailability of this code has been mitigated by a complex mechanism to have marshalling code generated by during AOT compilation. The [IL Linker][ilinker_link] is another tool that struggles with runtime generated code since it is unable to understand all potential used types without seeing what is generated.
|
||||||
|
|
||||||
|
|
|
@ -92,4 +92,4 @@ Most of the implementation is relatively straightforward given the design and be
|
||||||
|
|
||||||
1. The current call counter implementation is utterly naive and using the PreStub has a high per-invocation cost relative to other more sophisticated implementation options. We expected it would need to change sooner, but so far despite having some measurable cost it hasn't been reached the top of the priority list for performance gain vs. work necessary. Part of what makes it not as bad as it looks is that there is a bound on the number of times it can be called for any one method and relative to typical 100,000 cycle costs for jitting a method even an expensive call counter doesn't make a huge impact.
|
1. The current call counter implementation is utterly naive and using the PreStub has a high per-invocation cost relative to other more sophisticated implementation options. We expected it would need to change sooner, but so far despite having some measurable cost it hasn't been reached the top of the priority list for performance gain vs. work necessary. Part of what makes it not as bad as it looks is that there is a bound on the number of times it can be called for any one method and relative to typical 100,000 cycle costs for jitting a method even an expensive call counter doesn't make a huge impact.
|
||||||
|
|
||||||
2. Right now background compilation is limited to a single thread taken from the threadpool and used for up to 10ms. If we need more time than that we return the thread and request another. The goal is to be a good citizen in the threadpool's overall workload while still doing enough work in chunks that we get decent cache and thread quantum utilization. It's possible we could do better as the policy here hasn't been profiled much. Thus far we haven't profiled any performance issues that suggested we should be handling this differently.
|
2. Right now background compilation is limited to a single thread taken from the threadpool and used for up to 10ms. If we need more time than that we return the thread and request another. The goal is to be a good citizen in the threadpool's overall workload while still doing enough work in chunks that we get decent cache and thread quantum utilization. It's possible we could do better as the policy here hasn't been profiled much. Thus far we haven't profiled any performance issues that suggested we should be handling this differently.
|
||||||
|
|
|
@ -1,4 +1,4 @@
|
||||||
# `AssemblyLoadContext` unloadability
|
# `AssemblyLoadContext` unloadability
|
||||||
## Goals
|
## Goals
|
||||||
* Provide a building block for unloadable plug-ins
|
* Provide a building block for unloadable plug-ins
|
||||||
* Users can load an assembly and its dependencies into an unloadable `AssemblyLoadContext`.
|
* Users can load an assembly and its dependencies into an unloadable `AssemblyLoadContext`.
|
||||||
|
@ -100,7 +100,7 @@ Unloading is initialized by the user code calling `AssemblyLoadContext.Unload` m
|
||||||
* The `AssemblyLoadContext` fires the `Unloading` event to allow the user code to perform cleanup if required (e.g. stop threads running inside of the context, remove references and destroy handles, etc.)
|
* The `AssemblyLoadContext` fires the `Unloading` event to allow the user code to perform cleanup if required (e.g. stop threads running inside of the context, remove references and destroy handles, etc.)
|
||||||
* The `AssemblyLoadContext.InitiateUnload` method is called. It creates a strong GC handle referring to the `AssemblyLoadContext` to keep it around until the unload is complete. For example, finalizers of types that are loaded into the `AssemblyLoadContext` may need access to the `AssemblyLoadContext`.
|
* The `AssemblyLoadContext.InitiateUnload` method is called. It creates a strong GC handle referring to the `AssemblyLoadContext` to keep it around until the unload is complete. For example, finalizers of types that are loaded into the `AssemblyLoadContext` may need access to the `AssemblyLoadContext`.
|
||||||
* Then it calls `AssemblyNative::PrepareForAssemblyLoadContextRelease` method with that strong handle as an argument, which in turn calls `CLRPrivBinderAssemblyLoadContext::PrepareForLoadContextRelease`
|
* Then it calls `AssemblyNative::PrepareForAssemblyLoadContextRelease` method with that strong handle as an argument, which in turn calls `CLRPrivBinderAssemblyLoadContext::PrepareForLoadContextRelease`
|
||||||
* That method stores the passed in strong GC handle in `CLRPrivBinderAssemblyLoadContext::m_ptrManagedStrongAssemblyLoadContext`.
|
* That method stores the passed in strong GC handle in `CLRPrivBinderAssemblyLoadContext::m_ptrManagedStrongAssemblyLoadContext`.
|
||||||
* Then it decrements refcount of the `AssemblyLoaderAllocator` the `CLRPrivBinderAssemblyLoadContext` points to.
|
* Then it decrements refcount of the `AssemblyLoaderAllocator` the `CLRPrivBinderAssemblyLoadContext` points to.
|
||||||
* Finally, it destroys the strong handle to the managed `LoaderAllocator`. That allows the `LoaderAllocator` to be collected.
|
* Finally, it destroys the strong handle to the managed `LoaderAllocator`. That allows the `LoaderAllocator` to be collected.
|
||||||
### Second phase of unloading
|
### Second phase of unloading
|
||||||
|
|
|
@ -195,13 +195,13 @@ c) In section II.23.2.6 LocalVarSig, replace the diagram with production rules:
|
||||||
```ebnf
|
```ebnf
|
||||||
LocalVarSig ::=
|
LocalVarSig ::=
|
||||||
LOCAL_SIG Count LocalVarType+
|
LOCAL_SIG Count LocalVarType+
|
||||||
|
|
||||||
LocalVarType ::=
|
LocalVarType ::=
|
||||||
Type
|
Type
|
||||||
CustomMod* Constraint BYREF? Type
|
CustomMod* Constraint BYREF? Type
|
||||||
CustomMod* BYREF Type
|
CustomMod* BYREF Type
|
||||||
CustomMod* TYPEDBYREF
|
CustomMod* TYPEDBYREF
|
||||||
|
|
||||||
```
|
```
|
||||||
|
|
||||||
d) In section II.23.2.10 Param, replace the diagram with production rules:
|
d) In section II.23.2.10 Param, replace the diagram with production rules:
|
||||||
|
@ -227,7 +227,7 @@ f) In section II.23.2.12 Type, add a production rule to the definition of `Type`
|
||||||
|
|
||||||
```ebnf
|
```ebnf
|
||||||
Type ::= CustomMod* Type
|
Type ::= CustomMod* Type
|
||||||
|
|
||||||
```
|
```
|
||||||
|
|
||||||
g) In sections II.23.2.12 Type and II.23.2.14 TypeSpec replace production rule
|
g) In sections II.23.2.12 Type and II.23.2.14 TypeSpec replace production rule
|
||||||
|
|
|
@ -19,11 +19,11 @@ PE/COFF Specification defines the structure of Debug Directory in section 5.1.1.
|
||||||
| Offset | Size | Field | Description |
|
| Offset | Size | Field | Description |
|
||||||
|:-------|:-----|:---------------|----------------------------------------------------------------|
|
|:-------|:-----|:---------------|----------------------------------------------------------------|
|
||||||
| 0 | 4 | Signature | 0x52 0x53 0x44 0x53 (ASCII string: "RSDS") |
|
| 0 | 4 | Signature | 0x52 0x53 0x44 0x53 (ASCII string: "RSDS") |
|
||||||
| 4 | 16 | Guid | GUID (Globally Unique Identifier) of the associated PDB.
|
| 4 | 16 | Guid | GUID (Globally Unique Identifier) of the associated PDB.
|
||||||
| 20 | 4 | Age | Iteration of the PDB. The first iteration is 1. The iteration is incremented each time the PDB content is augmented.
|
| 20 | 4 | Age | Iteration of the PDB. The first iteration is 1. The iteration is incremented each time the PDB content is augmented.
|
||||||
| 24 | | Path | UTF-8 NUL-terminated path to the associated .pdb file |
|
| 24 | | Path | UTF-8 NUL-terminated path to the associated .pdb file |
|
||||||
|
|
||||||
Guid and Age are used to match PE/COFF image with the associated PDB.
|
Guid and Age are used to match PE/COFF image with the associated PDB.
|
||||||
|
|
||||||
The associated .pdb file may not exist at the path indicated by Path field. If it doesn't the Path, Guid and Age can be used to find the corresponding PDB file locally or on a symbol server. The exact search algorithm used by tools to locate the PDB depends on the tool and its configuration.
|
The associated .pdb file may not exist at the path indicated by Path field. If it doesn't the Path, Guid and Age can be used to find the corresponding PDB file locally or on a symbol server. The exact search algorithm used by tools to locate the PDB depends on the tool and its configuration.
|
||||||
|
|
||||||
|
@ -37,11 +37,11 @@ If the containing PE/COFF file is deterministic the Guid field above and DateTim
|
||||||
|
|
||||||
The entry doesn't have any data associated with it. All fields of the entry, but Type shall be zero.
|
The entry doesn't have any data associated with it. All fields of the entry, but Type shall be zero.
|
||||||
|
|
||||||
Presence of this entry indicates that the containing PE/COFF file is deterministic.
|
Presence of this entry indicates that the containing PE/COFF file is deterministic.
|
||||||
|
|
||||||
### Embedded Portable PDB Debug Directory Entry (type 17)
|
### Embedded Portable PDB Debug Directory Entry (type 17)
|
||||||
|
|
||||||
Declares that debugging information is embedded in the PE file at location specified by PointerToRawData.
|
Declares that debugging information is embedded in the PE file at location specified by PointerToRawData.
|
||||||
|
|
||||||
*Version Major=any, Minor=0x0100* of the data format:
|
*Version Major=any, Minor=0x0100* of the data format:
|
||||||
|
|
||||||
|
@ -49,7 +49,7 @@ Declares that debugging information is embedded in the PE file at location speci
|
||||||
|:-------|:---------------|:-----------------|-------------------------------------------------------|
|
|:-------|:---------------|:-----------------|-------------------------------------------------------|
|
||||||
| 0 | 4 | Signature | 0x4D 0x50 0x44 0x42 |
|
| 0 | 4 | Signature | 0x4D 0x50 0x44 0x42 |
|
||||||
| 4 | 4 | UncompressedSize | The size of decompressed Portable PDB image |
|
| 4 | 4 | UncompressedSize | The size of decompressed Portable PDB image |
|
||||||
| 8 | SizeOfData - 8 | PortablePdbImage | Portable PDB image compressed using Deflate algorithm |
|
| 8 | SizeOfData - 8 | PortablePdbImage | Portable PDB image compressed using Deflate algorithm |
|
||||||
|
|
||||||
|
|
||||||
If both CodeView and Embedded Portable PDB entries are present then they shall represent the same data.
|
If both CodeView and Embedded Portable PDB entries are present then they shall represent the same data.
|
||||||
|
@ -70,9 +70,9 @@ The value of Stamp field in the entry shall be 0.
|
||||||
|
|
||||||
Stores crypto hash of the content of the symbol file the PE/COFF file was built with.
|
Stores crypto hash of the content of the symbol file the PE/COFF file was built with.
|
||||||
|
|
||||||
The hash can be used to validate that a given PDB file was built with the PE/COFF file and not altered in any way.
|
The hash can be used to validate that a given PDB file was built with the PE/COFF file and not altered in any way.
|
||||||
|
|
||||||
More than one entry can be present, in case multiple PDBs were produced during the build of the PE/COFF file (e.g. private and public symbols).
|
More than one entry can be present, in case multiple PDBs were produced during the build of the PE/COFF file (e.g. private and public symbols).
|
||||||
|
|
||||||
*Version Major=0x0001, Minor=0x0000* of the entry data format is following:
|
*Version Major=0x0001, Minor=0x0000* of the entry data format is following:
|
||||||
|
|
||||||
|
@ -103,7 +103,7 @@ When validating that Portable PDB matches the debug directory record check that
|
||||||
|
|
||||||
If the symbol format is Windows PDB the checksum is calculated by hashing the entire content of the PDB file with the PDB signature comprising of 16B GUID and 4B timestamp zeroed.
|
If the symbol format is Windows PDB the checksum is calculated by hashing the entire content of the PDB file with the PDB signature comprising of 16B GUID and 4B timestamp zeroed.
|
||||||
|
|
||||||
When validating that Windows PDB matches the debug directory record check that the checksums match and that the PDB signature (both GUID and timestamp values) match the data in the corresponding [CodeView record](#WindowsCodeViewEntry).
|
When validating that Windows PDB matches the debug directory record check that the checksums match and that the PDB signature (both GUID and timestamp values) match the data in the corresponding [CodeView record](#WindowsCodeViewEntry).
|
||||||
|
|
||||||
> Note that when the debugger (or other tool) searches for the PDB only GUID and Age fields are used to match the PDB, but the timestamp of the CodeView debug directory entry does not need to match the timestamp stored in the PDB. Therefore, to verify byte-for-byte identity of the PDB, the timestamp field should also be checked.
|
> Note that when the debugger (or other tool) searches for the PDB only GUID and Age fields are used to match the PDB, but the timestamp of the CodeView debug directory entry does not need to match the timestamp stored in the PDB. Therefore, to verify byte-for-byte identity of the PDB, the timestamp field should also be checked.
|
||||||
|
|
||||||
|
|
|
@ -34,7 +34,7 @@ When debugging metadata is generated to a separate data blob "#Pdb" and "#~" str
|
||||||
#### <a name="PdbStream"></a>#Pdb stream
|
#### <a name="PdbStream"></a>#Pdb stream
|
||||||
|
|
||||||
The #Pdb stream has the following structure:
|
The #Pdb stream has the following structure:
|
||||||
|
|
||||||
| Offset | Size | Field | Description |
|
| Offset | Size | Field | Description |
|
||||||
|:-------|:-----|:---------------|----------------------------------------------------------------|
|
|:-------|:-----|:---------------|----------------------------------------------------------------|
|
||||||
| 0 | 20 | PDB id | A byte sequence uniquely representing the debugging metadata blob content. |
|
| 0 | 20 | PDB id | A byte sequence uniquely representing the debugging metadata blob content. |
|
||||||
|
@ -42,10 +42,10 @@ The #Pdb stream has the following structure:
|
||||||
| 24 | 8 | ReferencedTypeSystemTables | Bit vector of referenced type system metadata tables, let n be the number of bits that are 1. |
|
| 24 | 8 | ReferencedTypeSystemTables | Bit vector of referenced type system metadata tables, let n be the number of bits that are 1. |
|
||||||
| 32 | 4*n | TypeSystemTableRows | Array of n 4-byte unsigned integers indicating the number of rows for each referenced type system metadata table. |
|
| 32 | 4*n | TypeSystemTableRows | Array of n 4-byte unsigned integers indicating the number of rows for each referenced type system metadata table. |
|
||||||
|
|
||||||
#### #~ stream
|
#### #~ stream
|
||||||
|
|
||||||
"#~" stream shall only contain debugging information tables defined above.
|
"#~" stream shall only contain debugging information tables defined above.
|
||||||
|
|
||||||
References to heaps (strings, blobs, guids) are references to heaps of the debugging metadata. The sizes of references to type system tables are determined using the algorithm described in ECMA-335-II Chapter 24.2.6, except their respective row counts are found in _TypeSystemTableRows_ field of the #Pdb stream.
|
References to heaps (strings, blobs, guids) are references to heaps of the debugging metadata. The sizes of references to type system tables are determined using the algorithm described in ECMA-335-II Chapter 24.2.6, except their respective row counts are found in _TypeSystemTableRows_ field of the #Pdb stream.
|
||||||
|
|
||||||
### <a name="DocumentTable"></a>Document Table: 0x30
|
### <a name="DocumentTable"></a>Document Table: 0x30
|
||||||
|
@ -58,7 +58,7 @@ The Document table has the following columns:
|
||||||
|
|
||||||
The table is not required to be sorted.
|
The table is not required to be sorted.
|
||||||
|
|
||||||
There shall be no duplicate rows in the _Document_ table, based upon document name.
|
There shall be no duplicate rows in the _Document_ table, based upon document name.
|
||||||
|
|
||||||
_Name_ shall not be nil. It can however encode an empty name string.
|
_Name_ shall not be nil. It can however encode an empty name string.
|
||||||
|
|
||||||
|
@ -170,7 +170,7 @@ _InitialDocument_ is only present if the _Document_ field of the _MethodDebugInf
|
||||||
| _δILOffset_ | 0 | unsigned compressed |
|
| _δILOffset_ | 0 | unsigned compressed |
|
||||||
| _Document_ | Document row id | unsigned compressed |
|
| _Document_ | Document row id | unsigned compressed |
|
||||||
|
|
||||||
Each _SequencePointRecord_ represents a single sequence point. The sequence point inherits the value of _Document_ property from the previous record (_SequencePointRecord_ or _document-record_), from the _Document_ field of the _MethodDebugInformation_ table if it's the first sequence point of a method body that spans a single document, or from _InitialDocument_ if it's the first sequence point of a method body that spans multiple documents. The value of _IL Offset_ is calculated using the value of the previous sequence point (if any) and the value stored in the record.
|
Each _SequencePointRecord_ represents a single sequence point. The sequence point inherits the value of _Document_ property from the previous record (_SequencePointRecord_ or _document-record_), from the _Document_ field of the _MethodDebugInformation_ table if it's the first sequence point of a method body that spans a single document, or from _InitialDocument_ if it's the first sequence point of a method body that spans multiple documents. The value of _IL Offset_ is calculated using the value of the previous sequence point (if any) and the value stored in the record.
|
||||||
|
|
||||||
The values of _Start Line_, _Start Column_, _End Line_ and _End Column_ of a non-hidden sequence point are calculated based upon the values of the previous non-hidden sequence point (if any) and the data stored in the record.
|
The values of _Start Line_, _Start Column_, _End Line_ and _End Column_ of a non-hidden sequence point are calculated based upon the values of the previous non-hidden sequence point (if any) and the data stored in the record.
|
||||||
|
|
||||||
|
@ -253,14 +253,14 @@ There shall be no duplicate rows in the LocalConstant table, based upon owner an
|
||||||
The structure of the blob is
|
The structure of the blob is
|
||||||
|
|
||||||
Blob ::= CustomMod* (PrimitiveConstant | EnumConstant | GeneralConstant)
|
Blob ::= CustomMod* (PrimitiveConstant | EnumConstant | GeneralConstant)
|
||||||
|
|
||||||
PrimitiveConstant ::= PrimitiveTypeCode PrimitiveValue
|
PrimitiveConstant ::= PrimitiveTypeCode PrimitiveValue
|
||||||
PrimitiveTypeCode ::= BOOLEAN | CHAR | I1 | U1 | I2 | U2 | I4 | U4 | I8 | U8 | R4 | R8 | STRING
|
PrimitiveTypeCode ::= BOOLEAN | CHAR | I1 | U1 | I2 | U2 | I4 | U4 | I8 | U8 | R4 | R8 | STRING
|
||||||
|
|
||||||
EnumConstant ::= EnumTypeCode EnumValue EnumType
|
EnumConstant ::= EnumTypeCode EnumValue EnumType
|
||||||
EnumTypeCode ::= BOOLEAN | CHAR | I1 | U1 | I2 | U2 | I4 | U4 | I8 | U8
|
EnumTypeCode ::= BOOLEAN | CHAR | I1 | U1 | I2 | U2 | I4 | U4 | I8 | U8
|
||||||
EnumType ::= TypeDefOrRefOrSpecEncoded
|
EnumType ::= TypeDefOrRefOrSpecEncoded
|
||||||
|
|
||||||
GeneralConstant ::= (CLASS | VALUETYPE) TypeDefOrRefOrSpecEncoded GeneralValue? |
|
GeneralConstant ::= (CLASS | VALUETYPE) TypeDefOrRefOrSpecEncoded GeneralValue? |
|
||||||
OBJECT
|
OBJECT
|
||||||
|
|
||||||
|
@ -289,18 +289,18 @@ The encoding of the _PrimitiveValue_ and _EnumValue_ is determined based upon th
|
||||||
| ```U8``` | uint64 |
|
| ```U8``` | uint64 |
|
||||||
| ```R4``` | float32 |
|
| ```R4``` | float32 |
|
||||||
| ```R8``` | float64 |
|
| ```R8``` | float64 |
|
||||||
| ```STRING``` | A single byte 0xff (represents a null string reference), or a UTF-16 little-endian encoded string (possibly empty). |
|
| ```STRING``` | A single byte 0xff (represents a null string reference), or a UTF-16 little-endian encoded string (possibly empty). |
|
||||||
|
|
||||||
The numeric values of the type codes are defined by ECMA-335 §II.23.1.16.
|
The numeric values of the type codes are defined by ECMA-335 §II.23.1.16.
|
||||||
|
|
||||||
_EnumType_ must be an enum type as defined in ECMA-335 §II.14.3. The value of _EnumTypeCode_ must match the underlying type of the _EnumType_.
|
_EnumType_ must be an enum type as defined in ECMA-335 §II.14.3. The value of _EnumTypeCode_ must match the underlying type of the _EnumType_.
|
||||||
|
|
||||||
The encoding of the _GeneralValue_ is determined based upon the type expressed by _TypeDefOrRefOrSpecEncoded_ specified in _GeneralConstant_. _GeneralValue_ for special types listed in the table below has to be present and is encoded as specified. If the _GeneralValue_ is not present the value of the constant is the default value of the type. If the type is a reference type the value is a null reference, if the type is a pointer type the value is a null pointer, etc.
|
The encoding of the _GeneralValue_ is determined based upon the type expressed by _TypeDefOrRefOrSpecEncoded_ specified in _GeneralConstant_. _GeneralValue_ for special types listed in the table below has to be present and is encoded as specified. If the _GeneralValue_ is not present the value of the constant is the default value of the type. If the type is a reference type the value is a null reference, if the type is a pointer type the value is a null pointer, etc.
|
||||||
|
|
||||||
| Namespace | Name | _GeneralValue_ encoding |
|
| Namespace | Name | _GeneralValue_ encoding |
|
||||||
|:--------------|:---------|:-------------------------|
|
|:--------------|:---------|:-------------------------|
|
||||||
| System | Decimal | sign (highest bit), scale (bits 0..7), low (uint32), mid (uint32), high (uint32) |
|
| System | Decimal | sign (highest bit), scale (bits 0..7), low (uint32), mid (uint32), high (uint32) |
|
||||||
| System | DateTime | int64: ticks |
|
| System | DateTime | int64: ticks |
|
||||||
|
|
||||||
### <a name="ImportScopeTable"></a>ImportScope Table: 0x35
|
### <a name="ImportScopeTable"></a>ImportScope Table: 0x35
|
||||||
The ImportScope table has the following columns:
|
The ImportScope table has the following columns:
|
||||||
|
|
|
@ -12,7 +12,7 @@ dotnet/runtime issues and pull requests are a shared resource. As such, it will
|
||||||
|
|
||||||
Here are a few of the most salient components of working well together, and the FAQ has much more detail.
|
Here are a few of the most salient components of working well together, and the FAQ has much more detail.
|
||||||
## Scenarios where we all have to work together:
|
## Scenarios where we all have to work together:
|
||||||
- All incoming issues and pull requests will be automatically labeled with an `area-*` label. The bot will also assign the `untriaged` label to only issues, once they get created.
|
- All incoming issues and pull requests will be automatically labeled with an `area-*` label. The bot will also assign the `untriaged` label to only issues, once they get created.
|
||||||
- All issues and pull requests should have exactly 1 `area-*` label.
|
- All issues and pull requests should have exactly 1 `area-*` label.
|
||||||
- Issues are considered triaged when the `untriaged` label has been removed.
|
- Issues are considered triaged when the `untriaged` label has been removed.
|
||||||
- When issues have `area-*` labels switched, the `untriaged` label must be added. This prevents issues being lost in a `triaged` state when they have not actually been triaged by the area owner. In the future, a bot may automatically ensure this happens.
|
- When issues have `area-*` labels switched, the `untriaged` label must be added. This prevents issues being lost in a `triaged` state when they have not actually been triaged by the area owner. In the future, a bot may automatically ensure this happens.
|
||||||
|
|
|
@ -73,7 +73,7 @@ To install additional .NET Core runtimes or SDKs:
|
||||||
<add key="dotnet5" value="https://dnceng.pkgs.visualstudio.com/public/_packaging/dotnet5/nuget/v3/index.json" />
|
<add key="dotnet5" value="https://dnceng.pkgs.visualstudio.com/public/_packaging/dotnet5/nuget/v3/index.json" />
|
||||||
<add key="gRPC repository" value="https://grpc.jfrog.io/grpc/api/nuget/v3/grpc-nuget-dev" />
|
<add key="gRPC repository" value="https://grpc.jfrog.io/grpc/api/nuget/v3/grpc-nuget-dev" />
|
||||||
...
|
...
|
||||||
</packageSources>
|
</packageSources>
|
||||||
```
|
```
|
||||||
(Documentation for configuring feeds is [here](https://docs.microsoft.com/en-us/nuget/consume-packages/configuring-nuget-behavior).)
|
(Documentation for configuring feeds is [here](https://docs.microsoft.com/en-us/nuget/consume-packages/configuring-nuget-behavior).)
|
||||||
|
|
||||||
|
|
|
@ -1,4 +1,4 @@
|
||||||
Performance Tracing on Linux
|
Performance Tracing on Linux
|
||||||
============================
|
============================
|
||||||
|
|
||||||
When a performance problem is encountered on Linux, these instructions can be used to gather detailed information about what was happening on the machine at the time of the performance problem.
|
When a performance problem is encountered on Linux, these instructions can be used to gather detailed information about what was happening on the machine at the time of the performance problem.
|
||||||
|
|
|
@ -10,7 +10,7 @@ All .NET Core assemblies are [strong-named](https://docs.microsoft.com/en-us/dot
|
||||||
## FAQ
|
## FAQ
|
||||||
|
|
||||||
### 1. Microsoft strong-names their assemblies, should I?
|
### 1. Microsoft strong-names their assemblies, should I?
|
||||||
For the most part, the majority of applications do not need strong-names. Strong-names are left over from previous eras of .NET where [sandboxing](https://en.wikipedia.org/wiki/Sandbox_(computer_security)) needed to differentiate between code that was trusted, versus code that was untrusted. However in recent years, sandboxing via AppDomains, especially to [isolate ASP.NET web applications](https://support.microsoft.com/en-us/help/2698981/asp-net-partial-trust-does-not-guarantee-application-isolation), is no longer guaranteed and is not recommended.
|
For the most part, the majority of applications do not need strong-names. Strong-names are left over from previous eras of .NET where [sandboxing](https://en.wikipedia.org/wiki/Sandbox_(computer_security)) needed to differentiate between code that was trusted, versus code that was untrusted. However in recent years, sandboxing via AppDomains, especially to [isolate ASP.NET web applications](https://support.microsoft.com/en-us/help/2698981/asp-net-partial-trust-does-not-guarantee-application-isolation), is no longer guaranteed and is not recommended.
|
||||||
|
|
||||||
However, strong-names are still required in applications in some rare situations, most of which are called out on this page: [Strong-Named Assemblies](https://docs.microsoft.com/en-us/dotnet/framework/app-domains/strong-named-assemblies).
|
However, strong-names are still required in applications in some rare situations, most of which are called out on this page: [Strong-Named Assemblies](https://docs.microsoft.com/en-us/dotnet/framework/app-domains/strong-named-assemblies).
|
||||||
|
|
||||||
|
@ -21,6 +21,6 @@ There are three major problems that developers run into after strong naming thei
|
||||||
|
|
||||||
1. _Binding Policy_. When developers talk about strong-names, they are usually conflating it with the strict binding policy of the .NET Framework that kicks in _when_ you strong-name. This binding policy is problematic because it forces, by default, an exact match between reference and version, and requires developers to author complex [binding redirects](https://docs.microsoft.com/en-us/dotnet/framework/configure-apps/file-schema/runtime/bindingredirect-element) when they don't. In recent versions of Visual Studio, however, we've added [Automatic Binding Redirection](https://docs.microsoft.com/en-us/dotnet/framework/configure-apps/how-to-enable-and-disable-automatic-binding-redirection) as an attempt to reduce pain of this policy on developers. On top of this, all newer platforms, including _Silverlight_, _WinRT-based platforms_ (Phone and Store), _.NET Native_ and _ASP.NET 5_ this policy has been loosened, allowing later versions of an assembly to satisfy earlier references, thereby completely removing the need to ever write binding redirects on those platforms.
|
1. _Binding Policy_. When developers talk about strong-names, they are usually conflating it with the strict binding policy of the .NET Framework that kicks in _when_ you strong-name. This binding policy is problematic because it forces, by default, an exact match between reference and version, and requires developers to author complex [binding redirects](https://docs.microsoft.com/en-us/dotnet/framework/configure-apps/file-schema/runtime/bindingredirect-element) when they don't. In recent versions of Visual Studio, however, we've added [Automatic Binding Redirection](https://docs.microsoft.com/en-us/dotnet/framework/configure-apps/how-to-enable-and-disable-automatic-binding-redirection) as an attempt to reduce pain of this policy on developers. On top of this, all newer platforms, including _Silverlight_, _WinRT-based platforms_ (Phone and Store), _.NET Native_ and _ASP.NET 5_ this policy has been loosened, allowing later versions of an assembly to satisfy earlier references, thereby completely removing the need to ever write binding redirects on those platforms.
|
||||||
|
|
||||||
2. _Virality_. Once you've strong-named an assembly, you can only statically reference other strong-named assemblies.
|
2. _Virality_. Once you've strong-named an assembly, you can only statically reference other strong-named assemblies.
|
||||||
|
|
||||||
3. _No drop-in replacement_. This is a problem for open source libraries where the strong-name private key is not checked into the repository. This means that developers are unable to build to their own version of the library and then use it as a drop-in replacement without recompiling _all_ consuming libraries up stack to pick up the new identity. This is extremely problematic for libraries, such as Json.NET, which have large incoming dependencies. Firstly, we would recommend that these open source projects check-in their private key (remember, [strong-names are used for identity, and not for security](https://docs.microsoft.com/en-us/dotnet/framework/app-domains/strong-named-assemblies)). Failing that, however, we've introduced a new concept called [Public Signing](public-signing.md) that enables developers to build drop-in replacements without needing access to the strong-name private key. This is the mechanism that .NET Core libraries use by default.
|
3. _No drop-in replacement_. This is a problem for open source libraries where the strong-name private key is not checked into the repository. This means that developers are unable to build to their own version of the library and then use it as a drop-in replacement without recompiling _all_ consuming libraries up stack to pick up the new identity. This is extremely problematic for libraries, such as Json.NET, which have large incoming dependencies. Firstly, we would recommend that these open source projects check-in their private key (remember, [strong-names are used for identity, and not for security](https://docs.microsoft.com/en-us/dotnet/framework/app-domains/strong-named-assemblies)). Failing that, however, we've introduced a new concept called [Public Signing](public-signing.md) that enables developers to build drop-in replacements without needing access to the strong-name private key. This is the mechanism that .NET Core libraries use by default.
|
||||||
|
|
|
@ -57,4 +57,4 @@ The version we produce by our calculations is mainly used in two places:
|
||||||
- As the [Assembly File Version](https://msdn.microsoft.com/en-us/library/51ket42z(v=vs.110).aspx)
|
- As the [Assembly File Version](https://msdn.microsoft.com/en-us/library/51ket42z(v=vs.110).aspx)
|
||||||
- As the packages version number
|
- As the packages version number
|
||||||
|
|
||||||
To get more information on where are we doing the calculations for the versioning, you can [click here](https://github.com/dotnet/buildtools/blob/master/src/Microsoft.DotNet.Build.Tasks/PackageFiles/versioning.targets) to find the targets file where we create the versioning assets, and [here](https://github.com/dotnet/buildtools/blob/master/src/Microsoft.DotNet.Build.Tasks/GenerateCurrentVersion.cs) to see the code on where we calculate BuildNumberMajor and BuildNumberMinor.
|
To get more information on where are we doing the calculations for the versioning, you can [click here](https://github.com/dotnet/buildtools/blob/master/src/Microsoft.DotNet.Build.Tasks/PackageFiles/versioning.targets) to find the targets file where we create the versioning assets, and [here](https://github.com/dotnet/buildtools/blob/master/src/Microsoft.DotNet.Build.Tasks/GenerateCurrentVersion.cs) to see the code on where we calculate BuildNumberMajor and BuildNumberMinor.
|
||||||
|
|
|
@ -11,4 +11,4 @@ PerfView has significant documentation built-in, which includes:
|
||||||
|
|
||||||
To get started, download PerfView and use the links on the main screen to get help.
|
To get started, download PerfView and use the links on the main screen to get help.
|
||||||
|
|
||||||
If you have specific questions, please post them in an issue here.
|
If you have specific questions, please post them in an issue here.
|
||||||
|
|
|
@ -56,7 +56,7 @@ public async Task Headers_SetAfterRequestSubmitted_ThrowsInvalidOperationExcepti
|
||||||
```
|
```
|
||||||
|
|
||||||
# OuterLoop
|
# OuterLoop
|
||||||
This one is fairly simple but often used incorrectly. When running tests which depend on outside influences like e.g. Hardware (Internet, SerialPort, ...) and you can't mitigate these dependencies, you might consider using the `[OuterLoop]` attribute for your test.
|
This one is fairly simple but often used incorrectly. When running tests which depend on outside influences like e.g. Hardware (Internet, SerialPort, ...) and you can't mitigate these dependencies, you might consider using the `[OuterLoop]` attribute for your test.
|
||||||
With this attribute, tests are executed in a dedicated CI loop and won't break the default CI loops which get created when you submit a PR.
|
With this attribute, tests are executed in a dedicated CI loop and won't break the default CI loops which get created when you submit a PR.
|
||||||
To run OuterLoop tests locally you need to set the msbuild property "OuterLoop" to true: `/p:OuterLoop=true`.
|
To run OuterLoop tests locally you need to set the msbuild property "OuterLoop" to true: `/p:OuterLoop=true`.
|
||||||
To run OuterLoop tests in CI you need to mention dotnet-bot and identify the tests you want to run. See `@dotnet-bot help` for the exact loop names.
|
To run OuterLoop tests in CI you need to mention dotnet-bot and identify the tests you want to run. See `@dotnet-bot help` for the exact loop names.
|
||||||
|
|
|
@ -42,7 +42,7 @@ To build just one part you use the root build script (build.cmd/sh), and you add
|
||||||
|
|
||||||
## Configurations
|
## Configurations
|
||||||
|
|
||||||
You may need to build the tree in a combination of configurations. This section explains why.
|
You may need to build the tree in a combination of configurations. This section explains why.
|
||||||
|
|
||||||
A quick reminder of some concepts -- see the [glossary](../project/glossary.md) for more on these:
|
A quick reminder of some concepts -- see the [glossary](../project/glossary.md) for more on these:
|
||||||
|
|
||||||
|
@ -51,7 +51,7 @@ A quick reminder of some concepts -- see the [glossary](../project/glossary.md)
|
||||||
* **Release Configuration** -- Optimized code. Asserts are disabled. Runs at the best speed, and suitable for performance profiling. You will have limited debugging experience.
|
* **Release Configuration** -- Optimized code. Asserts are disabled. Runs at the best speed, and suitable for performance profiling. You will have limited debugging experience.
|
||||||
|
|
||||||
When we talk about mixing configurations, we're discussing the following sub-components:
|
When we talk about mixing configurations, we're discussing the following sub-components:
|
||||||
|
|
||||||
* **Runtime** is the execution engine for managed code and there are two different implementations available. Both are written in C/C++, therefore, easier to debug when built in a Debug configuration.
|
* **Runtime** is the execution engine for managed code and there are two different implementations available. Both are written in C/C++, therefore, easier to debug when built in a Debug configuration.
|
||||||
* CoreCLR is the comprehensive execution engine which if build in Debug Configuration it executes managed code very slowly. For example, it will take a long time to run the managed code unit tests. The code lives under [src/coreclr](../../src/coreclr).
|
* CoreCLR is the comprehensive execution engine which if build in Debug Configuration it executes managed code very slowly. For example, it will take a long time to run the managed code unit tests. The code lives under [src/coreclr](../../src/coreclr).
|
||||||
* Mono is portable and also slimmer runtime and it's not that sensitive to Debug Configuration for running managed code. You will still need to build it without optimizations to have good runtime debugging experience though. The code lives under [src/mono](../../src/mono).
|
* Mono is portable and also slimmer runtime and it's not that sensitive to Debug Configuration for running managed code. You will still need to build it without optimizations to have good runtime debugging experience though. The code lives under [src/mono](../../src/mono).
|
||||||
|
|
|
@ -185,7 +185,7 @@ You can iterate on `System.Private.CoreLib` by running:
|
||||||
build.cmd clr.corelib+clr.nativecorelib+libs.pretest -rc Release
|
build.cmd clr.corelib+clr.nativecorelib+libs.pretest -rc Release
|
||||||
```
|
```
|
||||||
|
|
||||||
When this `System.Private.CoreLib` will be built in Release mode, then it will be crossgen'd and we will update the testhost to the latest version of corelib.
|
When this `System.Private.CoreLib` will be built in Release mode, then it will be crossgen'd and we will update the testhost to the latest version of corelib.
|
||||||
|
|
||||||
You can use the same workflow for mono runtime by using `mono.corelib+libs.pretest` subsets.
|
You can use the same workflow for mono runtime by using `mono.corelib+libs.pretest` subsets.
|
||||||
|
|
||||||
|
|
|
@ -20,18 +20,18 @@ This is certainly undesirable and it should be avoided if possible.
|
||||||
```
|
```
|
||||||
mkdir ~/dotnet
|
mkdir ~/dotnet
|
||||||
cd ~/dotnet
|
cd ~/dotnet
|
||||||
curl https://dotnetcli.blob.core.windows.net/dotnet/Sdk/master/dotnet-sdk-latest-freebsd-x64.tar.gz | tar xfz -
|
curl https://dotnetcli.blob.core.windows.net/dotnet/Sdk/master/dotnet-sdk-latest-freebsd-x64.tar.gz | tar xfz -
|
||||||
```
|
```
|
||||||
if on 12.x you may also need to set `LD_PRELOAD` to `/usr/lib/libpthread.so` to avoid issue when cli freezes.
|
if on 12.x you may also need to set `LD_PRELOAD` to `/usr/lib/libpthread.so` to avoid issue when cli freezes.
|
||||||
|
|
||||||
|
|
||||||
As of summer 2019 this CLI is no longer good enough to build all repos. If that is your case jump to section [Updating CLI](#updating--bootstrap-cli)
|
As of summer 2019 this CLI is no longer good enough to build all repos. If that is your case jump to section [Updating CLI](#updating--bootstrap-cli)
|
||||||
Binary snapshot can be obtained from https://github.com/wfurt/blob as dotnet-sdk-freebsd-x64-latest.tgz
|
Binary snapshot can be obtained from https://github.com/wfurt/blob as dotnet-sdk-freebsd-x64-latest.tgz
|
||||||
|
|
||||||
## Getting sources
|
## Getting sources
|
||||||
master of source-build pulls in source code of specific snapshot instead of tip of master branches.
|
master of source-build pulls in source code of specific snapshot instead of tip of master branches.
|
||||||
That is generally OK but in case of FreeBSD it may miss some changes crucial for build.
|
That is generally OK but in case of FreeBSD it may miss some changes crucial for build.
|
||||||
(or pending un-submitted change)
|
(or pending un-submitted change)
|
||||||
|
|
||||||
```
|
```
|
||||||
git clone https://github.com/dotnet/source-build
|
git clone https://github.com/dotnet/source-build
|
||||||
|
@ -44,9 +44,9 @@ git submodule update
|
||||||
(cd src/coreclr ; git checkout master)
|
(cd src/coreclr ; git checkout master)
|
||||||
```
|
```
|
||||||
|
|
||||||
port change from
|
port change from
|
||||||
```https://github.com/dotnet/corefx/commit/037859ac403ef17879655bb2f2e821d52e6eb4f3```
|
```https://github.com/dotnet/corefx/commit/037859ac403ef17879655bb2f2e821d52e6eb4f3```
|
||||||
In ideal case we could sync up to **master** but that brings Arcade changes and **breaks** the build.
|
In ideal case we could sync up to **master** but that brings Arcade changes and **breaks** the build.
|
||||||
|
|
||||||
Bootstrap Arcade
|
Bootstrap Arcade
|
||||||
```
|
```
|
||||||
|
@ -86,8 +86,8 @@ index 81b8c7b..bb26868 100644
|
||||||
<BuildArguments>$(BuildArguments) -PortableBuild=$(PortableBuild)</BuildArguments>
|
<BuildArguments>$(BuildArguments) -PortableBuild=$(PortableBuild)</BuildArguments>
|
||||||
```
|
```
|
||||||
|
|
||||||
Depending of the day and moon phase you may need to get some updates as well.
|
Depending of the day and moon phase you may need to get some updates as well.
|
||||||
If build breaks look for pending PRs with FreeBSD tag or label and pull pending changes.
|
If build breaks look for pending PRs with FreeBSD tag or label and pull pending changes.
|
||||||
|
|
||||||
## Building
|
## Building
|
||||||
|
|
||||||
|
@ -106,7 +106,7 @@ export DOTNET_CLI_TELEMETRY_OPTOUT=1
|
||||||
```
|
```
|
||||||
|
|
||||||
In ideal situation this will build whole sdk. Right now it fails somewhere in cli.
|
In ideal situation this will build whole sdk. Right now it fails somewhere in cli.
|
||||||
There is problem with rebuild and build will attempt to patch files again and/or make git updates.
|
There is problem with rebuild and build will attempt to patch files again and/or make git updates.
|
||||||
|
|
||||||
```export SOURCE_BUILD_SKIP_SUBMODULE_CHECK=1```
|
```export SOURCE_BUILD_SKIP_SUBMODULE_CHECK=1```
|
||||||
|
|
||||||
|
@ -114,23 +114,23 @@ To build single repo again one can do:
|
||||||
```./build.sh /p:RootRepo=corefx /p:SkipRepoReferences=true ```
|
```./build.sh /p:RootRepo=corefx /p:SkipRepoReferences=true ```
|
||||||
|
|
||||||
## Resolving issues
|
## Resolving issues
|
||||||
Rebuild or source-build has issues.
|
Rebuild or source-build has issues.
|
||||||
Often running ```clean.sh``` from top helps. Be careful, that may undo any local pending changes.
|
Often running ```clean.sh``` from top helps. Be careful, that may undo any local pending changes.
|
||||||
|
|
||||||
Sometimes it would try to apply patches and it would fail.
|
Sometimes it would try to apply patches and it would fail.
|
||||||
You can pass
|
You can pass
|
||||||
```/p:SkipPatches=true``` to top level build.sh script.
|
```/p:SkipPatches=true``` to top level build.sh script.
|
||||||
|
|
||||||
|
|
||||||
## Running CoreFX tests
|
## Running CoreFX tests
|
||||||
|
|
||||||
Follow steps above to build at least corefx and it's dependencies.
|
Follow steps above to build at least corefx and it's dependencies.
|
||||||
|
|
||||||
TBD
|
TBD
|
||||||
|
|
||||||
## Updating bootstrap CLI.
|
## Updating bootstrap CLI.
|
||||||
|
|
||||||
As build changes, previous versions of CLI may not be good enough any more. Changes in runtime or build dependency on 3.0 JSON are some example of braking changes. Following steps outline steps to update published CLI to what build needs. It will require other system where builds is supported. As close similarity and availability Linux will be used in examples bellow but Windows or MacOS should also yield same result.
|
As build changes, previous versions of CLI may not be good enough any more. Changes in runtime or build dependency on 3.0 JSON are some example of braking changes. Following steps outline steps to update published CLI to what build needs. It will require other system where builds is supported. As close similarity and availability Linux will be used in examples bellow but Windows or MacOS should also yield same result.
|
||||||
|
|
||||||
Often build would ask for slightly different version without actually have real dependency on it (that is part of rolling updates across repos).
|
Often build would ask for slightly different version without actually have real dependency on it (that is part of rolling updates across repos).
|
||||||
One can cheat in this case and simply:
|
One can cheat in this case and simply:
|
||||||
|
@ -138,12 +138,12 @@ One can cheat in this case and simply:
|
||||||
ln -s ~/dotnet/sdk/old_version ~/dotnet/sdk/new_version
|
ln -s ~/dotnet/sdk/old_version ~/dotnet/sdk/new_version
|
||||||
```
|
```
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
### Finding versions and commit hashes
|
### Finding versions and commit hashes
|
||||||
First we need to find what version are are trying to recreate. That is 'sdk' section in global.json in each repo. As of preview9ih time, this is set to 3.0.100-preview6-012264 and such version will be used in examples. One advantage of using release branches is that it is in coherent state e.g. all repos should need exactly same version.
|
First we need to find what version are are trying to recreate. That is 'sdk' section in global.json in each repo. As of preview9ih time, this is set to 3.0.100-preview6-012264 and such version will be used in examples. One advantage of using release branches is that it is in coherent state e.g. all repos should need exactly same version.
|
||||||
|
|
||||||
Let's get SDK for supported OS. Sync code base to same version you are trying to build on FreeBSD.
|
Let's get SDK for supported OS. Sync code base to same version you are trying to build on FreeBSD.
|
||||||
```
|
```
|
||||||
./eng/common/build.sh --restore
|
./eng/common/build.sh --restore
|
||||||
Downloading 'https://dot.net/v1/dotnet-install.sh'
|
Downloading 'https://dot.net/v1/dotnet-install.sh'
|
||||||
|
@ -186,7 +186,7 @@ cd core-sdk
|
||||||
git checkout be3f0c1a03f80492d45396c9f5b855b10a8a0b79
|
git checkout be3f0c1a03f80492d45396c9f5b855b10a8a0b79
|
||||||
```
|
```
|
||||||
|
|
||||||
Set variables and assemble SKD without crossgen. (set DropSuffix=true to strip `preview6` from version).
|
Set variables and assemble SKD without crossgen. (set DropSuffix=true to strip `preview6` from version).
|
||||||
```
|
```
|
||||||
export DISABLE_CROSSGEN=true
|
export DISABLE_CROSSGEN=true
|
||||||
export CLIBUILD_SKIP_TESTS=true
|
export CLIBUILD_SKIP_TESTS=true
|
||||||
|
@ -212,7 +212,7 @@ cd coreclr
|
||||||
git checkout 7ec87b0097fdd4400a8632a2eae56612914579ef
|
git checkout 7ec87b0097fdd4400a8632a2eae56612914579ef
|
||||||
```
|
```
|
||||||
|
|
||||||
and build
|
and build
|
||||||
```
|
```
|
||||||
mkdir -p .dotnet
|
mkdir -p .dotnet
|
||||||
curl https://dotnetcli.blob.core.windows.net/dotnet/Sdk/master/dotnet-sdk-latest-freebsd-x64.tar.gz | tar xfz - -C .dotnet
|
curl https://dotnetcli.blob.core.windows.net/dotnet/Sdk/master/dotnet-sdk-latest-freebsd-x64.tar.gz | tar xfz - -C .dotnet
|
||||||
|
@ -247,7 +247,7 @@ git checkout d47cae744ddfb625db8e391cecb261e4c3d7bb1c
|
||||||
```
|
```
|
||||||
|
|
||||||
#### Building core-setup
|
#### Building core-setup
|
||||||
As this has very little platform dependency it is unlikely this needs to be touched.
|
As this has very little platform dependency it is unlikely this needs to be touched.
|
||||||
If we want to do this to pick up fix or for consistency than ... TBD
|
If we want to do this to pick up fix or for consistency than ... TBD
|
||||||
|
|
||||||
```
|
```
|
||||||
|
|
|
@ -14,7 +14,7 @@ export EMSDK_PATH=PATH_TO_SDK_INSTALL/emsdk
|
||||||
|
|
||||||
## Building everything
|
## Building everything
|
||||||
|
|
||||||
At this time no other build configurations are necessary to start building for WebAssembly. The CoreLib for WebAssembly build configurations will be built by default using the WebAssembly configuration shown below.
|
At this time no other build configurations are necessary to start building for WebAssembly. The CoreLib for WebAssembly build configurations will be built by default using the WebAssembly configuration shown below.
|
||||||
|
|
||||||
This document explains how to work on libraries. In order to work on library projects or run library tests it is necessary to have built the runtime to give the libraries something to run on. If you haven't already done so, please read [this document](../../README.md#Configurations) to understand configurations.
|
This document explains how to work on libraries. In order to work on library projects or run library tests it is necessary to have built the runtime to give the libraries something to run on. If you haven't already done so, please read [this document](../../README.md#Configurations) to understand configurations.
|
||||||
|
|
||||||
|
@ -90,7 +90,7 @@ The WebAssembly implementation files are built and made available in the artifac
|
||||||
|
|
||||||
For Linux and MacOSX:
|
For Linux and MacOSX:
|
||||||
```bash
|
```bash
|
||||||
./dotnet.sh build /p:Configuration=Debug|Release /p:TargetArchitecture=wasm /p:TargetOS=Browser src/libraries/src.proj /t:BuildWasmRuntimes
|
./dotnet.sh build /p:Configuration=Debug|Release /p:TargetArchitecture=wasm /p:TargetOS=Browser src/libraries/src.proj /t:BuildWasmRuntimes
|
||||||
```
|
```
|
||||||
|
|
||||||
__Note__: A `Debug` build sets the following environment variables by default. When built from the command line this way the `Configuration` value is case sensitive.
|
__Note__: A `Debug` build sets the following environment variables by default. When built from the command line this way the `Configuration` value is case sensitive.
|
||||||
|
@ -104,7 +104,7 @@ __Note__: A `Debug` build sets the following environment variables by default.
|
||||||
|
|
||||||
#### Example:
|
#### Example:
|
||||||
```
|
```
|
||||||
L: GC_MAJOR_SWEEP: major size: 752K in use: 39K
|
L: GC_MAJOR_SWEEP: major size: 752K in use: 39K
|
||||||
L: GC_MAJOR: (user request) time 3.00ms, stw 3.00ms los size: 0K in use: 0K
|
L: GC_MAJOR: (user request) time 3.00ms, stw 3.00ms los size: 0K in use: 0K
|
||||||
```
|
```
|
||||||
|
|
||||||
|
@ -125,7 +125,7 @@ First update emscripten version in the [webassembly Dockerfile](https://github.c
|
||||||
ENV EMSCRIPTEN_VERSION=1.39.16
|
ENV EMSCRIPTEN_VERSION=1.39.16
|
||||||
```
|
```
|
||||||
|
|
||||||
Submit a PR request with the updated version, wait for all checks to pass and for the request to be merged. A [master.json file](https://github.com/dotnet/versions/blob/master/build-info/docker/image-info.dotnet-dotnet-buildtools-prereqs-docker-master.json#L1126) will be updated with the a new docker image.
|
Submit a PR request with the updated version, wait for all checks to pass and for the request to be merged. A [master.json file](https://github.com/dotnet/versions/blob/master/build-info/docker/image-info.dotnet-dotnet-buildtools-prereqs-docker-master.json#L1126) will be updated with the a new docker image.
|
||||||
|
|
||||||
```
|
```
|
||||||
{
|
{
|
||||||
|
@ -154,4 +154,4 @@ container:
|
||||||
registry: mcr
|
registry: mcr
|
||||||
```
|
```
|
||||||
|
|
||||||
Open a PR request with the new image.
|
Open a PR request with the new image.
|
||||||
|
|
|
@ -13,14 +13,14 @@ Before proceeding further, please click on the link above that matches your mach
|
||||||
To build the Mono runtime, you must first do a complete runtime build (coreclr, libraries, and then mono). At the repo root, simply execute:
|
To build the Mono runtime, you must first do a complete runtime build (coreclr, libraries, and then mono). At the repo root, simply execute:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
./build.sh
|
./build.sh
|
||||||
```
|
```
|
||||||
or on Windows,
|
or on Windows,
|
||||||
```bat
|
```bat
|
||||||
build.cmd
|
build.cmd
|
||||||
```
|
```
|
||||||
Note that the debug configuration is the default option. It generates a 'debug' output and that includes asserts, fewer code optimizations, and is easier for debugging. If you want to make performance measurements, or just want tests to execute more quickly, you can also build the 'release' version which does not have these checks by adding the flag `-configuration release` (or `-c release`).
|
Note that the debug configuration is the default option. It generates a 'debug' output and that includes asserts, fewer code optimizations, and is easier for debugging. If you want to make performance measurements, or just want tests to execute more quickly, you can also build the 'release' version which does not have these checks by adding the flag `-configuration release` (or `-c release`).
|
||||||
|
|
||||||
|
|
||||||
Once you've built the whole runtime and assuming you want to work with just mono, you want to use the following command:
|
Once you've built the whole runtime and assuming you want to work with just mono, you want to use the following command:
|
||||||
|
|
||||||
|
@ -40,7 +40,7 @@ Here are a list of build arguments that may be of use:
|
||||||
|
|
||||||
`/p:MonoEnableLlvm=true /p:MonoLLVMDir=path/to/llvm` - Builds mono w/ LLVM from a custom path
|
`/p:MonoEnableLlvm=true /p:MonoLLVMDir=path/to/llvm` - Builds mono w/ LLVM from a custom path
|
||||||
|
|
||||||
`/p:MonoEnableLlvm=true /p:MonoLLVMDir=path/to/llvm /p:MonoLLVMUseCxx11Abi=true` - Builds mono w/ LLVM
|
`/p:MonoEnableLlvm=true /p:MonoLLVMDir=path/to/llvm /p:MonoLLVMUseCxx11Abi=true` - Builds mono w/ LLVM
|
||||||
from a custom path (and that LLVM was built with C++11 ABI)
|
from a custom path (and that LLVM was built with C++11 ABI)
|
||||||
|
|
||||||
For `build.sh`
|
For `build.sh`
|
||||||
|
@ -68,7 +68,7 @@ The following packages will be created under `artifacts\packages\<configuration>
|
||||||
- `transport.Microsoft.NETCore.Runtime.Mono.<version>-dev.<number>.1.nupkg`
|
- `transport.Microsoft.NETCore.Runtime.Mono.<version>-dev.<number>.1.nupkg`
|
||||||
- `transport.runtime.<OS>.Microsoft.NETCore.Runtime.Mono.<version>-dev.<number>.1.nupkg`
|
- `transport.runtime.<OS>.Microsoft.NETCore.Runtime.Mono.<version>-dev.<number>.1.nupkg`
|
||||||
|
|
||||||
## Important Notes
|
## Important Notes
|
||||||
|
|
||||||
Test binaries are not yet available for mono.
|
Test binaries are not yet available for mono.
|
||||||
|
|
||||||
|
|
|
@ -15,7 +15,7 @@ https://github.com/dotnet/runtime/issues/702 was opened as a way to simply view
|
||||||
|
|
||||||
#### Terminology
|
#### Terminology
|
||||||
|
|
||||||
In order to follow some of the terminology used, there is an expected familiarity of Azure DevOps required. For an in depth guide with Azure DevOps pipeline definitions, please see: https://docs.microsoft.com/en-us/azure/devops/pipelines/yaml-schema?view=azure-devops&tabs=schema.
|
In order to follow some of the terminology used, there is an expected familiarity of Azure DevOps required. For an in depth guide with Azure DevOps pipeline definitions, please see: https://docs.microsoft.com/en-us/azure/devops/pipelines/yaml-schema?view=azure-devops&tabs=schema.
|
||||||
|
|
||||||
The most common terminology and most important are the different containers work happens in.
|
The most common terminology and most important are the different containers work happens in.
|
||||||
|
|
||||||
|
@ -25,7 +25,7 @@ The most common terminology and most important are the different containers work
|
||||||
|
|
||||||
`Job`: Jobs are the smallest unit of work which happen on a unique machine. Jobs by default run in parallel, but may be set to depend on another job. **Every job executes its work on a unique machine**.
|
`Job`: Jobs are the smallest unit of work which happen on a unique machine. Jobs by default run in parallel, but may be set to depend on another job. **Every job executes its work on a unique machine**.
|
||||||
|
|
||||||
`Steps`: Steps are the smallest unit of work, they generally correspond to one command that will happen in a job. Normally a job contains steps, which execute serially.
|
`Steps`: Steps are the smallest unit of work, they generally correspond to one command that will happen in a job. Normally a job contains steps, which execute serially.
|
||||||
|
|
||||||
## CI Overview
|
## CI Overview
|
||||||
|
|
||||||
|
@ -157,7 +157,7 @@ This tracks the overall end to end run time of a pipeline. This graph is useful
|
||||||
Specifically the query is useful for finding out whether a specific Helix Queue (a group of machines) is overloaded or not. This is useful for diagnosing arm hardware issues, because we have a fixed amount that is easily overloaded.
|
Specifically the query is useful for finding out whether a specific Helix Queue (a group of machines) is overloaded or not. This is useful for diagnosing arm hardware issues, because we have a fixed amount that is easily overloaded.
|
||||||
|
|
||||||
```
|
```
|
||||||
WorkItems
|
WorkItems
|
||||||
| where QueueName == "ubuntu.1804.armarch.open"
|
| where QueueName == "ubuntu.1804.armarch.open"
|
||||||
| extend DaysAgo = datetime_diff('Day', now(), Queued)
|
| extend DaysAgo = datetime_diff('Day', now(), Queued)
|
||||||
| extend QueueTimeInSeconds = datetime_diff('Second', Started, Queued)
|
| extend QueueTimeInSeconds = datetime_diff('Second', Started, Queued)
|
||||||
|
|
|
@ -14,13 +14,13 @@ Debugging CoreFX build issues
|
||||||
|
|
||||||
(This documentation is work in progress.)
|
(This documentation is work in progress.)
|
||||||
|
|
||||||
I found the following process to help when investigating some of the build issues caused by incorrect packaging.
|
I found the following process to help when investigating some of the build issues caused by incorrect packaging.
|
||||||
|
|
||||||
To quickly validate if a given project compiles on all supported configurations use `dotnet build /t:RebuildAll`. This applies for running tests as well. For more information, see [Building individual libraries](../../building/libraries/README.md#building-individual-libraries)
|
To quickly validate if a given project compiles on all supported configurations use `dotnet build /t:RebuildAll`. This applies for running tests as well. For more information, see [Building individual libraries](../../building/libraries/README.md#building-individual-libraries)
|
||||||
|
|
||||||
Assuming the current directory is `\src\contractname\`:
|
Assuming the current directory is `\src\contractname\`:
|
||||||
|
|
||||||
1. Build the `\ref` folder: `dotnet build`
|
1. Build the `\ref` folder: `dotnet build`
|
||||||
|
|
||||||
|
|
||||||
Check the logs for output such as:
|
Check the logs for output such as:
|
||||||
|
@ -58,7 +58,7 @@ Use the same technique above to ensure that the binaries include the correct imp
|
||||||
|
|
||||||
Ensure that all Build Pivots are actually being built. This should build all .\ref and .\src variations as well as actually creating the NuGet packages.
|
Ensure that all Build Pivots are actually being built. This should build all .\ref and .\src variations as well as actually creating the NuGet packages.
|
||||||
|
|
||||||
Verify that the contents of the nuspec as well as the actual package is correct. You can find the packages by searching for the following pattern in the msbuild output:
|
Verify that the contents of the nuspec as well as the actual package is correct. You can find the packages by searching for the following pattern in the msbuild output:
|
||||||
|
|
||||||
```
|
```
|
||||||
GetPkgProjPackageDependencies:
|
GetPkgProjPackageDependencies:
|
||||||
|
|
|
@ -25,16 +25,16 @@ As Administrator:
|
||||||
windbg -I
|
windbg -I
|
||||||
```
|
```
|
||||||
|
|
||||||
You may need to do this for both x64 and x86 versions.
|
You may need to do this for both x64 and x86 versions.
|
||||||
Any application that crashes should now automatically start a WinDBG session.
|
Any application that crashes should now automatically start a WinDBG session.
|
||||||
|
|
||||||
## Debugging tests
|
## Debugging tests
|
||||||
To run a single test from command line:
|
To run a single test from command line:
|
||||||
|
|
||||||
* Locate the test binary folder based on the CSPROJ name.
|
* Locate the test binary folder based on the CSPROJ name.
|
||||||
|
|
||||||
For example: `src\System.Net.Sockets\tests\Functional\System.Net.Sockets.Tests.csproj` will build and output binaries at `bin\tests\Windows_NT.AnyCPU.Debug\System.Net.Sockets.Tests\netcoreapp1.0`.
|
For example: `src\System.Net.Sockets\tests\Functional\System.Net.Sockets.Tests.csproj` will build and output binaries at `bin\tests\Windows_NT.AnyCPU.Debug\System.Net.Sockets.Tests\netcoreapp1.0`.
|
||||||
|
|
||||||
* Execute the test
|
* Execute the test
|
||||||
|
|
||||||
Assuming that your repo is at `C:\corefx`:
|
Assuming that your repo is at `C:\corefx`:
|
||||||
|
@ -44,7 +44,7 @@ cd C:\corefx\bin\tests\Windows_NT.AnyCPU.Debug\System.Net.Sockets.Tests\netcorea
|
||||||
C:\corefx\bin\tests\Windows_NT.AnyCPU.Debug\System.Net.Sockets.Tests\netcoreapp1.0\CoreRun.exe xunit.console.dll System.Net.Sockets.Tests.dll -xml testResults.xml -notrait category=nonwindowstests -notrait category=OuterLoop -notrait category=failing
|
C:\corefx\bin\tests\Windows_NT.AnyCPU.Debug\System.Net.Sockets.Tests\netcoreapp1.0\CoreRun.exe xunit.console.dll System.Net.Sockets.Tests.dll -xml testResults.xml -notrait category=nonwindowstests -notrait category=OuterLoop -notrait category=failing
|
||||||
```
|
```
|
||||||
|
|
||||||
* If the test crashes or encounters a `Debugger.Launch()` method call, WinDBG will automatically start and attach to the `CoreRun.exe` process
|
* If the test crashes or encounters a `Debugger.Launch()` method call, WinDBG will automatically start and attach to the `CoreRun.exe` process
|
||||||
|
|
||||||
The following commands will properly configure the debugging extension and fix symbol and source-code references:
|
The following commands will properly configure the debugging extension and fix symbol and source-code references:
|
||||||
|
|
||||||
|
@ -129,7 +129,7 @@ Logs are going to be placed in %SYSTEMDRIVE%\sockets.etl.
|
||||||
|
|
||||||
1. Install [PerfView](https://github.com/Microsoft/perfview/blob/master/documentation/Downloading.md)
|
1. Install [PerfView](https://github.com/Microsoft/perfview/blob/master/documentation/Downloading.md)
|
||||||
2. Run PerfView as Administrator
|
2. Run PerfView as Administrator
|
||||||
3. Press Alt+C to collect events
|
3. Press Alt+C to collect events
|
||||||
4. Disable all other collection parameters
|
4. Disable all other collection parameters
|
||||||
5. Add Additional Providers (see below - Important: keep the "*" wildcard before the names.)
|
5. Add Additional Providers (see below - Important: keep the "*" wildcard before the names.)
|
||||||
|
|
||||||
|
@ -137,7 +137,7 @@ Logs are going to be placed in %SYSTEMDRIVE%\sockets.etl.
|
||||||
|
|
||||||
### Built-in EventSource tracing
|
### Built-in EventSource tracing
|
||||||
|
|
||||||
The following EventSources are built-in to CoreFX. The ones that are not marked as [__TestCode__] can be enabled in production scenarios for log collection.
|
The following EventSources are built-in to CoreFX. The ones that are not marked as [__TestCode__] can be enabled in production scenarios for log collection.
|
||||||
|
|
||||||
#### Global
|
#### Global
|
||||||
* `*System.Diagnostics.Eventing.FrameworkEventSource {8E9F5090-2D75-4d03-8A81-E5AFBF85DAF1}`: Global EventSource used by multiple namespaces.
|
* `*System.Diagnostics.Eventing.FrameworkEventSource {8E9F5090-2D75-4d03-8A81-E5AFBF85DAF1}`: Global EventSource used by multiple namespaces.
|
||||||
|
@ -169,5 +169,5 @@ Helper scripts are available at https://github.com/dotnet/runtime/tree/master/sr
|
||||||
* `*System.Threading.Tasks.Parallel.EventSource`: Provides an event source for tracing TPL information.
|
* `*System.Threading.Tasks.Parallel.EventSource`: Provides an event source for tracing TPL information.
|
||||||
* `*System.Threading.Tasks.Dataflow.DataflowEventSource {16F53577-E41D-43D4-B47E-C17025BF4025}`: Provides an event source for tracing Dataflow information.
|
* `*System.Threading.Tasks.Dataflow.DataflowEventSource {16F53577-E41D-43D4-B47E-C17025BF4025}`: Provides an event source for tracing Dataflow information.
|
||||||
|
|
||||||
## Notes
|
## Notes
|
||||||
* You can find the test invocation command-line by looking at the logs generated after the `dotnet build /t:test` within the test folder.
|
* You can find the test invocation command-line by looking at the logs generated after the `dotnet build /t:test` within the test folder.
|
||||||
|
|
|
@ -84,10 +84,10 @@ profiles:
|
||||||
cores: 12
|
cores: 12
|
||||||
jobs:
|
jobs:
|
||||||
application:
|
application:
|
||||||
endpoints:
|
endpoints:
|
||||||
- http://asp-perf-win:5001
|
- http://asp-perf-win:5001
|
||||||
load:
|
load:
|
||||||
endpoints:
|
endpoints:
|
||||||
- http://asp-perf-load:5001
|
- http://asp-perf-load:5001
|
||||||
|
|
||||||
aspnet-physical-lin:
|
aspnet-physical-lin:
|
||||||
|
@ -96,18 +96,18 @@ profiles:
|
||||||
cores: 12
|
cores: 12
|
||||||
jobs:
|
jobs:
|
||||||
application:
|
application:
|
||||||
endpoints:
|
endpoints:
|
||||||
- http://asp-perf-lin:5001
|
- http://asp-perf-lin:5001
|
||||||
load:
|
load:
|
||||||
endpoints:
|
endpoints:
|
||||||
- http://asp-perf-load:5001
|
- http://asp-perf-load:5001
|
||||||
```
|
```
|
||||||
|
|
||||||
Now, what does this configuration mean and how is it applied? Let's go over
|
Now, what does this configuration mean and how is it applied? Let's go over
|
||||||
the most important fields to understand its main functionality.
|
the most important fields to understand its main functionality.
|
||||||
|
|
||||||
* **Imports**: These are external tools hosted in the Benchmarks repo.
|
* **Imports**: These are external tools hosted in the Benchmarks repo.
|
||||||
In this case, we only need `wrk`, which is a tool that loads and tests
|
In this case, we only need `wrk`, which is a tool that loads and tests
|
||||||
performance in Web applications.
|
performance in Web applications.
|
||||||
|
|
||||||
* **Jobs**: Here go the job descriptions. A job in this context is the set of
|
* **Jobs**: Here go the job descriptions. A job in this context is the set of
|
||||||
|
|
|
@ -88,4 +88,4 @@ Or simply open `logcat` window in Android Studio or Visual Stuido.
|
||||||
### Existing Limitations
|
### Existing Limitations
|
||||||
- `-os Android` is not supported for Windows yet (`WSL` can be used instead)
|
- `-os Android` is not supported for Windows yet (`WSL` can be used instead)
|
||||||
- XHarness.CLI is not able to boot emulators yet (so you need to boot via `AVD Manager` or IDE)
|
- XHarness.CLI is not able to boot emulators yet (so you need to boot via `AVD Manager` or IDE)
|
||||||
- AOT and Interpreter modes are not supported yet
|
- AOT and Interpreter modes are not supported yet
|
||||||
|
|
|
@ -7,14 +7,14 @@ In order to be able to run tests, the following JavaScript engines should be ins
|
||||||
|
|
||||||
They can be installed as a part of [jsvu](https://github.com/GoogleChromeLabs/jsvu).
|
They can be installed as a part of [jsvu](https://github.com/GoogleChromeLabs/jsvu).
|
||||||
|
|
||||||
Please make sure that a JavaScript engine binary is available via command line,
|
Please make sure that a JavaScript engine binary is available via command line,
|
||||||
e.g. for V8:
|
e.g. for V8:
|
||||||
```bash
|
```bash
|
||||||
$ v8
|
$ v8
|
||||||
V8 version 8.5.62
|
V8 version 8.5.62
|
||||||
```
|
```
|
||||||
|
|
||||||
If you use `jsvu`, first add its location to PATH variable
|
If you use `jsvu`, first add its location to PATH variable
|
||||||
e.g. for V8
|
e.g. for V8
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
|
@ -39,7 +39,7 @@ The following shows how to run tests for a specific library
|
||||||
```
|
```
|
||||||
|
|
||||||
### Running tests using different JavaScript engines
|
### Running tests using different JavaScript engines
|
||||||
It's possible to set a JavaScript engine explicitly by adding `/p:JSEngine` property:
|
It's possible to set a JavaScript engine explicitly by adding `/p:JSEngine` property:
|
||||||
|
|
||||||
```
|
```
|
||||||
./dotnet.sh build /t:Test src/libraries/System.AppContext/tests /p:TargetOS=Browser /p:TargetArchitecture=wasm /p:Configuration=Release /p:JSEngine=SpiderMonkey
|
./dotnet.sh build /t:Test src/libraries/System.AppContext/tests /p:TargetOS=Browser /p:TargetArchitecture=wasm /p:Configuration=Release /p:JSEngine=SpiderMonkey
|
||||||
|
|
|
@ -1,10 +1,10 @@
|
||||||
# Running Tests using Mono Runtime
|
# Running Tests using Mono Runtime
|
||||||
|
|
||||||
## Running Runtime Tests
|
## Running Runtime Tests
|
||||||
We currently only support running tests against coreclr. There are additional mono runtime tests in mono/mono, but they
|
We currently only support running tests against coreclr. There are additional mono runtime tests in mono/mono, but they
|
||||||
have not been moved over yet. Simply run the following command:
|
have not been moved over yet. Simply run the following command:
|
||||||
|
|
||||||
```
|
```
|
||||||
dotnet build /t:RunCoreClrTests $(REPO_ROOT)/src/mono/mono.proj
|
dotnet build /t:RunCoreClrTests $(REPO_ROOT)/src/mono/mono.proj
|
||||||
```
|
```
|
||||||
|
|
||||||
|
@ -36,8 +36,8 @@ dotnet build /t:Test /p:RuntimeFlavor=mono
|
||||||
```
|
```
|
||||||
|
|
||||||
# Patching Local dotnet (.dotnet-mono)
|
# Patching Local dotnet (.dotnet-mono)
|
||||||
Another way to test mono out is by 'patching' a local dotnet with our runtime bits. This is a good way to write simple
|
Another way to test mono out is by 'patching' a local dotnet with our runtime bits. This is a good way to write simple
|
||||||
test programs and get a glimpse of how mono will work with the dotnet tooling.
|
test programs and get a glimpse of how mono will work with the dotnet tooling.
|
||||||
|
|
||||||
To generate a local .dotnet-mono, execute this command:
|
To generate a local .dotnet-mono, execute this command:
|
||||||
|
|
||||||
|
@ -51,4 +51,4 @@ You can then, for example, run our HelloWorld sample via:
|
||||||
dotnet build -c Release $(REPO_ROOT)/src/mono/netcore/sample/HelloWorld
|
dotnet build -c Release $(REPO_ROOT)/src/mono/netcore/sample/HelloWorld
|
||||||
MONO_ENV_OPTIONS="" COMPlus_DebugWriteToStdErr=1 \
|
MONO_ENV_OPTIONS="" COMPlus_DebugWriteToStdErr=1 \
|
||||||
$(REPO_ROOT)/.dotnet-mono/dotnet $(REPO_ROOT)/src/mono/netcore/sample/HelloWorld/bin/HelloWorld.dll
|
$(REPO_ROOT)/.dotnet-mono/dotnet $(REPO_ROOT)/src/mono/netcore/sample/HelloWorld/bin/HelloWorld.dll
|
||||||
```
|
```
|
||||||
|
|
|
@ -1,18 +1,18 @@
|
||||||
# Working in dotnet/runtime using Visual Studio
|
# Working in dotnet/runtime using Visual Studio
|
||||||
|
|
||||||
Visual Studio is a great tool to use when working in the dotnet/runtime repo.
|
Visual Studio is a great tool to use when working in the dotnet/runtime repo.
|
||||||
|
|
||||||
Almost all its features should work well, but there are a few special considerations to bear in mind:
|
Almost all its features should work well, but there are a few special considerations to bear in mind:
|
||||||
|
|
||||||
## Test Explorer
|
## Test Explorer
|
||||||
|
|
||||||
You can run tests from the Visual Studio Test Explorer, but there are a few settings you need:
|
You can run tests from the Visual Studio Test Explorer, but there are a few settings you need:
|
||||||
- Enable `Auto detect runsettings Files` (`Test Explorer window -> Settings button -> Options`). Test parameters (like which `dotnet` host to use) are persisted in an auto-generated .runsettings file, and it's important that Visual Studio knows to use it.
|
- Enable `Auto detect runsettings Files` (`Test Explorer window -> Settings button -> Options`). Test parameters (like which `dotnet` host to use) are persisted in an auto-generated .runsettings file, and it's important that Visual Studio knows to use it.
|
||||||
- Set `Processor Architecture for AnyCPU project` to `auto` (`Test Explorer window -> Settings button`).
|
- Set `Processor Architecture for AnyCPU project` to `auto` (`Test Explorer window -> Settings button`).
|
||||||
- Consider whether to disable `Discover tests in real time from C# and Visual Basic .NET source files` (`Test explorer window -> Settings button -> Options`).
|
- Consider whether to disable `Discover tests in real time from C# and Visual Basic .NET source files` (`Test explorer window -> Settings button -> Options`).
|
||||||
- You may want it enabled if you're actively writing new tests and want them to show up in Test Explorer without building first.
|
- You may want it enabled if you're actively writing new tests and want them to show up in Test Explorer without building first.
|
||||||
- You may want it disabled if you're mostly running existing tests, and some of them have conditional attributes. Many of our unit tests have attributes, like `[SkipOnTargetFramework]`, to indicate that they're only valid in certain configurations. Because the real-time discovery feature does not currently recognize these attributes the tests will show up in Test Explorer as well, and fail or possibly hang when you try to run them.
|
- You may want it disabled if you're mostly running existing tests, and some of them have conditional attributes. Many of our unit tests have attributes, like `[SkipOnTargetFramework]`, to indicate that they're only valid in certain configurations. Because the real-time discovery feature does not currently recognize these attributes the tests will show up in Test Explorer as well, and fail or possibly hang when you try to run them.
|
||||||
- Consider whether to enable `Run tests in Parallel` (`Test Explorer window -> Settings button`).
|
- Consider whether to enable `Run tests in Parallel` (`Test Explorer window -> Settings button`).
|
||||||
- You may want it enabled if some of the unit tests you're working with run slowly or there's many of them.
|
- You may want it enabled if some of the unit tests you're working with run slowly or there's many of them.
|
||||||
- You may want it disabled if you want to simplify debugging or viewing debug output.
|
- You may want it disabled if you want to simplify debugging or viewing debug output.
|
||||||
|
|
||||||
|
|
|
@ -5,7 +5,7 @@ Provides reusable docker build infrastructure for the dotnet/runtime repo.
|
||||||
## libraries-sdk Dockerfiles
|
## libraries-sdk Dockerfiles
|
||||||
|
|
||||||
The `libraries-sdk` Dockerfiles can be used to build dotnet sdk docker images
|
The `libraries-sdk` Dockerfiles can be used to build dotnet sdk docker images
|
||||||
that contain the current libraries built from source.
|
that contain the current libraries built from source.
|
||||||
These images can be used to build dockerized dotnet services that target the current libraries.
|
These images can be used to build dockerized dotnet services that target the current libraries.
|
||||||
Currently, debian and windows nanoserver sdk's are supported.
|
Currently, debian and windows nanoserver sdk's are supported.
|
||||||
|
|
||||||
|
|
|
@ -56,4 +56,4 @@ internal.yml -> platform-matrix.yml -------> build-job.yml -------> xplat-job.ym
|
||||||
| (passed-in jobTemplate) | (arcade)
|
| (passed-in jobTemplate) | (arcade)
|
||||||
\------> test-job.yml ------/
|
\------> test-job.yml ------/
|
||||||
\------> format-job.yml ----/
|
\------> format-job.yml ----/
|
||||||
```
|
```
|
||||||
|
|
|
@ -4,4 +4,4 @@ but if that changes we can always create a little nicer tooling for it.
|
||||||
dump\_helper\_resource.bin is used to populate the DUMP\_HELPER resource inside coreclr.dll on Windows. When an application crashes, Windows MinidumpWriteDump is planning to scan
|
dump\_helper\_resource.bin is used to populate the DUMP\_HELPER resource inside coreclr.dll on Windows. When an application crashes, Windows MinidumpWriteDump is planning to scan
|
||||||
modules looking for this resource. The content of the resource is expected to be the name of a dll in the same folder, encoded in UTF8, null terminated, that implements the
|
modules looking for this resource. The content of the resource is expected to be the name of a dll in the same folder, encoded in UTF8, null terminated, that implements the
|
||||||
CLRDataCreateInterface function. For OS security purposes MinidumpWriteDump will do an authenticode signing check before loading the indicated binary, however if your build isn't
|
CLRDataCreateInterface function. For OS security purposes MinidumpWriteDump will do an authenticode signing check before loading the indicated binary, however if your build isn't
|
||||||
signed you can get around this limitation by registering it at HKLM\Software\Microsoft\WindowsNT\CurrentVersion\MiniDumpAuxilliaryDlls.
|
signed you can get around this limitation by registering it at HKLM\Software\Microsoft\WindowsNT\CurrentVersion\MiniDumpAuxilliaryDlls.
|
||||||
|
|
|
@ -9,4 +9,4 @@ for midl.exe which did that conversion so we work around the issue by doing:
|
||||||
- If needed, adjust any of the .cpp files in src\pal\prebuilt\idl\ by hand, using the corresponding artifacts\obj\Windows_NT.x64.Debug\src\inc\idls_out\*_i.c as a guide. Typically
|
- If needed, adjust any of the .cpp files in src\pal\prebuilt\idl\ by hand, using the corresponding artifacts\obj\Windows_NT.x64.Debug\src\inc\idls_out\*_i.c as a guide. Typically
|
||||||
this is just adding MIDL_DEFINE_GUID(...) for any new classes/interfaces that have been added to the idl file.
|
this is just adding MIDL_DEFINE_GUID(...) for any new classes/interfaces that have been added to the idl file.
|
||||||
|
|
||||||
Include these src changes with the remainder of your work when you submit a PR.
|
Include these src changes with the remainder of your work when you submit a PR.
|
||||||
|
|
|
@ -1 +1 @@
|
||||||
README
|
README
|
||||||
|
|
|
@ -1,3 +1,3 @@
|
||||||
# ILVerification
|
# ILVerification
|
||||||
|
|
||||||
The ILVerification library is part of the ILVerify project. See details under [src/coreclr/src/tools/ILVerify](../ILVerify).
|
The ILVerification library is part of the ILVerify project. See details under [src/coreclr/src/tools/ILVerify](../ILVerify).
|
||||||
|
|
|
@ -32,7 +32,7 @@ Note, this tool requires MethodDetails events which are produced by the .NET 5.0
|
||||||
```
|
```
|
||||||
"dotnet trace collect -p 73060 --providers Microsoft-Windows-DotNETRuntime:0x6000080018:5"
|
"dotnet trace collect -p 73060 --providers Microsoft-Windows-DotNETRuntime:0x6000080018:5"
|
||||||
```
|
```
|
||||||
|
|
||||||
- Capture events from process 73060 where we capture only JIT events using EventPipe tracing
|
- Capture events from process 73060 where we capture only JIT events using EventPipe tracing
|
||||||
```
|
```
|
||||||
"dotnet trace collect -p 73060 --providers Microsoft-Windows-DotNETRuntime:0x4000080018:5"
|
"dotnet trace collect -p 73060 --providers Microsoft-Windows-DotNETRuntime:0x4000080018:5"
|
||||||
|
|
|
@ -1,7 +1,7 @@
|
||||||
# Experiments towards a Profile Data pipeline for .NET
|
# Experiments towards a Profile Data pipeline for .NET
|
||||||
-----
|
-----
|
||||||
The .NET Runtime has a long history of providing instrumentation based profile guided optimization
|
The .NET Runtime has a long history of providing instrumentation based profile guided optimization
|
||||||
for use internally at Microsoft, and for scenarios involving extremely high value customers. To
|
for use internally at Microsoft, and for scenarios involving extremely high value customers. To
|
||||||
this end the team built the IBC (instrumented block count) infrastructure into the runtime/ngen,
|
this end the team built the IBC (instrumented block count) infrastructure into the runtime/ngen,
|
||||||
and IBCMerge as a tool for manipulating .ibc files. Over the last few years, the structure of these
|
and IBCMerge as a tool for manipulating .ibc files. Over the last few years, the structure of these
|
||||||
technologies and tools has shown that they are not ideal for customer use or even internal use, and
|
technologies and tools has shown that they are not ideal for customer use or even internal use, and
|
||||||
|
@ -33,19 +33,19 @@ Profile guided optimization in .NET is used to provide benefits for 3 major conc
|
||||||
Startup time for an application is primarily improved by avoiding the use of the JIT by ahead of time
|
Startup time for an application is primarily improved by avoiding the use of the JIT by ahead of time
|
||||||
compiling methods in the application. In addition a profile can allow determination of which methods
|
compiling methods in the application. In addition a profile can allow determination of which methods
|
||||||
are hot vs cold, and group methods commonly used together with others. This has been the primary use
|
are hot vs cold, and group methods commonly used together with others. This has been the primary use
|
||||||
of pgo in .NET historically.
|
of pgo in .NET historically.
|
||||||
|
|
||||||
Pgo is used to address size on disk concerns of R2R binaries where the default R2R strategy is too
|
Pgo is used to address size on disk concerns of R2R binaries where the default R2R strategy is too
|
||||||
aggressive and produces binaries that are excessively large. The idea in that case is to only generate
|
aggressive and produces binaries that are excessively large. The idea in that case is to only generate
|
||||||
the functions specifically referenced in some profile instead of every method the heuristic indicates
|
the functions specifically referenced in some profile instead of every method the heuristic indicates
|
||||||
may be interesting.
|
may be interesting.
|
||||||
|
|
||||||
Application throughput performance has historically been the primary use of pgo data for C++ compilers.
|
Application throughput performance has historically been the primary use of pgo data for C++ compilers.
|
||||||
.NET has history with the use of instrumented per block counts, but this data is not generally processed
|
.NET has history with the use of instrumented per block counts, but this data is not generally processed
|
||||||
in an effective manner by the JIT. This proposal aims to revitalize efforts to make good use of profile
|
in an effective manner by the JIT. This proposal aims to revitalize efforts to make good use of profile
|
||||||
guided data to improve code quality. Over time, it is expected that not only will profile data be used at
|
guided data to improve code quality. Over time, it is expected that not only will profile data be used at
|
||||||
build time, but that it will also be used to do runtime profile instrumentation.
|
build time, but that it will also be used to do runtime profile instrumentation.
|
||||||
|
|
||||||
# Proposal Contents
|
# Proposal Contents
|
||||||
Profile guided optimization is a combination of effort across a swath of components.
|
Profile guided optimization is a combination of effort across a swath of components.
|
||||||
|
|
||||||
|
@ -59,7 +59,7 @@ And there are a series of components that need to be modified
|
||||||
2. Instrumenting jit (clrjit)
|
2. Instrumenting jit (clrjit)
|
||||||
3. Trace processing tool (dotnet-pgo)
|
3. Trace processing tool (dotnet-pgo)
|
||||||
4. AOT compilation tool (crossgen2)
|
4. AOT compilation tool (crossgen2)
|
||||||
6. Consuming runtime (coreclr)
|
6. Consuming runtime (coreclr)
|
||||||
7. Diagnostic tools (r2rdump, dotnet-pgo)
|
7. Diagnostic tools (r2rdump, dotnet-pgo)
|
||||||
|
|
||||||
## Conceptual model of `InstrumentationData`
|
## Conceptual model of `InstrumentationData`
|
||||||
|
@ -68,7 +68,7 @@ statically, and instead is determined through instrumentation of the code. The f
|
||||||
is expected to be defined by the JIT team, and be specific to the probes inserted, and may very well
|
is expected to be defined by the JIT team, and be specific to the probes inserted, and may very well
|
||||||
change over time. It is composed of two sections
|
change over time. It is composed of two sections
|
||||||
|
|
||||||
1. The descriptor used to describe the probes, this is fixed at JIT time, and describes the meaning of the data.
|
1. The descriptor used to describe the probes, this is fixed at JIT time, and describes the meaning of the data.
|
||||||
2. The data gathered as counts, and values that will be used to perform further optimization.
|
2. The data gathered as counts, and values that will be used to perform further optimization.
|
||||||
|
|
||||||
Both of these data blocks are able to contain type and method data, where the concept is that it is
|
Both of these data blocks are able to contain type and method data, where the concept is that it is
|
||||||
|
@ -78,7 +78,7 @@ but there are also plausible cases for gathering each kind of data in both secti
|
||||||
be made general to support both. Instrumentation Data shall have a version number independent of the
|
be made general to support both. Instrumentation Data shall have a version number independent of the
|
||||||
general R2R versioning scheme. The intention is for this form of `InstrumentationData` to become
|
general R2R versioning scheme. The intention is for this form of `InstrumentationData` to become
|
||||||
useable for both out of line instrumentation as described in this document, as well as only tiered
|
useable for both out of line instrumentation as described in this document, as well as only tiered
|
||||||
compilation rejit scenarios with in process profiling.
|
compilation rejit scenarios with in process profiling.
|
||||||
|
|
||||||
## Trace data format
|
## Trace data format
|
||||||
Runtime instrumentation will be accomplished through 4 events, 2 of which are already existing
|
Runtime instrumentation will be accomplished through 4 events, 2 of which are already existing
|
||||||
|
@ -149,7 +149,7 @@ Profile data shall be encoded into the R2R FileFormat in a new section named `RE
|
||||||
This section shall hold a version number, and a single `NativeHashtable` that contains a mapping from type/method
|
This section shall hold a version number, and a single `NativeHashtable` that contains a mapping from type/method
|
||||||
to the pair of Desc and Data. TODO define how Desc and Data are encoded. The intention is to store exactly the
|
to the pair of Desc and Data. TODO define how Desc and Data are encoded. The intention is to store exactly the
|
||||||
same data as is stored in the PGO data file, except that the instrumentation data version must be the same for
|
same data as is stored in the PGO data file, except that the instrumentation data version must be the same for
|
||||||
all data chunks.
|
all data chunks.
|
||||||
|
|
||||||
## Instrumenting Runtime
|
## Instrumenting Runtime
|
||||||
The runtime shall be responsible for choosing when to execute instrumentation, allocating the tracing buffers
|
The runtime shall be responsible for choosing when to execute instrumentation, allocating the tracing buffers
|
||||||
|
@ -197,7 +197,7 @@ data that may be embedded into the R2R file format for possible consumption by t
|
||||||
## Trace processing tool
|
## Trace processing tool
|
||||||
The trace processing tool is responsible for reading the trace files as produced by perfview/dotnet trace, and
|
The trace processing tool is responsible for reading the trace files as produced by perfview/dotnet trace, and
|
||||||
producing .MIBC files. The process should be a straightforward format translation for instrumentation data. The
|
producing .MIBC files. The process should be a straightforward format translation for instrumentation data. The
|
||||||
`FunctionTouchOrder` and existence of the method shall be based on the `JitStarted` and `R2EEntryPoint` events.
|
`FunctionTouchOrder` and existence of the method shall be based on the `JitStarted` and `R2EEntryPoint` events.
|
||||||
|
|
||||||
## AOT Compilation tool
|
## AOT Compilation tool
|
||||||
AOT compilation shall use the profile guided data in several ways.
|
AOT compilation shall use the profile guided data in several ways.
|
||||||
|
@ -210,7 +210,7 @@ data for the method being compiled, and for both the uninstantiated method and i
|
||||||
as are present. The jit is responsible for merging these multiple data sources.
|
as are present. The jit is responsible for merging these multiple data sources.
|
||||||
|
|
||||||
In addition the JIT may optionally choose to generate a profile guided data block for association with the precompiled
|
In addition the JIT may optionally choose to generate a profile guided data block for association with the precompiled
|
||||||
code for use in re-jit scenarios, and information about related method code layout for the code, and optionally a
|
code for use in re-jit scenarios, and information about related method code layout for the code, and optionally a
|
||||||
portion of the function body which is to be placed into a cold code section. The intention here it to allow some
|
portion of the function body which is to be placed into a cold code section. The intention here it to allow some
|
||||||
algorithm such as Pettis-Hansen or a more modern variant (eg https://research.fb.com/wp-content/uploads/2017/01/cgo2017-hfsort-final1.pdf)
|
algorithm such as Pettis-Hansen or a more modern variant (eg https://research.fb.com/wp-content/uploads/2017/01/cgo2017-hfsort-final1.pdf)
|
||||||
to be used to optimize code layout.
|
to be used to optimize code layout.
|
||||||
|
@ -219,7 +219,7 @@ to be used to optimize code layout.
|
||||||
If present in an R2R file, when a method is rejitted, the runtime shall provide a means for the jit to see instrumentation
|
If present in an R2R file, when a method is rejitted, the runtime shall provide a means for the jit to see instrumentation
|
||||||
data from either previous compiles in process, and/or from the R2R file. This shall provide a means for the JIT to choose
|
data from either previous compiles in process, and/or from the R2R file. This shall provide a means for the JIT to choose
|
||||||
whether or not the method should be recompiled, or possibly to inform it about optimization opportunities that are
|
whether or not the method should be recompiled, or possibly to inform it about optimization opportunities that are
|
||||||
too expensive to compute at jit time, but could be computed by the AOT compiler, or other such ideas.
|
too expensive to compute at jit time, but could be computed by the AOT compiler, or other such ideas.
|
||||||
|
|
||||||
As a means of doing this, options such as the following will be given to the jit to provide custom behavior.
|
As a means of doing this, options such as the following will be given to the jit to provide custom behavior.
|
||||||
1. Ignore the profile data and rejit.
|
1. Ignore the profile data and rejit.
|
||||||
|
@ -235,4 +235,4 @@ would be to use this as a means for adaptive or speculative optimization.
|
||||||
The tools r2rdump and dotnet-pgo shall provide a means for dumping their inputs. For most forms of data this is
|
The tools r2rdump and dotnet-pgo shall provide a means for dumping their inputs. For most forms of data this is
|
||||||
fairly straightforward, but for `InstrumentationData`, there shall be a common dump tool written in managed code
|
fairly straightforward, but for `InstrumentationData`, there shall be a common dump tool written in managed code
|
||||||
that can provide a human readable dump of the data. r2rdump, dotnet-pgo, and possibly sos will all be able to share
|
that can provide a human readable dump of the data. r2rdump, dotnet-pgo, and possibly sos will all be able to share
|
||||||
this codebase for examination of the data structures in r2r files, traces, and runtime environments respectively.
|
this codebase for examination of the data structures in r2r files, traces, and runtime environments respectively.
|
||||||
|
|
|
@ -11,9 +11,9 @@ HostModel is a library used by the [SDK](https://github.com/dotnet/sdk) to perfo
|
||||||
The HostModel library is in the Runtime repo because:
|
The HostModel library is in the Runtime repo because:
|
||||||
|
|
||||||
* The implementations of the host and HostModel are closely related, which facilitates easy development, update, and testing.
|
* The implementations of the host and HostModel are closely related, which facilitates easy development, update, and testing.
|
||||||
* Separating the HostModel implementation from SDK repo repo aligns with code ownership, and facilitates maintenance.
|
* Separating the HostModel implementation from SDK repo repo aligns with code ownership, and facilitates maintenance.
|
||||||
|
|
||||||
The build targets/tasks that use the HostModel library are in the SDK repo because:
|
The build targets/tasks that use the HostModel library are in the SDK repo because:
|
||||||
|
|
||||||
* This facilitates the MSBuild tasks to be multi-targeted.
|
* This facilitates the MSBuild tasks to be multi-targeted.
|
||||||
* It helps generate localized error messages, since SDK repo has the localization infrastructure.
|
* It helps generate localized error messages, since SDK repo has the localization infrastructure.
|
||||||
|
|
|
@ -2,33 +2,33 @@ This project has the purpose to automate verification test for .NET Runtime and
|
||||||
|
|
||||||
To have this test running in your local machine do the following steps:
|
To have this test running in your local machine do the following steps:
|
||||||
1. Download VerificationTestOnDocker.sh, RuntimeInstallation.sh, SdkInstallation.sh, images.txt in the same folder
|
1. Download VerificationTestOnDocker.sh, RuntimeInstallation.sh, SdkInstallation.sh, images.txt in the same folder
|
||||||
2. Update images.txt with images name you want to run the installation test
|
2. Update images.txt with images name you want to run the installation test
|
||||||
3. Run $ ./VerificationTestOnDocker.sh \<package> \<version> \<command>
|
3. Run $ ./VerificationTestOnDocker.sh \<package> \<version> \<command>
|
||||||
|
|
||||||
The options are:
|
The options are:
|
||||||
|
|
||||||
* \<package>
|
* \<package>
|
||||||
* runtime - verification test for .NET Runtime Linux packages
|
* runtime - verification test for .NET Runtime Linux packages
|
||||||
* sdk - verification test for .NET SDK Linux packages
|
* sdk - verification test for .NET SDK Linux packages
|
||||||
* \<version>
|
* \<version>
|
||||||
* latest - install the latest available .NET package from our master repository
|
* latest - install the latest available .NET package from our master repository
|
||||||
* \<number> - install the package corresponding to this version number
|
* \<number> - install the package corresponding to this version number
|
||||||
* \<command>
|
* \<command>
|
||||||
* install - verification test for install
|
* install - verification test for install
|
||||||
* install uninstall - verification test for install and uninstall
|
* install uninstall - verification test for install and uninstall
|
||||||
|
|
||||||
|
|
||||||
The script VerificationTestOnDocker.sh is responsible for read a file (images.txt) containing docker images and run a docker container for each image specified in that file. Inside each container it will be executed the script to install .NET Runtime (RuntimeInstallation.sh) or .NET SDK (SdkInstallation.sh).
|
The script VerificationTestOnDocker.sh is responsible for read a file (images.txt) containing docker images and run a docker container for each image specified in that file. Inside each container it will be executed the script to install .NET Runtime (RuntimeInstallation.sh) or .NET SDK (SdkInstallation.sh).
|
||||||
|
|
||||||
Both scripts RuntimeInstallation.sh and SdkInstallation.sh automatically identify what distro and version is running in the current machine and can install and uninstall the latest version of .NET Runtime/Sdk packages corresponding to that distro & version. The installation's stdout for all containers is redirected to a single file (logfile.txt). In the end of this file (logfile.txt) it's also displayed the results of the test, printing for each distro and version the result 'failed' or 'passed'.
|
Both scripts RuntimeInstallation.sh and SdkInstallation.sh automatically identify what distro and version is running in the current machine and can install and uninstall the latest version of .NET Runtime/Sdk packages corresponding to that distro & version. The installation's stdout for all containers is redirected to a single file (logfile.txt). In the end of this file (logfile.txt) it's also displayed the results of the test, printing for each distro and version the result 'failed' or 'passed'.
|
||||||
|
|
||||||
.NET packages are downloaded from the blob https://dotnetcli.blob.core.windows.net/dotnet
|
.NET packages are downloaded from the blob https://dotnetcli.blob.core.windows.net/dotnet
|
||||||
|
|
||||||
This project takes in account:
|
This project takes in account:
|
||||||
-> dotnet-sdk depends on dotnet-runtime and aspnet-runtime
|
-> dotnet-sdk depends on dotnet-runtime and aspnet-runtime
|
||||||
-> aspnet-runtime depends on dotnet-runtime (can be different to what dotnet-sdk depends on)
|
-> aspnet-runtime depends on dotnet-runtime (can be different to what dotnet-sdk depends on)
|
||||||
-> dotnet-runtime-deps depends on system packages
|
-> dotnet-runtime-deps depends on system packages
|
||||||
-> .NET runtime carries: dotnet-runtime-deps, dotnet-host, dotnet-hostfxr and dotnet-runtime.
|
-> .NET runtime carries: dotnet-runtime-deps, dotnet-host, dotnet-hostfxr and dotnet-runtime.
|
||||||
|
|
||||||
Changes on how dotnet runtime packages are structured or modification on the packages dependencies may affect the verification test result.
|
Changes on how dotnet runtime packages are structured or modification on the packages dependencies may affect the verification test result.
|
||||||
|
|
||||||
|
@ -37,13 +37,13 @@ This verification test depends on docker images and the test result can be a fal
|
||||||
|
|
||||||
The script allows automated test only for the following distro & version:
|
The script allows automated test only for the following distro & version:
|
||||||
|
|
||||||
| Distro | Version |
|
| Distro | Version |
|
||||||
|--------|---------|
|
|--------|---------|
|
||||||
| Ubuntu | 14.04, 16.04, 18.04 |
|
| Ubuntu | 14.04, 16.04, 18.04 |
|
||||||
| Debian | 8, 9 |
|
| Debian | 8, 9 |
|
||||||
| Centos | 7 |
|
| Centos | 7 |
|
||||||
| Fedora | 27 |
|
| Fedora | 27 |
|
||||||
| OpenSUSE | 42 |
|
| OpenSUSE | 42 |
|
||||||
| Oracle Linux | 7 |
|
| Oracle Linux | 7 |
|
||||||
| RHEL | 7 |
|
| RHEL | 7 |
|
||||||
| SLES | 12 |
|
| SLES | 12 |
|
||||||
|
|
|
@ -1,4 +1,4 @@
|
||||||
Contracts such as NameResolution and Sockets require internal access to Primitive types. Binary copies of these types have been made within the System.Net.Internals namespace using #ifdef pragmas (source code is reused).
|
Contracts such as NameResolution and Sockets require internal access to Primitive types. Binary copies of these types have been made within the System.Net.Internals namespace using #ifdef pragmas (source code is reused).
|
||||||
|
|
||||||
An adaptation layer between .Internals and public types exists within the Extensions classes.
|
An adaptation layer between .Internals and public types exists within the Extensions classes.
|
||||||
|
|
||||||
|
|
|
@ -1,7 +1,7 @@
|
||||||
# Enterprise Scenario Testing
|
# Enterprise Scenario Testing
|
||||||
|
|
||||||
## What Are Enterprise Scenarios?
|
## What Are Enterprise Scenarios?
|
||||||
There are many definitions for enterprise scenarios. But generally in terms of how .NET Core networking APIs are used, enterprise scenarios are those networking scenarios that are fundamentally used by businesses (a.k.a enterprises) compared with consumers. As such, they use networking components, protocols, and security authentication mechanisms that are not used by most consumers using their home networking and Internet connections.
|
There are many definitions for enterprise scenarios. But generally in terms of how .NET Core networking APIs are used, enterprise scenarios are those networking scenarios that are fundamentally used by businesses (a.k.a enterprises) compared with consumers. As such, they use networking components, protocols, and security authentication mechanisms that are not used by most consumers using their home networking and Internet connections.
|
||||||
|
|
||||||
## Networking Components of Enterprise Scenarios
|
## Networking Components of Enterprise Scenarios
|
||||||
Enterprise scenarios typically see the following kinds of components/protocols/security:
|
Enterprise scenarios typically see the following kinds of components/protocols/security:
|
||||||
|
|
|
@ -10,7 +10,7 @@ Contains source files for the networking test servers in Azure or a private IIS
|
||||||
|
|
||||||
Note: the `config.ps1` file has been added to .gitignore to prevent it being updated in the master branch.
|
Note: the `config.ps1` file has been added to .gitignore to prevent it being updated in the master branch.
|
||||||
|
|
||||||
### Build the server applications
|
### Build the server applications
|
||||||
|
|
||||||
Prepare the $DOTNET_TEST_NET_CLIENT_Machine as any Dev station following the instructions at https://github.com/dotnet/runtime/blob/master/docs/workflow/requirements/windows-requirements.md. Ensure that you can build and test CoreFX on this machine.
|
Prepare the $DOTNET_TEST_NET_CLIENT_Machine as any Dev station following the instructions at https://github.com/dotnet/runtime/blob/master/docs/workflow/requirements/windows-requirements.md. Ensure that you can build and test CoreFX on this machine.
|
||||||
In addition, you will also need to install the _Azure development_ workload for Visual Studio 2017.
|
In addition, you will also need to install the _Azure development_ workload for Visual Studio 2017.
|
||||||
|
@ -30,7 +30,7 @@ You should now find a folder named `IISApplications` within the Deployment folde
|
||||||
Skip this step if previously completed and all machines are already part of a domain to which you have Administrator rights.
|
Skip this step if previously completed and all machines are already part of a domain to which you have Administrator rights.
|
||||||
This will join all machines to a test Active Directory and enable Windows Remoting.
|
This will join all machines to a test Active Directory and enable Windows Remoting.
|
||||||
|
|
||||||
1. Copy the Deployment folder to each of the machines.
|
1. Copy the Deployment folder to each of the machines.
|
||||||
2. Run the .\setup.ps1 script on the machine designated to become the Domain Controller. Once complete, the machine will reboot.
|
2. Run the .\setup.ps1 script on the machine designated to become the Domain Controller. Once complete, the machine will reboot.
|
||||||
3. Run the .\setup.ps1 script on all other domain joined machines. Once complete, the machines will reboot.
|
3. Run the .\setup.ps1 script on all other domain joined machines. Once complete, the machines will reboot.
|
||||||
|
|
||||||
|
|
|
@ -5,4 +5,4 @@ The library is effectively archived.
|
||||||
|
|
||||||
The library and supporting language features are mature and no longer evolving, and the risk of code change likely exceeds the benefit.
|
The library and supporting language features are mature and no longer evolving, and the risk of code change likely exceeds the benefit.
|
||||||
We will consider changes that address significant bugs or regressions, or changes that are necessary to continue shipping the binaries.
|
We will consider changes that address significant bugs or regressions, or changes that are necessary to continue shipping the binaries.
|
||||||
Other changes will be rejected.
|
Other changes will be rejected.
|
||||||
|
|
|
@ -12,4 +12,4 @@ Contains common DI abstractions that ASP.NET Core and Entity Framework Core use.
|
||||||
* [**LightInject**](https://github.com/seesharper/LightInject.Microsoft.DependencyInjection)
|
* [**LightInject**](https://github.com/seesharper/LightInject.Microsoft.DependencyInjection)
|
||||||
* [**StructureMap**](https://github.com/structuremap/StructureMap.Microsoft.DependencyInjection)
|
* [**StructureMap**](https://github.com/structuremap/StructureMap.Microsoft.DependencyInjection)
|
||||||
* [**Stashbox**](https://github.com/z4kn4fein/stashbox-extensions-dependencyinjection)
|
* [**Stashbox**](https://github.com/z4kn4fein/stashbox-extensions-dependencyinjection)
|
||||||
* [**Unity**](https://www.nuget.org/packages/Unity.Microsoft.DependencyInjection/)
|
* [**Unity**](https://www.nuget.org/packages/Unity.Microsoft.DependencyInjection/)
|
||||||
|
|
|
@ -5,4 +5,4 @@ The library is effectively archived.
|
||||||
|
|
||||||
The library and supporting language features are mature and no longer evolving, and the risk of code change likely exceeds the benefit.
|
The library and supporting language features are mature and no longer evolving, and the risk of code change likely exceeds the benefit.
|
||||||
We will consider changes that address significant bugs or regressions, or changes that are necessary to continue shipping the binaries.
|
We will consider changes that address significant bugs or regressions, or changes that are necessary to continue shipping the binaries.
|
||||||
Other changes will be rejected.
|
Other changes will be rejected.
|
||||||
|
|
|
@ -1,6 +1,6 @@
|
||||||
# MEF1 vs. MEF2
|
# MEF1 vs. MEF2
|
||||||
|
|
||||||
* MEF1 is used to compose different pieces of code together for dynamic extensions. It has attributes like `Export` and `Import`, hooking up import classes to exports. MEF has a catalog that keeps track of exports and looks at assembly, directory, type catalogs to discover exports. Container instantiates objects and satisfies imports. We do a series of object instantiations, catalog lookups, and finding exports to get the model that is handed out from a container.
|
* MEF1 is used to compose different pieces of code together for dynamic extensions. It has attributes like `Export` and `Import`, hooking up import classes to exports. MEF has a catalog that keeps track of exports and looks at assembly, directory, type catalogs to discover exports. Container instantiates objects and satisfies imports. We do a series of object instantiations, catalog lookups, and finding exports to get the model that is handed out from a container.
|
||||||
|
|
||||||
* MEF2 is purely type-based and in order to compose the graph it creates a catalog at compile-time. It builds graphs using expression trees. The expression trees in MEF2 fundamentally cannot have cycles in them. This is a limitation you get when you go for MEF2 that is an optimzed version of MEF1. This was done to improve performance and to have the graph known already at compile-time.
|
* MEF2 is purely type-based and in order to compose the graph it creates a catalog at compile-time. It builds graphs using expression trees. The expression trees in MEF2 fundamentally cannot have cycles in them. This is a limitation you get when you go for MEF2 that is an optimzed version of MEF1. This was done to improve performance and to have the graph known already at compile-time.
|
||||||
|
|
||||||
|
|
Some files were not shown because too many files have changed in this diff Show more
Loading…
Add table
Add a link
Reference in a new issue