Enable embedding install location options in apphost (#104749)

- Add placeholder value in `apphost` for .NET install search options. Format:
  - `<search_location_flags> /0 <app_relative_dotnet_path>`
- Make `apphost` conditionally look at app-local, app-relative, environment variables, and global locations based on configured behaviour
  - Default (placeholder not changed) is to look at app-local, environment variables, and global locations.
- Update error and tracing messages to include information about any configured search options
- Allow specifying search options in `HostWriter.CreateAppHost`
- Add unit/integration tests

Part of https://github.com/dotnet/designs/blob/main/proposed/apphost-embed-install-location.md.
There still needs to be a corresponding change on the SDK side to allow configuration via `AppHostDotNetSearch` and `AppHostRelativeDotNet` properties.

docs/design/features/host-error-codes.md

@@ -31,7 +31,7 @@ Note that the exit code returned by running an application via `dotnet.exe` or `
 | `LibHostInvalidArgs`                  | `0x80008092` | `-2147450734`          | `146`         | Arguments to `hostpolicy` are invalid. This is used in three unrelated places in the `hostpolicy`, but in all cases it means the component calling `hostpolicy` did something wrong: <ul><li> Command line arguments for the app - the failure would typically mean that wrong argument was passed or such. For example if the application main assembly is not specified on the command line. On its own this should not happen as `hostfxr` should have parsed and validated all command line arguments. </li><li> `hostpolicy` context's `get_delegate` - if the requested delegate enum value is not recognized. Again this would mean `hostfxr` passed the wrong value. </li><li> `corehost_resolve_component_dependencies` - if something went wrong initializing `hostpolicy` internal structures. Would happen for example when the `component_main_assembly_path` argument is wrong. </li></ul> |
 | `InvalidConfigFile`                   | `0x80008093` | `-2147450733`          | `147`         | The `.runtimeconfig.json` file is invalid. The reasons for this failure can be among these: <ul><li> Failure to read from the file </li><li> Invalid JSON </li><li> Invalid value for a property (for example number for property which requires a string) </li><li> Missing required property </li><li> Other inconsistencies (for example `rollForward` and `applyPatches` are not allowed to be specified in the same config file) </li><li> Any of the above failures reading the `.runtimecofig.dev.json` file </li><li> Self-contained `.runtimeconfig.json` used in `hostfxr_initialize_for_runtime_config`. Note that missing `.runtimconfig.json` is not an error (means self-contained app). This error code is also used when there is a problem reading the CLSID map file in `comhost`. </li></ul> |
 | `AppArgNotRunnable`                   | `0x80008094` | `-2147450732`          | `148`         | Used internally when the command line for `dotnet.exe` doesn't contain path to the application to run. In such case the command line is considered to be a CLI/SDK command. This error code should never be returned to external caller. |
-| `AppHostExeNotBoundFailure`           | `0x80008095` | `-2147450731`          | `149`         | `apphost` failed to determine which application to run. This can mean: <ul><li> The `apphost` binary has not been imprinted with the path to the app to run (so freshly built `apphost.exe` from the branch will fail to run like this) </li><li> The `apphost` is a bundle (single-file exe) and it failed to extract correctly. </li></ul> |
+| `AppHostExeNotBoundFailure`           | `0x80008095` | `-2147450731`          | `149`         | `apphost` failed to determine which application to run. This can mean: <ul><li> The `apphost` binary has not been imprinted with the path to the app to run (so freshly built `apphost.exe` from the branch will fail to run like this) </li><li> The `apphost` is a bundle (single-file exe) and it failed to extract correctly. </li><li>The `apphost` binary has been imprinted with invalid .NET search options</li></ul> |
 | `FrameworkMissingFailure`             | `0x80008096` | `-2147450730`          | `150`         | It was not possible to find a compatible framework version. This originates in `hostfxr` (`resolve_framework_reference`) and means that the app specified a reference to a framework in its `.runtimeconfig.json` which could not be resolved. The failure to resolve can mean that no such framework is available on the disk, or that the available frameworks don't match the minimum version specified or that the roll forward options specified excluded all available frameworks. Typically this would be used if a 3.0 app is trying to run on a machine which has no 3.0 installed. It would also be used for example if a 32bit 3.0 app is running on a machine which has 3.0 installed but only for 64bit. |
 | `HostApiFailed`                       | `0x80008097` | `-2147450729`          | `151`         | Returned by `hostfxr_get_native_search_directories` if the `hostpolicy` could not calculate the `NATIVE_DLL_SEARCH_DIRECTORIES`. |
 | `HostApiBufferTooSmall`               | `0x80008098` | `-2147450728`          | `152`         | Returned when the buffer specified to an API is not big enough to fit the requested value. Can be returned from: <ul><li> `hostfxr_get_runtime_properties` </li><li> `hostfxr_get_native_search_directories` </li><li> `get_hostfxr_path` </li></ul> |

src/installer/managed/Microsoft.NET.HostModel/AppHost/AppHostExceptions.cs

@@ -80,10 +80,38 @@ namespace Microsoft.NET.HostModel.AppHost
     {
         public string LongName { get; }
 
-        internal AppNameTooLongException(string name)
-            : base($"The name of the app is too long (must be less than 1024 bytes). Name: {name}")
+        internal AppNameTooLongException(string name, int maxSize)
+            : base($"The name of the app is too long (must be less than {maxSize} bytes when encoded in UTF-8). Name: {name}")
         {
             LongName = name;
         }
     }
+
+    /// <summary>
+    /// App-relative .NET path is an absolute path
+    /// </summary>
+    public sealed class AppRelativePathRootedException : AppHostUpdateException
+    {
+        public string Path { get; }
+
+        internal AppRelativePathRootedException(string path)
+            : base($"The app-relative .NET path should not be an absolute path. Path: {path}")
+        {
+            Path = path;
+        }
+    }
+
+    /// <summary>
+    /// App-relative .NET path is too long to be embedded in the apphost
+    /// </summary>
+    public sealed class AppRelativePathTooLongException : AppHostUpdateException
+    {
+        public string Path { get; }
+
+        internal AppRelativePathTooLongException(string path, int maxSize)
+            : base($"The app-relative .NET path is too long (must be less than {maxSize} bytes when encoded in UTF-8). Path: {path}")
+        {
+            Path = path;
+        }
+    }
 }

src/installer/managed/Microsoft.NET.HostModel/AppHost/HostWriter.cs

@@ -3,7 +3,6 @@
 
 using System;
 using System.ComponentModel;
-using System.Diagnostics;
 using System.IO;
 using System.IO.MemoryMappedFiles;
 using System.Runtime.InteropServices;
@@ -23,6 +22,36 @@ namespace Microsoft.NET.HostModel.AppHost
         private const string AppBinaryPathPlaceholder = "c3ab8ff13720e8ad9047dd39466b3c8974e592c2fa383d4a3960714caef0c4f2";
         private static readonly byte[] AppBinaryPathPlaceholderSearchValue = Encoding.UTF8.GetBytes(AppBinaryPathPlaceholder);
 
+        // See placeholder array in corehost.cpp
+        private const int MaxAppBinaryPathSizeInBytes = 1024;
+
+        /// <summary>
+        /// Value embedded in default apphost executable for configuration of how it will search for the .NET install
+        /// </summary>
+        private const string DotNetSearchPlaceholder = "\0\019ff3e9c3602ae8e841925bb461a0adb064a1f1903667a5e0d87e8f608f425ac";
+        private static readonly byte[] DotNetSearchPlaceholderSearchValue = Encoding.UTF8.GetBytes(DotNetSearchPlaceholder);
+
+        // See placeholder array in hostfxr_resolver.cpp
+        private const int MaxDotNetSearchSizeInBytes = 512;
+        private const int MaxAppRelativeDotNetSizeInBytes = MaxDotNetSearchSizeInBytes - 3; // -2 for search location + null, -1 for null terminator
+
+        public class DotNetSearchOptions
+        {
+            // Keep in sync with fxr_resolver::search_location in fxr_resolver.h
+            [Flags]
+            public enum SearchLocation : byte
+            {
+                Default,
+                AppLocal = 1 << 0,
+                AppRelative = 1 << 1,
+                EnvironmentVariable = 1 << 2,
+                Global = 1 << 3,
+            }
+
+            public SearchLocation Location { get; set; } = SearchLocation.Default;
+            public string AppRelativeDotNet { get; set; }
+        }
+
         /// <summary>
         /// Create an AppHost with embedded configuration of app binary location
         /// </summary>
@@ -33,6 +62,7 @@ namespace Microsoft.NET.HostModel.AppHost
         /// <param name="assemblyToCopyResourcesFrom">Path to the intermediate assembly, used for copying resources to PE apphosts.</param>
         /// <param name="enableMacOSCodeSign">Sign the app binary using codesign with an anonymous certificate.</param>
         /// <param name="disableCetCompat">Remove CET Shadow Stack compatibility flag if set</param>
+        /// <param name="dotNetSearchOptions">Options for how the created apphost should look for the .NET install</param>
         public static void CreateAppHost(
             string appHostSourceFilePath,
             string appHostDestinationFilePath,
@@ -40,20 +70,31 @@ namespace Microsoft.NET.HostModel.AppHost
             bool windowsGraphicalUserInterface = false,
             string assemblyToCopyResourcesFrom = null,
             bool enableMacOSCodeSign = false,
-            bool disableCetCompat = false)
+            bool disableCetCompat = false,
+            DotNetSearchOptions dotNetSearchOptions = null)
         {
-            var bytesToWrite = Encoding.UTF8.GetBytes(appBinaryFilePath);
-            if (bytesToWrite.Length > 1024)
+            byte[] appPathBytes = Encoding.UTF8.GetBytes(appBinaryFilePath);
+            if (appPathBytes.Length > MaxAppBinaryPathSizeInBytes)
             {
-                throw new AppNameTooLongException(appBinaryFilePath);
+                throw new AppNameTooLongException(appBinaryFilePath, MaxAppBinaryPathSizeInBytes);
             }
 
+            byte[] searchOptionsBytes = dotNetSearchOptions != null
+                ? GetSearchOptionBytes(dotNetSearchOptions)
+                : null;
+
             bool appHostIsPEImage = false;
 
             void RewriteAppHost(MemoryMappedFile mappedFile, MemoryMappedViewAccessor accessor)
             {
                 // Re-write the destination apphost with the proper contents.
-                BinaryUtils.SearchAndReplace(accessor, AppBinaryPathPlaceholderSearchValue, bytesToWrite);
+                BinaryUtils.SearchAndReplace(accessor, AppBinaryPathPlaceholderSearchValue, appPathBytes);
+
+                // Update the .NET search configuration
+                if (searchOptionsBytes != null)
+                {
+                    BinaryUtils.SearchAndReplace(accessor, DotNetSearchPlaceholderSearchValue, searchOptionsBytes);
+                }
 
                 appHostIsPEImage = PEUtils.IsPEImage(accessor);
 
@@ -239,6 +280,29 @@ namespace Microsoft.NET.HostModel.AppHost
             return headerOffset != 0;
         }
 
+        private static byte[] GetSearchOptionBytes(DotNetSearchOptions searchOptions)
+        {
+            if (Path.IsPathRooted(searchOptions.AppRelativeDotNet))
+                throw new AppRelativePathRootedException(searchOptions.AppRelativeDotNet);
+
+            byte[] pathBytes = searchOptions.AppRelativeDotNet != null
+                ? Encoding.UTF8.GetBytes(searchOptions.AppRelativeDotNet)
+                : [];
+
+            if (pathBytes.Length > MaxAppRelativeDotNetSizeInBytes)
+                throw new AppRelativePathTooLongException(searchOptions.AppRelativeDotNet, MaxAppRelativeDotNetSizeInBytes);
+
+            // <search_location> 0 <app_relative_dotnet_root> 0
+            byte[] searchOptionsBytes = new byte[pathBytes.Length + 3]; // +2 for search location + null, +1 for null terminator
+            searchOptionsBytes[0] = (byte)searchOptions.Location;
+            searchOptionsBytes[1] = 0;
+            searchOptionsBytes[searchOptionsBytes.Length - 1] = 0;
+            if (pathBytes.Length > 0)
+                pathBytes.CopyTo(searchOptionsBytes, 2);
+
+            return searchOptionsBytes;
+        }
+
         [LibraryImport("libc", SetLastError = true)]
         private static partial int chmod([MarshalAs(UnmanagedType.LPStr)] string pathname, int mode);
     }

src/installer/tests/HostActivation.Tests/FrameworkDependentAppLaunch.cs

@@ -317,10 +317,8 @@ namespace Microsoft.DotNet.CoreSetup.Test.HostActivation
                 result.Should().Fail()
                     .And.HaveStdErrContaining($"https://aka.ms/dotnet-core-applaunch?{expectedUrlQuery}")
                     .And.HaveStdErrContaining($"&rid={TestContext.BuildRID}")
-                    .And.HaveStdErrContaining(expectedStdErr);
-
-                // Some Unix systems will have 8 bit exit codes.
-                Assert.True(result.ExitCode == expectedErrorCode || result.ExitCode == (expectedErrorCode & 0xFF));
+                    .And.HaveStdErrContaining(expectedStdErr)
+                    .And.ExitWith(expectedErrorCode);
             }
         }
 

src/installer/tests/HostActivation.Tests/InstallLocation.cs

@@ -3,19 +3,21 @@
 
 using System;
 using System.IO;
-
+using FluentAssertions;
 using Microsoft.DotNet.Cli.Build.Framework;
 using Microsoft.DotNet.CoreSetup.Test;
 using Microsoft.DotNet.CoreSetup.Test.HostActivation;
+using Microsoft.NET.HostModel.AppHost;
 using Xunit;
+using static Microsoft.NET.HostModel.AppHost.HostWriter.DotNetSearchOptions;
 
 namespace HostActivation.Tests
 {
-    public class MultiArchInstallLocation : IClassFixture<MultiArchInstallLocation.SharedTestState>
+    public class InstallLocation : IClassFixture<InstallLocation.SharedTestState>
     {
         private SharedTestState sharedTestState;
 
-        public MultiArchInstallLocation(SharedTestState fixture)
+        public InstallLocation(SharedTestState fixture)
         {
             sharedTestState = fixture;
         }
@@ -288,6 +290,163 @@ namespace HostActivation.Tests
             }
         }
 
+        [Theory]
+        [InlineData(SearchLocation.AppLocal)]
+        [InlineData(SearchLocation.AppRelative)]
+        [InlineData(SearchLocation.EnvironmentVariable)]
+        [InlineData(SearchLocation.Global)]
+        public void SearchOptions(SearchLocation searchLocation)
+        {
+            TestApp app = sharedTestState.App.Copy();
+
+            if (searchLocation == SearchLocation.AppLocal)
+            {
+                // Copy a mock hostfxr to imitate an app-local install
+                File.Copy(Binaries.HostFxr.FilePath, Path.Combine(app.Location, Binaries.HostFxr.FileName));
+            }
+
+            // Create directories for all install locations
+            string appLocalLocation = $"{app.Location}{Path.DirectorySeparatorChar}";
+            string appRelativeLocation = Directory.CreateDirectory(Path.Combine(app.Location, "rel")).FullName;
+            string envLocation = Directory.CreateDirectory(Path.Combine(app.Location, "env")).FullName;
+            string globalLocation = Directory.CreateDirectory(Path.Combine(app.Location, "global")).FullName;
+
+            app.CreateAppHost(dotNetRootOptions: new HostWriter.DotNetSearchOptions()
+            {
+                Location = searchLocation,
+                AppRelativeDotNet = Path.GetRelativePath(app.Location, appRelativeLocation)
+            });
+            CommandResult result;
+            using (var installOverride = new RegisteredInstallLocationOverride(app.AppExe))
+            {
+                installOverride.SetInstallLocation([(TestContext.BuildArchitecture, globalLocation)]);
+                result = Command.Create(app.AppExe)
+                    .EnableTracingAndCaptureOutputs()
+                    .ApplyRegisteredInstallLocationOverride(installOverride)
+                    .DotNetRoot(envLocation)
+                    .Execute();
+            }
+
+            switch (searchLocation)
+            {
+                case SearchLocation.AppLocal:
+                    result.Should().HaveUsedAppLocalInstallLocation(appLocalLocation);
+                    break;
+                case SearchLocation.AppRelative:
+                    result.Should().HaveUsedAppRelativeInstallLocation(appRelativeLocation);
+                    break;
+                case SearchLocation.EnvironmentVariable:
+                    result.Should().HaveUsedDotNetRootInstallLocation(envLocation, TestContext.BuildRID);
+                    break;
+                case SearchLocation.Global:
+                    result.Should().HaveUsedGlobalInstallLocation(globalLocation);
+                    break;
+            }
+        }
+
+        [Fact]
+        public void AppHost_AppRelative_MissingPath()
+        {
+            TestApp app = sharedTestState.App.Copy();
+            app.CreateAppHost(dotNetRootOptions: new HostWriter.DotNetSearchOptions()
+            {
+                Location = SearchLocation.AppRelative
+            });
+            Command.Create(app.AppExe)
+                .CaptureStdErr()
+                .CaptureStdOut()
+                .Execute()
+                .Should().Fail()
+                .And.HaveStdErrContaining("The app-relative .NET path is not embedded.")
+                .And.ExitWith(Constants.ErrorCode.AppHostExeNotBoundFailure);
+        }
+
+        [Theory]
+        [InlineData("./dir")]
+        [InlineData("../dir")]
+        [InlineData("..\\dir")]
+        [InlineData("dir1/dir2")]
+        [InlineData("dir1\\dir2")]
+        public void SearchOptions_AppRelative_PathVariations(string relativePath)
+        {
+            TestApp app = sharedTestState.App.Copy();
+            string installLocation = Path.Combine(app.Location, relativePath);
+            Directory.CreateDirectory(installLocation);
+            using (var testArtifact = new TestArtifact(installLocation))
+            {
+                app.CreateAppHost(dotNetRootOptions: new HostWriter.DotNetSearchOptions()
+                {
+                    Location = HostWriter.DotNetSearchOptions.SearchLocation.AppRelative,
+                    AppRelativeDotNet = relativePath
+                });
+                Command.Create(app.AppExe)
+                    .EnableTracingAndCaptureOutputs()
+                    .Execute()
+                    .Should().HaveUsedAppRelativeInstallLocation(Path.GetFullPath(installLocation));
+            }
+        }
+
+        [Theory]
+        [InlineData(SearchLocation.AppLocal)]
+        [InlineData(SearchLocation.AppRelative)]
+        [InlineData(SearchLocation.EnvironmentVariable)]
+        [InlineData(SearchLocation.Global)]
+        public void SearchOptions_Precedence(SearchLocation expectedResult)
+        {
+            TestApp app = sharedTestState.App.Copy();
+
+            // Create directories for the install locations for the expected result and for those that should
+            // have a lower precedence than the expected result (they should not be used)
+            string appLocalLocation = $"{app.Location}{Path.DirectorySeparatorChar}";
+            if (expectedResult == SearchLocation.AppLocal)
+                File.Copy(Binaries.HostFxr.FilePath, Path.Combine(app.Location, Binaries.HostFxr.FileName));
+
+            string appRelativeLocation = Path.Combine(app.Location, "rel");
+            if (expectedResult <= SearchLocation.AppRelative)
+                Directory.CreateDirectory(appRelativeLocation);
+
+            string envLocation = Path.Combine(app.Location, "env");
+            if (expectedResult <= SearchLocation.EnvironmentVariable)
+                Directory.CreateDirectory(envLocation);
+
+            string globalLocation = Path.Combine(app.Location, "global");
+            if (expectedResult <= SearchLocation.Global)
+                Directory.CreateDirectory(globalLocation);
+
+            app.CreateAppHost(dotNetRootOptions: new HostWriter.DotNetSearchOptions()
+            {
+                // Search all locations
+                Location = SearchLocation.AppLocal | SearchLocation.AppRelative | SearchLocation.EnvironmentVariable | SearchLocation.Global,
+                AppRelativeDotNet = Path.GetRelativePath(app.Location, appRelativeLocation)
+            });
+            CommandResult result;
+            using (var installOverride = new RegisteredInstallLocationOverride(app.AppExe))
+            {
+                installOverride.SetInstallLocation([(TestContext.BuildArchitecture, globalLocation)]);
+                result = Command.Create(app.AppExe)
+                    .EnableTracingAndCaptureOutputs()
+                    .ApplyRegisteredInstallLocationOverride(installOverride)
+                    .DotNetRoot(envLocation)
+                    .Execute();
+            }
+
+            switch (expectedResult)
+            {
+                case SearchLocation.AppLocal:
+                    result.Should().HaveUsedAppLocalInstallLocation(appLocalLocation);
+                    break;
+                case SearchLocation.AppRelative:
+                    result.Should().HaveUsedAppRelativeInstallLocation(appRelativeLocation);
+                    break;
+                case SearchLocation.EnvironmentVariable:
+                    result.Should().HaveUsedDotNetRootInstallLocation(envLocation, TestContext.BuildRID);
+                    break;
+                case SearchLocation.Global:
+                    result.Should().HaveUsedGlobalInstallLocation(globalLocation);
+                    break;
+            }
+        }
+
         public class SharedTestState : IDisposable
         {
             public TestApp App { get; }

src/installer/tests/HostActivation.Tests/InstallLocationCommandResultExtensions.cs

@@ -43,7 +43,17 @@ namespace HostActivation.Tests
 
         public static AndConstraint<CommandResultAssertions> HaveUsedGlobalInstallLocation(this CommandResultAssertions assertion, string installLocation)
         {
-            return assertion.HaveStdErrContaining($"Using global installation location [{installLocation}]");
+            return assertion.HaveStdErrContaining($"Using global install location [{installLocation}]");
+        }
+
+        public static AndConstraint<CommandResultAssertions> HaveUsedAppLocalInstallLocation(this CommandResultAssertions assertion, string installLocation)
+        {
+            return assertion.HaveStdErrContaining($"Using app-local location [{installLocation}]");
+        }
+
+        public static AndConstraint<CommandResultAssertions> HaveUsedAppRelativeInstallLocation(this CommandResultAssertions assertion, string installLocation)
+        {
+            return assertion.HaveStdErrContaining($"Using app-relative location [{installLocation}]");
         }
 
         public static AndConstraint<CommandResultAssertions> HaveLookedForDefaultInstallLocation(this CommandResultAssertions assertion, string installLocationPath)

src/installer/tests/HostActivation.Tests/InvalidHost.cs

@@ -30,19 +30,8 @@ namespace HostActivation.Tests
                 .Execute(expectedToFail: true);
 
             result.Should().Fail()
-                .And.HaveStdErrContaining("This executable is not bound to a managed DLL to execute.");
-
-            int exitCode = result.ExitCode;
-            const int AppHostExeNotBoundFailure = unchecked((int)0x80008095);
-            if (OperatingSystem.IsWindows())
-            {
-                exitCode.Should().Be(AppHostExeNotBoundFailure);
-            }
-            else
-            {
-                // Some Unix flavors filter exit code to ubyte.
-                (exitCode & 0xFF).Should().Be(AppHostExeNotBoundFailure & 0xFF);
-            }
+                .And.HaveStdErrContaining("This executable is not bound to a managed DLL to execute.")
+                .And.ExitWith(Constants.ErrorCode.AppHostExeNotBoundFailure);
         }
 
         [Fact]
@@ -84,19 +73,8 @@ namespace HostActivation.Tests
                 .Execute(expectedToFail: true);
 
             result.Should().Fail()
-                .And.HaveStdErrContaining($"Error: cannot execute dotnet when renamed to {Path.GetFileNameWithoutExtension(sharedTestState.RenamedDotNet)}");
-
-            int exitCode = result.ExitCode;
-            const int EntryPointFailure = unchecked((int)0x80008084);
-            if (OperatingSystem.IsWindows())
-            {
-                exitCode.Should().Be(EntryPointFailure);
-            }
-            else
-            {
-                // Some Unix flavors filter exit code to ubyte.
-                (exitCode & 0xFF).Should().Be(EntryPointFailure & 0xFF);
-            }
+                .And.HaveStdErrContaining($"Error: cannot execute dotnet when renamed to {Path.GetFileNameWithoutExtension(sharedTestState.RenamedDotNet)}")
+                .And.ExitWith(Constants.ErrorCode.EntryPointFailure);
         }
 
         public class SharedTestState : IDisposable

src/installer/tests/HostActivation.Tests/NativeHosting/Nethost.cs

@@ -357,7 +357,7 @@ namespace Microsoft.DotNet.CoreSetup.Test.HostActivation.NativeHosting
                 .EnvironmentVariable(Constants.TestOnlyEnvironmentVariables.GloballyRegisteredPath, sharedState.ValidInstallRoot)
                 .DotNetRoot(null)
                 .Execute()
-                .Should().NotHaveStdErrContaining($"Using global installation location [{sharedState.ValidInstallRoot}] as runtime location.");
+                .Should().NotHaveStdErrContaining($"Using global install location [{sharedState.ValidInstallRoot}] as runtime location.");
         }
 
         public class SharedTestState : SharedTestStateBase

src/installer/tests/Microsoft.NET.HostModel.Tests/AppHost/CreateAppHost.cs

@@ -25,6 +25,12 @@ namespace Microsoft.NET.HostModel.AppHost.Tests
         private const string AppBinaryPathPlaceholder = "c3ab8ff13720e8ad9047dd39466b3c8974e592c2fa383d4a3960714caef0c4f2";
         private readonly static byte[] AppBinaryPathPlaceholderSearchValue = Encoding.UTF8.GetBytes(AppBinaryPathPlaceholder);
 
+        /// <summary>
+        /// Value embedded in default apphost executable for configuration of how it will search for the .NET install
+        /// </summary>
+        private const string DotNetSearchPlaceholder = "\0\019ff3e9c3602ae8e841925bb461a0adb064a1f1903667a5e0d87e8f608f425ac";
+        private static readonly byte[] DotNetSearchPlaceholderValue = Encoding.UTF8.GetBytes(DotNetSearchPlaceholder);
+
         [Fact]
         public void EmbedAppBinaryPath()
         {
@@ -96,6 +102,54 @@ namespace Microsoft.NET.HostModel.AppHost.Tests
             }
         }
 
+        [Fact]
+        public void AppRelativePathRooted_Fails()
+        {
+            using (TestArtifact artifact = CreateTestDirectory())
+            {
+                string sourceAppHostMock = PrepareAppHostMockFile(artifact.Location);
+                string destinationFilePath = Path.Combine(artifact.Location, "DestinationAppHost.exe.mock");
+                HostWriter.DotNetSearchOptions options = new()
+                {
+                    Location = HostWriter.DotNetSearchOptions.SearchLocation.AppRelative,
+                    AppRelativeDotNet = artifact.Location
+                };
+
+                Assert.Throws<AppRelativePathRootedException>(() =>
+                    HostWriter.CreateAppHost(
+                        sourceAppHostMock,
+                        destinationFilePath,
+                        "app.dll",
+                        dotNetSearchOptions: options));
+
+                File.Exists(destinationFilePath).Should().BeFalse();
+            }
+        }
+
+        [Fact]
+        public void AppRelativePathTooLong_Fails()
+        {
+            using (TestArtifact artifact = CreateTestDirectory())
+            {
+                string sourceAppHostMock = PrepareAppHostMockFile(artifact.Location);
+                string destinationFilePath = Path.Combine(artifact.Location, "DestinationAppHost.exe.mock");
+                HostWriter.DotNetSearchOptions options = new()
+                {
+                    Location = HostWriter.DotNetSearchOptions.SearchLocation.AppRelative,
+                    AppRelativeDotNet = new string('p', 1024)
+                };
+
+                Assert.Throws<AppRelativePathTooLongException>(() =>
+                    HostWriter.CreateAppHost(
+                        sourceAppHostMock,
+                        destinationFilePath,
+                        "app.dll",
+                        dotNetSearchOptions: options));
+
+                File.Exists(destinationFilePath).Should().BeFalse();
+            }
+        }
+
         [Fact]
         public void GUISubsystem_WindowsPEFile()
         {
@@ -399,11 +453,11 @@ namespace Microsoft.NET.HostModel.AppHost.Tests
             // The only customization which we do on non-Windows files is the embedding
             // of the binary path, which works the same regardless of the file format.
 
-            int size = WindowsFileHeader.Length + AppBinaryPathPlaceholderSearchValue.Length;
+            int size = WindowsFileHeader.Length + AppBinaryPathPlaceholderSearchValue.Length + DotNetSearchPlaceholderValue.Length;
             byte[] content = new byte[size];
             Array.Copy(WindowsFileHeader, 0, content, 0, WindowsFileHeader.Length);
             Array.Copy(AppBinaryPathPlaceholderSearchValue, 0, content, WindowsFileHeader.Length, AppBinaryPathPlaceholderSearchValue.Length);
-
+            Array.Copy(DotNetSearchPlaceholderValue, 0, content, WindowsFileHeader.Length + AppBinaryPathPlaceholderSearchValue.Length, DotNetSearchPlaceholderValue.Length);
             customize?.Invoke(content);
 
             string filePath = Path.Combine(directory, "SourceAppHost.exe.mock");

src/installer/tests/TestUtils/Assertions/CommandResultAssertions.cs

@@ -21,6 +21,10 @@ namespace Microsoft.DotNet.CoreSetup.Test
 
         public AndConstraint<CommandResultAssertions> ExitWith(int expectedExitCode)
         {
+            // Some Unix systems will have 8 bit exit codes
+            if (!OperatingSystem.IsWindows())
+                expectedExitCode = expectedExitCode & 0xFF;
+
             Execute.Assertion.ForCondition(Result.ExitCode == expectedExitCode)
                 .FailWith($"Expected command to exit with {expectedExitCode} but it did not.{GetDiagnosticsInfo()}");
             return new AndConstraint<CommandResultAssertions>(this);

src/installer/tests/TestUtils/Constants.cs

@@ -116,11 +116,13 @@ namespace Microsoft.DotNet.CoreSetup.Test
             public const int Success = 0;
             public const int InvalidArgFailure = unchecked((int)0x80008081);
             public const int CoreHostLibMissingFailure = unchecked((int)0x80008083);
+            public const int EntryPointFailure = unchecked((int)0x80008084);
             public const int ResolverInitFailure = unchecked((int)0x8000808b);
             public const int ResolverResolveFailure = unchecked((int)0x8000808c);
             public const int LibHostInvalidArgs = unchecked((int)0x80008092);
             public const int InvalidConfigFile = unchecked((int)0x80008093);
             public const int AppArgNotRunnable = unchecked((int)0x80008094);
+            public const int AppHostExeNotBoundFailure = unchecked((int)0x80008095);
             public const int FrameworkMissingFailure = unchecked((int)0x80008096);
             public const int FrameworkCompatFailure = unchecked((int)0x8000809c);
             public const int BundleExtractionFailure = unchecked((int)0x8000809f);

src/installer/tests/TestUtils/TestApp.cs

@@ -77,13 +77,13 @@ namespace Microsoft.DotNet.CoreSetup.Test
             builder.Build(this);
         }
 
-        public void CreateAppHost(bool isWindowsGui = false, bool copyResources = true, bool disableCetCompat = false)
-            => CreateAppHost(Binaries.AppHost.FilePath, isWindowsGui, copyResources, disableCetCompat);
+        public void CreateAppHost(bool isWindowsGui = false, bool copyResources = true, bool disableCetCompat = false, HostWriter.DotNetSearchOptions dotNetRootOptions = null)
+            => CreateAppHost(Binaries.AppHost.FilePath, isWindowsGui, copyResources, disableCetCompat, dotNetRootOptions);
 
-        public void CreateSingleFileHost(bool isWindowsGui = false, bool copyResources = true, bool disableCetCompat = false)
-            => CreateAppHost(Binaries.SingleFileHost.FilePath, isWindowsGui, copyResources, disableCetCompat);
+        public void CreateSingleFileHost(bool isWindowsGui = false, bool copyResources = true, bool disableCetCompat = false, HostWriter.DotNetSearchOptions dotNetRootOptions = null)
+            => CreateAppHost(Binaries.SingleFileHost.FilePath, isWindowsGui, copyResources, disableCetCompat, dotNetRootOptions);
 
-        private void CreateAppHost(string hostSourcePath, bool isWindowsGui, bool copyResources, bool disableCetCompat)
+        private void CreateAppHost(string hostSourcePath, bool isWindowsGui, bool copyResources, bool disableCetCompat, HostWriter.DotNetSearchOptions dotNetRootOptions)
         {
             // Use the live-built apphost and HostModel to create the apphost to run
             HostWriter.CreateAppHost(
@@ -92,7 +92,8 @@ namespace Microsoft.DotNet.CoreSetup.Test
                 Path.GetFileName(AppDll),
                 windowsGraphicalUserInterface: isWindowsGui,
                 assemblyToCopyResourcesFrom: copyResources ? AppDll : null,
-                disableCetCompat: disableCetCompat);
+                disableCetCompat: disableCetCompat,
+                dotNetSearchOptions: dotNetRootOptions);
         }
 
         public enum MockedComponent

src/native/corehost/apphost/standalone/hostfxr_resolver.cpp

@@ -8,6 +8,70 @@
 #include "trace.h"
 #include "hostfxr_resolver.h"
 
+namespace
+{
+    // SHA-256 of "dotnet-search" in UTF-8
+    #define EMBED_DOTNET_SEARCH_HI_PART_UTF8 "19ff3e9c3602ae8e841925bb461a0adb"
+    #define EMBED_DOTNET_SEARCH_LO_PART_UTF8 "064a1f1903667a5e0d87e8f608f425ac"
+
+    // <fxr_resolver::search_location_default> \0 <app_relative_dotnet_placeholder>
+    #define EMBED_DOTNET_SEARCH_FULL_UTF8    ("\0\0" EMBED_DOTNET_SEARCH_HI_PART_UTF8 EMBED_DOTNET_SEARCH_LO_PART_UTF8)
+
+    // Get the .NET search options that should be used
+    // Returns false if options are invalid - for example, app-relative search was specified, but the path is invalid or not embedded
+    bool try_get_dotnet_search_options(fxr_resolver::search_location& out_search_location, pal::string_t& out_app_relative_dotnet)
+    {
+        constexpr int EMBED_SIZE = 512;
+        static_assert(sizeof(EMBED_DOTNET_SEARCH_FULL_UTF8) / sizeof(EMBED_DOTNET_SEARCH_FULL_UTF8[0]) < EMBED_SIZE, "Placeholder value for .NET search options longer than expected");
+
+        // Contains the EMBED_DOTNET_SEARCH_FULL_UTF8 value at compile time or app-relative .NET path written by the SDK (dotnet publish).
+        static char embed[EMBED_SIZE] = EMBED_DOTNET_SEARCH_FULL_UTF8;
+
+        out_search_location = (fxr_resolver::search_location)embed[0];
+        assert(embed[1] == 0); // NUL separates the search location and embedded .NET root value
+        if ((out_search_location & fxr_resolver::search_location_app_relative) == 0)
+            return true;
+
+        // Get the embedded app-relative .NET path
+        std::string binding(&embed[2]); // Embedded path is null-terminated
+
+        // Check if the path exceeds the max allowed size
+        constexpr int EMBED_APP_RELATIVE_DOTNET_MAX_SIZE = EMBED_SIZE - 3; // -2 for search location + null, -1 for null terminator
+        if (binding.size() > EMBED_APP_RELATIVE_DOTNET_MAX_SIZE)
+        {
+            trace::error(_X("The app-relative .NET path is longer than the max allowed length (%d)"), EMBED_APP_RELATIVE_DOTNET_MAX_SIZE);
+            return false;
+        }
+
+        // Check if the value is empty or the same as the placeholder
+        // Since the single static string is replaced by editing the executable, a reference string is needed to do the compare.
+        // So use two parts of the string that will be unaffected by the edit.
+        static const char hi_part[] = EMBED_DOTNET_SEARCH_HI_PART_UTF8;
+        static const char lo_part[] = EMBED_DOTNET_SEARCH_LO_PART_UTF8;
+        size_t hi_len = (sizeof(hi_part) / sizeof(hi_part[0])) - 1;
+        size_t lo_len = (sizeof(lo_part) / sizeof(lo_part[0])) - 1;
+        if (binding.empty()
+            || (binding.size() >= (hi_len + lo_len)
+                && binding.compare(0, hi_len, &hi_part[0]) == 0
+                && binding.compare(hi_len, lo_len, &lo_part[0]) == 0))
+        {
+            trace::error(_X("The app-relative .NET path is not embedded."));
+            return false;
+        }
+
+        pal::string_t app_relative_dotnet;
+        if (!pal::clr_palstring(binding.c_str(), &app_relative_dotnet))
+        {
+            trace::error(_X("The app-relative .NET path could not be retrieved from the executable image."));
+            return false;
+        }
+
+        trace::info(_X("Embedded app-relative .NET path: '%s'"), app_relative_dotnet.c_str());
+        out_app_relative_dotnet = std::move(app_relative_dotnet);
+        return true;
+    }
+}
+
 hostfxr_main_bundle_startupinfo_fn hostfxr_resolver_t::resolve_main_bundle_startupinfo()
 {
     assert(m_hostfxr_dll != nullptr);
@@ -34,7 +98,23 @@ hostfxr_main_fn hostfxr_resolver_t::resolve_main_v1()
 
 hostfxr_resolver_t::hostfxr_resolver_t(const pal::string_t& app_root)
 {
-    if (!fxr_resolver::try_get_path(app_root, &m_dotnet_root, &m_fxr_path))
+    fxr_resolver::search_location search_location = fxr_resolver::search_location_default;
+    pal::string_t app_relative_dotnet;
+    pal::string_t app_relative_dotnet_path;
+    if (!try_get_dotnet_search_options(search_location, app_relative_dotnet))
+    {
+        m_status_code = StatusCode::AppHostExeNotBoundFailure;
+        return;
+    }
+
+    trace::info(_X(".NET root search location options: %d"), search_location);
+    if (!app_relative_dotnet.empty())
+    {
+        app_relative_dotnet_path = app_root;
+        append_path(&app_relative_dotnet_path, app_relative_dotnet.c_str());
+    }
+
+    if (!fxr_resolver::try_get_path(app_root, search_location, &app_relative_dotnet_path, &m_dotnet_root, &m_fxr_path))
     {
         m_status_code = StatusCode::CoreHostLibMissingFailure;
     }

src/native/corehost/corehost.cpp

@@ -59,15 +59,23 @@ bool is_exe_enabled_for_execution(pal::string_t* app_dll)
         return false;
     }
 
+    std::string binding(&embed[0]);
+
+    // Check if the path exceeds the max allowed size
+    if (binding.size() > EMBED_MAX - 1) // -1 for null terminator
+    {
+        trace::error(_X("The managed DLL bound to this executable is longer than the max allowed length (%d)"), EMBED_MAX - 1);
+        return false;
+    }
+
+    // Check if the value is the same as the placeholder
     // Since the single static string is replaced by editing the executable, a reference string is needed to do the compare.
     // So use two parts of the string that will be unaffected by the edit.
     size_t hi_len = (sizeof(hi_part) / sizeof(hi_part[0])) - 1;
     size_t lo_len = (sizeof(lo_part) / sizeof(lo_part[0])) - 1;
-
-    std::string binding(&embed[0]);
-    if ((binding.size() >= (hi_len + lo_len)) &&
-        binding.compare(0, hi_len, &hi_part[0]) == 0 &&
-        binding.compare(hi_len, lo_len, &lo_part[0]) == 0)
+    if (binding.size() >= (hi_len + lo_len)
+        && binding.compare(0, hi_len, &hi_part[0]) == 0
+        && binding.compare(hi_len, lo_len, &lo_part[0]) == 0)
     {
         trace::error(_X("This executable is not bound to a managed DLL to execute. The binding value is: '%s'"), app_dll->c_str());
         return false;

src/native/corehost/fxr/hostfxr.cpp

@@ -380,7 +380,7 @@ SHARED_API int32_t HOSTFXR_CALLTYPE hostfxr_get_dotnet_environment_info(
     {
         if (pal::get_dotnet_self_registered_dir(&dotnet_dir) || pal::get_default_installation_dir(&dotnet_dir))
         {
-            trace::info(_X("Using global installation location [%s]."), dotnet_dir.c_str());
+            trace::info(_X("Using global install location [%s]."), dotnet_dir.c_str());
         }
         else
         {

src/native/corehost/fxr_resolver.cpp

@@ -50,33 +50,70 @@ namespace
 
         return false;
     }
+
+    // The default is defined as app-local, environment variables, and global install locations
+    const fxr_resolver::search_location s_default_search = static_cast<fxr_resolver::search_location>(
+        fxr_resolver::search_location_app_local | fxr_resolver::search_location_environment_variable | fxr_resolver::search_location_global);
 }
 
 bool fxr_resolver::try_get_path(const pal::string_t& root_path, pal::string_t* out_dotnet_root, pal::string_t* out_fxr_path)
+{
+    return try_get_path(root_path, search_location_default, nullptr, out_dotnet_root, out_fxr_path);
+}
+
+bool fxr_resolver::try_get_path(
+    const pal::string_t& root_path,
+    search_location search,
+    /*opt*/ pal::string_t* app_relative_dotnet_root,
+    /*out*/ pal::string_t* out_dotnet_root,
+    /*out*/ pal::string_t* out_fxr_path)
 {
 #if defined(FEATURE_APPHOST) || defined(FEATURE_LIBHOST)
+    if (search == search_location_default)
+        search = s_default_search;
+
     // For apphost and libhost, root_path is expected to be a directory.
     // For libhost, it may be empty if app-local search is not desired (e.g. com/ijw/winrt hosts, nethost when no assembly path is specified)
     // If a hostfxr exists in root_path, then assume self-contained.
-    if (root_path.length() > 0 && file_exists_in_dir(root_path, LIBFXR_NAME, out_fxr_path))
+    bool search_app_local = (search & search_location_app_local) != 0;
+    if (search_app_local && root_path.length() > 0 && file_exists_in_dir(root_path, LIBFXR_NAME, out_fxr_path))
     {
+        trace::info(_X("Using app-local location [%s] as runtime location."), root_path.c_str());
         trace::info(_X("Resolved fxr [%s]..."), out_fxr_path->c_str());
         out_dotnet_root->assign(root_path);
         return true;
     }
 
-    // For framework-dependent apps, use DOTNET_ROOT_<ARCH>
+    // Check in priority order (if specified by search location options):
+    //   - App-relative .NET root location
+    //   - Environment variables: DOTNET_ROOT_<ARCH> and DOTNET_ROOT
+    //   - Global installs:
+    //      - self-registered install location
+    //      - default install location
+    bool search_app_relative = (search & search_location_app_relative) != 0 && app_relative_dotnet_root != nullptr && !app_relative_dotnet_root->empty();
+    bool search_env = (search & search_location_environment_variable) != 0;
+    bool search_global = (search & search_location_global) != 0;
     pal::string_t default_install_location;
     pal::string_t dotnet_root_env_var_name;
-    if (get_dotnet_root_from_env(&dotnet_root_env_var_name, out_dotnet_root))
+    if (search_app_relative && pal::realpath(app_relative_dotnet_root))
+    {
+        trace::info(_X("Using app-relative location [%s] as runtime location."), app_relative_dotnet_root->c_str());
+        out_dotnet_root->assign(*app_relative_dotnet_root);
+        if (file_exists_in_dir(*app_relative_dotnet_root, LIBFXR_NAME, out_fxr_path))
+        {
+            trace::info(_X("Resolved fxr [%s]..."), out_fxr_path->c_str());
+            return true;
+        }
+    }
+    else if (search_env && get_dotnet_root_from_env(&dotnet_root_env_var_name, out_dotnet_root))
     {
         trace::info(_X("Using environment variable %s=[%s] as runtime location."), dotnet_root_env_var_name.c_str(), out_dotnet_root->c_str());
     }
-    else
+    else if (search_global)
     {
         if (pal::get_dotnet_self_registered_dir(&default_install_location) || pal::get_default_installation_dir(&default_install_location))
         {
-            trace::info(_X("Using global installation location [%s] as runtime location."), default_install_location.c_str());
+            trace::info(_X("Using global install location [%s] as runtime location."), default_install_location.c_str());
             out_dotnet_root->assign(default_install_location);
         }
         else
@@ -89,40 +126,78 @@ bool fxr_resolver::try_get_path(const pal::string_t& root_path, pal::string_t* o
     pal::string_t fxr_dir = *out_dotnet_root;
     append_path(&fxr_dir, _X("host"));
     append_path(&fxr_dir, _X("fxr"));
-    if (!pal::directory_exists(fxr_dir))
+    if (pal::directory_exists(fxr_dir))
+        return get_latest_fxr(std::move(fxr_dir), out_fxr_path);
+
+    // Failed to find hostfxr
+    if (trace::is_enabled())
     {
-        if (default_install_location.empty())
-        {
-            pal::get_dotnet_self_registered_dir(&default_install_location);
-        }
-        if (default_install_location.empty())
-        {
-            pal::get_default_installation_dir(&default_install_location);
-        }
+        trace::verbose(_X("The required library %s could not be found. Search location options [0x%x]"), LIBFXR_NAME, search);
+        if (search_app_local)
+            trace::verbose(_X("  app-local: [%s]"), root_path.c_str());
 
-        pal::string_t self_registered_config_location = pal::get_dotnet_self_registered_config_location(get_current_arch());
-        trace::verbose(_X("The required library %s could not be found. Searched with root path [%s], environment variable [%s], default install location [%s], self-registered config location [%s]"),
-            LIBFXR_NAME,
-            root_path.c_str(),
-            dotnet_root_env_var_name.c_str(),
-            default_install_location.c_str(),
-            self_registered_config_location.c_str());
+        if (search_app_relative)
+            trace::verbose(_X("  app-relative: [%s]"), app_relative_dotnet_root->c_str());
 
-        pal::string_t host_path;
-        pal::get_own_executable_path(&host_path);
-        trace::error(
-            MISSING_RUNTIME_ERROR_FORMAT,
-            INSTALL_NET_ERROR_MESSAGE,
-            host_path.c_str(),
-            get_current_arch_name(),
-            _STRINGIFY(HOST_VERSION),
-            _X("Not found"),
-            get_download_url().c_str(),
-            _STRINGIFY(HOST_VERSION));
-        return false;
+        if (search_env)
+            trace::verbose(_X("  environment variable: [%s]"), dotnet_root_env_var_name.c_str());
+
+        if (search_global)
+        {
+            if (default_install_location.empty())
+            {
+                pal::get_dotnet_self_registered_dir(&default_install_location);
+            }
+            if (default_install_location.empty())
+            {
+                pal::get_default_installation_dir(&default_install_location);
+            }
+
+            pal::string_t self_registered_config_location = pal::get_dotnet_self_registered_config_location(get_current_arch());
+            trace::verbose(_X("  global install location [%s]\n  self-registered config location [%s]"),
+                default_install_location.c_str(),
+                self_registered_config_location.c_str());
+        }
     }
 
-    return get_latest_fxr(std::move(fxr_dir), out_fxr_path);
+    pal::string_t host_path;
+    pal::get_own_executable_path(&host_path);
+
+    pal::string_t location = _X("Not found");
+    if (search != s_default_search)
+    {
+        location.append(_X(" - search options: ["));
+        if (search_app_local)
+            location.append(_X(" app_local"));
+
+        if (search_app_relative)
+            location.append(_X(" app_relative"));
+
+        if (search_env)
+            location.append(_X(" environment_variable"));
+
+        if (search_global)
+            location.append(_X(" global"));
+
+        location.append(_X(" ]"));
+        if (search_app_relative)
+        {
+            location.append(_X(", app-relative path: "));
+            location.append(app_relative_dotnet_root->c_str());
+        }
+    }
+
+    trace::error(
+        MISSING_RUNTIME_ERROR_FORMAT,
+        INSTALL_NET_ERROR_MESSAGE,
+        host_path.c_str(),
+        get_current_arch_name(),
+        _STRINGIFY(HOST_VERSION),
+        location.c_str(),
+        get_download_url().c_str(),
+        _STRINGIFY(HOST_VERSION));
+    return false;
+
 #else // !FEATURE_APPHOST && !FEATURE_LIBHOST
     // For non-apphost and non-libhost (i.e. muxer), root_path is expected to be the full path to the host
     pal::string_t host_dir;

src/native/corehost/fxr_resolver.h

@@ -12,7 +12,18 @@
 
 namespace fxr_resolver
 {
+    // Keep in sync with DotNetRootOptions.SearchLocation in HostWriter.cs
+    enum search_location : uint8_t
+    {
+        search_location_default = 0,
+        search_location_app_local = 1 << 0,             // Next to the app
+        search_location_app_relative = 1 << 1,          // Path relative to the app read from the app binary
+        search_location_environment_variable = 1 << 2,  // DOTNET_ROOT[_<arch>] environment variables
+        search_location_global = 1 << 3,                // Registered and default global locations
+    };
+
     bool try_get_path(const pal::string_t& root_path, pal::string_t* out_dotnet_root, pal::string_t* out_fxr_path);
+    bool try_get_path(const pal::string_t& root_path, search_location search, /*opt*/ pal::string_t* embedded_dotnet_root, pal::string_t* out_dotnet_root, pal::string_t* out_fxr_path);
     bool try_get_path_from_dotnet_root(const pal::string_t& dotnet_root, pal::string_t* out_fxr_path);
     bool try_get_existing_fxr(pal::dll_t *out_fxr, pal::string_t *out_fxr_path);
 }