External manifest seems to be unrecognized by Windows

Question

I am using an external manifest for a very simple program I have written, program.exe. I have an external manifest, program.exe.1.manifest, in the same directory. These are the contents:

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<assembly
    xmlns="urn:schemas-microsoft-com:asm.v1"
    xmlns:asmv3="urn:schemas-microsoft-com:asm.v3"
    manifestVersion="1.0"
    >
  <assemblyIdentity
      name="TestC.Testos.TestEx"
      processorArchitecture="amd64"
      type="win32"
      version="0.1.0.0"
      />
  <description>TestTool</description>
  <file
      loadFrom="%homepath%\Desktop\source\payload.dll"
      name="payload.dll"
      />
</assembly>

Now, the simple program simply calls LoadLibraryW(L"payload.dll"). The payload.dll is located in the Windows directory of the system, however, I am trying to load a version of payload.dll from a folder called source on my desktop. This works when I embed the manifest in the program directly, but does not when it is an external manifest. Why does an external manifest not work?