P/Invoke/DllImport via AssemblyLoadContext find Library inside .framework file

Question

currently I try to DllImport a library from a framework file (dylib). Currently inside the framework there are multiple other library files which are dependencies of the library I want to dllimport.

I actually have setup code that can actually import the framework, however when I try to somehow link to the library file, I always get a Reason: image not found

Inside my AssemblyLoadContext I've overwritten LoadUnmanagedDll like that:

protected override IntPtr LoadUnmanagedDll(string unmanagedDllName)
    {
        if (unmanagedDllName == KernelApiDll)
        {
            if (RuntimeInformation.IsOSPlatform(OSPlatform.OSX))
            {
                return LoadUnmanagedDllFromPath("CUSTOM_FRAMEWORK_DIR_UNDER_USERS/Frameworks/Nuance-OmniPage-CSDK-RunTime.framework/Versions/A/Libraries/libkernelapi.dylib");
            }
            // other os stuff

        return IntPtr.Zero;
    }

is there a good way to actually correctly load all necessary files inside the framework?

In a example makefile (for C code) the -rpath will be set to load the library:

OCRLIBPATH = ../Frameworks/Nuance-OmniPage-CSDK-RunTime.framework/Versions/Current/Libraries
OCRLIBS = -L$(OCRLIBPATH) -lkernelapi -lrecapiplus -lrecpdf -Wl,-rpath,$(OCRRUNPATH)

EDIT:

Here is the full error I get:

System.DllNotFoundException: Unable to load shared library '/REDACTED_PATH/Nuance-OmniPage-CSDK-RunTime.framework/Versions/A/Libraries/libkernelapi.dylib' or one of its dependencies. In order to help diagnose loading problems, consider setting the DYLD_PRINT_LIBRARIES environment variable: dlopen(/REDACTED_PATH/Nuance-OmniPage-CSDK-RunTime.framework/Versions/A/Libraries/libkernelapi.dylib, 1): Library not loaded: @rpath/Nuance-OmniPage-CSDK-RunTime.framework/Versions/A/Libraries/librecdiag.dylib
Referenced from: /REDACTED_PATH/Nuance-OmniPage-CSDK-RunTime.framework/Versions/A/Libraries/libkernelapi.dylib