Weird 'Native linking failed, undefined Objective-C class' when Xamarin-iOS nuget is generated from Azure Pipeline (but not when it's built locally)

Question

I have Xamarin-iOS binding-project which upon being built generates a nuget. Said nuget works as intended in Xamarin-iOS applications if and only if I built it on my Mac.

However, when I build this nuget via Azure Pipelines using MacOS-12 as the host (+ iphone16.2 sdk + sharpie 3.5.61 + clang-1400.0.29.202 exactly as in my localdev Mac) even though the build succeeds in generating the nuget it is poisoned in the sense that upon trying to build a Xamarin Application with it I get the following errors:

/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang -framework CoreFoundation -framework Security -framework VisionKit -framework UserNotificationsUI -framework UniformTypeIdentifiers -framework ThreadNetwork -framework WatchConnectivity [...] -u _BrotliEncoderHasMoreOutput -u _BrotliEncoderDestroyInstance -u _BrotliEncoderCompress -u _mono_pmip

Undefined symbols for architecture arm64:
  "_OBJC_CLASS_$__TtC17McuMgrBindingsiOS17IOSDeviceResetter", referenced from:
      objc-class-ref in registrar.o
  "_OBJC_CLASS_$__TtC17McuMgrBindingsiOS17IOSFirmwareEraser", referenced from:
      objc-class-ref in registrar.o
  "_OBJC_CLASS_$__TtC17McuMgrBindingsiOS19IOSFirmwareUpgrader", referenced from:
      objc-class-ref in registrar.o

ld: symbol(s) not found for architecture arm64

I have inspected the generated dll that lives inside both nugets and the symbols 'TtC17McuMgrBindingsiOS17IOSDeviceResetter', 'TtC17McuMgrBindingsiOS17IOSFirmwareEraser' and 'TtC17McuMgrBindingsiOS19IOSFirmwareUpgrader' do indeed exist on both the local and the azure nuget.

The azure pipeline hosted on MacOS-12 seems to be employing Mono ver. 16.10.1 for the build which is exactly what my localdev has.

I noticed that 'clang' in Azure targets x86 instead of arm64 - maybe this relates to the error observed somehow?

(localdev)
Apple clang version 14.0.0 (clang-1400.0.29.202)
Target: arm64-apple-darwin21.6.0
Thread model: posix
InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin

(azure)
Apple clang version 14.0.0 (clang-1400.0.29.102)
Target: x86_64-apple-darwin21.6.0
Thread model: posix
InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin

The build script I'm using to invoke xcodebuild, sharpie and lipo is this one:

#!/usr/bin/env bash

# Builds a fat library for a given xcode project (framework)
#
# Derived from https://github.com/xamcat/xamarin-binding-swift-framework/blob/master/Swift/Scripts/build.fat.sh#L3-L14

IOS_SDK_VERSION="${IOS_SDK_VERSION:-16.2}" # xcodebuild -showsdks

SWIFT_PROJECT_NAME="McuMgrBindingsiOS"
SWIFT_BUILD_PATH="./$SWIFT_PROJECT_NAME/build"
SWIFT_OUTPUT_PATH="./VendorFrameworks/swift-framework-proxy"
SWIFT_BUILD_SCHEME="McuMgrBindingsiOS"
SWIFT_PROJECT_PATH="./$SWIFT_PROJECT_NAME/$SWIFT_PROJECT_NAME.xcodeproj"
SWIFT_PACKAGES_PATH="./packages"
SWIFT_BUILD_CONFIGURATION="Release"

XAMARIN_BINDING_PATH="Xamarin/SwiftFrameworkProxy.Binding"

function print_macos_sdks() {
  xcodebuild -showsdks
}

function build() {
  echo "** Build iOS framework for simulator and device"

  echo "**** (Build 1/5) Cleanup any possible traces of previous builds"

  rm -Rf "$SWIFT_BUILD_PATH"
  rm -Rf "$SWIFT_PACKAGES_PATH"
  rm -Rf "$XAMARIN_BINDING_PATH"

  echo "**** (Build 2/5) Restore packages for 'iphoneos$IOS_SDK_VERSION'"

  xcodebuild \
    -sdk "iphoneos$IOS_SDK_VERSION" \
    -arch arm64 \
    -scheme "$SWIFT_BUILD_SCHEME" \
    -project "$SWIFT_PROJECT_PATH" \
    -configuration "$SWIFT_BUILD_CONFIGURATION" \
    -clonedSourcePackagesDirPath "$SWIFT_PACKAGES_PATH" \
    -resolvePackageDependencies

  if [ $? -ne 0 ]; then
    echo "** [FAILED] Failed to download dependencies for 'iphoneos$IOS_SDK_VERSION'"
    exit 1
  fi

  echo "**** (Build 3/5) Build for 'iphoneos$IOS_SDK_VERSION'"

  # https://stackoverflow.com/a/74478244/863651
  xcodebuild \
    -sdk "iphoneos$IOS_SDK_VERSION" \
    -arch arm64 \
    -scheme "$SWIFT_BUILD_SCHEME" \
    -project "$SWIFT_PROJECT_PATH" \
    -configuration "$SWIFT_BUILD_CONFIGURATION" \
    -derivedDataPath "$SWIFT_BUILD_PATH" \
    -clonedSourcePackagesDirPath "$SWIFT_PACKAGES_PATH" \
    CODE_SIGN_IDENTITY="" \
    CODE_SIGNING_ALLOWED=NO \
    CODE_SIGNING_REQUIRED=NO

  if [ $? -ne 0 ]; then
    echo "** [FAILED] Failed to build 'iphoneos$IOS_SDK_VERSION'"
    exit 1
  fi

  echo "**** (Build 4/5) Restore packages for 'iphonesimulator$IOS_SDK_VERSION'"

  xcodebuild \
    -sdk "iphonesimulator$IOS_SDK_VERSION" \
    -arch arm64 \
    -scheme "$SWIFT_BUILD_SCHEME" \
    -project "$SWIFT_PROJECT_PATH" \
    -configuration "$SWIFT_BUILD_CONFIGURATION" \
    -clonedSourcePackagesDirPath "$SWIFT_PACKAGES_PATH" \
    -resolvePackageDependencies

  if [ $? -ne 0 ]; then
    echo "** [FAILED] Failed to download dependencies for 'iphonesimulator$IOS_SDK_VERSION'"
    exit 1
  fi

  echo "**** (Build 5/5) Build for 'iphonesimulator$IOS_SDK_VERSION'"

  # https://stackoverflow.com/a/74478244/863651
  # https://stackoverflow.com/a/64026089/863651
  xcodebuild \
    -sdk "iphonesimulator$IOS_SDK_VERSION" \
    -scheme "$SWIFT_BUILD_SCHEME" \
    -project "$SWIFT_PROJECT_PATH" \
    -configuration "$SWIFT_BUILD_CONFIGURATION" \
    -derivedDataPath "$SWIFT_BUILD_PATH" \
    -clonedSourcePackagesDirPath "$SWIFT_PACKAGES_PATH" \
    EXCLUDED_ARCHS="arm64" \
    CODE_SIGN_IDENTITY="" \
    CODE_SIGNING_ALLOWED=NO \
    CODE_SIGNING_REQUIRED=NO

  if [ $? -ne 0 ]; then
    echo "** [FAILED] Failed to build 'iphonesimulator$IOS_SDK_VERSION'"
    exit 1
  fi
}

function create_fat_binaries() {
  echo "** Create fat binaries for Release-iphoneos and Release-iphonesimulator configuration"

  echo "**** (FatBinaries 1/8) Copy one build as a fat framework"

  cp \
    -R \
    "$SWIFT_BUILD_PATH/Build/Products/Release-iphoneos" \
    "$SWIFT_BUILD_PATH/Release-fat"
  if [ $? -ne 0 ]; then
    echo "** [FAILED] Failed to copy"
    exit 1
  fi

  echo "**** (FatBinaries 2/8) Combine modules from another build with the fat framework modules"

  cp \
    -R \
    "$SWIFT_BUILD_PATH/Build/Products/Release-iphonesimulator/$SWIFT_PROJECT_NAME.framework/Modules/$SWIFT_PROJECT_NAME.swiftmodule/" \
    "$SWIFT_BUILD_PATH/Release-fat/$SWIFT_PROJECT_NAME.framework/Modules/$SWIFT_PROJECT_NAME.swiftmodule/"
  if [ $? -ne 0 ]; then
    echo "** [FAILED] Failed to copy"
    exit 1
  fi

  echo "**** (FatBinaries 3/8) Combine iphoneos + iphonesimulator configuration as fat libraries"

  lipo \
    -create \
    -output "$SWIFT_BUILD_PATH/Release-fat/$SWIFT_PROJECT_NAME.framework/$SWIFT_PROJECT_NAME" \
    "$SWIFT_BUILD_PATH/Build/Products/Release-iphoneos/$SWIFT_PROJECT_NAME.framework/$SWIFT_PROJECT_NAME" \
    "$SWIFT_BUILD_PATH/Build/Products/Release-iphonesimulator/$SWIFT_PROJECT_NAME.framework/$SWIFT_PROJECT_NAME"
  if [ $? -ne 0 ]; then
    echo "** [FAILED] Failed to combine configurations"
    exit 1
  fi

  echo "**** (FatBinaries 4/8) Verify results"
  lipo \
    -info \
    "$SWIFT_BUILD_PATH/Release-fat/$SWIFT_PROJECT_NAME.framework/$SWIFT_PROJECT_NAME"
  if [ $? -ne 0 ]; then
    echo "** [FAILED] Failed to verify results"
    exit 1
  fi

  echo "**** (FatBinaries 5/8) Copy fat frameworks to the output folder"

  rm -Rf "$SWIFT_OUTPUT_PATH" &&
    mkdir -p "$SWIFT_OUTPUT_PATH" &&
    cp -Rf \
      "$SWIFT_BUILD_PATH/Release-fat/$SWIFT_PROJECT_NAME.framework" \
      "$SWIFT_OUTPUT_PATH"
  if [ $? -ne 0 ]; then
    echo "** [FAILED] Failed to copy fat frameworks"
    exit 1
  fi

  echo "**** (FatBinaries 6/8) Generating binding api definition and structs"
  sharpie \
    bind \
    --sdk="iphoneos$IOS_SDK_VERSION" \
    --scope="$SWIFT_OUTPUT_PATH/$SWIFT_PROJECT_NAME.framework/Headers/" \
    --output="$SWIFT_OUTPUT_PATH/XamarinApiDef" \
    --namespace="$SWIFT_PROJECT_NAME" \
    "$SWIFT_OUTPUT_PATH/$SWIFT_PROJECT_NAME.framework/Headers/$SWIFT_PROJECT_NAME-Swift.h"
  if [ $? -ne 0 ]; then
    echo "** [FAILED] Failed to generate binding api definitions and structs"
    exit 1
  fi

  echo "**** (FatBinaries 7/8) Replace existing metadata with the updated"

  mkdir -p "$XAMARIN_BINDING_PATH/" &&
    cp \
      -Rf \
      "$SWIFT_OUTPUT_PATH/XamarinApiDef/." \
      "$XAMARIN_BINDING_PATH/"
  if [ $? -ne 0 ]; then
    echo "** [FAILED] Failed to replace existing metadata with the updated"
    exit 1
  fi

  echo "**** (FatBinaries 8/8) Replace NativeHandle -> IntPtr in the generated c# files"

  # replace nativehandle -> intptr
  find \
    "$XAMARIN_BINDING_PATH/" \
    -type f \
    -exec sed -i.bak "s/NativeHandle[ ]/IntPtr /gi" {} \;

  # also need to get rid of stupid autogenerated [verify(...)] attributes which are intentionally placed there
  # by sharpie to force manual verification of the .cs files that have been autogenerated
  #
  # https://learn.microsoft.com/en-us/xamarin/cross-platform/macios/binding/objective-sharpie/platform/verify
  find \
    "$XAMARIN_BINDING_PATH/" \
    -type f \
    -exec sed -i.bak 's/\[Verify\s*\(.*\)\]//gi' {} \;

  # adding [model] to the interfaces seems to be mandatory for the azure pipelines to generate a valid nuget for ios   if we
  # omit adding this attribute then the nuget generated by the azure pipelines gets poisoned and it causes a very cryptic runtime error
  # so I'm not 100% sure why the [model] attribute does away with the observed error but it does the trick of solving the problem somehow
  #
  #  find \
  #    "$XAMARIN_BINDING_PATH/" \
  #    -type f \
  #    -exec sed -i.bak 's/interface IOSDeviceResetter/[Model] interface IOSDeviceResetter/gi' {} \;
  #  find \
  #    "$XAMARIN_BINDING_PATH/" \
  #    -type f \
  #    -exec sed -i.bak 's/interface IOSFirmwareEraser/[Model] interface IOSFirmwareEraser/gi' {} \;
  #  find \
  #    "$XAMARIN_BINDING_PATH/" \
  #    -type f \
  #    -exec sed -i.bak 's/interface IOSFirmwareUpgrader/[Model] interface IOSFirmwareUpgrader/gi' {} \;

  # https://stackoverflow.com/a/49477937/863651   its vital to add [BaseType] to the interface otherwise compilation will fail
  find \
    "$XAMARIN_BINDING_PATH/" \
    -type f \
    -exec sed -i.bak 's/interface IOSListenerForDeviceResetter/[BaseType(typeof(NSObject))] [Model] interface IOSListenerForDeviceResetter/gi' {} \;
  find \
    "$XAMARIN_BINDING_PATH/" \
    -type f \
    -exec sed -i.bak 's/interface IOSListenerForFirmwareEraser/[BaseType(typeof(NSObject))] [Model] interface IOSListenerForFirmwareEraser/gi' {} \;
  find \
    "$XAMARIN_BINDING_PATH/" \
    -type f \
    -exec sed -i.bak 's/interface IOSListenerForFirmwareUpgrader/[BaseType(typeof(NSObject))] [Model] interface IOSListenerForFirmwareUpgrader/gi' {} \;
}

function main() {
  print_macos_sdks
  build
  create_fat_binaries

  echo "** Done!"
}

main "$@"

I can provide you with the working and the non-working nugets for you to compare them yourselves if you want - maybe a more experienced pair of eyes can spot something that I can't.

PS: I tried adding [Protocol] before each generated interface in 'ApiDefinition.cs' but even though this solved the original problem it also caused another problem:

Upon attempting to invoke any of the methods of the instantiated class I now get an exception ala 'Foundation.You_Should_Not_Call_base_In_This_Method'

Answer 1

I figured out what was truly wrong quite a few months down the road. In my MSBuild I had:

    <!-- McuMgrBindingsiOS.framework -->
    <NativeReference Include="Frameworks/McuMgrBindingsiOS.framework">
        <Kind>Framework</Kind>
        <SmartLink>False</SmartLink>
        <Frameworks>Foundation</Frameworks>
    </NativeReference>

The thing is that the folder 'Frameworks/McuMgrBindingsiOS.framework/' did not initially exist when MSBuild.exe was getting invoked in the pipeline. This folder was getting created dynamically through the build process a few minutes later but at that point it would be too late because MSBuild had already "made up its mind" over the fact that it would not include this native reference because the folder didn't exist the moment MSBuild.exe was invoked initially. The solution? Simply force the folder to exist pre-emptively by committing to git:

 touch   Frameworks/McuMgrBindingsiOS.framework/.keepme_in_git_otherwise_azure_pipelines_will_break
 git add -f Frameworks/McuMgrBindingsiOS.framework/.keepme_in_git_otherwise_azure_pipelines_will_break
 git commit ...

With this in place the build system started taking the folder into account properly and embedded it in the resulting .dll and nuget.