run dll files on notebook pySpark

148 views Asked by At

I want to run dll file based on C# code into my pySpark notebook (in databricks) using 'clr' method. Unfortunately I can't figure out what should be my 'Assembly Name/ .NET reference'. The file is in correct location and there is no typo when I try to use C# 'Namespace.Class.Function' or direct location to the file (I've been trying multiple ways but still same problem).

Here is my approach:

!pip install pythonnet

%sh
sudo apt-get update
sudo apt-get install -y mono-complete

%sh
echo '#!/bin/bash
sudo apt-get update
sudo apt-get install -y mono-complete' > conf/install_mono.sh
import os
import clr
import sys

# Set environment variables
os.environ["PYTHONNET_PYDLL"] = "dbfs:/FileStore/shared_uploads/[email protected]/SampleClassLibraryProject.dll"

# Add a reference to your .NET assembly
clr.AddReference("SampleClassLibraryProject") #Here is the problem!!!

# Set the .NET runtime based on environment variables
sys.path.append("/Users/admin/anaconda3/lib/python3.11/site-packages")  # Replace with your Python home path

# Now try importing the necessary module
from SampleClassLibraryProject.Algebra import Addition  # Import specific namespace and class

# Example usage:
result = Addition.Add(2, 3)
print(result)

Error message :Error Message File path location: File path

1

There are 1 answers

4
Alex Ott On

Instead of dbfs:/ use /dbfs/ as a prefix for the file path - the reason for that .Net doesn't understand the DBFS file system URL, and needs the local file path.

Update: on Community Edition, the DBFS fuse isn't available (see this answer), so you need to use dbutils.fs.cp to copy file from DBFS to the local disk, and then point to that copy, like this:

local_file = 'file:/tmp/SampleClassLibraryProject.dll'
dbutils.fs.cp(
  'dbfs:/FileStore/shared_uploads/[email protected]/SampleClassLibraryProject.dll', 
  local_file)
os.environ["PYTHONNET_PYDLL"] = local_file