I have a project, where I am given an id, and then using that ID look up files paths and process them... these files are on various mounted drives, so I am using the SMBJ java libraries to access them.
The problem I am having is that some (most) of the files are using a DFS mountpoint... Now, this in and of itself is NOT a problem per se, but apparently the SMBJ libraries appear to create nested sessions for each distinct DFS location. So even though I am closing the actual FILE after I am done reading it the DiskSession object is holding onto all these nested sessions... and eventually either through the DFS config settings, or through these libraries I am hitting some point where it just blows up and stops allowing more sessions to be created.
I am processing hundreds of thousands of records, and the "crash" appears to happen somewhere around 500ish records(session) being processed. I do not see anything obvious looking at the code to explicitly close these nested sessions.. in fact I see no external access to them at all externally from the DiskShare object.
Is there some sort of setting I am missing that maximizes the sessions that this is holding onto? Other than me managing some sort of my own counter around this, and closing and reopening sessions/connections I am at a loss how to handle this.
Does anyone know what I am missing here?
Code below:
public class Smb {
private static SMBClient client;
private static String[] DFSMounts = {"DFS1","dfs1"};
private static final Logger Log = LoggerFactory.getLogger(Smb.class);
private static HashMap<String,DiskShare> shares = new HashMap<>();
private static HashMap<String,Connection> connections = new HashMap<>();
private static HashMap<Connection,Session> sessions = new HashMap<>();
private synchronized static SMBClient getClient(){
if (client == null){
SmbConfig cfg = SmbConfig.builder().withDfsEnabled(true).build();
client = new SMBClient(cfg);
}
return client;
}
private synchronized static Connection getConnection(String realDomainName) throws IOException{
Log.info("DOMAIN NAME "+realDomainName);
Connection connection = (connections.get(realDomainName) == null) ? client.connect(realDomainName) : connections.get(realDomainName);
if(!connection.isConnected()) {
connection.close();
sessions.remove(connection);
connection = client.connect(realDomainName);
}
// connection = client.connect(realDomainName);
connections.put(realDomainName,connection);
return connection;
}
private synchronized static Session getSession(Connection connection,SMBClient client){
Session session = sessions.get(connection);
if(session==null) {
PropertiesCache props = PropertiesCache.getInstance();
String sambaUsername = props.getProperty("smb.user");
String sambaPass = props.getProperty("smb.password");
String sambaDomain = props.getProperty("smb.domain");
Log.info("CLIENT " + client);
session = (sessions.get(connection) != null) ? sessions.get(connection) : connection.authenticate(new AuthenticationContext(sambaUsername, sambaPass.toCharArray(), sambaDomain));
sessions.put(connection, session);
}
return session;
}
@SuppressWarnings("UnusedReturnValue")
public synchronized static DiskShare getShare(String domainName, String shareName) throws SmbException
{
DiskShare share = shares.get(domainName+"/"+shareName);
if((share!=null)&&(!share.isConnected())) share=null;
if(share == null){
try {
PropertiesCache props = PropertiesCache.getInstance();
String sambaUsername = props.getProperty("smb.user");
String sambaPass = props.getProperty("smb.password");
String sambaDomain = props.getProperty("smb.domain");
String dfsIP = props.getProperty("smb.sambaIP");
SMBClient client = getClient();
String realDomainName = (Arrays.stream(DFSMounts).anyMatch(domainName::equals)) ? dfsIP: domainName;
Connection connection = getConnection(realDomainName);
Session session = getSession(connection,client);
share = (DiskShare) session.connectShare(shareName);
shares.put(domainName+"/"+shareName,share);
}
catch (Exception e){
Log.info("EXCEPTION E "+e);
Log.info("EX "+e.getMessage());
throw new SmbException();
}
}
return(share);
}
public static String fixFilename(String filename){
String[] parts = filename.split("\\\\");
ArrayList<String> partsList = new ArrayList<>(Arrays.asList(parts));
partsList.remove(0);
partsList.remove(0);
partsList.remove(0);
partsList.remove(0);
return String.join("/",partsList);
}
public static File open(String filename) throws SmbException {
String[] parts = filename.split("\\\\");
String domainName = parts[2];
String shareName = parts[3];
DiskShare share = getShare(domainName,shareName);
Set<SMB2ShareAccess> s = new HashSet<>();
s.add(SMB2ShareAccess.ALL.iterator().next());
filename = fixFilename(filename);
return(share.openFile(filename, EnumSet.of(AccessMask.GENERIC_READ), null, s, SMB2CreateDisposition.FILE_OPEN, null));
}
}
And here is how the OPEN is being used (to show it is closing the file after use):
String filename = documents.get(0).getUNCPath();
try (File f = Smb.open(filename)){
Process the file code...
f.closeSilently();
}
And:
while(i.hasNext()){
String filename = (String)i.next();
Log.info("FILENAME "+filename);
try(File f = Smb.open(filename)){
Process the file stuff here
}
}
I have created a PR for SMBJ which changes this. It will reuse the nested session for same host. I have successfully used it myself to avoid the exact same problem you are having. https://github.com/hierynomus/smbj/pull/489