VB6: Single-instance application across all user sessions

Question

I have an application that needs to be a single-instance app across all user sessions on a Windows PC. My research thus far has centered around using a mutex to accomplish this, but I am having an issue that I am not sure is really an issue, this is really a best-practice question I believe.

Here's the code first of all:

Private Const AppVer = "Global\UNIQUENAME" ' This is not what i am using but the name is unique

Public Sub Main()

    Dim mutexValue As Long

    mutexValue = CreateMutex(ByVal 0&, 1, AppVer)
    If (Err.LastDllError = ERROR_ALREADY_EXISTS) Then
        SaveTitle$ = App.Title
        App.Title = "... duplicate instance."
        MsgBox "A duplicate instance of this program exists."
        CloseHandle mutexValue
        Exit Sub
    End If
    ' Else keep on truckin'

Now, based on this article I believe I understand that by passing the NULL pointer to the CreateMutex function as I am above I'm basically assigning whatever security descriptor is associated with the currently logged in user.

If that means what I think it does (I may need more guidance here) that tells me that other users who log in will not be able to "see" the mutex created under the original user's session, nor will they be able to create a mutex with the same name.

Now, emperical evidence seems to back this up. I used a message box to pop the "LastDLLError" I was receiving, and when another user attempted to launch the application (while it was already running under another user account) I would receive an ERROR_ACCESS_DENIED code. I am OK with testing against this along with the ERROR_ALREADY_EXISTS code and just exiting on either/or. However, this feels sort of hackish and I'm wondering if someone can suggest an alternative. The "right" thing to do seems to be to pass the proper pointer to the CreateMutex function such that any user has the proper permissions to view any existing mutexes (mutices?), but I'm not so sure this is possible without the currently logged in user being an admin (which is unacceptible). Any assistance/guidance is greatly appreciated. Thanks in advance!

Answer 1

I was looking for a similar solution in VB6 late last year. At the time I was unable to find any examples of VB6 apps communicating across the user boundary, so I had to write my own.

See: Interprocess Communication via Semaphores

You can use the class to create and check for a global semaphore which will tell you if your app is already running under any user. I didn't look at the Mutex APIs but their usage is very similar. The GetSecurityDescriptor function is what you'll want to transpose if you've already got some Mutex code written.

Answer 2

I think your instincts are exactly right. I don't know any reason why it wouldn't be safe to infer from ERROR_ACCESS_DENIED that some other process has the mutex, so effectively it's the same as ERROR_ALREADY_EXISTS (in this context.) But at the same time, it doesn't feel quite right.

As you suggest, setting a proper security descriptor is indeed the right way to do it. MSDN says that granting MUTEX_ALL_ACCESS privileges increases the risk that the user will have to be an admin, and I think you do need MUTEX_ALL_ACCESS. But in my experience it works fine for non-admins.

Your question intrigued me enough do a quick test. That means I have some source code, and so here it is:

int wmain(int argc, wchar_t* argv[])
{
    ACL *existing_dacl = NULL;
    ACL *new_dacl = NULL;
    PSECURITY_DESCRIPTOR security_descriptor = NULL;

    bool owner = false;
    HANDLE mutex = CreateMutex(NULL,FALSE,L"Global\\blah");
    if(mutex == NULL)
        wprintf(L"CreateMutex failed: 0x%08x\r\n",GetLastError());
    if(GetLastError() == ERROR_ALREADY_EXISTS)
        wprintf(L"Got handle to existing mutex\r\n");
    else
    {
        wprintf(L"Created new mutex\r\n");
        owner = true;
    }

    if(owner)
    {
        //  Get the DACL on the mutex
        HRESULT hr = GetSecurityInfo(mutex,SE_KERNEL_OBJECT,
                                    DACL_SECURITY_INFORMATION,NULL,NULL,
                                    &existing_dacl,NULL,
                                    &security_descriptor);
        if(hr != S_OK)
            wprintf(L"GetSecurityInfo failed: 0x%08x\r\n",hr);

        //  Add an ACE to the ACL
        EXPLICIT_ACCESSW ace;
        memset(&ace,0,sizeof(ace));
        ace.grfAccessPermissions = MUTEX_ALL_ACCESS;
        ace.grfAccessMode = GRANT_ACCESS;
        ace.grfInheritance = NO_INHERITANCE;
        ace.Trustee.pMultipleTrustee = NULL;
        ace.Trustee.MultipleTrusteeOperation = NO_MULTIPLE_TRUSTEE;
        ace.Trustee.TrusteeForm = TRUSTEE_IS_NAME;
        ace.Trustee.TrusteeType = TRUSTEE_IS_WELL_KNOWN_GROUP;
        ace.Trustee.ptstrName = L"EVERYONE";
        hr = SetEntriesInAcl(1,&ace,existing_dacl,&new_dacl);
        if(hr != S_OK)
            wprintf(L"SetEntriesInAcl failed: 0x%08x\r\n",hr);

        //  Set the modified DACL on the mutex
        hr = SetSecurityInfo(mutex,SE_KERNEL_OBJECT,
                            DACL_SECURITY_INFORMATION,NULL,NULL,new_dacl,NULL);
        if(hr != S_OK)
            wprintf(L"SetSecurityInfo failed: 0x%08x\r\n",hr);
        else
            wprintf(L"Changed ACL\r\n");

        LocalFree(existing_dacl);
        LocalFree(new_dacl);
        LocalFree(security_descriptor);
    }

    wprintf(L"Press any key...");
    _getch();
    CloseHandle(mutex);
    return 0;
}

Answer 3

You don't need admin priveleges to set security on you own mutexes. Here is a simple demo app that basicly gives Everyone/Full control to the mutex.

Option Explicit

Private Const STANDARD_RIGHTS_REQUIRED              As Long = &HF0000
Private Const SYNCHRONIZE                           As Long = &H100000
Private Const MUTANT_QUERY_STATE                    As Long = &H1
Private Const MUTANT_ALL_ACCESS                     As Long = (STANDARD_RIGHTS_REQUIRED Or SYNCHRONIZE Or MUTANT_QUERY_STATE)
Private Const SECURITY_DESCRIPTOR_REVISION          As Long = 1
Private Const DACL_SECURITY_INFORMATION             As Long = 4

Private Declare Function CreateMutex Lib "kernel32" Alias "CreateMutexA" (lpMutexAttributes As Any, ByVal bInitialOwner As Long, ByVal lpName As String) As Long
Private Declare Function OpenMutex Lib "kernel32" Alias "OpenMutexA" (ByVal dwDesiredAccess As Long, ByVal bInheritHandle As Long, ByVal lpName As String) As Long
Private Declare Function CloseHandle Lib "kernel32" (ByVal hObject As Long) As Long
Private Declare Function InitializeSecurityDescriptor Lib "advapi32.dll" (pSecurityDescriptor As Any, ByVal dwRevision As Long) As Long
Private Declare Function SetSecurityDescriptorDacl Lib "advapi32.dll" (pSecurityDescriptor As Any, ByVal bDaclPresent As Long, pDacl As Any, ByVal bDaclDefaulted As Long) As Long
Private Declare Function SetKernelObjectSecurity Lib "advapi32.dll" (ByVal Handle As Long, ByVal SecurityInformation As Long, pSecurityDescriptor As SECURITY_DESCRIPTOR) As Long

Private Type SECURITY_DESCRIPTOR
    Revision            As Byte
    Sbz1                As Byte
    Control             As Long
    Owner               As Long
    Group               As Long
    pSacl               As Long
    pDacl               As Long
End Type

Private Const MUTEX_NAME            As String = "Global\20b70e57-1c2e-4de9-99e5-20f3961e6812"

Private m_hCurrentMutex         As Long

Private Sub Form_Load()
    Dim hMutex          As Long
    Dim uSec            As SECURITY_DESCRIPTOR

    hMutex = OpenMutex(MUTANT_ALL_ACCESS, 0, MUTEX_NAME)
    If hMutex <> 0 Then
        Call CloseHandle(hMutex)
        MsgBox "Already running", vbExclamation
        Unload Me
        Exit Sub
    End If
    m_hCurrentMutex = CreateMutex(ByVal 0&, 1, MUTEX_NAME)
    Call InitializeSecurityDescriptor(uSec, SECURITY_DESCRIPTOR_REVISION)
    Call SetSecurityDescriptorDacl(uSec, 1, ByVal 0, 0)
    Call SetKernelObjectSecurity(m_hCurrentMutex, DACL_SECURITY_INFORMATION, uSec)
End Sub

Private Sub Form_Unload(Cancel As Integer)
    If m_hCurrentMutex <> 0 Then
        Call CloseHandle(m_hCurrentMutex)
        m_hCurrentMutex = 0
    End If
End Sub