What is "Locked ownable synchronizers" in thread dump?

Question

I am trying to understand what does Locked ownable synchronizers refer to in a thread dump?

I started using ReentrantReadWriteLock have a thread in WAITING state, waiting for a ReentrantReadWriteLock$FairSync in the "locked ownable synchronizers" list of another thread in WAITING state (a ThreadPoolExecutor).

I couldn't find much information about that. Is it some kind of locks "passed onto" the thread? I'm trying to figure out where my deadlock comes from and I can't see any thread actively locking those (i.e. no corresponding - locked <0x...> in any stack trace).

Answer 1

TL;DR: write locks appear in the "ownable synchronizers" list, read locks don't.

I ended up with the following MVCE to try and understand what's with "ownable synchronizer". The idea was to have two threads locking/unlocking read/write reentrant locks and see the effect on different thread dumps at different timings (taken in jVisualVM while the Eclipse project was paused in breakpoints at specific lines).

Here is the code:

package lock;

public class LockTest {

    static ReentrantReadWriteLock lock = new ReentrantReadWriteLock(true);

    public static void main(String[] args) {
        lock.readLock().lock();
        System.out.println(Thread.currentThread().getName()+": read hold "+lock.getReadHoldCount()+" read lock "+lock.getReadLockCount());
        new Th().start();
        synchronized (LockTest.class) {
            try { LockTest.class.wait(); } catch (InterruptedException e) { }
        }
        lock.readLock().unlock();
        System.out.println(Thread.currentThread().getName()+": unlocked read lock. Read hold "+lock.getReadHoldCount()+" read lock "+lock.getReadLockCount()+". Getting write lock");
        lock.writeLock().lock();
        System.out.println(Thread.currentThread().getName()+": got write lock. Unlocking (=>Thread dump #3)"); // Take thead dump #3 here ("main" has a write lock, "other" has died)
        lock.writeLock().unlock();
    }

    static class Th extends Thread {
        Th() { super("other"); }

        public void run() {
            System.out.println(Thread.currentThread().getName()+": read hold "+lock.getReadHoldCount()+" read lock "+lock.getReadLockCount());
            if (!lock.writeLock().tryLock())
                System.out.println(Thread.currentThread().getName()+": cannot lock write");
            else {
                System.out.println(Thread.currentThread().getName()+": lock write taken");
                lock.writeLock().unlock();
            }
            System.out.println(Thread.currentThread().getName()+": trying to unlock read lock");
            try {
                lock.readLock().unlock();
                System.out.println(Thread.currentThread().getName()+": successfully unlocked read lock. Read hold "+lock.getReadHoldCount()+" read lock "+lock.getReadLockCount());
            } catch (IllegalMonitorStateException e) {
                System.out.println(Thread.currentThread().getName()+": cannot unlock read lock: "+e.getMessage());
            }
            synchronized (LockTest.class) {
                System.out.println(Thread.currentThread().getName()+": notifying write lock take (=>Thread dump #1)");
                LockTest.class.notify(); // Take thead dump #1 here ("main" has a read lock)
            }
            System.out.println(Thread.currentThread().getName()+": locking write lock");
            lock.writeLock().lock();
            System.out.println(Thread.currentThread().getName()+": unlocking write lock (=>Thread dump #2)"); // Take thead dump #2 here ("other" has a write lock)
            lock.writeLock().unlock();
        }
    }
}

Here is the output:

main: read hold 1 read lock 1
other: read hold 0 read lock 1
other: cannot lock write
other: trying to unlock read lock
other: cannot unlock read lock: attempt to unlock read lock, not locked by current thread
other: notifying write lock take (=>Thread dump #1)
other: locking write lock
main: unlocked read lock. Read hold 0 read lock 0. Getting write lock
other: unlocking write lock (=>Thread dump #2)
main: got write lock. Unlocking (=>Thread dump #3)

Now, thread dumps.

Thread dump #1 is taken when thread "main" got a read lock. As we can see, no "ownable synchronizer" is owned by the thread:

"main" prio=10 tid=0x00007fea5c00d000 nid=0x1866 in Object.wait() [0x00007fea65bd5000]
   java.lang.Thread.State: WAITING (on object monitor)
    at java.lang.Object.wait(Native Method)
    - waiting on <0x00000007acf62620> (a java.lang.Class for lock.LockTest)
    at java.lang.Object.wait(Object.java:503)
    at lock.LockTest.main(LockTest.java:14)
    - locked <0x00000007acf62620> (a java.lang.Class for lock.LockTest)

   Locked ownable synchronizers:
    - None

"other" prio=10 tid=0x00007fea5c0e0800 nid=0x1883 at breakpoint[0x00007fea3abe8000]
   java.lang.Thread.State: RUNNABLE
    at lock.LockTest$Th.run(LockTest.java:46)
    - locked <0x00000007acf62620> (a java.lang.Class for lock.LockTest)

   Locked ownable synchronizers:
    - None

Thread dump #2 is taken after thread "other" has taken the write lock. It appears in the "ownable synchronizers":

"main" prio=10 tid=0x00007fea5c00d000 nid=0x1866 waiting on condition [0x00007fea65bd5000]
   java.lang.Thread.State: WAITING (parking)
    at sun.misc.Unsafe.park(Native Method)
    - parking to wait for  <0x00000007acf63278> (a java.util.concurrent.locks.ReentrantReadWriteLock$FairSync)
    at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
    at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:834)
    at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:867)
    at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1197)
    at java.util.concurrent.locks.ReentrantReadWriteLock$WriteLock.lock(ReentrantReadWriteLock.java:945)
    at lock.LockTest.main(LockTest.java:18)

   Locked ownable synchronizers:
    - None

"other" prio=10 tid=0x00007fea5c0e0800 nid=0x1883 at breakpoint[0x00007fea3abe8000]
   java.lang.Thread.State: RUNNABLE
    at lock.LockTest$Th.run(LockTest.java:51)

   Locked ownable synchronizers:
    - <0x00000007acf63278> (a java.util.concurrent.locks.ReentrantReadWriteLock$FairSync)

Thread dump #3 is taken after thread "other" has released the write lock (and died), and thread "main" has taken it:

"main" prio=10 tid=0x00007fea5c00d000 nid=0x1866 at breakpoint[0x00007fea65bd5000]
   java.lang.Thread.State: RUNNABLE
    at lock.LockTest.main(LockTest.java:19)

   Locked ownable synchronizers:
    - <0x00000007acf63278> (a java.util.concurrent.locks.ReentrantReadWriteLock$FairSync)

So write locks will appear in the list of "locked ownable synchronizers", when read locks won't. Even though getReadHoldCount() shows the number of read locks taken by the current thread, a read "locking" doesn't seem to belong to a particular thread and is therefore absent from the list. And that makes it difficult to debug deadlocks (or let's say "not as easy as with jVisualVM").

EDIT: To help figuring out copy/paste errors with locks taken and not released such as in:

myLock.readLock().lock();
try {
    // ...
} finally {
    myLock.readLock().lock(); // Oops! Should be "unlock()"
}

you can use the following Linux command line at the root of your source directory:

find . -name '*.java' -exec grep -Hn 'myLock.readLock().lock();' {} \; | wc -l

will display how many read locks are taken, and:

find . -name '*.java' -exec grep -Hn 'myLock.readLock().unlock();' {} \; | wc -l

will display how many read locks are released. If numbers don't match, remove the | wc -l to show the details of file names (grep -H) and line number (grep -n).

Answer 2

Correct use ReentrantLock it isn't so easy as it seems. It has several pitfalls. If we speak about deadlocks i think you need to know :

1.

The main explanation we found at this point is associated with the usage of the ReentrantLock READ lock. The read locks are normally not designed to have a notion of ownership. Since there is not a record of which thread holds a read lock, this appears to prevent the HotSpot JVM deadlock detector logic to detect deadlock involving read locks.

Some improvements were implemented since then but we can see that the JVM still cannot detect this special deadlock scenario.

It's from nice article "Java concurrency: the hidden thread deadlocks"

If you have access to source code getReadHoldCount() method can help in investigation deadlocks.

2. Correct upgrade from readLock to writeLock - "Java ReentrantReadWriteLocks - how to safely acquire write lock?"

Answer 3

From Java 7 documentation:

An ownable synchronizer is a synchronizer that may be exclusively owned by a thread and uses AbstractOwnableSynchronizer (or its subclass) to implement its synchronization property. ReentrantLock and ReentrantReadWriteLock are two examples of ownable synchronizers provided by the platform.