Deadlock checking screen DPI (with AWT) while inside GlimpsePainter.paint()

[stacktracer]

Deadlock can occur if a method that grabs the AWT lock (e.g. java.awt.Toolkit.getScreenResolution()) is called from inside GlimpsePainter.paint().

Event thread:

• Holds AWT lock (while handling a mouse event, e.g.)
• Waits for GlimpseLayout lock (to determine which GlimpseTarget the mouse is in, e.g.)

Render thread:

• Holds GlimpseLayout lock (while rendering)
• Waits for AWT lock (to get screen resolution, e.g.)

This causes issues with SVG Salamander, because it checks screen DPI when loading an SVG.

[stacktracer]

Maybe the GlimpseLayout lock could become a read/write lock? Hold the write lock only while doing re-layout, then downgrade to read lock during paint? (Assuming the event thread only needs a read lock.) That way you can implement paint (common case) without having to worry about the AWT lock.

[stacktracer]

Well, I can't get it to happen again.

Stack traces of the deadlocked threads during the original occurrence are here.

In the tests I'm running now, the event thread does not hold the AWT lock while handling events. Also, event handling stacktraces are now starting at DefaultEDTUtil.java:326, where in the deadlock it was DefaultEDTUtil.java:349. I guess JOGL 2 has two different code paths for event dispatch ... and apparently one of them can end up owning the AWT lock.

[stacktracer]

In the render thread:

• CanvasA is already visible
• render-thread is painting CanvasA
  • holds GlimpseLayout lock
  • needs AWT lock (to get screen resolution)

Meanwhile, in the event thread:

• CanvasB is being created and added to the AWT window
• event-thread is making CanvasB visible
• event-thread calls WindowImpl.createNative, which does:
  • lockSurface
    • grabs the AWT lock
  • waitForVisible
    • spin-waits
    • while spin-waiting, calls dispatchMessages
      • dispatch triggers a re-layout, which tries to grab the GlimpseLayout lock
      • DEADLOCK
  • unlockSurface
    • would release the AWT lock, but we never make it here

So the event thread can indeed hold the AWT lock while event listeners run. But only while a new canvas is being created.

[stacktracer]

Added WaitForVisibleDeadlockExample in wfv-deadlock branch.

To summarize ... you can get deadlock if: you move your mouse in an existing glimpse-canvas while a new glimpse-canvas is becoming visible, and a painter simultaneously tries to acquire the AWT lock (e.g. by calling getScreenResolution).

[stacktracer]

Changing the GlimpseLayout lock to a read/write lock would fix this specific deadlock, but would not make everything work in general -- any painter that uses its own lock during event handling and rendering would encounter this same problem.

Current lock acquisition order goes ("AWT" = AWT lock, "Layout" = GlimpseLayout lock, "Painter" = hypothetical lock in a painter):

• Event Thread: AWT, Layout, Painter
• Render Thread: Layout, Painter, AWT

I see the following ways to avoid deadlock:

1. Change JOGL, somehow, to avoid holding the AWT lock during event dispatch
   • Event Thread: AWT, Layout, Painter
   • Render Thread: Layout, Painter, AWT
2. Painters never request the AWT lock during paint
   • Event Thread: AWT, Layout, Painter
   • Render Thread: Layout, Painter, AWT
3. Render thread preemptively grabs the AWT lock before grabbing the layout lock
   • Event Thread: AWT, Layout, Painter
   • Render Thread: AWT, Layout, Painter, AWT(reentrant)

Not clear whether Solution 1 is possible.

Solution 2 would be an onerous restriction.

Solution 3 would make all canvases unresponsive while a new one was being added, and would mean that only one canvas could render at a time.

[ghost]

Do you reproduce this deadlock with NewtCanvasAWT?

[stacktracer]

Yes, reproducible with NewtCanvasAWT: https://gist.github.com/stacktracer/11266465

In that snippet, the NewtSwingGlimpseCanvas contains a NewtCanvasAWT. (The snippet depends on Glimpse 2.1.1 ... sorry it's not a more self-contained example.) Glimpse 2.1.1 depends on JOGL 2.1.3, but deadlock still occurs if I bump the JOGL dependency up to JOGL 2.1.5-01.

Run that snippet, move your mouse around in the window, and it deadlocks consistently.