How can I get a reference to the UI element that is currently hovered by an XRRayInteractor?

Question

I am working with XR Interaction Toolkit 2.2.0 and Unity 2022.2.7f1 and I have a Complete XR Origin Set Up Prefab in my Scene with some UI elements to click on (buttons with Tracked Device Ray Caster Script). I have a XR Device Simulator to move the controllers and the camera and everything works fine, the cursor turns blue when I hover a button and the button also changes color.

I need to get some properties from the hovered UI elements to make some changes on the controller appearence. Thus, it would be nice to have a script attached to the controller that can access the UI element currently being hovered.

I made a test script that I attached to the Ray Interactor to see if I could do it in a simple way :

using UnityEngine;
using UnityEngine.EventSystems;
using UnityEngine.XR.Interaction.Toolkit;

[RequireComponent(typeof(XRRayInteractor))]
public class AccessCurrentlyHoveredElement : MonoBehaviour
{

    private XRRayInteractor rayInteractor = null;

    void Start(){
        rayInteractor = GetComponent<XRRayInteractor>();
        rayInteractor.hoverEntered.AddListener(Test);
    }

    private void Test(HoverEnterEventArgs arg0)
    {
        Debug.Log("Hover Enter Triggered");
    }

    void Update(){
        if(rayInteractor.IsOverUIGameObject()){
            Debug.Log(rayInteractor.interactablesHovered.Count);
            Debug.Log(rayInteractor.GetOldestInteractableHovered());
        }
    }
}

My problem is that when I test it, the IsOverUIGameObject method works fine but the hoverEntered event is never triggered, the interactablesHovered is always empty and the GetOldestInteractableHovered always returns null . So I know when a UI element is being hovered but I cannot access it.

Is there a way to do it ?

Answer 1

So, one solution could be to attach an event trigger to each UI element that you want to track? That way you can call a custom function in your AccessCurrentlyHoveredElement script whenever the hover event is triggered.

You'd have to do this for each UI element:

• Select the UI button in the Scene view or the Hierarchy window and in the Inspector window, click the "Add Component" button and search for "Event Trigger".
• In the Event Trigger component, click the "+" button to add a new event and select "Pointer Enter" from the Event Type dropdown menu.
• Drag and drop the GameObject with the AccessCurrentlyHoveredElement script onto the "None (Object)" field.
• Then select the AccessCurrentlyHoveredElement script from the dropdown menu and select the "Test" function (or the function you want to call when the hover event is triggered) from the function dropdown menu.

Now, whenever the pointer enters the UI button, the Test function in your AccessCurrentlyHoveredElement script will be called, and you can access the currently hovered UI element from there.

To get info/ data from the UI element, you should be able to get everything you need using the eventData parameter of the event trigger method. So, you'd need to modify the Test method like so:

private void Test(BaseEventData eventData)
{
    // Get the GameObject that raised the event
    GameObject hoveredObject = eventData.selectedObject;

    // Do something with the hoveredObject, like print the name
    Debug.Log("Currently hovered object: " + hoveredObject.name);
}

Answer 2

Trying it out I see the problem myself, oops!

Re-thinking the whole thing, I went and did some searing about the UnityEngine.XR.Interaction.Toolkit

By using the XRUIInputModule (which is a component that comes with the XR Interaction Toolkit and is responsible for handling input events on UI elements), you can use this to get the currently hovered UI element and its properties. Make sure to have an object in your scene with an EventSystem and a XRUI Input Module component.

You can then create a script that references the XRUIInputModule and uses its GetCurrentRaycastResult method to get the RaycastResult of the currently hovered UI element. Here's an example script that you can attach to your controller:

using UnityEngine;
using UnityEngine.EventSystems;
using UnityEngine.XR.Interaction.Toolkit;
using UnityEngine.XR.Interaction.Toolkit.UI;

public class AccessCurrentlyHoveredElement : MonoBehaviour
{
    [SerializeField] private XRUIInputModule uiInputModule;
    [SerializeField] private XRRayInteractor rayInteractor;

    private void Start()
    {
        // You could find the references to the XRUIInputModule and XRRayInteractor components rather than setting them in the inspector
        //uiInputModule = FindObjectOfType<XRUIInputModule>();
        //rayInteractor = GetComponent<XRRayInteractor>();

        // Subscribe to the hoverEntered event of the rayInteractor
        rayInteractor.hoverEntered.AddListener(OnHoverEntered);
    }

    private void OnHoverEntered(HoverEnterEventArgs args)
    {
        // Get the current UI element that is being hovered
        GameObject hoveredObject = uiInputModule.GetCurrentRaycastResult().gameObject;

        // Do something with the hovered object, e.g. change its appearance
        hoveredObject.GetComponent<RectTransform>().sizeDelta = new Vector2(200, 200);
    }

    private void OnDestroy()
    {
        // Unsubscribe from the hoverEntered event to avoid memory leaks
        rayInteractor.hoverEntered.RemoveListener(OnHoverEntered);
    }
}

This script uses the XRUIInputModule component, which is added to the EventSystem object by the XR Interaction Toolkik. It then subscribes to the hoverEntered event of the XRRayInteractor component, and in the event handler it uses the GetCurrentRaycastResult method of the XRUIInputModule to get the RaycastResult of the currently hovered UI element. Finally, it does something with the hovered object

Answer 3

So I got it working in the end by modifying the code exemple from the UI Interaction Tool Kit UIInputModule doc . (The UIInputModule comes from the EventSystem object from the Complete Xr Origin Set Up). Here is the final code :

using UnityEngine;
using UnityEngine.EventSystems;
using UnityEngine.XR.Interaction.Toolkit.UI;

public class AccessCurrentlyHoveredElement: MonoBehaviour
{
    [SerializeField]
    UIInputModule inputModule;


    private void OnEnable()
    {
        if (inputModule != null)
        {
            inputModule.pointerEnter += OnDevicePointerEnter;

        }
    }

    private void OnDisable()
    {
        if (inputModule != null)
        {
            inputModule.pointerEnter -= OnDevicePointerEnter;
        }
    }

    // This method will fire after registering with the UIInputModule callbacks. The UIInputModule will
    // pass the PointerEventData for the device responsible for triggering the callback and can be used to
    // find the pointerId registered with the EventSystem for that device-specific event.
    private void OnDevicePointerEnter(GameObject selected, PointerEventData pointerData)
    {
        if (EventSystem.current.IsPointerOverGameObject(pointerData.pointerId))
        {
            Debug.Log($"Entered {EventSystem.current.currentSelectedGameObject} with pointer Id {pointerData.pointerId}", this);
        }
    }
}

One thing that is really important is to disable the Enable Mouse Input inside the XR UI Input Module component of the EventSystem if your are using XR Device Simulator because otherwise the EventSystem.current.currentSelectedGameObject will always be null for Pointers that are not the mouse.