More efficient way to call ListAdapter's submitList to update RecyclerView after drag and drop

Question

I am using the ItemTouchHelper class to support drag and drop in my RecyclerView. I have an implementation that works as intended but it seems to be very inefficient. In my onMoved method, I get the current version of the list, swap the item that is dragged, and then call submitList with the new version of the list. The problem is, onMoved is called each time a swap occurs, and swaps execute one by one as they are dragged in the list. So if an item at position 1 gets dragged to position 11, onMoved is called at each swap from (1 -> 2) , (2 -> 3) and so on. This means I get a new version of the list and call submitList 10 times by the time the drag is complete.

What I've tried:

I've tried calling submitList with the updated list once the drag is complete but it doesn't change. I have logged the adapter's getCurrentList() before calling submitList and it shows the original unchanged list (which is expected). I have logged the MainActivity's currentList variable and it shows the updated swapped list. I pass the currentList variable to submitList then log the adapter's getCurrentList() and it still shows the original unchanged list. I have read this post but none of the suggested solutions work. Any help would be appreciated!

EDIT: I am also using DiffUtil in the adapter.

MainActivity.java

...

private void createItemTouchHelper() {
    new ItemTouchHelper(new ItemTouchHelper.SimpleCallback(
        ItemTouchHelper.UP | ItemTouchHelper.DOWN,
        ItemTouchHelper.START | ItemTouchHelper.END) {

    // To track changes when user drags
    List<ListItem> currentList;

    // This method is called when user starts dragging an item
    public void onSelectedChanged(@Nullable RecyclerView.ViewHolder viewHolder, int actionState) {
      if (actionState == ItemTouchHelper.ACTION_STATE_DRAG) {

        // get the current list from the adapter to modify
        // have to create new list because getCurrentList() returns an unmodifiable list
        currentList = new ArrayList<>(adapterMain.getCurrentList()); 
      }
    }

    // This method gets called when user releases the dragged item
    public void clearView(@NonNull RecyclerView recyclerView,
                            @NonNull RecyclerView.ViewHolder viewHolder) { 

      adapterMain.submitList(currentList); // Does not update the list in the adapter
    }

    // gets called each time a row is dragged by one position
    public void onMoved(@NonNull RecyclerView recyclerView,
                          @NonNull RecyclerView.ViewHolder source, int fromPos,
                          @NonNull RecyclerView.ViewHolder target, int toPos, int x, int y) {

        Collections.swap(currentList, toPos, fromPos); // update the local ArrayList

        // This "visually" moves the item in RecyclerView but it doesn't update the list in the adapter
        adapterMain.notifyItemMoved(fromPos, toPos); 
    }
  }).attachToRecyclerView(recyclerMain);
}

This is the working version of onMoved method. I don't have to put any code in onSelectedChanged() or clearView() to make it work. Seems highly inefficient though...

public void onMoved(@NonNull RecyclerView recyclerView,
                          @NonNull RecyclerView.ViewHolder source, int fromPos,
                          @NonNull RecyclerView.ViewHolder target, int toPos, int x, int y) {


        // adapterMain.getCurrentList() returns an unmodifiable list, convert to new ArrayList
        currentList = new ArrayList<>(adapterMain.getCurrentList());
        Collections.swap(currentList, toPos, fromPos);
        adapterMain.submitList(currentList); // why does submitList work here but not in the clearView method?
        // adapterMain.notifyItemMoved(fromPos, toPos); // this is not needed when I call submitList here
      }

RecyclerAdapterMain.java

public class RecyclerAdapterMain extends ListAdapter<ListItem, RecyclerAdapterMain.ListItemHolder> {

  public RecyclerAdapterMain() {
    super(DIFF_CALLBACK);
  }

    private static final DiffUtil.ItemCallback<ListItem> DIFF_CALLBACK = new DiffUtil.ItemCallback<ListItem>() {
    @Override
    public boolean areItemsTheSame(@NonNull ListItem oldItem, @NonNull ListItem newItem) {
      return oldItem.getId() == newItem.getId();
    }

    @Override
    public boolean areContentsTheSame(@NonNull ListItem oldItem, @NonNull ListItem newItem) {
      return oldItem.equals(newItem);
    }
  };

...

}