Android RecyclerView with ViewBinding - notifyItemChanged only works the first time

Question

So I have my RecyclerView setup with ViewDataBinding and I'm showing a ProgressBar on click of a button inside the item. The ProgressBar shows and pops a dialog, on cancelling dialog I can notifyItemChanged to hide the ProgressBar, it only works the first time.

Here is my layout of the item:

    <?xml version="1.0" encoding="utf-8"?>
<layout>
    <data>
        <import type="android.view.View"/>

        <variable
            name="inProgress"
            type="boolean" />

        <variable
            name="userModel"
            type="com.smartsco.recaru.room.dataModels.UserModel" />

        <variable
            name="messageModel"
            type="com.smartsco.recaru.room.dataModels.MessageModel" />

        <variable
            name="clickHandler"
            type="com.smartsco.recaru.interfaces.OnClickHandler" />
    </data>
<RelativeLayout
    xmlns:android="http://schemas.android.com/apk/res/android"
    xmlns:app="http://schemas.android.com/apk/res-auto"
    android:layout_width="match_parent"
    android:layout_height="wrap_content"
    android:paddingTop="8dp"
    android:paddingBottom="8dp">

    <androidx.cardview.widget.CardView
        android:id="@+id/ll_invoice"
        android:layout_width="@dimen/ml_invoice_width"
        android:layout_height="wrap_content"
        android:layout_alignParentStart="true"
        android:layout_alignParentTop="true"
        android:layout_marginStart="8dp"
        android:clipChildren="false"
        android:clipToPadding="false"
        android:orientation="vertical"
        app:cardBackgroundColor="@color/bg_ml_invoice"
        app:cardCornerRadius="2dp"
        app:cardElevation="3dp"
        app:cardUseCompatPadding="true">

        <LinearLayout
            android:layout_width="match_parent"
            android:layout_height="wrap_content"
            android:background="@color/bg_ml_invoice"
            android:gravity="center_horizontal"
            android:orientation="vertical"
            android:padding="4dp">


            <RelativeLayout
                android:layout_width="match_parent"
                android:layout_height="wrap_content">

                <LinearLayout
                    android:id="@+id/ll_invoice_buttons"
                    android:layout_width="match_parent"
                    android:layout_height="wrap_content"
                    android:clipChildren="false"
                    android:clipToPadding="false"
                    android:orientation="horizontal"
                    android:padding="8dp"
                    android:visibility="@{!inProgress &amp;&amp; messageModel.InvoiceData.status.equalsIgnoreCase(`pending`) ? View.VISIBLE : View.INVISIBLE}"
                    android:weightSum="2">

                    <Button
                        style="@style/Button_Invoice_Positive"
                        android:id="@+id/btn_agree"
                        android:layout_width="match_parent"
                        android:layout_height="@dimen/btn_invoice_height"
                        android:layout_weight="1"
                        android:text="@string/agree"
                        android:onClick="@{(v)-> clickHandler.onClick(v)}"/>

                    <Space
                        android:layout_width="6dp"
                        android:layout_height="0dp"/>

                    <Button
                        android:id="@+id/btn_disagree"
                        style="@style/Button_Invoice_Negative"
                        android:layout_width="match_parent"
                        android:layout_height="@dimen/btn_invoice_height"
                        android:layout_weight="1"
                        android:text="@string/disagree"
                        android:onClick="@{(v)-> clickHandler.onClick(v)}" />

                </LinearLayout>

                <ProgressBar
                    android:id="@+id/pb_invoice"
                    android:layout_width="40dp"
                    android:layout_height="40dp"
                    android:layout_centerInParent="true"
                    android:padding="8dp"
                    android:visibility="@{inProgress ? View.VISIBLE : View.INVISIBLE}" />

            </RelativeLayout>

        </LinearLayout>

    </androidx.cardview.widget.CardView>

</RelativeLayout>
</layout>

Here is how the ViewHolder looks like:

    private class InvoiceMessageHolder extends RecyclerView.ViewHolder implements OnClickHandler {
    private RowRecyclerViewChatInvoiceMessageBinding binding;

    InvoiceMessageHolder(RowRecyclerViewChatInvoiceMessageBinding binding) {
        super(binding.getRoot());
        this.binding = binding;
    }

    void bind(MessageModel message, UserModel userModel) {
        binding.setMessageModel(message);
        binding.setUserModel(userModel);
        binding.setClickHandler(this);
    }

    @Override
    public void onClick(View v) {
        switch (v.getId()){
            case R.id.btn_agree:
            case R.id.btn_disagree:
                binding.setInProgress(true);
                invoiceClickHandler.onClick(v, getAdapterPosition(), binding.getMessageModel());
                break;
        }
    }
}

as you can see I have created a variable inside the layout itself to toggle inProgress boolean which works perfectly except won't reset itself on 2nd call of notifyItemChanged.

Another weird behaviour I have witnessed is that sometimes I click on button inside one item and it shows progress bar in all the viewholders for that itemViewType.

P.S.On calling notifyItemChanged, the view animates as if it was notified of a change but is only effective the first time.

Thanks