Prevent TextField from resigning as first responder

Question

I'm have a login view with 2 text fields, username and password.

I am trying to create a functionality where I have a button to show / hide the password in the password field.

I set this up as follows:

ZStack(alignment: .trailing) {
    Group {
        if viewModel.isPasswordVisible {
            SecureField("", text: $viewModel.text)
                .keyboardType(.default)
                .focused($focusField, equals: .secure)
        }
        else {
            TextField("", text: $viewModel.text)
                .keyboardType(.emailAddress)
                .focused($focusField, equals: .plain)
        }
    }
    .autocorrectionDisabled()
    .textInputAutocapitalization(.none)
    .font(.system(size: Constants.Font.regularSize))
    .padding()
    .padding(.trailing, textFieldPaddingTrailing + numberOfSymbolsInTextField)
    
    HStack {
        if viewModel.isSecureField {
            Button(action: {
                viewModel.isPasswordVisible.toggle()
            }) {
                Text(Image(systemName: viewModel.isPasswordVisible ? Constants.Image.eye : Constants.Image.eyeSlash))
                    .font(.system(size: Constants.Font.symbolSize))
                    .foregroundColor(DMSColor.SwiftUIColor.primaryText)
            }
        }
    }
    .padding(.trailing)
}

Logic wise, things work as expected with the password field showing or hiding as the button is tapped, see below

The issue I have is that the keyboard also gets hidden as a result of my implementation.

I believe it is because when I toggle the isPasswordVisible, the view gets redrawn to show a regular text field and this isn't focused.

How can I implement this show / hide mechanism while also maintaining the current first responder. This means the keyboard and focus of the textfield should remain the same after the show / hide button was tapped.