In my code below:
I have 5 buttons added into a vertical scrollView. Each button is constrained to the scrollViews's top + 20 ,leading, trailing edges and its height. I have created a b1HeightConstraint variable. It's there to hold the heightConstraint of the b1 button. 
In a button click, I'm trying to remove this constraint. Yet I'm facing an odd issue:
When I log the constraints I only see 2 constraints, even though I've added 4 constraints to it. My the view debug hierarchy is like below:
import UIKit
import Foundation
class ViewController: UIViewController {
    var filterView: UIView!
    var scrollView: UIScrollView!
    var containerView: UIView!
    override func loadView() {
        filterView = UIView()
        view = filterView
        view.backgroundColor = #colorLiteral(red: 0.909803926944733, green: 0.47843137383461, blue: 0.643137276172638, alpha: 1.0)
        scrollView = UIScrollView()
        scrollView.backgroundColor = #colorLiteral(red: 0.474509805440903, green: 0.839215695858002, blue: 0.976470589637756, alpha: 1.0)
        view.addSubview(scrollView)
        scrollView.translatesAutoresizingMaskIntoConstraints = false
        scrollView.topAnchor.constraint(equalTo: view.topAnchor, constant: 0).isActive = true
        scrollView.leadingAnchor.constraint(equalTo: view.leadingAnchor).isActive = true
        scrollView.widthAnchor.constraint(equalTo: view.widthAnchor).isActive = true
        scrollView.heightAnchor.constraint(equalTo: view.heightAnchor, multiplier: 1).isActive = true
        scrollView.isScrollEnabled = true
        containerView = UIView()
        containerView.backgroundColor = #colorLiteral(red: 0.176470592617989, green: 0.498039215803146, blue: 0.756862759590149, alpha: 1.0)
        scrollView.addSubview(containerView)
        containerView.translatesAutoresizingMaskIntoConstraints = false
        // This is key:  connect all four edges of the containerView to
        // to the edges of the scrollView
        containerView.topAnchor.constraint(equalTo: scrollView.topAnchor).isActive = true
        containerView.leadingAnchor.constraint(equalTo: scrollView.leadingAnchor).isActive = true
        containerView.trailingAnchor.constraint(equalTo: scrollView.trailingAnchor).isActive = true
        containerView.bottomAnchor.constraint(equalTo: scrollView.bottomAnchor).isActive = true
        // Making containerView and scrollView the same height means the
        // content will not scroll vertically
        containerView.widthAnchor.constraint(equalTo: scrollView.widthAnchor).isActive = true
    }
    let b1 = Buttons(titleText: "one")
    let b2 = Buttons(titleText: "two")
    let b3 = Buttons(titleText: "three")
    let b4 = Buttons(titleText: "four")
    let b5 = Buttons(titleText: "five")
    var b1HeightConstraint : NSLayoutConstraint?
    override func viewDidLoad() {
        super.viewDidLoad()
        let buttonArray = [b1, b2, b3, b4, b5]
        b1.button.addTarget(self, action: #selector(ViewController.shrink(_:)), for: .touchUpInside)
        var startPoint = containerView.topAnchor
        for btn in buttonArray {
            let theBtn = btn.button
            containerView.addSubview(theBtn)
            theBtn.translatesAutoresizingMaskIntoConstraints = false
            theBtn.topAnchor.constraint(equalTo: startPoint, constant: 20).isActive = true
            theBtn.leadingAnchor.constraint(equalTo: containerView.leadingAnchor).isActive = true
            theBtn.trailingAnchor.constraint(equalTo: containerView.trailingAnchor).isActive = true
            theBtn.heightAnchor.constraint(equalTo: scrollView.heightAnchor).isActive = true
            startPoint = theBtn.bottomAnchor
            let btnHeight = theBtn.heightAnchor.constraint(equalTo: scrollView.heightAnchor)
            if btn == b1{
                b1HeightConstraint = btnHeight
            }
        }
        containerView.bottomAnchor.constraint(equalTo: startPoint, constant: 20).isActive = true
    }
    @objc func shrink(_ sender: Any){
        guard let btn = sender as? UIButton else{
            return
        }
        print("count is: \(btn.constraints.count)")
        btn.removeConstraint(b1HeightConstraint!)
        containerView.removeConstraint(b1HeightConstraint!)
        print("count is: \(btn.constraints.count)")
        containerView.updateConstraintsIfNeeded()
        containerView.updateConstraints()
        scrollView.updateConstraintsIfNeeded()
        scrollView.updateConstraints()
    }
}
class Buttons : NSObject {
    let button = UIButton()
    init(titleText: String) {
        button.backgroundColor = #colorLiteral(red: 0.976470589637756, green: 0.850980401039124, blue: 0.549019634723663, alpha: 1.0)
        button.setTitle(titleText, for: .normal)
    }
}
The code is ready to just be dumpped in the ViewController class. Works out of the box. My code is a spinoff of the code written here

                        
Here are several comments about your code:
removeConstraintwhen you didn't calladdConstraint). Your control over constraints is activating and deactivating them. Leave the adding and removing to iOS.These lines of code:
are creating a new constraint and assigning it to
b1HeightConstraint, but you never activated this constraint, so it hasn't have been added to any view at all. So trying to remove it was never going to work, because that constraint exists only in yourb1HeightConstraintproperty. Since it was never activated, it isn't actually constraining anything.If you want to shrink a button, you need to do one of these: a) modify the
constantproperty of its height constraint OR b) set its height constraint'sisActiveproperty tofalseand then give it a new height constraint OR c) modify the priorities of the active constraints to have Auto Layout choose to use different constraints.In your view debug hierarchy, all the constraints shown are active constraints (meaning they are available to be used by Auto Layout). The grayed out ones are the ones Auto Layout chose not to use because a higher priority constraint had precedence over it. This causes no conflict. The
self.height = 34 (content size)constraint is added by the system to account for content compression and content hugging.UIButtons resist compression with priority750and resist expansion with priority250. Theself.height = 34 (content size)constraint is grayed out because content hugging has a priority of250and another higher priority constraint was used instead (the constraint which sets the button's height equal to the scrollView's height has priority1000).Updated Code:
Here is your modified code. I changed two things:
b1HeightConstraintwas an activated constraint.shrinkmethod to deactivate the old height constraint and then create and activate a new one.Updated code
Options for shrinking the button's height
Setting the
constantproperty of the height constraintDeactivate the old constraint and create and activate a new one
Change the priority of the height constraint
Note: For this to work, you have to give the height constraint an initial priority less than
1000(999works nicely) because Auto Layout will not let you change the priority of an active constraint if it is required (priority1000).OR