Angularjs forms dynamic validity states of elements

Question

Angularjs 1.3-beta.8

This is a table of inputs, which are embedded in a form that contains other form elements. This question is in regards to this table only. Here are a few things to keep in mind.

There are no other nested forms, just one master form, which we will call masterForm and contains ALL form elements, not just this table.

Rows are added dynamically to the form using a button you cannot see. Rows can also be deleted by the red circle on the left of the table.
There may be hundreds of rows, or only one; but there must be at least one row.

The only rows that are relevant are the ones that have a payment amount entered.
Each input may have no validation or some validation (dynamically assigned from the server per input).

This means that each individual input will differ and could be this

$scope.masterForm.thisInput.$error = {required: true, format: true, somethingnew: true, etc...}

or it could be nothing as seen in the table picture; only the 'Account Number' and 'Total Due' fields have validation errors.

Question: How can I disregard the input validations in each row input, THAT HAS VALIDATION SET, and only when the 'Payment Amount' is filled in?

The question is a little misleading, as I know how to trigger a state change in Angularjs, however everything I have tried does not change the form validation.

Here is what I have tried -

I did search SO, but did not find an answer that worked in this case.

When the form is submitted, it flags all inputs that do not pass validation. I have hijacked the submit function to first run through the form object and $setValidity state of each "named" input, like so

loop through $scope.masterForm and look for input name fields
  When you find the relevant fields of table inputs that have no payment amounts
    then run through each input.$error and then input.$setValidity of each
      $error (like required: true), set these to false (like required: false) - thanks

This does set each input to valid -

input.$valid (is true)
input.$invalid (is false)
input.$error (are all set to false)

However, all the inputs still show invalid as shown in the table.

There is also the masterForm object which does contain each input name object. In the form object there are also masterForm.$error objects which are the same as the input.$error objects, but just that ALL the form $error objects are gathered together. I have also tried to change these values, which does happen to the inputs I wanted to change, but the form is still invalid.

I have also tried to ng-if each row, meaning I removed them on submit if they didn't meet requirements, but still the form see's them and states the form is still invalid.

Even though I can setValidity for each input, the form does not care. Is this because I have to change the $error state in the form AND the input object? Shouldn't they change state together?

Is this just because I'm using 1.3-beta.8? Is there an easier way to to this? Could I have more control using the $validators pipeline available in 1.3-beta.12+?

There is no need for code as I have none that works. If you know how to do this please provide YOUR example.