There is a project on GitHub with AppVeyor check enabled for pull requests. When I create a pull request and wait for the check to complete, the PR is marked as green:
In the same time there is a possibility to merge a PR when the check is in progress. In this case the PR is displayed as failed, but the changes (potentially undesirable) are merged:
Is it possible to make the check required and prevent unvalidated pull requests from being completed?
I think protected branches is what you need, if I understood you correctly.