Apache .htaccess is not listening to whitelisting

Question

I've setup an .htaccess file to implement a password protected environment. The idea is that every user that lands on this domain will be asked to login with credentials (using .htpasswd). The only exception is that when I go to that domain, I'm granted access immediately. At this moment, everyone has to give credentials, so my whitelisting is not working properly..

Can anybody spot the mistake I've made?

        AuthType Basic
        AuthName "Name"
        AuthUserFile /var/www/user/magento2/pub/.htpasswd

        ## IP Whitelist
        # Allow access for this ip
        <RequireAny>
          Require valid-user
          Require ip <my ip>
        </RequireAny>