Exclude logo from being affect by `mix-blend-mode: difference` in header

Question

My header contains navigation and a logo. I've set mix-blend-mode on the .header so as the page is scrolled, the text within the header will change colour depending on the background.

The problem with this is, all the child elements are affected by the difference declaration. More importantly the logo, which should maintain the same colour/styling as it has by default.

Is there a way to exclude this for the mix-blend-mode property so it doesn't change? I've tried setting a pseudo :before / :after element behind the logo with absolute positioning to 'block out' the area behind the logo but everything I've tried so far doesn't work.

Is there a way to do this without removing the logo from the .header entirely?

I'm under the assumption that the mix-blend-mode property needs to be on the top-level parent and applying it to the nav alone will not work (which it doesn't).

body {
  background: white;
  margin: 0;
  padding: 0;
}

.header {
  color: white;
  display: flex;
  align-items: center;
  justify-content: space-between;
  height: 64px;
  mix-blend-mode: difference;
  position: sticky;
  top: 0;
  z-index: 10;
}

.site-logo svg {
  height: 64px;
  width: 64px;
}

.header ul {
  display: flex;
  list-style: none;
}

.header li {
  margin-right: 32px;
}

<header class="header">
  <a href="#" class="site-logo">
    <svg xmlns="http://www.w3.org/2000/svg" xml:space="preserve" id="Layer_1" x="0" y="0" style="enable-background:new 0 0 64 64" version="1.1" viewBox="0 0 64 64"><style>.st0{fill:#fff}</style><path d="M0 0h64v64H0z"/><path d="M9.46 22.06h1.96V35.6h6.49v1.64H9.46V22.06zM29.58 31.7c0 4.03-2.79 5.79-5.43 5.79-2.95 0-5.23-2.16-5.23-5.61 0-3.65 2.39-5.79 5.41-5.79 3.13 0 5.25 2.27 5.25 5.61zm-8.65.11c0 2.39 1.37 4.19 3.31 4.19 1.89 0 3.31-1.78 3.31-4.23 0-1.85-.92-4.19-3.27-4.19-2.34-.01-3.35 2.16-3.35 4.23zM41.47 26.34c-.05.79-.09 1.67-.09 3v6.33c0 2.5-.5 4.03-1.55 4.98-1.06.99-2.59 1.31-3.96 1.31-1.31 0-2.75-.32-3.63-.9l.5-1.51c.72.45 1.85.86 3.2.86 2.03 0 3.51-1.06 3.51-3.81v-1.22h-.05c-.61 1.01-1.78 1.82-3.47 1.82-2.7 0-4.64-2.3-4.64-5.32 0-3.69 2.41-5.79 4.91-5.79 1.89 0 2.93.99 3.4 1.89h.05l.09-1.64h1.73zm-2.05 4.3c0-.34-.02-.63-.11-.9-.36-1.15-1.33-2.09-2.77-2.09-1.89 0-3.24 1.6-3.24 4.12 0 2.14 1.08 3.92 3.22 3.92 1.22 0 2.32-.77 2.75-2.03.11-.34.16-.72.16-1.06v-1.96zM54.54 31.7c0 4.03-2.79 5.79-5.43 5.79-2.95 0-5.23-2.16-5.23-5.61 0-3.65 2.39-5.79 5.41-5.79 3.13 0 5.25 2.27 5.25 5.61zm-8.65.11c0 2.39 1.37 4.19 3.31 4.19 1.89 0 3.31-1.78 3.31-4.23 0-1.85-.92-4.19-3.27-4.19-2.34-.01-3.35 2.16-3.35 4.23z" class="st0"/></svg>
  </a>
  <ul>
    <li>Link 1</li>
    <li>Link 2</li>
    <li>Link 3</li>
  </ul>
</header>

<main>
  <img src="https://images.pexels.com/photos/18235303/pexels-photo-18235303/free-photo-of-autumn-s-reflection.jpeg">
  <img src="https://images.pexels.com/photos/9367323/pexels-photo-9367323.jpeg">

</main>