powershell ignore character escaping with output from a different process

Question

I have a process running powershell commands with input from a config file. (In casu it's a drone CI executor on Windows with a drone YML file). This process sets an environment variable but I think this environment variable contains backticks which are interpreted as escape characters by powershell.

Since it's an OpenSSH key, this corrupts the key. Is there a way to treat the contents of the environment variable entirely as raw? Either during setting or reading (because since I don't know the key I do not know where it happens).

Drone CI config:

  - name: copy to archive on remote host
    commands:
      - Write-Output $env:SSH_KEY > ssh_key
      - scp -i ssh_key file.zip user@host:/path/to/file.zip
      - rm ssh_key
    environment:
      SSH_KEY:
        from_secret: ssh_key

I know it's a functional key because it works on my linux containers. But on Windows the format is deemed invalid.

Answer 1

The problem is the behavior of >, the redirection operator, in PowerShell, which is in effect an alias of the Out-File cmdlet:

• Character encoding:

  • In Windows PowerShell (powershell.exe), it defaults to creating UTF16-LE ("Unicode") files, which most command-line utilities do not recognize.

  • In PowerShell (Core) 7+ (pwsh), it now - fortunately - defaults to BOM-less UTF-8, which is now the consistent default for all cmdlets, as well as for the PowerShell engine itself with respect to readings source code.

  • As an aside: In Windows PowerShell and PowerShell (Core) up to v7.3.x, the PowerShell pipeline and > never act as a raw byte conduit for relaying raw data from external programs: such data is invariably decoded into a .NET string first, which > / Out-File then write to a file based on their default encoding. See this answer for background.

• Trailing newline:

  • In both editions, a platform-native newline is appended to the input (each input object) by default on writing the file, which can be suppressed with the -NoNewLine switch.

---

The simplest solution is to use New-Item instead, as it - surprisingly - creates BOM-less UTF-8 files even in Windows PowerShell, and never appends a trailing newline (-ItemType File is implied):^[1]

$env:SSH_KEY | New-Item -Force ssh_key

Note:

• -Force is needed in order to ensure that a preexisting file by that name is overwritten. However, -Force also creates a non-existent target directory on demand, which > / Out-File won't do.

• In Windows PowerShell, using Out-File (or Set-Content, which is generally preferable for text-only input)^[2] with -Encoding Utf8 is not an option, because the resulting file is invariably created with a BOM.

• Given that PowerShell (Core) 7+ defaults to BOM-less UTF-8, there you could alternatively use:

  $env:SSH_KEY | Set-Content -NoNewLine ssh_key
  

  • Assuming that the input string contains ASCII-range characters only, this will also work in Windows PowerShell, because Set-Content defaults to the ANSI code page, as determined by the legacy system locale, which is a superset of ASCII.^[3]

---

^{[1] Strictly speaking, this only applies if PowerShell's current location is one backed by the file-system provider - however, in practice it almost always is. If it isn't and you still want to create a file, either specify a full file-system path or prefix a relative one with FileSystem:: in order to target the file-system provider's current directory.}

^{[2] See this answer for background.}

^{[3] This implies, unfortunately, that in Windows PowerShell different cmdlets have different default encodings - see the bottom section of this answer for background.}