BASH: Process Substitution in a Variable

Question

What works is

vimdiff -c 'set diffopt+=iwhite' <(curl -sS '...') <(curl -sS '...')

Now I want to make a script that allows defining between two or four URLs and then generate the above command.

What I have for now:

#!/bin/bash

args=(-c 'set diffopt+=iwhite')
while [ -n "$1" ]; do
  args+=(<(curl -sS "$1"))
  shift
done

vimdiff "${args[@]}"

When run with two URLs it results in vimdiff showing two tabs without content and with the same filename at the bottom, e.g. "/proc/20228/fd/63". In other words the process substitution doesn't seem to be applied and/or wrong as I'd at least expect to see two different file descriptors in the filenames.

I guess that the problem is either in the args+=... line or in the last line.

Answer 1

This will do the trick, and it is very close to your original proposal:

#!/bin/bash

args=(-c 'set diffopt+=iwhite')
while [ -n "$1" ]; do
  exec {fdnum}< <(curl -sS "$1")
  args+=(/dev/fd/$fdnum)
  shift
done

vimdiff "${args[@]}"

So you'd first create all file descriptors via process substitution. By accessing their content via /dev/fd, you can easily collect them in an array and pass that to vimdiff.

Answer 2

This looks like a better approach to me:

declare -p args=()
while [ -n "$1" ]; do
  args+=("<(curl -sS ${1})")
  shift
done

bash -c "diff `declare -p args | grep -o -P '"<(.*?)"' | tr '\n' ' ' | tr -d '"'`"

This works :)

If you run your script using set -x at the beginning, you can see that when it arrives to executing the command, the two <(xxx) are inside a quote, thus interpreted as a string. This solves the issue. Clearly, using bash -c poses some security issues... however if the script is just for you, this should not be a big problem.

However, the script that uses the fifo posted in another answer seems cleaner :)

Answer 3

I think it's better to use named pipes (FIFOs) instead of <(...) to create temporary files for vimdiff, in my example,FIFOs are created for each URL provided and the output of the curl command is redirected to the corresponding named pipe and the named pipe paths are then passed as arguments to vimdiff!

#!/bin/bash

args=(-c 'set diffopt+=iwhite')
fifo_files=()

cleanup() {
  rm -f "${fifo_files[@]}"
}

trap cleanup EXIT

while [ -n "$1" ]; do
  #creating a named pipe (FIFO) and add it to the array
  fifo=$(mktemp -u)
  mkfifo "$fifo"
  fifo_files+=("$fifo")

  #using curl to write the output to the named pipe
  curl -sS "$1" > "$fifo" &

  shift
done

#pass the named pipes as arguments to vimdiff
vimdiff "${args[@]}" "${fifo_files[@]}"

Answer 4

I don't consider myself the Unix-/BASH-pro, so FIFO-pipes and the pipe-sequence are kind of beyond the solution. Don't get me wrong, it's nice to learn things.

Anyway, I just wanted to get my rather simple approach to work as it's basically just concatenating stuff and then running that, so here's my eval(-hell!!!):

args="vimdiff -c 'set diffopt+=iwhite'"
for url in "$@"; do
  args="$args <(curl -sS \"$url\")"
done

eval "$args"

It's not meant for production, just to check and compare.

Answer 5

This Shellcheck-clean code uses a recursive function to build a list of process substitutions:

#! /bin/bash -p

function vimdiff_urls
{
    local -r nurls=$1

    if (( nurls == 0 )); then
        vimdiff -c 'set diffopt+=iwhite' "${@:2}"
    else
        vimdiff_urls "$((nurls-1))" "${@:3}" <(curl -sS "$2")
    fi
}

vimdiff_urls "$#" "$@"

• The arguments to the function are a count of URLs, followed by a list of URLs, followed by a list of paths for process substitutions for curl fetching a URL.
• While any URLs remain, the function removes the first URL and adds a process substitution path for the URL to the end of the list of process substitution paths.
• This works because each process substitution path remains live until the function call where it was created completes.