I want to assemble a curl invocation with two variables (one for common arguments and one for the URL) and capture the output via command substitution. This breaks when I add parameters that need enclosing quotes, like say -H "Accept: text/html".
Here's a small script to demonstrate the problem, which calls a public echo service to show the sent request headers:
#!/bin/bash
COMMON_ARGS="--http1.1 -H \"Accept: text/html\""
URL="http://scooterlabs.com/echo"
RETVAL=$(curl -s $COMMON_ARGS $URL | grep Accept)
echo "1 $RETVAL"
RETVAL=$(curl -s --http1.1 -H "Accept: text/html" $URL | grep Accept)
echo "2 $RETVAL"
Output:
1 [Accept] => */*
2 [Accept] => text/html
So the first attempt where I try to supply the header via a variable, is not working (curl sends the default Accept: */* ), whereas the second attempt, where I manually added the header, works. Neither using single quotes nor quote escaping did help.
Update: I learnt at least a part of my lesson here when I stumbled upon a BashFaq/050 just before submitting the question. The problem is word splitting performed by the shell's parser before it invokes a given command. The correct way to do it my case is by supplying the parameters as an array (older bashes may not support this):
COMMON_ARGS=(--http1.1 -H "Accept: text/html")
RETVAL=$(curl -s "${COMMON_ARGS[@]}" $URL | grep Accept)
echo "3 $RETVAL"
Output:
3 [Accept] => text/html
But there's still one thing I couldn't figure out: If word splitting by default acts on a space, a tab, and a newline. then why doesn't this variant work - there's no space to wrongly split?
COMMON_ARGS="--http1.1 -H 'Accept:text/html'"
RETVAL=$(curl -s $COMMON_ARGS $URL | grep Accept)
echo "4 $RETVAL"
Output:
4 [Accept] => */*
But there are the single quotes which would be normally removed in quote removal before curl saw the header. By including them in double quotes, they're preserved, but curl doesn't know what to do with them.