Using JSON output in curl bash script

Question

I want to upload a file automatically to rackspace files which requires an auth-token that is updated daily, so I want to create a script which gets the auth token and then uses that in the script to upload the file.

This is the command to get the auth token which outputs the key perfectly:

curl -s -X POST https://auth.api.rackspacecloud.com/v2.0/tokens\
     -d '{ "auth":{ "RAX-KSKEY:apiKeyCredentials":{ "username":"USER", "apiKey":"KEY" } } }'\
     -H "Content-type: application/json" | python -mjson.tool |\
python -c 'import sys, json;\
           print json.load(sys.stdin)[sys.argv[1]][sys.argv[2]][sys.argv[3]]'\
access token id

This is the command to upload the file:

curl -X PUT -T file.xml -D - \
-H "Content-Type: text/xml" \
-H "X-Auth-Token: TOKENGOESHERE" \
URL

I need to get the token from the first command into the TOKENGOESHERE place in the second command.

What I have tried so far is:

token = curl -s -X POST https://auth.api.rackspacecloud.com/v2.0/tokens -d '{ "auth":{ "RAX-KSKEY:apiKeyCredentials":{ "username":"USER", "apiKey":"KEY" } } }' -H "Content-type: application/json" | python -mjson.tool | python -c 'import sys, json; print json.load(sys.stdin)[sys.argv[1]][sys.argv[2]][sys.argv[3]]' access token id

curl -X PUT -T file.xml -D - \
-H "Content-Type: text/xml" \
-H "X-Auth-Token: $token" \
URL

but it didn't work and I am guessing it has something to do with the quotes but I don't know enough about bash to know what the problem is.

Thanks!

Answer 1

This should work:

token=$(curl -s -X POST https://auth.api.rackspacecloud.com/v2.0/tokens \
    -d '{ "auth":{ "RAX-KSKEY:apiKeyCredentials":{ "username":"USER", "apiKey":"KEY" } } }' \
    -H "Content-type: application/json" \
    | python -mjson.tool \
    | python -c 'import sys, json; print json.load(sys.stdin)["access"]["token"]["id"]')

curl -X PUT -T file.xml -D - \
    -H "Content-Type: text/xml" \
    -H "X-Auth-Token: $token" \
    URL

Answer 2

This is the pattern which you should be using:

token=`cat /etc/passwd`
echo "file contents: $token"

Note, as triplee points out, that you must not have spaces on either side of the = sign.

Answer 3

I highly recommend skipping curl and using one of the language specific SDKs found on http://developer.rackspace.com

They all handle authentication easily and reauthentication for long lived processes. They all have examples of how to upload files too.

Answer 4

I know it's a bit off topic, but I wanted to share my 'workflow' which may help a lot of people.

If you download these two cool toys (replacement for curl and python's json):

• https://github.com/jkbr/httpie
• http://stedolan.github.io/jq/

Then you can do all these fun things:

(Just replace USER and KEY with your real user and key in the 1st line, and all the others are copy and paste-able.

Get the json:

json=$(echo '{ "auth":{ "RAX-KSKEY:apiKeyCredentials":{ "username":"USER", "apiKey":"KEY" } } }' | http POST https://auth.api.rackspacecloud.com/v2.0/tokens)

Get token with http:

token=$(echo $json | jq '.access | .token | .id' | sed s/\"//g)

Easy token usage for later:

auth="X-Auth-Token:$token"

Get endpoint for Sydney cloud files (change SYD for your favorite Datacenter) (change publicURL to internalURL if you're running from inside the DC):

url=$(echo $json | jq '.access | .serviceCatalog | .[] | select(.name == "cloudFiles") | .endpoints | .[] | select(.region == "SYD") | .publicURL' | sed s/\"//g)

-- Hard work is done. Now it gets easy --

Get list of containers:

http "$url" $auth

Create a container:

http PUT "$url/my_container" $auth

Upload a file:

cat python1.JPG | http PUT "$url/my_container/python1.jpg" $auth

List files:

http "$url/my_container"

Get CDN API URL (not the one for downloading, that's later):

cdn_url=$(echo $json | jq ' .access | .serviceCatalog | .[] | select(.name == "cloudFilesCDN") | .endpoints | .[] | select(.region == "SYD") | .publicURL' | sed s/\"//g)

CDN enable the container:

http PUT "$cdn_url/my_container" $auth "X-Cdn-Enabled: True"

Get public CDN url for my_container:

pub_url=$(http -h HEAD "$cdn_url/my_container" $auth  | awk '/X-Cdn-Uri/{print $2;}')

View your file:

firefox "$pub_url/python1.jpg"

All the API docs are here: http://docs.rackspace.com/files/api/v1/cf-devguide/content/API_Operations_for_Storage_Services-d1e942.html

Enjoy :)