Built-in/portable method of appending string to file via bash without redirection

80 views Asked by At

I've got a debug() function written in bash which aims to:

  1. be portable across GNU linux boxes and MacOS
  2. be usable inside loops such as while ...; do ...; done < foo where redirection is used
  3. write messages to /dev/stderr OR a log file specified by $DEBUG_LOG

At present, it basically follows the (simplified) format:

# @description Write debug messages to $DEBUG_LOG if set, else /dev/stderr
# @usage debug <message>
function debug() {
    if [[ "${DEBUG}" == "true" ]]; then
        # Use $DEBUG_LOG if set, else /dev/stderr
        local log_file="${DEBUG_LOG:-/dev/stderr}"

        # Join all arguments using \n and prepend every line with "[debug] "
        local debug_message=$(printf "%s\n" "${@}" | sed 's/^/[debug] /')

        # Append the debug message to the log file
        echo "${debug_message}" | dd of="${log_file}" conv=notrunc oflag=append status=none
    fi
}

debug example

full code for the curious

Main problem

The obvious echo "${debug_message}" >> "${log_file}" isn't an option. It fails on scenario #2. When debug foo is called inside a while ... done < file.txt loop, using redirection inside the debug() function will eat STDIN and break the loop.

From that, I've determined that I need some command that I can instead pipe to and append to either a log file or STDERR, and dd worked great until I tried using it on MacOS Apparently the BSD version of dd doesn't allow the oflag=append option, which is fairly crucial for appending to a log file...


Ideal solution

Ideally, there would be some basic command append-stuff such that I can do one of the following without redirection:

echo "${debug_message}" | append-stuff "${DEBUG_LOG:-/dev/stderr}"
append-stuff "${debug_message}" "${DEBUG_LOG:-/dev/stderr}"

Approaches examined thus far

debug_message=$'hello world\nsome |[]& weird ch4r4ct3r$!\nfoo bar'

sed

I've tried variations on:

sed -e '$a '"${debug_message}" "${DEBUG_LOG:-/dev/stderr}"
# sed: -e expression #1, char 42: unterminated `s' command

sed -e '$a helloworld' /dev/stderr
# hangs

echo "${debug_message}" | sed -e '$r /dev/stdin' "${DEBUG_LOG:-/dev/stderr}"
# hangs

awk -i inplace

I've played with a few different variations of echo "${debug_message}" | awk -i inplace ... /dev/stdin ... /dev/stderr, but they all either printed warnings and/or hung the shell. More importantly, the inplace extension doesn't seem to be built-in / portable.

dd

echo "${debug_message}" \
  | dd of="${DEBUG_LOG:-/dev/stderr}" conv=notrunc oflag=append status=none
# works, but oflag=append is not available on MacOS, and there doesn't seem
# to be an equivalent flag in the BSD version of dd

I don't know if there's a suitable append-stuff sort of command like I'm looking for that's either built-in to bash or part of the standard suite of *nix tools one can reasonably expect to find on any box, but if someone here knows of such a thing, I'd super appreciate it

1

There are 1 answers

1
Diego Torres Milano On

Using your own example

#! /bin/bash

# @description Write debug messages to $DEBUG_LOG if set, else /dev/stderr
# @usage debug <message>
function debug() {
    if [[ "${DEBUG}" == "true" ]]; then
        # Use $DEBUG_LOG if set, else /dev/stderr
        local log_file="${DEBUG_LOG:-/dev/stderr}"

        # Join all arguments using \n and prepend every line with "[debug] "
        local debug_message=$(printf "%s\n" "${@}" | sed 's/^/[debug] /')

        # Append the debug message to the log file
        #echo "${debug_message}" | dd of="${log_file}" conv=notrunc oflag=append status=none
        echo "${debug_message}" >> "${log_file}"
    fi
}


DEBUG=true
DEBUG_LOG=$(mktemp)

yes | head -5 > foo

i=5
while (( i > 0 )); do
    debug "$i"
    read f
    printf '%s\n' "$f"
    (( i-- ))
done < foo

cat "$DEBUG_LOG"
rm "$DEBUG_LOG"

and you'll see the debug messages

[debug] 5
[debug] 4
[debug] 3
[debug] 2
[debug] 1

printed.