Scoping anomalies with 'typeset -r' and 'readonly' keywords in ksh

Question

First off, I am aware of the general scoping differences (dynamic/static) between bash and ksh when declaring functions using the function keyword vs. myfunction() and this post only discusses scoping issues with regards to readonly variables.

Today I stumbled across something that confuses me, though. I have a script that sources files from within a function declared with the function keyword (and thus I was not immediately aware of why the following happened at all, as I looked at those separate files through "global-scope goggles"). As part of a recent cleanup I made various variables inside those sourced files readonly and noticed that some parts of the code stopped working under ksh93, depending on how I marked the variables as readonly. More specifically, if I used readonly FOO=bar, ${FOO} would be unset for the remaining parts of the sourced file.

This demonstrates the issue(s):

(Note: The behaviour is the same with inlined code (vs. a second script that gets sourced), but since it saves some lines here and the post is pretty long already I kept it as it is)

readonly_test_sourced.sh:

readonly   foo=function
typeset -r bar=function
typeset    baz=function
readonly   baz
qux=function
readonly   qux
quux=function
typeset -r quux

readonly_test.sh:

function f
{
    . ./readonly_test_sourced.sh
    printf "foo=${foo}\nbar=${bar}\nbaz=${baz}\nqux=${qux}\nquux=${quux}\n"
}

g()
{
    . ./readonly_test_sourced.sh
    printf "foo=${foo}\nbar=${bar}\nbaz=${baz}\nqux=${qux}\nquux=${quux}\n"
}

[ -n "${KSH_VERSION}" ]  && echo "ksh: ${KSH_VERSION}"
[ -n "${BASH_VERSION}" ] && echo "bash: ${BASH_VERSION}"

for var in foo bar baz qux quux; do
    unset "${var}"
    eval "$var=global" # don't do this at home, there are better ways
done

func="${1:-f}"
echo
echo "inside function ${func}"
echo '----------------'
${func}

echo
echo "global scope after calling ${func}"
echo '----------------------------'
printf "foo=${foo}\nbar=${bar}\nbaz=${baz}\nqux=${qux}\nquux=${quux}\n"

This is the output I get with ksh93u+:

$ ksh ./readonly_test.sh f
ksh: Version JM 93u+ 2012-02-29

inside function f
----------------
foo=
bar=function
baz=function
qux=
quux=

global scope after calling f
----------------------------
foo=function
bar=global
baz=global
qux=function
quux=function

$ ksh ./readonly_test.sh g
ksh: Version JM 93u+ 2012-02-29

inside function g
----------------
foo=function
bar=function
baz=function
qux=function
quux=function

global scope after calling g
----------------------------
foo=function
bar=function
baz=function
qux=function
quux=function

This is what I get with bash 4.2:

$ bash ./readonly_test.sh f
bash: 4.2.42(1)-release

inside function f
----------------
foo=function
bar=function
baz=function
qux=function
quux=

global scope after calling f
----------------------------
foo=function
bar=global
baz=global
qux=function
quux=function

$ bash ./readonly_test.sh g
bash: 4.2.42(1)-release

inside function g
----------------
foo=function
bar=function
baz=function
qux=function
quux=

global scope after calling g
----------------------------
foo=function
bar=global
baz=global
qux=function
quux=function

I also ran it through mksh (which implements dynamic scoping according to its manpage, and it does yield the same results as with bash):

$ mksh ./readonly_test.sh f
ksh: @(#)MIRBSD KSH R40 2012/03/20

inside function f
----------------
foo=function
bar=function
baz=function
qux=function
quux=

global scope after calling f
----------------------------
foo=function
bar=global
baz=global
qux=function
quux=function

$ mksh ./readonly_test.sh g
ksh: @(#)MIRBSD KSH R40 2012/03/20

inside function g
----------------
foo=function
bar=function
baz=function
qux=function
quux=

global scope after calling g
----------------------------
foo=function
bar=global
baz=global
qux=function
quux=function

Observations:

• readonly and typeset -r are sometimes synonymous, sometimes not

• bash and mksh

  • there are 3 possible different scenarios (both for f and g):

    • [1] readonly foo=function assigns 'function' to the global variable foo and does not declare a new local variable (identical to [4])
    • [2] typeset -r bar=function assigns 'function' to the new local variable bar (identical to [3])
    • [3] typeset baz=function; readonly baz assigns 'function' to the new local variable baz (identical to [2])
    • [4] qux=function; readonly qux assigns 'function' to the global variable qux and does not declare a new local variable (identical to [1]). qux is readonly in both local as well as global scope, but this is expected because it marks the global variable as readonly and due to dynamic scoping it becomes readonly in the function as well (side note: see also).
    • [5] quux=function; typeset -r quux assigns 'function' to the global variable quux, then declares a new local variable quux without value

  • readonly never declares a new (local) variables (as expected). For [1] and [4] it marks the global variables readonly, for [3] the new local variable. This is totally what I would expect and means the scope that readonly operates in is the same as for the respective variable itself, i.e.

    • x=y; if $x is local then readonly x would mark the local variable x readonly
    • x=y; if $x is global then readonly x would mark the global variable x readonly

  • consistency between [1] / [2] and [4] / [5], respectively

• in ksh (discussing f; g behaves as expected):

  • there are also 3 possible different scenarios:
    • [6] readonly foo=function: similar to [5], [8] and [9] but way more confusing as this is a single command (as opposed to: assignment first, readonly/typeset -r later). Apparently readonly declares a new local variable foo without value but sets the global variable foo to 'function'. ?!?. foo becomes readonly in both function as well as global scope.
    • [7] typeset -r bar=function assigns 'function' to the new local variable bar (identical to [8])
    • [8] typeset baz=function; readonly baz assigns 'function' to the new local variable baz (identical to [7]). baz becomes readonly only in function scope
    • [9] qux=function; readonly qux assigns 'function' to the global variable qux, then declares a new local variable qux without value (identical to [5], [6], [10]). qux becomes readonly only in function scope
    • [10] quux=function; typeset -r quux assigns 'function' to the global variable quux, then declares a new local variable quux without value (identical to [5], [9], [10]). quux becomes readonly only in function scope.
  • readonly does seem to declare new local variables in [6] and [9], but not in [8].

The behaviour of bash is expected. typeset (= declare) creates/modifies in the scope of the function (bash has the -g option to force creation/modification in global scope even when used inside functions), readonly really only "marks" existing variables and never introduces new local variables. [5] puzzled me a little as I never had to declare an unset readonly variable before and I would have assumed that in case a global variable of the same name exists it modifies that, but I can absolutely live with this behaviour as it is consistent with the other scenarios and the existence of -g.

But, to my understanding, the ksh man page fails to completely explain all of the above scenarios and the subtle differences between the readonly and typeset -r keywords, unless I missed something while re-reading the respective sections in questions. What is most confusing to me is that nowhere does it mention the keyword readonly near the explanation of the scoping differences between foo() and function bar, and the short explanation of the readonly inbuilt does not mention anything about this either. Based on that I would never assume that readonly introduces new, statically scoped variables like typeset -r does and it seems really unexpected that it does so in some (but not even all) scenarios.

The most confusing scenario for me is [6] and I fail to understand what exactly happens there (this is also the specific scenario that broke my code).

Are my observations correct and can someone shed light on the behaviour of ksh93? (I hope this question is acceptably scoped (no pun intended))

Answer 1

Welcome to the world of shell incompatibilities :)

If I understood correctly the question, it's about the difference between

function blah { }

and

blah() { }

Looking ad man ksh(1) (on Solaris if that makes any difference)

 Functions defined by the function name syntax and called  by
 name execute in the same process as the caller and share all
 files and present working directory with the  caller.
 ...
 Ordinarily,  variables
 are  shared  between  the  calling program and the function.
 However, the typeset special built-in command used within  a
 function  defines  local  variables whose scope includes the
 current function. They can be passed to functions that  they

versus

 Functions defined  with  the  name()  syntax  and  functions
 defined  with the function name syntax that are invoked with
 the . special built-in are executed in the caller's environ-
 ment  and  share  all  variables  and traps with the caller.
 Errors within these function  executions  cause  the  script
 that contains them to abort.

Answer 2

The mistake is as follows, there is no "local" scope for functions when you dont use "local" or make the functions a subshell!

So, use f() ( <code> ) instead of f() { <code> } to get your local scope!

Still, you have one point!

The difference between "readonly <var>" and "declare -r <var>"!

# ./readonly_test.sh
bash: 3.00.16(1)-release

inside function
----------------
foo=function
bar=function
baz=function
qux=function
quux=

global scope after calling
----------------------------
foo=global
bar=global
baz=global
qux=global
quux=global