Rules of the standard and question
The standard says the following about the rescanning phase (after #/##-processing and parameter substitution) of macro expansion (C17 draft, 6.10.3.4 ¶2):
If the name of the macro being replaced is found during this scan of the replacement list (not including the rest of the source file’s preprocessing tokens), it is not replaced. Furthermore, if any nested replacements encounter the name of the macro being replaced, it is not replaced. These nonreplaced macro name preprocessing tokens are no longer available for further replacement even if they are later (re)examined in contexts in which that macro name preprocessing token would otherwise have been replaced.
Let me take this opportunity to summarize how macro replacement interacts with #/##-processing and parameter substitution for an object- or function-like macro M:
- Parameters after
#or adjacent to##are replaced verbatim (possibly as a placemarker token) and then handled by#and##.- Note that the relative evaluation order of
#and##is not specified by the standard. Whether this ever matters is unclear.
- Note that the relative evaluation order of
- For each other parameter, first its corresponding argument is fully macro-expanded, and then the parameter is replaced by the result.
- The macro expansion of these arguments happens as if they existed in isolation, ie function arguments in parentheses after the parameter in the replacement list are not considered, unlike function arguments supplied as part of the parameter's argument.
- All placemarker tokens resulting from
##are deleted. - The resulting token sequence S is rescanned, along with all subsequent preprocessing tokens of the source file, for more macros to evaluate.
- During this process, further occurrences of M within S or expansions of S are not expanded, even if examined within another context.
(This algorithmic description is not identical to that in the standard, but it should be sufficiently equivalent, at least for the purpose of this post.)
Now the question: How does application of the token concatenation operator ## interact with the prohibition against recursive macro expansion? Specifically, how does it affect the boundary of the area in which certain recursive expansions are blocked? It seems like concatenation with a placemarker is treated differently from ordinary concatenation by both GCC and MSVC.
Example
Let's consider the following example:
#define RECURSIONTEST(a, b, c) a ## c + b ## c
#define AC A
#define A AC A
#define CALL_RC(x, y, z) RECURSIONTEST(x, y, z)
CALL_RC(AC, A, C)
CALL_RC(AC, A, )
(The preprocessor can be run as follows: cpp or gcc -E (GCC), cl /E (MSVC).)
The two macros evaluate as follows
CALL_RC(AC, A, C) -> AC AC A + A AC A
CALL_RC(AC, A, ) -> AC A + A A
with both GCC and MSVC.
Let's calculate manually what should happen for CALL_RC(AC, A, C):
CALL_RC(AC, A, C)
[def of CALL_RC(x, y, z): RECURSIONTEST(x, y, z)]
arg x: AC -> A /*blk:AC*/ -> AC A /*blk:AC,A*/
arg y: A -> AC A /*blk:A*/ -> A A /*blk:A,AC*/
arg z: C
after arg subst:
RECURSIONTEST(AC A /*blk:AC,A*/, A A /*blk:A,AC*/, C)
// rescan for add'l macros:
RECURSIONTEST(AC A /*blk:AC,A*/, A A /*blk:A,AC*/, C) /*blk: CALL_RC*/
[def of RECURSIONTEST(a, b, c): a ## c + b ## c]
##-arg a: AC A /*blk:AC,A*/
##-arg b: A A /*blk:A,AC*/
##-arg c: C
concat 1: AC AC
// blocked for the left-hand AC: AC, A
// Are A and AC blocked for the right-hand AC?
concat 2: A AC
// blocked for A: A, AC
// Are A and AC blocked only for A or also for AC?
after ##:
AC AC + A AC
// rescan for add'l macros:
AC AC + A AC
// blocked globally: CALL_RC, RECURSIONTEST
// blocked for the 1st AC: AC, A
// perhaps (*) blocked locally for the 2nd and 3rd AC: A, AC
// blocked for A: A, AC
after macro expansion:
case 1 (blocking for (*)):
result: AC AC + A AC /*blk:AC,A*/
case 2 (no blocking for (*)):
2nd/3rd AC: AC -> A /*blk:AC*/ -> AC A /*blk:AC,A*/
result: AC AC A + A AC A /*blk:A,AC*/
// no further expansion possible in either case
Here, "blk" means "macros which are blocked for recursive expansion" in the preceding expression. Local blocks (for subexpressions) are specially indicated.
Now let's do the same calculation for CALL_RC(AC, A, ):
CALL_RC(AC, A, )
[def of CALL_RC(x, y, z): RECURSIONTEST(x, y, z)]
arg x: AC -> A /*blk:AC*/ -> AC A /*blk:AC,A*/
arg y: A -> AC A /*blk:A*/ -> A A /*blk:A,AC*/
arg z: <empty>
after arg subst:
RECURSIONTEST(AC A /*blk:AC,A*/, A A /*blk:A,AC*/, )
// rescan for add'l macros:
RECURSIONTEST(AC A /*blk:AC,A*/, A A /*blk:A,AC*/, ) /*blk: CALL_RC*/
[def of RECURSIONTEST(a, b, c): a ## c + b ## c]
##-arg a: AC A /*blk:AC,A*/
##-arg b: A A /*blk:A,AC*/
##-arg c: <placemarker>
concat 1: AC A
// blocked for AC: AC, A
// Are A and AC blocked for A?
concat 2: A A
// blocked for the left-hand A: A, AC
// Are A and AC blocked for the right-hand A?
after ##:
AC A + A A
// rescan for add'l macros:
AC A + A A
// blocked globally: CALL_RC, RECURSIONTEST
// blocked for AC: AC, A
// perhaps (**) blocked locally for the 1st and 3rd A: A, AC
// blocked for the 2nd A: A, AC
after macro expansion:
case 1 (blocking for (**)):
result: AC A + A A /*blk:AC,A*/
case 2 (no blocking for (**)):
1st/3rd A: A -> AC A /*blk:A*/ -> A A /*blk:A,AC*/
result: AC A A + A A A /*blk:A,AC*/
// no further expansion possible in either case
Analysis
CALL_RC(AC, A, C):
The output of GCC and MSVC (AC AC A + A AC A) corresponds to case 2 ("no blocking"). Specifically, the two instances of AC resulting from token concatenation have their blocking lists reset, allowing AC and A to be expanded afresh.
CALL_RC(AC, A, ):
The output of GCC and MSVC (AC A + A A) corresponds to case 1 ("blocking"). Specifically, the two instances of A resulting from token concatenation with a placemarker (ie a token adjacent to ## resulting from an empty argument) don't have their blocking lists reset, preventing further expansion.
Granted, it makes sense for these two cases to be treated differently, but does this actually follow from the standard, or is the standard vague about this? I am specifically referring to the wording in 6.10.3.4 ¶2 of the C17 draft (quoted at the very top of this post).
Arguably the answer depends on whether a token is still "the same" token after concatenation with a placemarker (C17 draft, 6.10.3.3 ¶3).
There is a similar question on this site: Is a repeated macro invocation via token concatenation unspecified behavior?
Note:
- Its example is convoluted.
- Its answer doesn't go into any detail.
- My question attempts to show detailed step-by-step evaluations.
- My question shows two macro expansions which should (conceivably) be treated the same but aren't.