Document the use of "base methods" to resolve hierarchy conflicts

[lwaern-intel]

This is much more realistic and better use of TQMIC:s than those I originally thought of, and the one pattern that have actually ended up being used. It's high time we document it proper.

Thanks to Gustav for coming up with it! I've requested him as reviewer to double-check that what I've written is correct and also understandable.

[syssimics]

PR Verification: ✅ success

[mandolaerik]

I think the text is now very heavy. If this new case is the main TQMIC pattern that's used in practice, then I think we can throw out the current example text, and focus on a single hands-on example that is easy to consume.

I would suggest starting off with your example code, without an abstract description of when the pattern is applicable. Something like this might be sufficient to present your example:

In the following example, two templates write_1_clears and gated_write both override the write method, which causes an ambiguity when both are instantiated on the same field. To fix this, we can define a combined template gated_write_1_clears with a write method that resolves the ambiguity; TQMIC allows you to reuse existing methods when implementing this method. In this case, we also have to factor out a separate method, base_write_of_gated_write, to make this work.

[mandolaerik]

I think the code snippet should explain itself only, no what-ifs. If this remark is needed, then we can place it after the snippet instead.

[lwaern-intel]

The point is to touch upon the "thought is needed as to ordering" remark with a concrete example as to how it may matter.

[lwaern-intel]

This is explained more briefly following the main example instead, now.

[lwaern-intel]

I think the text is now very heavy. If this new case is the main TQMIC pattern that's used in practice, then I think we can throw out the current example text, and focus on a single hands-on example that is easy to consume.

I would suggest starting off with your example code, without an abstract description of when the pattern is applicable. Something like this might be sufficient to present your example:

In the following example, two templates write_1_clears and gated_write both override the write method, which causes an ambiguity when both are instantiated on the same field. To fix this, we can define a combined template gated_write_1_clears with a write method that resolves the ambiguity; TQMIC allows you to reuse existing methods when implementing this method. In this case, we also have to factor out a separate method, base_write_of_gated_write, to make this work.

Your suggestion has merit. I'm slightly hesitant to discard the first case I document: if it's possible to leverage, then it is the simplest and best solution. Most notably, it's suitable for conflicts to write_action/read_action over in VP and, to a lesser extent, resets. But yes, today only case 2 occurs (together with some other ad-hoc uses of TQMICs.) So I feel conflicted about it.
Case 3 can be imagined to sometimes be needed, but case 2 should trump it basically always. I'm fine scrapping the documentation of that one.

[mandolaerik]

I'm slightly hesitant to discard the first case I document

Then maybe start with documenting case 2 end-to-end, and add a remark afterward that you sometimes can get away with a somewhat simpler/cleaner solution, and show the example from case 1.

I don't think we need to describe the exact abstract circumstances when the pattern is applicable; this is tips-and-tricks rather than formal reference, and it should be easy for the reader to extrapolate from examples.

[lwaern-intel]

Then maybe start with documenting case 2 end-to-end, and add a remark afterward that you sometimes can get away with a somewhat simpler/cleaner solution, and show the example from case 1.

That was exactly what I was going to suggest before my internet dropped out

I don't think we need to describe the exact abstract circumstances when the pattern is applicable; this is tips-and-tricks rather than formal reference, and it should be easy for the reader to extrapolate from examples.

Well. This is the DML reference manual. But I agree an elaborate description would be redundant.

[mandolaerik]

This is the DML reference manual.
Indeed, so the details of formal requirements can be inferred from other parts of the document.

[syssimics]

PR Verification: ❌ failure

[mandolaerik]

The added value of this note is questionable -- I expect users would read it as "as usual, you sometimes need to add a shared annotation; the compiler will let you know". This is already true for all method declarations.

[mandolaerik]

It doesn't scale nicely though: with three templates A, B and C, you'd in principle need templates for A+B, A+C, B+C, and A+B+C, and with N templates the worst-case would be exponential, up to 2**N - N - 1 templates. There would also be a crazy amount of diamond expansion: the rank of the A+B+C template would need to beat that of A+B, A+C and B+C, which adds extra copies of the A/B/C vtables, again in an exponential manner (unless cough the #if(false) trick cough).

[lwaern-intel]

Your point being? You want me to change anything?

unless cough the #if(false) trick cough).

No, it doesn't help. The #if (false) trick only allows one to make one override more specific than a number of others. It doesn't help any in the question of combining the implementations.

[mandolaerik]

Regarding if false: If we have

template abc is (a, b, c) { method m() default { .. } }
template ab is (a, b) { method m() default { .. } }
template ac is (a, c) { method m() default { .. } }
template bc is (b, c) { method m() default { .. } }
group g is (a,b,c);

and apply in-each, then all four templates will be applied into g, which gives an ambiguity on g. You can use the abominable trick to make abc win; without it you need to spell out template abc is (ab, ac, bc) which explodes worse (although admittedly the exponential punishment only applies with 4 or more templates).

[mandolaerik]

Your point being? You want me to change anything?

Yes, your remark is incorrect, it only extends to N conflicting templates if you are ready to write O(2N) code and pay O(4N) memory. I'd say this is questionable for N==3, and unreasonable for N>3. So it's not honest to say "extends to any number of".

I don't know what to do about it; it feels like a genuine problem with this code pattern. Unless I'm missing something.

[lwaern-intel]

It is honest -- at least, with the intention I had behind the statement. The point of it is only to clarify that the approach is not limited to two conflicting templates. I can, however, bring up the fact that is scales poorly:

This approach extends to any number of conflicting templates — as long as
all but (optionally) one can be made to offer overridable base methods (with
distinct names.) However, if one wishes to leverage the above approach to write code supporting arbitrary combinations across those templates, then the amount of code needed scales exponentially, becoming untenable past 4 possible conflicting templates being in play simultaneously. Because of that, in practice it is better to only apply the pattern for the specific conflicting combinations that prove necessary to support.

[mandolaerik]

Oh, it's not exponential, it's factorial. without #if (false), it's a simple N!; with #if (false) this is multiplied by sum(i/(factorial(N-i)*factorial(i)) for i in range(N)), which only pays off with N>=5, when the base vtables are multiplied by 80 instead of 120.

[lwaern-intel]

Oh, it's not exponential, it's factorial.

Factorial is exponential though, if I remember my complexity theory correctly. As in they simplify to the same complexity class.

[lwaern-intel]

I do not. I thought I learned factorial was equivalent to n^n, but not even that's true.

[mandolaerik]

If we want to summarize what we know in one sentence, then I would say "it is a pattern that scales poorly on conflicts that involve more than two templates". This is sufficient -- the message is that the pattern can NOT be extended without pain and extra thought. What probably matters the most is that you usually can find special-case circumstances that make multi-template conflicts solvable, and this is out of scope for this doc.

[lwaern-intel]

@mandolaerik But only saying "it is a pattern that scales poorly on conflicts that involve more than two templates" is wrong. Or, at least, misleading. The pattern works well as a workaround for specific cases of conflicting combinations, even those involving more than two templates -- it doesn't work well as something you apply in order to take care of every possible situation as a just-in-case. Hence,

, if one wishes to leverage the above approach to write code supporting arbitrary combinations across those templates

and

Because of that, in practice it is better to only apply the pattern for the specific conflicting combinations that prove necessary to support.

You're saying "oh, people should find special-case circumstances" -- applying the pattern for only a particular combination is a special-case circumstance that works well.

[mandolaerik]

The triple mdash in one sentence doesn't look good. Maybe split the sentence and replace them with commas?

[lwaern-intel]

Yeah. I'll give it a think, I had a particular reason to do it this way, but it's not exactly pretty.

[mandolaerik]

Unexpected context switch: we were on example usage, and now go back to detailed reference docs. Maybe put the example last, perhaps even under a named subsection?

[lwaern-intel]

The problem with that doing so is that it will bore and confuse the reader to tears before they gain any understanding of how TQMICs may actually be used.

Doing it this way is also not completely without precedence, see e.g. explicit_param_decls. Though admittedly, it is rare and the examples presented in other sections are smaller.

[lwaern-intel]

I'm not sure what to do about this. A named subsection feels distasteful to me. Nothing else in the reference manual does it.

[mandolaerik]

would have

[lwaern-intel]

Are we that stuffy? This feels like on the level of replacing every "don't" with "do not".

But bah, fine.