Skip to content

Document the use of "base methods" to resolve hierarchy conflicts #392

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 2 commits into
base: main
Choose a base branch
from
Open

Document the use of "base methods" to resolve hierarchy conflicts #392

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
e536d54
Document the use of "base methods" to resolve hierarchy conflicts
lwaern-intel Oct 29, 2025
b7b52f2
Address Erik's feedback
lwaern-intel Oct 30, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
179 changes: 96 additions & 83 deletions doc/1.4/language.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3395,116 +3395,129 @@ the implementations are provided by hierarchically unrelated templates such that
`default` can't be used (see [Resolution of
overrides](#resolution-of-overrides).) In particular, this typically allows for
ergonomically resolving conflicts introduced when multiple orthogonal templates
are instantiated, as long as all conflicting implementations are overridable,
and one of the following is true:
* The implementations can be combined together by calling each one of them, as
long as that can be done without risking e.g. side-effects being duplicated.
* The implementations can be combined by choosing one particular template's
implementation to invoke (typically the one most complex), and then adding
code around that implementation call in order to replicate the behaviour of
the implementations of the other templates. Ideally, the other templates would
provide methods that may be leveraged so that their behaviour may be
replicated without the need for excessive boilerplate.

The following is an example of the first case:
```
template alter_write is write {
method write(uint64 written) {
default(alter_write(written));
}

method alter_write(uint64 curr, uint64 written) -> (uint64);
}
are instantiated, as long as all conflicting implementations are overridable.

template gated_write is alter_write {
method write_allowed() -> (bool) default {
The following example demonstrates the most common kind of conflict that
hierarchically unrelated templates may introduce, and how template-qualified
method implementation calls may be leveraged to resolve it. Consider the
following templates:
```
template gated_write is write {
method write_allowed(uint64 val) -> (bool) default {
return true;
}

method alter_write(uint64 curr, uint64 written) -> (uint64) default {
return write_allowed() ? written : curr;
method write(uint64 val) default {
if (write_allowed(val)) {
default(val);
}
}
}

template write_1_clears is alter_write {
method alter_write(uint64 curr, uint64 written) -> (uint64) default {
return curr & ~written;
template write_1_clears is write {
method write(uint64 val) default {
default(get() & ~val);
}
}

template gated_write_1_clears is (gated_write, write_1_clears) {
method alter_write(uint64 curr, uint64 written) default {
local uint64 new = this.templates.write_1_clears.alter_write(
curr, written);
return this.templates.gated_write.alter_write(curr, new);
}
}

// Resolve the conflict introduced whenever the two orthogonal templates are
// instantiated by also instantiating gated_write_1_clears when that happens
in each (gated_write, write_1_clears) { is gated_write_1_clears; }
```

The following is an example of the second case:
```
template very_complex_register is register {
method write_register(uint64 written, uint64 enabled_bytes,
void *aux) default {
... // An extremely complicated implementation
}
}

template gated_register is register {
method write_allowed() -> (bool) default {
If one would like to instantiate both templates for a particular `field`,
attempting to do so would cause DMLC to reject the model, as the choice of
`write` implementation then becomes ambiguous.
The typical solution to implementation conflicts between templates —
making one template inherit from the other — is not appropriate in this
situation, as the operation of each template is orthogonal to the other,
and they may be used individually in other contexts.

Instead, what one may do is to modify one or both templates to offer an
overridable "base" method that is called instead of `default` within the
template's implementation of `write`. This additional flexibility enables a way
to situationally resolve the conflict: if both templates are in play, override
the base method of one template to call the `write` implementation of the
other. This effectively defines the chain in which the conflicting
implementations are to be called from one another, combining their behaviour.

The default implementations of the base methods can be to invoke the `write`
implementation of their parent template, which makes calling them the same as
calling `default`. This captures the regular case where no conflicting templates
are in play.

The below shows this approach being applied to the example above, modifying
`gated_write` to offer a base method, and leveraging a new template and an
`in each` declaration to automatically resolve the conflict wherever it would
occur in the model.
```
template gated_write is write {
method write_allowed(uint64 val) -> (bool) default {
return true;
}

method on_write_attempted_when_not_allowed() default {
log spec_viol: "%s was written to when not allowed", qname;
method write(uint64 val) default {
if (write_allowed(val)) {
base_write_of_gated_write(val); // instead of default()
}
}

method write_register(uint64 written, uint64 enabled_bytes,
void *aux) default {
if (write_allowed()) {
default(written, enabled_bytes, aux);
} else {
on_write_attempted_when_not_allowed();
}
method base_write_of_gated_write(uint64 val) default {
// This is the implementation that default() would've resolved to
Copy link
Contributor

@mandolaerik mandolaerik Oct 30, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

would have

Copy link
Contributor Author

@lwaern-intel lwaern-intel Oct 31, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are we that stuffy? This feels like on the level of replacing every "don't" with "do not".

But bah, fine.

this.templates.write.write(val);
}
}

template very_complex_gated_register is (very_complex_register,
gated_register) {
// No sensible way to combine the two implementations by calling both.
// Even if there were, calling both implementations would cause each field
// of the register to be written to multiple times, potentially duplicating
// side-effects, which is undesirable.
// Instead, very_complex_register is chosen as the base implementation
// called, and the behaviour of gated_register is replicated around that
// call.
method write_register(uint64 written, uint64 enabled_bytes,
void *aux) default {
if (write_allowed()) {
this.templates.very_complex_register.write_register(
written, enabled_bytes, aux);
} else {
on_write_attempted_when_not_allowed();
}
}
template write_1_clears is (write, get) { // unchanged
...
}

in each (gated_register, very_complex_register) {
is very_complex_gated_register;
template gated_write_1_clears is (gated_write, write_1_clears) {
// Resolve the conflict by providing an unambiguously most specific
// write implementation
method write(uint64 val) default {
// This makes gated_write the first link in the call chain...
this.templates.gated_write.write(val);
}

// ... and by overriding gated_write's base method, we make write_1_clears
// the second (and final) link in the chain
method base_write_of_gated_write(uint64 val) default {
this.templates.write_1_clears.write(val);
}
}

in each (gated_write, write_1_clears) { is gated_write_1_clears; }
```
> [!NOTE]
> The example given above applies regardless of whether the method
> implementations in the original templates are `shared` or not. However, the
> implementations in the template defined to resolve the conflicts might need to
> be non-`shared` if some of the implementations involved are not `shared`;
> see the final paragraph of this subsection.
Comment on lines +3488 to +3493
Copy link
Contributor

@mandolaerik mandolaerik Oct 30, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The added value of this note is questionable -- I expect users would read it as "as usual, you sometimes need to add a shared annotation; the compiler will let you know". This is already true for all method declarations.


This approach extends to any number of conflicting templates — as long as
all but (optionally) one can be made to offer overridable base methods (with
distinct names.)
Comment on lines +3495 to +3497
Copy link
Contributor

@mandolaerik mandolaerik Oct 30, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It doesn't scale nicely though: with three templates A, B and C, you'd in principle need templates for A+B, A+C, B+C, and A+B+C, and with N templates the worst-case would be exponential, up to 2**N - N - 1 templates. There would also be a crazy amount of diamond expansion: the rank of the A+B+C template would need to beat that of A+B, A+C and B+C, which adds extra copies of the A/B/C vtables, again in an exponential manner (unless cough the #if(false) trick cough).

Copy link
Contributor Author

@lwaern-intel lwaern-intel Oct 31, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Your point being? You want me to change anything?

unless cough the #if(false) trick cough).

No, it doesn't help. The #if (false) trick only allows one to make one override more specific than a number of others. It doesn't help any in the question of combining the implementations.

Copy link
Contributor

@mandolaerik mandolaerik Oct 31, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Regarding if false: If we have

template abc is (a, b, c) { method m() default { .. } }
template ab is (a, b) { method m() default { .. } }
template ac is (a, c) { method m() default { .. } }
template bc is (b, c) { method m() default { .. } }
group g is (a,b,c);

and apply in-each, then all four templates will be applied into g, which gives an ambiguity on g. You can use the abominable trick to make abc win; without it you need to spell out template abc is (ab, ac, bc) which explodes worse (although admittedly the exponential punishment only applies with 4 or more templates).

Copy link
Contributor

@mandolaerik mandolaerik Oct 31, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Your point being? You want me to change anything?

Yes, your remark is incorrect, it only extends to N conflicting templates if you are ready to write O(2N) code and pay O(4N) memory. I'd say this is questionable for N==3, and unreasonable for N>3. So it's not honest to say "extends to any number of".

I don't know what to do about it; it feels like a genuine problem with this code pattern. Unless I'm missing something.

Copy link
Contributor Author

@lwaern-intel lwaern-intel Oct 31, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is honest -- at least, with the intention I had behind the statement. The point of it is only to clarify that the approach is not limited to two conflicting templates. I can, however, bring up the fact that is scales poorly:

This approach extends to any number of conflicting templates — as long as
all but (optionally) one can be made to offer overridable base methods (with
distinct names.) However, if one wishes to leverage the above approach to write code supporting arbitrary combinations across those templates, then the amount of code needed scales exponentially, becoming untenable past 4 possible conflicting templates being in play simultaneously. Because of that, in practice it is better to only apply the pattern for the specific conflicting combinations that prove necessary to support.

Copy link
Contributor

@mandolaerik mandolaerik Oct 31, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, it's not exponential, it's factorial. without #if (false), it's a simple N!; with #if (false) this is multiplied by sum(i/(factorial(N-i)*factorial(i)) for i in range(N)), which only pays off with N>=5, when the base vtables are multiplied by 80 instead of 120.

Copy link
Contributor Author

@lwaern-intel lwaern-intel Oct 31, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, it's not exponential, it's factorial.

Factorial is exponential though, if I remember my complexity theory correctly. As in they simplify to the same complexity class.

Copy link
Contributor Author

@lwaern-intel lwaern-intel Oct 31, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I do not. I thought I learned factorial was equivalent to n^n, but not even that's true.

Copy link
Contributor

@mandolaerik mandolaerik Oct 31, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we want to summarize what we know in one sentence, then I would say "it is a pattern that scales poorly on conflicts that involve more than two templates". This is sufficient -- the message is that the pattern can NOT be extended without pain and extra thought. What probably matters the most is that you usually can find special-case circumstances that make multi-template conflicts solvable, and this is out of scope for this doc.

Copy link
Contributor Author

@lwaern-intel lwaern-intel Nov 3, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mandolaerik But only saying "it is a pattern that scales poorly on conflicts that involve more than two templates" is wrong. Or, at least, misleading. The pattern works well as a workaround for specific cases of conflicting combinations, even those involving more than two templates -- it doesn't work well as something you apply in order to take care of every possible situation as a just-in-case. Hence,

, if one wishes to leverage the above approach to write code supporting arbitrary combinations across those templates

and

Because of that, in practice it is better to only apply the pattern for the specific conflicting combinations that prove necessary to support.

You're saying "oh, people should find special-case circumstances" -- applying the pattern for only a particular combination is a special-case circumstance that works well.


Note that the order of implementations in the call chain might matter. If, in the
example above, `write_1_clears` were instead modified to offer a base method
and been made the first link in the chain, then what it would have passed down
as the written value to `gated_write` would not have been the original value
— but rather, the register's current value with bits cleared, which might
violate the expectations of `gated_write`'s implementation.

The approach given above is not the only way in which template-qualified method
implementation calls may be utilized to resolve conflicts, only the most
commonly applicable one. In fact, the most simple approach, if viable, is to
have the implementation introduced to resolve the conflict simply call each
conflicting implementation in turn — if that doesn't cause side-effects
to be duplicated in an undesirable way — or even only call one particular
implementation — if it makes sense to prefer it ahead of every other.
Copy link
Contributor

@mandolaerik mandolaerik Oct 30, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The triple mdash in one sentence doesn't look good. Maybe split the sentence and replace them with commas?

Copy link
Contributor Author

@lwaern-intel lwaern-intel Oct 31, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah. I'll give it a think, I had a particular reason to do it this way, but it's not exactly pretty.


A template-qualified method implementation call is resolved by using
the method implementation provided to the object by the named template.
If no such implementation is provided (whether it be because the template does
not specify one, or specifies one which is not provided to the object due to its
If no such implementation exists (whether it be because the template does not
specify one, or specifies one which is not provided to the object due to its
definition being eliminated by an [`#if`](#conditional-objects)), then the
ancestor templates of the named template are recursively searched for the
Comment on lines 3514 to 3519
Copy link
Contributor

@mandolaerik mandolaerik Oct 30, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unexpected context switch: we were on example usage, and now go back to detailed reference docs. Maybe put the example last, perhaps even under a named subsection?

Copy link
Contributor Author

@lwaern-intel lwaern-intel Oct 31, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The problem with that doing so is that it will bore and confuse the reader to tears before they gain any understanding of how TQMICs may actually be used.

Doing it this way is also not completely without precedence, see e.g. explicit_param_decls. Though admittedly, it is rare and the examples presented in other sections are smaller.

Copy link
Contributor Author

@lwaern-intel lwaern-intel Oct 31, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure what to do about this. A named subsection feels distasteful to me. Nothing else in the reference manual does it.

highest-rank (most specific) implementation provided by them. If the ancestor
highest-ranking (most specific) implementation provided by them. If the ancestor
templates provide multiple hierarchically unrelated implementations, then the
choice is ambiguous and the call will be rejected by the compiler. In this case,
the modeller must refine the template-qualified method implementation call to
Expand All @@ -3513,7 +3526,7 @@ name the ancestor template whose implementation they would like to use.
A template-qualified method implementation call done via [a value of template
type](#templates-as-types) functions differently compared to compile-time
object references. In particular, `this.templates` within the bodies of `shared`
methods functions differently. The specified template must be an ancestor
methods functions differently. The specified template must either be an ancestor
template of the value's template type, the <tt>object</tt> template, or the
template type itself; furthermore, the specified template **must provide or
inherit a `shared` implementation of the named method**. It is not sufficient
Expand Down