Permit discard identifier for layout members

Author: lwaern-intel

RELEASENOTES-1.4.md

@@ -370,7 +370,7 @@
 - `note 6` '`_`' can no longer be used as the name for arbitrary declarations.
   Instead, it is now only permitted in particular contexts, and in these
   contexts, '`_`' will affect the declaration in a way suitable for when the
-  declaration is _unused_. These contexts are:
+  declaration is _unused_ in some particular way. These contexts are:
     - Method-local bindings (e.g. variables and input parameters.)
 
       When a method-local binding is given the name '`_`', it will not be added
@@ -384,6 +384,12 @@
       definition that gives the index variable a different name. This is useful
       when defining an object array specification which does not depend on the
       index.
+    - Layout member names
+
+      When a layout member is given the name '`_`', that member will not be
+      referencable within DML code, but will still affect the memory
+      representation of the layout. This is useful to represent e.g. reserved
+      or padding bytes.
 
   Note that as a consequence of these semantics, any reference to '`_`' in code
   will _always_ resolve to the discard reference.

doc/1.4/language.md

@@ -90,6 +90,7 @@ declaration special semantics. Currently, these contexts are:
 see that section for more information.
 * Index parameters for object arrays. See the documentation for the
   [`object` template](dml-builtins.html#object) for more information.
+* As the name of one or more members of a [layout type](#layouts).
 
 </dd><dt>
 
@@ -1480,6 +1481,7 @@ typedef struct { <em>member declarations</em> } <em>name</em>;
 
 Layouts
 </dt><dd>
+<a id="layouts"/>
 
 A layout is similar to a struct in many ways.  The important
 difference is that there is a well-defined mapping between a layout
@@ -1514,6 +1516,24 @@ integer members (and arrays of similar) are translated
 to endian integers (or arrays of such) of similar size,
 with endianness matching the layout. Layout and endian integer
 members are accessed normally.
+
+The *discard identifer* "\_" may be used as the name of any number of members
+within a layout, making these *anonymous*. Anonymous layout members cannot be
+referenced within DML code, but will still influence the underlying memory
+representation of the layout in the same way as regular members.
+This is useful to represent reserved or padding bytes, or bytes that the device
+otherwise doesn't study or manipulate.
+
+Note that when a compound initializer is given for a variable of layout type,
+an initializer must still be given for each anonymous member:
+```
+local layout "little-endian" { uint32 x; uint32 _; uint32 y} = {1,0,2};
+```
+... unless designated initializers are used, in which case anonymous members
+can (and must) be omitted:
+```
+local layout "little-endian" { uint32 x; uint32 _; uint32 y} = {.x = 1, .y = 2};
+```
 </dd><dt>
 
 Bitfields
@@ -3184,27 +3204,23 @@ local (bool a, int i) = m();
 In the absence of explicit initializer expressions, a default
 "all zero" initializer will be applied to each declared object.
 
-"\_" may be used as an identifier for local variables, as well as other
-method-local bindings such as the method parameters, the bound identifier
-in `foreach`/`#foreach`/`#select` statements, and message component parameters
-of [hook-bound after statements](#hook-bound-after-statements). Any method-local
-binding named "\_" *will not be added to scope.* This is useful for when
-a method parameter is unused, or if you perform a method call where only a
-subset of returned values are of interest:
+The *discard identifier* "\_" may be used as an identifier for local variables,
+as well as other method-local bindings such as the method parameters, the bound
+identifier in `foreach`/`#foreach`/`#select` statements, and message component
+parameters of [hook-bound after statements](#hook-bound-after-statements).
+Any method-local binding named "\_" *will not be added to scope.* This is useful
+for when a method parameter is unused, or if you perform a method call where
+only a subset of returned values are of interest:
 ```
 local (bool a, int _) = m();
 // No conflicts since "_" is not added to scope
 local (bool a, int _, float _) = returns_three_vals();
 ```
 
-An alternative to this pattern is to leverage the
-[discard reference](#discard-reference)
-```
-local bool a;
-(a, _, _) = returns_three_vals();
-```
-... which does not require you to specify the types of the discarded values,
-may require multiple lines.
+An alternative to this pattern is to leverage the [discard
+reference](#discard-reference) ``` local bool a; (a, _, _) =
+returns_three_vals(); ``` ... which does not require you to specify the types of
+the discarded values, may require multiple lines.
 
 ### Session Statements
 <pre>

py/dml/codegen.py

@@ -1487,17 +1487,20 @@ def eval_type(asttype, site, location, scope, extern=False, typename=None,
                 raise ELAYOUT(site, "extern layout not permitted,"
                               + " use 'struct { }' instead")
             endian, fields = info
-            members = {}
-            for (_, msite, name, type_ast) in fields:
+            member_decls = []
+            for (_, msite, ident, type_ast) in fields:
                 (member_struct_defs, member_type) = eval_type(
                     type_ast, msite, location, scope, False)
                 if isinstance(member_type, TFunction):
                     raise EFUNSTRUCT(msite)
-                members[name] = (msite, member_type)
+                member_decls.append((
+                    msite,
+                    ident.args[0] if ident.kind == 'variable' else None,
+                    member_type))
                 struct_defs.extend(member_struct_defs)
-            if not members:
+            if not member_decls:
                 raise EEMPTYSTRUCT(site)
-            etype = TLayout(endian, members, label=typename)
+            etype = TLayout(endian, member_decls, label=typename)
             struct_defs.append((site, etype))
         elif tag == 'bitfields':
             width, fields = info
@@ -1737,9 +1740,9 @@ def check_designated_initializers(site, etype, init_asts, allow_partial):
     shallow_real_etype = safe_realtype_shallow(etype)
     duplicates = set()
     bad_fields = set()
-    remaining = set(shallow_real_etype.members)
+    remaining = set(shallow_real_etype.named_members)
     for (field, init) in init_asts:
-        if field not in shallow_real_etype.members:
+        if field not in shallow_real_etype.named_members:
             bad_fields.add(field)
         elif field not in remaining:
             duplicates.add(field)
@@ -1900,14 +1903,14 @@ def do_eval(etype, astinit):
             init = tuple(do_eval(etype.base, e) for e in init_asts)
             return CompoundInitializer(site, init)
         elif isinstance(etype, TStruct):
-            if len(etype.members) != len(init_asts):
+            members = list(etype.members_qualified)
+            if len(members) != len(init_asts):
                 raise EDATAINIT(site, 'mismatched number of fields')
             init = tuple(do_eval(mt, e)
-                         for ((_, mt), e) in zip(etype.members_qualified,
-                                                 init_asts))
+                         for ((mn, mt), e) in zip(members, init_asts))
             return CompoundInitializer(site, init)
         elif isinstance(etype, TExternStruct):
-            if len(etype.members) != len(init_asts):
+            if len(etype.named_members) != len(init_asts):
                 raise EDATAINIT(site, 'mismatched number of fields')
             init = {mn: do_eval(mt, e)
                     for ((mn, mt), e) in zip(etype.members_qualified,

py/dml/ctree_test.py

@@ -1400,8 +1400,8 @@ def const_types(self):
             types.TExternStruct({}, 'struct_t', 'struct_t'),
             types.TStruct({'x': types.TBool()}, 'struct_label'),
             types.TLayout(
-                'big-endian', {
-                    'x': (site, types.TEndianInt(24, True, 'big-endian'))},
+                'big-endian', [(site, 'x',
+                                types.TEndianInt(24, True, 'big-endian'))],
                 'struct_label'),
             types.THook([]),
         ]

py/dml/dmlparse.py

@@ -1482,7 +1482,13 @@ def typeof(t):
 def check_struct_namecoll(member_decls):
     sites_by_name = {}
     for decl in member_decls:
-        (name, _) = decl.args
+        if decl.kind == 'cdecl':
+            (name, _) = decl.args
+        else:
+            (ident, _) = decl.args
+            if ident.kind == 'discard':
+                continue
+            (name,) = ident.args
         if name in sites_by_name:
             report(ENAMECOLL(decl.site, sites_by_name[name], name))
         else:
@@ -1512,14 +1518,21 @@ def layout_decl(t):
         field_names = set()
         fields = []
         for cdecl in t[4]:
-            (name, typ) = cdecl.args
-            if name in field_names:
-                while (name in field_names
-                       or any(name == d.args[0] for d in t[4])):
-                    name = '_' + name
-                cdecl = ast.cdecl(cdecl.site, name, typ)
+            (ident, typ) = cdecl.args
+            if ident.kind == 'variable':
+                (name,) = ident.args
+                if name in field_names:
+                    while (name in field_names
+                           or any(name == d.args[0].args[0]
+                                  for d in t[4]
+                                  if d.args[0].kind == 'variable')):
+                        name = '_' + name
+                    cdecl = ast.cdecl_maybe_discarded(
+                        cdecl.site,
+                        ast.variable(ident.site, name),
+                        typ)
+                field_names.add(name)
             fields.append(cdecl)
-            field_names.add(name)
     else:
         fields = t[4]
         check_struct_namecoll(fields)
@@ -1536,7 +1549,7 @@ def layout(t):
 
 @prod
 def layout_decls(t):
-    'layout_decls : layout_decls named_cdecl SEMI'
+    'layout_decls : layout_decls named_cdecl_maybe_discarded SEMI'
     t[0] = t[1] + (t[2],)
 
 @prod
@@ -2757,10 +2770,11 @@ def objident_discard(t):
 def discard_error(site):
     report(ESYNTAX(site,
                    "_",
-                   "'_' can only be used as an expression or as an unused "
-                   + "identifier for an index variable of an object array or "
-                   + "a method-local binding (e.g. local variable or method "
-                   + "parameter)"))
+                   "can't use the name '_' (the discard identifier) in this "
+                   + "context. See the description of 'Identifiers' within "
+                   + "the Lexical Structure section of the DML 1.4 Reference "
+                   + "Manual for an overview of when '_' may be used as a "
+                   + "name."))
 
 @prod
 def ident_or_discard_ident(t):

py/dml/serialize.py

@@ -99,6 +99,25 @@ def prepare_array_de_serialization(site, t):
     dimsizes_expr = expr.mkLit(site, dimsizes_lit, TPtr(TInt(32, False)))
     return (base, dims, sizeof_base, dimsizes_expr)
 
+def mkSubRefLit(site, expr, sub, typ, op):
+    real_etype = safe_realtype_shallow(expr.ctype())
+
+    if isinstance(real_etype, TPtr):
+        if op == '.':
+            raise ENOSTRUCT(site, expr)
+        basetype = real_etype.base
+        real_basetype = safe_realtype(basetype)
+    else:
+        if op == '->':
+            raise ENOPTR(site, expr)
+        real_basetype = safe_realtype(etype)
+
+    real_basetype = real_basetype.resolve()
+
+    return ctree.StructMember(site, expr, sub,
+                              conv_const(real_basetype.const, typ), op)
+
+
 # This works on the assumption that args do not need to be hard-cast
 # to fit the actual fun signature
 def apply_c_fun(site, fun, args, rettype):
@@ -354,7 +373,7 @@ def map_dmltype_to_attrtype(site, dmltype):
         return 'f'
     if isinstance(real_type, TStruct):
         return '[%s]' % "".join([map_dmltype_to_attrtype(site, mt)
-                                 for mt in real_type.members.values()])
+                                 for (_, mt) in real_type.members])
     if isinstance(real_type, TArray):
         assert real_type.size.constant
         arr_attr_type = map_dmltype_to_attrtype(site, real_type.base)
@@ -375,7 +394,7 @@ def mark_for_serialization(site, dmltype):
     '''
     real_type = safe_realtype(dmltype)
     if isinstance(real_type, TStruct):
-        for mt in real_type.members.values():
+        for (_, mt) in real_type.members:
             mark_for_serialization(site, mt)
     elif isinstance(real_type, TArray):
         # Can only serialize constant-size arrays
@@ -496,9 +515,12 @@ def generate_serialize(real_type):
         in_arg_decl.toc()
         out_arg_decl.toc()
         if isinstance(real_type, TStruct):
-            sources = ((ctree.mkSubRef(site, in_arg, name, "->"),
-                        safe_realtype(typ))
-                       for (name, typ) in real_type.members.items())
+            sources = (
+                (mkSubRefLit(
+                    site, in_arg, name or TStruct.anon_member_cident(i),
+                    typ, "->"),
+                 safe_realtype(typ))
+                for (i, (name, typ)) in enumerate(real_type.members))
             serialize_sources_to_list(site, sources, out_arg)
         elif isinstance(real_type, TVector):
             raise ICE(site, "TODO: serialize vector")
@@ -619,9 +641,12 @@ def error_out(exc, msg):
                            else ctree.mkCast(site, tmp_out_ref, TPtr(void)))
             cleanup.append(ctree.mkDelete(site, cleanup_ref))
             tmp_out_decl.toc()
-            targets = tuple((ctree.mkSubRef(site, tmp_out_ref, name, "->"),
-                             conv_const(real_type.const, safe_realtype(typ)))
-                            for (name, typ) in real_type.members.items())
+            targets = tuple(
+                (mkSubRefLit(
+                    site, tmp_out_ref,
+                    name or TStruct.anon_member_cident(i), typ, "->"),
+                 conv_const(real_type.const, safe_realtype(typ)))
+                for (i, (name, typ)) in enumerate(real_type.members))
             def error_out_at_index(_i, exc, msg):
                 return error_out(exc, msg)
             deserialize_list_to_targets(site, in_arg, targets,

py/dml/structure.py

@@ -325,7 +325,7 @@ def type_deps(t, include_structs, expanded_typedefs):
         deps = []
         if include_structs:
             deps.append(t.label)
-        for (mn, mt) in t.members.items():
+        for (_, mt) in t.members:
             deps.extend(type_deps(mt, True, expanded_typedefs))
         return deps
     elif isinstance(t, TArray):