Discussion for proposal 2016-2

[matt rice]

It'd be nice to define the ways we query external standards and basis defined optional modules

[John Reppy]

I think that such a mechanism is part of the build system (e.g., CM or MLB). In SML/NJ's CM tool, we have some support for conditional compilation (you can even test if a given module is defined), but MLB is lacking such a mechanism.

[EDIT] I see now that you are asking for a smackage hook.

[John Reppy]

There are a few places where this proposal could use some clarification:

• what does it mean for a module's name field to be NONE?
• what useful information does the optional field convey? It it is false, then obviously the module needs to be in the list. If it is true, the module may be optional, but the implementation provides it anyway.
• the discussion mentions a definedBy field, but there is no such field in the record type.

[John Reppy]

Sorry about closing the issue; that was a click on the wrong button.

[John Reppy]

While I think that a standard mechanism to support packaging and portability of code across implementations is a good idea, it is not obvious to me that such a mechanism should be embedded in the Basis Library. Taking that approach has a few of disadvantages:

1. any tool using this implementation would have to be able to compile and run bits of SML code in a non-portable way (e.g., compare using this mechanism in SML/NJ vs. MLton), so there is little portability advantage to defining the mechanism in SML.
2. the proposal is unclear on what information an implementation would be expected to provide. A sample implementation might help here.
3. some build tools, such as CM and MLB, load libraries on demand, thus the information available from this mechanism would likely be incomplete.

I think that a better approach would probably be to standardize on some packaging tool, like smackage.

[matt rice]

Thanks for the comments John, I will try and edit the proposal, and work on a prototype,
In the meantime some answers to your questions here:

• what does it mean for a module's name field to be NONE?

That the record should be taken as reflecting upon the entire module_group.

It was intended to provide a simplified single record in the case that you implement every module specified by the module_group, you could just provide one record covering the entire module group.

• what useful information does the optional field convey? It it is false, then obviously the module needs to be in the list. If it is true, the module may be optional, but the implementation provides it anyway.

If the optional field is false, and the implemented field is false, this would reflect non-conformance or partial conformance, (some portion of the module group is implemented, but this portion is not)

• the discussion mentions a definedBy field, but there is no such field in the record type.

Ahh, sorry I believe this was superseeded by the 'standard' field but got left in the proposal.

it is not obvious to me that such a mechanism should be embedded in the Basis Library.

Entirely agree, this was more of an RFC, to spur discussion on how we can make module/library level packaging better, taking into account that some libraries, like the Basis, and smlnj-lib, are usually/sometimes provided by the compiler.

And whether any of it belongs in the basis (E.g. should the basis provide the signature for the record, and relegate the value portions, to a basis-meta library, and smlnj-lib-meta library?

Or does the entire thing belong outside of the basis.

1. any tool using this implementation would have to be able to compile and run bits of SML code in a non-portable way (e.g., compare using this mechanism in SML/NJ vs. MLton), so there is little portability advantage to defining the mechanism in SML.

The main thought was that, while they need to compile/run bits in SML, the module query program could be written once... I suppose you are recommending doing that, but using an external file format that basis libraries could provide, rather than SML records?

2. A sample implementation might help here.

Will work on it.

3. some build tools, such as CM and MLB, load libraries on demand

Indeed, its a bit tricky, since e.g. MLB I know, has multiple versions of the basis to choose from as well.

[John Reppy]

Thanks for the info. Here are a are more questions:

Presumably for the module-group mechanism to be useful, there would have to be an agreed specification of what modules belong in a given group?

Likewise, for a partially implemented group, is it correct that records for all modules in the group would have to be provided (even if some would have the implemented field set to false)?

SML/NJ also supports multiple versions of the Basis Library as of a couple of releases ago.

Perhaps there should also be a "version" field in the records? CM supports a library versioning mechanism (although it hasn't been put to much use).

If you stick with this API, you should change the basisModules to a function, since making it a list is not compatible with SML/NJ's export mechanism (the value would be determined at the time that the sampling program is compiled, instead of when it is run).

[John Reppy]

I also have comments about the general problem.

The design should be driven by an understanding of what kind of mechanisms are needed by a tool, such as smackage, to aid distribution. Providing one or two small example scenarios would help a lot. Without such motivating examples, I don't think that the resulting design will be useful.

Based on my own experience of writing autoconf tests for MLton and SML/NJ, I suspect that what we really need is a mechanism to query (perhaps from the command line) whether a given library or module is available, and, perhaps what version is available. It would also be nice to be able to query properties of the implementation, such as the size of the default integer type. The main difficulty with these kinds of tests right now is that they are implementation dependent hacks. For example, here is how I determine if MLton is 32 or 64-bits in a configure.ac script.

AC_MSG_CHECKING([native size of mlton executables])
MLTON_OBJPTR_REP=`$MLTON -show path-map | grep OBJPTR_REP | sed -e s/OBJPTR_REP// | sed -e s/\.sml//`
case x"$MLTON_OBJPTR_REP" in
  *rep32)
    AC_MSG_RESULT([32 bit])
    CFLAGS="$CFLAGS -m32"
    AC_DEFINE([WORDS_32], [], [Define to 1 if system is 32 bit])
  ;;
  *rep64)
    AC_MSG_RESULT([64 bit])
    CFLAGS="$CFLAGS -m64"
    AC_DEFINE([WORDS_64], [], [Define to 1 if system is 64 bit])
  ;;
  *) AC_MSG_ERROR([unable to determine mlton's word size]) ;;
esac

This test only works for MLton and might break if the internals are changed.

[matt rice]

Sorry, I kind of closed the wrong bug report (#20), but I don't think that one can really make any progress until this one is hammered out.

Like i said there, I see this overlapping to some extent with Matthias Blume's "Portable library descriptions for standard ML", e.g. If the basis library comes with a companion library which implements the Portable library description, and Compilers/distributors of Basis implementations generate a library description from their implementation of basis, A tool could then be written to compare the two library implementations for differences.

What you bring up, The desire for e.g. size of the default integer type, I'm unsure if that as-is would currently handle constant symbols, or if it would need some modification.