## MLton 20241230+git20251029+dfsg-5

# MLNLFFIImplementation

MLton’s implementation(s) of the MLNLFFI library differs from the SML/NJ implementation in two important ways:

- MLton cannot utilize the `Unsafe.cast` "cheat" described in Section 3.7 of Blume01. (MLton’s representation of closures and aggressive representation optimizations make an `Unsafe.cast` even more "unsafe" than in other implementations.)

  We have considered two solutions:

  - One solution is to utilize an additional type parameter (as described in Section 3.7 of Blume01):

    >     signature C = sig
    >         type ('t, 'f, 'c) obj
    >         eqtype ('t, 'f, 'c) obj'
    >         ...
    >         type ('o, 'f) ptr
    >         eqtype ('o, 'f) ptr'
    >         ...
    >         type 'f fptr
    >         type 'f ptr'
    >         ...
    >         structure T : sig
    >             type ('t, 'f) typ
    >             ...
    >         end
    >     end

    > The rule for `('t, 'f, 'c) obj`,`('t, 'f, 'c) ptr`, and also `('t, 'f) T.typ` is that whenever `F fptr` occurs within the instantiation of `'t`, then `'f` must be instantiated to `F`. In all other cases, `'f` will be instantiated to `unit`.

    (In the actual MLton implementation, an abstract type `naf` (not-a-function) is used instead of `unit`.)

    While this means that type-annotated programs may not type-check under both the SML/NJ implementation and the MLton implementation, this should not be a problem in practice. Tools, like `ml-nlffigen`, which are necessarily implementation dependent (in order to make calls through a C function pointer), may be easily extended to emit the additional type parameter. Client code which uses such generated glue-code (e.g., Section 1 of Blume01) need rarely write type-annotations, thanks to the magic of type inference.

  - The above implementation suffers from two disadvantages.

    First, it changes the MLNLFFI Library interface, meaning that the same program may not type-check under both the SML/NJ implementation and the MLton implementation (though, in light of type inference and the richer `MLRep` structure provided by MLton, this point is mostly moot).

    Second, it appears to unnecessarily duplicate type information. For example, an external C variable of type `int (* f[3])(int)` (that is, an array of three function pointers), would be represented by the SML type `(((sint -> sint) fptr, dec dg3) arr, sint -> sint, rw) obj`. One might well ask why the `'f` instantiation (`sint -> sint` in this case) cannot be extracted from the `'t` instantiation (`((sint -> sint) fptr, dec dg3) arr` in this case), obviating the need for a separate function-type type argument. There are a number of components to an complete answer to this question. Foremost is the fact that Standard ML supports neither (general) type-level functions nor intensional polymorphism.

    A more direct answer for MLNLFFI is that in the SML/NJ implemention, the definition of the types `('t, 'c) obj` and `('t, 'c) ptr` are made in such a way that the type variables `'t` and `'c` are phantom (not contributing to the run-time representation of an `('t, 'c) obj` or `('t, 'c) ptr` value), despite the fact that the types `((sint -> sint) fptr, rw) ptr` and `((double -> double) fptr, rw) ptr` necessarily carry distinct (and type incompatible) run-time (C-)type information (RTTI), corresponding to the different calling conventions of the two C functions. The `Unsafe.cast` "cheat" overcomes the type incompatibility without introducing a new type variable (as in the first solution above).

    Hence, the reason that function-type type cannot be extracted from the `'t` type variable instantiation is that the type of the representation of RTTI doesn’t even see the (phantom) `'t` type variable. The solution which presents itself is to give up on the phantomness of the `'t` type variable, making it available to the representation of RTTI.

    This is not without some small drawbacks. Because many of the types used to instantiate `'t` carry more structure than is strictly necessary for `'t`’s RTTI, it is sometimes necessary to wrap and unwrap RTTI to accommodate the additional structure. (In the other implementations, the corresponding operations can pass along the RTTI unchanged.) However, these coercions contribute minuscule overhead; in fact, in a majority of cases, MLton’s optimizations will completely eliminate the RTTI from the final program.

  The implementation distributed with MLton uses the second solution.

  Bonus question: Why can’t one use a universal type to eliminate the use of `Unsafe.cast`?

  - Answer: ???

- MLton (in both of the above implementations) provides a richer `MLRep` structure, utilizing `Int<N>` and `Word<N>` structures.

      structure MLRep = struct
          structure Char =
             struct
                structure Signed = Int8
                structure Unsigned = Word8
                (* word-style bit-operations on integers... *)
                structure <<SignedBitops#>> = IntBitOps(structure I = Signed
                                                   structure W = Unsigned)
             end
          structure Short =
             struct
                structure Signed = Int16
                structure Unsigned = Word16
                (* word-style bit-operations on integers... *)
                structure <<SignedBitops#>> = IntBitOps(structure I = Signed
                                                   structure W = Unsigned)
             end
          structure Int =
             struct
                structure Signed = Int32
                structure Unsigned = Word32
                (* word-style bit-operations on integers... *)
                structure <<SignedBitops#>> = IntBitOps(structure I = Signed
                                                   structure W = Unsigned)
             end
          structure Long =
             struct
                structure Signed = Int32
                structure Unsigned = Word32
                (* word-style bit-operations on integers... *)
                structure <<SignedBitops#>> = IntBitOps(structure I = Signed
                                                   structure W = Unsigned)
             end
          structure <<LongLong#>> =
             struct
                structure Signed = Int64
                structure Unsigned = Word64
                (* word-style bit-operations on integers... *)
                structure <<SignedBitops#>> = IntBitOps(structure I = Signed
                                                   structure W = Unsigned)
             end
          structure Float = Real32
          structure Double = Real64
      end

  This would appear to be a better interface, even when an implementation must choose `Int32` and `Word32` as the representation for smaller C-types.