Getting the README example running

[seanhinde]

Readme example
I'm trying to get the server on the README page running. Two things so far:

1. Process.make - I'm thinking this should be Process.spawn now?
2. I can't figure out how to get erlang versions of the stdlib runtime modules

To Reproduce
Steps to reproduce the behavior:

1. Create a new rebar3 app
2. Add the caramel README code to src/holder_annotated.ml
3. caramel compile holder_annotated.ml
4. rebar3 shell
5. holder_annotated:start().
6. The process erlang module is (obviously) not found

Expected behavior
The server should run

What I've tried
I tried the obvious of generating .erl files from the stdlib .ml by running caramel compile *.ml in stdlib/{beam|ocaml} but get circular dependency errors:

~/local/bin/stdlib/beam$ caramelc.exe compile *.ml
File "_none_", line 1:
Error: cycle in dependencies. End of list is not sorted.
beam.ml: Beam.ml Stdlib_binary.ml Stdlib_calendar.ml Stdlib_erlang.ml Stdlib_ets.ml Stdlib_gen_server.ml Stdlib_gen_statem.ml Stdlib_io.ml Stdlib_lists.ml Stdlib_maps.ml Stdlib_process.ml Stdlib_supervisor.ml Stdlib_sys.ml Stdlib_timer.ml

This is with the tagged release 0.0.6 that includes the stdlib ml files in the bistro.

Any pointers welcome. I'll have a go at a rebar3 plugin if I can figure out a working manual workflow.

Cool project !

[Leandro Ostera]

Thanks for bringing this up! Excellent point. I'll have to add a flag so there could be multiple compilation targets at one go. Then Caramel can ship its .erl stdlib as well so you can just point to it with erlc's -pa path flags.

Working out a rebar3 plugin would be fantastic 🙌🏼 I'm thinking the plugin could just download the appropriate binary and then you have full control over the paths of the stdlib as well.

[seanhinde]

Sounds great!

[Leandro Ostera]

Oops! Apparently the word "fix" closes the issue 😅

[seanhinde]

Seeing some compile errors in the generated stdlib_process.erl:

===> Compiling src/stdlib_process.erl failed
src/stdlib_process.erl:17: type variable 'A' is only used once (is unbound)
src/stdlib_process.erl:19: function '__caramel_recv'/0 undefined
src/stdlib_process.erl:25: type variable 'A' is only used once (is unbound)
src/stdlib_process.erl:29: function recv/0 undefined
src/stdlib_process.erl:36: ambiguous call of overridden pre R14 auto-imported BIF spawn/1

• use erlang:spawn/1 or "-compile({no_auto_import,[spawn/1]})." to resolve name clash

I'm on OTP 23

[Leandro Ostera]

I managed to clear out all those errors you mentioned, except the recv/0 function being undefined. This is a blocking issue I need to fix first: #10

[seanhinde]

Do let me know if you have stuff you want to get sorted before dealing with this. I can come back :)

[Leandro Ostera]

I think #10 is a good next thing to get done. After that this issue might be resolved.

I'll make a new issue for allowing compiler annotations that can be turned into module attributes, to allow for -compile({no_auto_import,[spawn/1]}) to be set. This seems useful as well.

[seanhinde]

Some more progress today. I have the README example running. Hello Joe! (RIP)

A few things needed:

1. Copied process.erl and beam_io.erl into my project and manually fixed up the errors and warnings in both. The other .erl files (maps.erl etc) fight with the OTP modules with the same names - I think they need to go back to being prefixed even if it means lots of name mangling / unmangling.

2. Provide a stdlib-path. The default stdlib-path for caramelc isn't right any more. I had to use --stdlib-path /home/sean/.opam/default/lib/caramel/stdlib

3. Added a .merlin file to my erlang project to get ocaml editor support. Io wasn't found - I had to change the code to Beam_io:format for merlin to find the definition.

Super nice :)

[Leandro Ostera]

Regarding the --stdlib-path flag, keep an eye on this build -- it fixes the paths so you can use make install and it'll find the stdlib there too. I tried it locally and it seemed to work on Ubuntu 20.04.

[Leandro Ostera]

Regarding the additional .erl files -- currently the only one that is included in the release tarball is process.erl, since the rest should be empty or point to types that should be mapped to OTP types.

[seanhinde]

Closer and closer. stdlib-path default now works for me after make install from latest head.

Still a few smaller things I'm sure you already have on your list:

1. The generated process.erl still has type variables the erlang compiler doesn't like
2. I kind of feel the need for an Io.ml so merlin can work. You have beam_io.ml - I guess this is a workaround for some other issue?

I will have a go at writing some more programs and exploring the possibilities

[Leandro Ostera]

@seanhinde yup! getting there 🚀

The latest head should produce the following process.erl, which for me compiles without warnings on Erlang 23. Could you try on a clean build and see if you get the same file?

% Source code generated with Caramel.
-module(process).
-export_type([after_time/0]).
-export_type([recv/1]).

-export([contramap/2]).
-export([make/1]).
-export([recv/1]).
-export([send/2]).

-type after_time() :: infinity
                    | {bounded, integer()}
                    .

-type recv(M) :: fun((after_time()) -> option:t(M)).

-spec recv(after_time()) -> option:t(any()).
recv(Timeout) ->
  F = fun (T) -> receive X -> {some, X} after T -> none end end,
  case Timeout of
    infinity -> F(infinity);
    {bounded, T} -> F(T)
  end.

-spec make(fun((erlang:pid(), recv(_m)) -> _a)) -> erlang:pid().
make(F) -> erlang:spawn(fun
  () ->
  Pid = erlang:self(),
  F(Pid, fun recv/1)
end).

-spec send(erlang:pid(), _m) -> ok.
send(Proc, Msg) -> erlang:send(Proc, Msg).

-spec contramap(fun((_b) -> _a), erlang:pid()) -> erlang:pid().
contramap(F, Pid) -> make(fun
  (_self, Recv) ->
  case Recv(infinity) of
    {some, A} -> send(Pid, F(A));
    none -> ok
  end
end).

For 2) could you try using the full path? Beam.Io.format? The Beam module is opened by default, but otherwise should still be available.

[seanhinde]

process.erl looking good. no more warnings :)

Beam.Io.format fixes merlin, but compiles to:

beam__io:format(<<"current_state: ~p\n">>, [State | []])

beam__io is not an erlang module in the tree. beam_io.erl exists in your generated files but uses a non existing '__unsafe_fmt'(S, X).

As an experiment I renamed beam_io.ml to io.ml and changed the dune files and beam.ml to match. Everything now seems to work.. but I'm not experienced with ocaml so don't know what else I might have broken.

[Leandro Ostera]

Sweet! Yeah there'll be some .erl files that are not included in the actual release (beam_io.erl being one of them), since they are just compilation garbage.

Could you try making a local alias?

module Io = Beam.Io

Or with an explicit open?

open Beam

[Leandro Ostera]

@seanhinde I figured what was breaking with the module name and now its called just io.ml :) so Merlin should work just fine now.

Quite a ride! 🎢 -- thanks for all the feedback btw, it helped me get the release in order and I'm a lot happier with it now. We covered:

• fixing the stdlib path to work with dune install structure
• ship process.erl as part of the stdlib and make sure it compiles without errors
• fix the Io module name

Let me know if you give that rebar3 plugin a shot :) if rebar3 supports plugins that are nested within a git repo, i'd be happy to include it in here too.

Closing this for now, feel free to open other issues if you find anything 🙌🏼

[seanhinde]

Thanks so much. Super cool. I'll take a look at a rebar3 plugin - no promises when though - I will also spend some time using the system.