Better end of user input conditions

[Hannes Mehnert]

First of all thanks for down! This is a great tool, much easier to use than other OCaml REPL.

I'm usually humble and make typos all the way (and since I forget about the multi-line string literal syntax, I need a toplevel for testing), so sorry for bothering you. This is with down 0.0.1.

a good one:

# let foo = {|foo
  bar|};;
val foo : string = "foo\nbar"
#

an unexpected result:

# let foo = {|foo
  bar;;
# let foo = {|foo
  bar|};;
val foo : string = "foo\nbar;;\nlet foo = {|foo\nbar"
# 

it looks like down accepted the initial ;; (part of the multi-line string) as end-of-input (and down draws a new prompt), while this was inside of a multi-line string literal..

# let foo = {|foo
  bar"|};;
  

it looks like down expected a closing " (though this was part of the multi-line string) -- the return after ;; doesn't finish the input.

# let foo = {|foo
  bar"|};;
  
  
  ";;
val foo : string = "foo\nbar\""
# 

another " and now there's an expression (but where did the second " go?)

there may be other examples with multi-line strings that fail -- the above I tested against OCaml toplevel (that behaves much more what I expected).

[Daniel Bünzli]

Ah crap. Thanks for the report.

Basically there is this function to determine when we have an answer to give to the OCaml toploop. While it does take care to handle (multiline) string literals it does not handle quoted strings.

More precisely the has_semisemi function needs to be made aware of quoted strings and disable semisemi detection while it's inside one.

[Hannes Mehnert]

since you pointed to your implementation, it is also wrong for let foo = "\\";;

FWIW, here's an attempt that seems to work reasonably well for my test cases

    let has_semisemi s =
      let rec loop s max i in_str = match i > max with
      | true -> false
      | false ->
          let in_str', more = match in_str, s.[i] with
          | in_str, '\\' -> in_str, true (* skip \ *)
          | `No, ';' -> `Semi, false
          | `Semi, ';' -> `Fin, false
          | (`No | `Semi), '"' -> `Dquote, false
          | (`No | `Semi), '{' -> `Id [], false
          | (`No | `Semi), _ -> `No, false
          | `Quoted cs, '|' -> `Await (cs, cs), false
          | `Quoted cs, _ -> `Quoted cs, false
          | `Dquote, '"' -> `No, false
          | `Dquote, _ -> `Dquote, false
          | `Id id, ('a'..'z'|'_' as c) -> `Id (c :: id), false
          | `Id id, '|' -> `Quoted (List.rev id), false
          | `Id _, _ -> `No, false
          | `Await (_, []), '}' -> `No, false
          | `Await (cs, c :: xs), y when y = c -> `Await (cs, xs), false
          | `Await (cs, _), _ -> `Await (cs, cs), false
          | `Fin, _ -> assert false
          in
          match in_str' with
          | `Fin -> true
          | _ -> loop s max (i + if more then 2 else 1) in_str'
      in
      loop s (String.length s - 1) 0 `No
    in

[Daniel Bünzli]

I'm wondering if maybe the most robust answer to this problem wouldn't be to simply invoke the input with parse_toplevel_phrase until it no longer exn in your face.

[Daniel Bünzli]

Forget about my last suggestion it's stupid. I think Down is trying to be too subtle here. What about: Down considers your input finished when the String.trimmed ends with ;; ?

[Hannes Mehnert]

That should work as well -- I thought your clever loop which detects any ;; (not only at the end of input) was intentional in its semantics. to me, String,trim sounds like a robust solution (according to its documentation it copies, but that should be fine)!

[Daniel Bünzli]

This would break if you try to input e.g. a literal string that has the sequence ;;\n but if you input that by hand it shouldn't be too puzzling as why it happened and you can work around.

It will however break the ability to paste certain chunks of code (namely those that happen to have exactly that sequence of characters), unless #16 can be robustly fixed.

I think I'll go with this less clever way of doing, I'm not too keen on reimplementing a partial OCaml parser in Down.

[Daniel Bünzli]

(namely those that happen to have exactly that sequence of characters),

and in a literal string if ocaml/ocaml#8813 eventually gets fixed.

[Daniel Bünzli]

There's another thing that is mildly annoying is when you refine a multiline input and want to add a new line in it. Basically for doing so you are forced to go at the end to remove the ;; otherwise the input will be accepted when you try to insert a newline.

It would be nice to be able to distinguish between return and enter... but it seems the distinction has been lost somewhere. And at least in my terminal I can't detect any modifier on return which would have been a natural way of inserting a newline unconditionally. An alternative would be to accept only input when the cursor is at or after the ;; but it's rather surprising w.r.t. how most prompts work.

Anyways pushed something in 4dc4050, you can test it via opam pin down --dev.

[Daniel Bünzli]

Let me know if you still see problems on master.