It appears that strings are not auto-converted

[Katherine Cox-Buday]

This appears to work:

(py4cl:python-eval "\"foo\"")

This appears not to work:

(let ((my-str "foo"))
  (py4cl:python-eval my-str))

My expectation was that the library would handle converting my-str to a python string so that I can do things like pass it into functions, etc. Am I doing something wrong?

[Katherine Cox-Buday]

I was also trying to simplify the error for the sake of reporting the bug, but I have found perhaps what may be my real issue:

I use format a lot to concatenate bits of strings. I noticed today a difference between how output returned from format is pythonized, and how output from say, concatenate is pythonized:

CL-USER> (py4cl::pythonize (format nil "foo"))
"#A((3) BASE-CHAR . \"foo\")"

CL-USER> (py4cl::pythonize (concatenate 'string "foo"))
"\"foo\""

At this point I'm even more convinced I'm doing something unexpected or wrong, but I'm not sure what that might be yet.

[Ben Dudson]

Thanks for trying this out, sorry for the problems. It's one of the things which may be a problem with the interface to python-eval: How to distinguish an input string which is part of a python command, from strings to be passed as arguments? Currently python-eval passes all arguments through pythonize except strings:https://github.com/bendudson/py4cl/blob/master/src/callpython.lisp#L96

This is a workaround but I'm not very happy with it:

(let ((my-str "foo"))
    (py4cl:python-eval (py4cl::pythonize my-str)))

Perhaps pythonize should be exported, or maybe have a function named something like arglist or values which runs its inputs through pythonize so it would be

(let ((my-str "foo"))
    (py4cl:python-eval (py4cl:values my-str)))

or to use it as arguments to a function:

(let ((my-str "foo"))
    (py4cl:python-eval  "len(" (py4cl:values my-str) ")" ))

Any suggestions welcome.

Handling of format output

The type returned by format is simple-base-string rather than string. It seems that this gets dispatched here as it should: https://github.com/bendudson/py4cl/blob/master/src/writer.lisp#L59 but then write-to-string outputs it as a readable array.
Using coerce to string doesn't work, but coerce to (vector character) does. I'll make a PR and add a test or two.

[Katherine Cox-Buday]

First of all, no worries about the "problems"! You've provided a useful library, and it's OK if it has some rough edges :)

I borrowed a page from parenscript and had a go at defining a chain function which might make some of this less pythonic, and more lispish:

(defmacro chain (&rest chain)
  `(apply
    #'py4cl:python-eval
    (list
     ,@(let* ((obj (car chain))
              (result (list obj)))
         (dolist (link (cdr chain))
           (setf result
                 (append result
                         (if (consp link)
                             (append (list (format nil ".~(~a~)(" (car link)))
                                     (cdr (loop for e in (mapcar #'py4cl::pythonize (cdr link))
                                                collect ","
                                                collect e))
                                     (list ")"))
                             (format nil "\".~a\"" link)))))
         result))))

I'm afraid I haven't thought that deeply about the problem since I'm clearing a path while learning something else. I'm hoping this might spark an idea for you though.

This allows both of these things to work:

(chain "\"hello {0}\"" (format "world"))

(let* ((csv-path "/my/csv/path")
       (dat (pd:read_csv csv-path)))
  (format t "~a" (chain dat (describe) (to_string))))

Of course there is still unexpected behavior.

[Ben Dudson]

That is great! Yes, I hadn't thought of that; parenscript looks like a good source source of ideas/inspiration for a nicer more lispy interface over the low-level python-eval. Thanks very much!

[Ben Dudson]

Thanks again for this, it is a much nicer way to do this. I have made some modifications, hopefully it works as you would expect. If you come across problems please let me know.

[Katherine Cox-Buday]

That's awesome work @bendudson! I am currently hacking on ghollisjr/cl-ana. After I get some data in usable shape, I will probably be reaching for this to pipe it into tensorflow. Thanks very much :)