Giters

Shen-Language / shen-cl

Shen for Common Lisp (Unmaintained)

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Reintroducing Shen code

rkoeninger opened this issue · comments

commented

Brought up in Shen-Language/shen-sources#82 (comment)

When moving this codebase into GitHub, I replaced files written in Shen, like backend.shen with their CL equivalents as Shen binaries were not as readily accessible and bootstrapping made the build process more difficult.

Now that there are hosted binaries for all major platforms, Shen code can be re-introduced to this repo. backend is a clear choice for being written in Shen, as it originally was, and is written in a way that is idiotic idiomatic to Shen and noisy in CL.

Considering another goal was mine was to:

  • produce a source distribution of shen-cl, a shen.lisp or package of .lisp files (shen -l build-kernel.shen)
  • that shen.lisp can then be loaded up by SBCL and use SB-EXT:SAVE-LISP-AND-DIE to generate an executable (sbcl --load build-executable.lisp)

Given this was the design, we need a program that generates CL from KL and writes it to a series of files. This should be doable without having any CL source code.

Could the project be 100% Shen?

commented

You probably still need Common Lisp for the primitives. Such code doesn't benefit from being written in Shen anyway. The compiler itself on the other hand (backend.shen) does, probably not worth it to go much further than that.

commented

Maybe some code wouldn't benefit from being written in Shen, but in the alternate design I refer to, I don't think you need CL source checked in the repo. All CL would get generated. That's at least possible if not desirable?

We're starting with a shen executable that has a full working shen environment, including primitives, which then generates code that defines the primitives. This could would probably just look like:

(write-lisp Stream
  [(protect DEFUN) shen.add [x y] [+ [shen.double-precision x] [shen.double-precision y]]])

to generate the primitive we have now:

(DEFUN shen.add (X Y)
  (+ (shen.double-precision X) (shen.double-precision Y)))

One minor benefit of generating this literal code with Shen is the package macro could be used to automatically qualify these functions with shen-cl.. Granted, the code is being emitted almost verbatim.

(package shen-cl []
  \\ of course, there would be other emit functions and more primitives
  (define emit-primitives
    Stream -> (emit-lisp [(protect DEFUN) add [(protect X) (protect Y)]
                           [+ [double-precision (protect X)]
                              [double-precision (protect Y)]]]))
commented

Sure, but whats the advantage of doing that instead of just writing the primitive in a .lsp file that you load? Writing the first version is more cumbersome than writing the lisp code itself, and also reads worse.

For the stuff that is written in Shen, yes, I would not include the generated lisp. In Shen/Scheme I use this script to go from the Shen code in the repo to the .scm files, the same would work here and is probably easier because there is no need to adjust things to work with the Scheme module system.

commented

@tizoc Do you use a shen-scheme executable to render Shen as a part of the Shen/Scheme build or can you use any shen exe?

commented

You can use any Shen that behaves correctly, both the compiler and the build script are fully portable Shen.

I used to do it manually with either Shen/Scheme or Shen/SBCL interchangeably, I now use the binary of the Shen/Scheme 0.18 release, see the makefile. But the plan is to keep it portable.

Also, the .travis.yml file

commented

@rkoeninger backend.lsp behaves just like backend.shen did right? other than code cleanups and making everything live in the ":SHEN" package and handling the lisp. prefix in the compiler instead of using a macro, I think not much changed, but I'm not 100% sure.

I think the easiest way to start is by just reintroducing backend.shen as it was and start from there (at least thats my current plan)

Some useful lists of commits:

commented

Both backend.shen and backend.lsp were in the original zip file downloaded from shenlanguage.org. I saw the .shen file wasn't being used and didn't come up with a way to conveniently work it into the build process, so deleted it.

In order to make this fit into the build process, we're going to have the Makefile download the previous build of shen-cl from GitHub automatically and store it adjacent to the bin/ I guess? Like in shen-sources?

commented

Yes, thats the process I follow with Shen/Scheme. I download a binary from an earlier release and use that to bootstrap the current version.

commented

When I removed that backend.shen file (that was a couple years ago now) I was trying to find a way to simplify the dependencies between repos. I wanted to get it to a point there it was a Directed Acyclic Graph (diagram) where there would be minimal bootstraping and if there were cyclic dependencies, they would be kept inside a repo. Re-adding Shen code makes it much more complicated (diagram). Oh, I see there's already a PR open.

Eh, I have a lot of thoughts on this and Shen, but they don't really matter since it sounds like Tarver doesn't want Shen to change in any substantive way. Don't mind me, just drifting through an existential dilemma.

There's no better language at manipulating Shen than Shen and it would probably be better if ports had more similar implementation approaches.

commented

@rkoeninger the backend.lsp file can still be part of the repo, it is not like the backend changes that often. Each time backend.shen changes, backend.lsp can be regenerated and added back to the repository, leaving the dependency complexity the same as it is now.

There's no better language at manipulating Shen than Shen and it would probably be better if ports had more similar implementation approaches.

This is the motivation behind this change, working with backend.lsp is much much harder than working with backend.shen. That means that in its current form it is far likely for meaningful work to happen to the compiler of this port.

The first version of the Shen/Scheme compiler was written in Scheme, I even used a pattern matching library that made things better, but one day I decided to rewrite it in Shen, which turned out to be far easier, and in the process I was also able to generate better code (because it was much easier to experiment and handle Klambda code correctly).

In Shen/Scheme I don't include the generated .scm code, but thats just once choice, doesn't have to be the same way for this repository.

commented

Eh, I have a lot of thoughts on this and Shen, but they don't really matter since it sounds like Tarver doesn't want Shen to change in any substantive way. Don't mind me, just drifting through an existential dilemma.

Missed this. But I think either the list (or direct email) is a better place for that topic than this issue.

As far as the open source version goes, It doesn't have anything to do with Tarver, it is about keeping compatibility with SP, what is in the books (the main documentation for Shen) and the official website. If compatibility is broken, then the language is not Shen anymore, it is something else. I don't know what you have in mind, but I think most things can be done either without having to touch the kernel at all (as extensions), or by doing so without breaking compatibility.

commented

Solved by #39