Now we have a DEFSTRUCT

Cody, with his usual speed and critical eye for details, has finished the work on DEFSTRUCT. The major addition is the :include option. While we're stomping bugs that crept in (not many), this seems to be solid. As usual in CLforJava, this is a hybrid of Lisp and Java coding. For example, the instances themselves are instances of Java classes, where each slot is a Java field. There are Java methods to access and alter instance values via Java (as well, of course, via Lisp). General information about a struct class, for example the Lisp names of struct slots,  is stored in Java Annotations.

Structs with included components are consist across redefinitions. For example, a struct BAR that includes FOO such that the type is BAR and FOO. If FOO is redefined, a BAR instance will still be a FOO instance. New instances of BAR will use the redefined FOO struct. Both the new and old BAR instances are FOOs. All access functions will still work for both instances. It is up the user to take care of the use of accessors under redefinition. A FOO-A accessor that worked on the old FOO may not work with an instance of a redefined BAR that uses the changed FOO.

One Last Patch...

With regards to Compiler Birdie...

Ok, I swore that I'd leave the current compiler be (hoping it be trampled by the new Lisp-based compiler). But in looking (I shouldn't have) in how I handle bindings for special variables in LET, I found a way to stuff in lexical bindings for special parameters. It's not the right way, but it's good enough to let me compile the LOOP macro.

Here's I'm going to do. Let's say that I have a form (lambda (x) (declare (special x)) ..do stuff..). A function application of this lambda would look like:

(let ((#:g42 (some form))) 
  (%function-application the-lambda-reference #:g42)))

At this point, the compile ignores the 'special' declaration - because I didn't bother in the early days. The current compiler would turn this into a Java method call to the function instance with one argument. I propose to add one more transform in the current compiler. When it notices the 'special' declaration, it will the form into this:

(lambda (x) (let ((x x)) (declare (special x)) ..do stuff..)).

The current compiler does know how to deal with a LET binding for special variables. Easy. What's the catch? Not so easy. The elements in the parameter are not being evaluation at the wrong time. If the lambda list looked like:

(lambda (&optional (x 42) (y (- (incf x 42)))) 
  (declare (special x y)) 
  (+ x y))

The function application form would be:

(let ((#:g42 42) (#:g43 (incf x 42))) 
  (%function-application the-lambda-reference #:g42 #:g43)))

The new transform would make the lambda:

(lambda (&optional (x 42) (y (- (incf x 42)))) 

  (let ((x x) (y y)) 

    (declare (special x y)) 

    (+ x y)))

In this instance, x would not be bound until the LET form in the body. So, a quick hack. But not a long-term fix.

What's left?

As of now, we have a working Lisp system, although it's missing some big pieces: CLOS, Conditions, and a full Type system (it doesn't handle subtypes). The new compiler will fix problems with we have with compiling Loop and the Pretty Printer. It will also cut down the amount of JVM code by at least a factor of 2 and make it run much faster than that. The new compiler is built around Structure Analysis algorithms. This enables of the good things a compiler should have: tail-calls, better closure handling, dead code elimination, loop optimization, etc.

CLOS is the key to access to Java libraries. The package system is extended to handle Java packages. Later the FIND-CLASS function will locate Java classes directly as will FIND-SYMBOL and Reader will locate Java fields and methods.

Other pieces are to bring CLforJava up to the current version of Unicode (it is at Unicode 4.0 now), better documentation, and a help system based on Java Help.

And lots and lots and lots of tests!

What's going now

This semester I have 3 students working on some intricate components.

• A full implementation of DEFSTRUCT. This is a hybrid component of Lisp code and some interesting JVM code. The JVM code also uses Annotations in a rather novel way to link included structs.
• A TRACE function using the Java Proxy system.
• A useful GUI. This brings the issue of threading since the GUI is built in Swing. As we like to say: non-trivial.

For my part, I fixed a rather intricate set of bugs in our current compiler. Took from July to make it work, but it's amazing that worked that well. This was our original bootstrap compiler, never intended for heavy work. But we've patched, and patched, and patched. Now we're a the point where we can make a new, modern compiler in Lisp (work in process).

Welcome to CLforJava

Finally, we have a blog set up. It's only taken us 8 years, but some things take some amount of aging.

For those of you who don't know, CLforJava is a project to create a new implementation of Common Lisp from the ground up and using CS undergraduates at the College of Charleston. CLforJava differs from other CL implementations that run on the Java Virtual Machine in that it is intertwined with the Java language. It's major goals are that a CL user can access any Java library via CLOS with no Foreign Function Interface needed and a Java programmer can access CL as a Java library. Since it is being built in a CS department, it has the side-effect of teaching students how to work in a development environment that mimics that of modern software engineering processes.

CLforJava is an Open Source project and under the MIT license. For more information, go to http://clforjava.org/. There you can find more information about the project in the Wiki. There is also a discussion forum you are welcome to join in as well as read several reviewed papers.