Email comment to my previous post from someone wishing to stay anonymous:
The GC for IBM's RVM (Java VM) is written in Java The reason
to do it in Lisp is because GC needs to be much smarter than it
is today For example, profiling memory usage to split objects
into "hot" and "cold" parts Specializing the allocater on the
fly Dynamically allocating stack based GC regions Working
*with* the application to improve memory behavior, rather than
cleaning the heap blindly and dumbly
This intimate interplay between an application and it's runtime is
already happening in the JVM and CLR Shouldn't Lisp be ahead of the
curve?
The short answer:
I was writing in the context of what is practical for SBCL, not what should be done for the platonic ideal of a CL implementation.
The long answer:
As far as I know, nobody is using a Common Lisp implementation as the platform for implementation research, which is where one would most likely find the sorts of optimizations described above. Nor is anyone financing a team of tens or hundreds of developers for writing and maintaining a CL implementation unlike with the JVM and CLR.
By my estimate there's been 1-2 man-years of time spent on SBCL development this year. A large amount of that time is spent maintaining old code instead of writing exciting new features. I'd be surprised if any of the other 6 (?) free and active CLs had any more time spent on them. It's hard to estimate how much time is spent on the core implementation of LispWorks and Allegro (as opposed to the IDE, Lisp-friendly object databases, Prolog implementations in Lisp, ODBC and Corba bindings, or whatever that commercial implementors need to do to remain commercially viable), but surely it's of the same magnitude.
So, no. Lisp shouldn't be ahead of the curve. With the resources that are available, I doubt one could use these exotic techniques and still end up with a production quality implementation. If there were more resources available for making a production quality implementation, there's plenty of lower hanging fruit to be picked first. OTOH if the resources were available for making the research without making it production quality (stable, maintainable, performant, etc) then that code wouldn't have a place in SBCL. Probably not in any other currently existing CL implementation either.
Please note that I'm not saying that SBCL development is going to stop since the Lisp Community Central Resource Planning Committee isn't allocating sufficient resources to it or that the sky is falling in some other way. The point is rather that trying to justify a hazardous rewrite by how it would make some future development easier makes no sense when that future development is almost certainly never going to happen in any case.