My First Impressions of Smalltalk

As a disclaimer, I've only been using (Squeak) Smalltalk for a couple weeks, and haven't written anything major in it yet, so this is currently just my first impressions. But wow, what a first impression. The "turtles all the way down" object model is absolutely delightful to work with, and having the whole development environment so tightly linked is incredible. It's like going from a complied-only language to a language with a REPL and realizing how clunky and slow your development flow used to be: Having not just a repl, but a whole organic environment that you can play with and introspect into...The analogy that came to mind was that the difference between programming in Smalltalk and other programming in languages is like the difference between building a bridge and gardening.

I had briefly tried mucking about with Smalltalk previously, but my Emacs-centricism made me avoid Squeak in favour of GNU Smalltalk which, while doubtless a fine implementation, did not really demonstrate to me the joy of Smalltalk the same way Squeak does. Now that I've started to use Squeak a little more (went through "Squeak By Example", now in the process of trying to write some Seaside stuff), I'm starting to understand why Smalltalk is held up there with Lisp in terms of languages that smart people like.

For me, the hard part about learning Smalltalk was wrapping my brain around the idea of your all your code being in an image, as a series of living objects, rather than existing only in flat text files which are briefly given life when run, then filt out of existence. It was initially kind of hard to accept what seemed a disgusting loss of control: I want to be able to have all my code in front of me dammit, not scattered to the winds, where I have to use some kind of GUI Browser (gross!) to see my code, and even then only being able to look at one object or method at a time! I think I've come to realize, however, that this paradigm has definite advantages. For one, being physically unable to view massive blocks of code all at once forces you to write well-factored, conceptually-separated code right from the get-go.

I haven't had the chance to make anything real yet in Smalltalk, I'm thinking of starting to write some webapps using Seaside, which seems quite enjoyable...Smalltalk seems well on it's way to becoming one of my favourite languages—While I've been a Smug Lisp Weenie since high school, I think I may very well soon become a Smug Smalltalk Weenie :)

Lisp, According to Me

Many other people have talked about Lisp at far greater length and in far more depth than I can. I therefore begin by assuming the reader has at least passing familiarity with Lisp and the oceans of ink spilled in its name.

My Road to Lisp

I started learning Lisp in high school, with (Gambit) Scheme, after being inspired by Steve Yegge's Tour de babel. I read On Lisp and wrote reams of Scheme to implement all sorts of fun things (looking through my old stuff, it seems I managed to generate more than 7 kLOC of Scheme during grade 12). Eventually, however, I realized that I was merely writing libraries, and not actually doing anything with them. This, along with a certain sense of curiosity, led to me taking up Common Lisp.

While I appreciate the conceptual purity of Scheme, I also find complex, powerful systems to be deeply intriguing (more on this in my upcoming post on Perl), so I immediately fell in love with CL. The sheer number of things you can do with it, even beyond macros...CLOS (probably my favourite object system), its unique (AFAIK, but I'm sure I'll be corrected if mistaken) and incredibly system of restarts and exceptions, ASDF, optional type tags...I could go on for ages. As Steve Yegge put it (albeit in a manner not meant to be flattering to CL):

• Scheme is an exotic sports car. Fast. Manual transmission. No radio.
• Common Lisp is Howl's Moving Castle.

While it's true that Common Lisp may not be exactly elegant...which would you rather have, a sports car, or a gigantic, semi-sentient, ambulatory castle with magic powers? I rest my case.

I have used Clojure a little bit, but mostly as a means of learning my way around Java without actually having to learn Java. Quite pleasant, and very nice to see a Lisp dialect that doesn't shackle itself to the legacy of the crufty CL standard, nor to the extreme minimalism of Scheme.

That being said, my language of choice eventually shifted from (Common) Lisp to Factor. I found that pretty much anything I could do in CL I could do at least as easily in Factor, but in a much cleaner manner, in a much more tightly-knit community, and crucially, I could see the evolution of the language over time: Unlike Common Lisp, which has been basically unchanged for longer than I've been alive, Factor is still growing and evolving, and knowing I can (and have, in some small way, I may flatter myself to think) help shape the future direction of the language is a powerful incentive.

Well, that went off on a bit of a tangent (mayhaps I should do an entry of Factor?), so let me try to summarize my attitude towards Lisp. While I was once convinced that Lisp was the language, more experience with other languages has led me to begin to question that. It is indeed a very enjoyable language to program in, and macros are certainly the cat's pajama's, but I'm not sure if it's really that much better than some of the interesting newer languages on the scene, such as Haskell, Factor, et cetera. So...a very nice language, one that I had a long infatuation with, but not necessarily the end-all be-all of languages.

Been a While...

Well, it's been ages since I wrote anything here, as school as been consuming my life...but summer is nearly here, so hopefully I'll be able to start writing again. In the meantime, a brief summary of what I've been up to:

• I've been doing a bit of factor stuff on the side, but nothing worth contributing.
• I wrote about 5 kLOC of PIC assembler for a school project, to control a "robot". Not the most pleasant experience.
• I started playing with perl's Catalyst, and found it to be pretty nice...more grokkable than RoR, for me at least.
• Just started trying to learn Squeak Smalltalk, which is quite a beauty thus far.

Not much, other than that...second-year engsci barely gives one time to sleep, much less have fun. I have been considering taking advice I was given concerning this blog and starting to write about various other topics, besides programming. I think I may enjoy that..

Postscript

I take a great deal of pride in knowing a large number of programming languages...so, in order to give myself more motivation to write, I'm going to soon be starting an n-part series of my impressions, opinions, and snarky comments about as many different languages as I can. Should be fun!

Advising Factor (Aspect Oriented Programming, if you're nasty)

I've been pretty busy the last couple of weeks with school, so I haven't been able to get any real (read: non-school related...I am not a fan of Verilog) programming done. So, the other day, I decided to try and implement something neat in Factor. After thinking about it for a bit, I decided to try to implement what is known as "advice" in Lisp and "aspects" in Java.

I had tried to do something similiar to this in Scheme a couple of years ago, but that was not pretty: Since functions in Scheme are just symbols bound to functions, I had to keep track of the advice to call around each function externally (e.g. in a globally-scoped table). The alternative was to redefine the function to add the advice directly into the body, which was even uglier, and didn't allow you to undefine advice. Gross. In Factor, however, this is made much, much, much easier by the fact that words are tuples. This allows me to easily annotate and add properties to words (and I do mean easily--the whole vocab is only fifty-six lines (plus 21 of docs and 63 of tests (more tests than code, yeah!))).

How it works

The implementation of advice is actually quite simple: When we first advise a function, we annotate it to call call-before, call-after, and call-around. These words look properties of the word (named before, after and around, respectively), which contain hash-tables of the given type of advice for the function and call them. The before and after type advice is quite easy to implement (just call a list of quotations, either before or after the main body (which, as Slava points out, really should be a macro or some such for performance reasons)), but around is a little trickier...But, I am very tired now and I have a calculus midterm tomorrow morning, so I'll write about that part sometime soon.

Oh, and if you're interested, you can grab my code from git://factorcode.org/git/jamesnvc

Hopefully not much more on extra/lisp

I've been working on it for far too long, but I think I'm almost done the Factor side of extra/lisp. I've done a little more work since I last wrote and now the only thing remaining is, unfortunately, something of an open problem; Namely, the propagation of locals into literals (more specifically, cond within lambdas).

So, what have I done?

I've only really made three changes since I last wrote, which are as follows:

1. Allowing multiple forms in the body of an expression.

   This was something that was really just due to my carelessness in the first place: When I wrote the s-exp to quotation translator, it would only take the first form. This first manifested itself as a problem when trying to write begin as a macro.

2. Begin as a special form

   I originally had begin as a macro, but soon realized that this was untenable, since the stack needed to be cleared after each form: That is, we want (begin 1 2 3) to leave just a three on the stack, but if done as a macro, the stack would look like 1 2 3. So, begin is now a special form, which essentially just wraps with-datastack and a drop around each form in the begin.

3. Changing macro-expansion time

   Oh boy. This one is embarrassing. Originally macros were expanded as the forms were being called. Given that macros being called at compile-, not run-time is really the whole point of macros, I have no excuse. But anyway, it's been fixed now.

So, what's left?

At this point, the only thing not working satisfactorily is locals and cond. Since I've defined if in terms of cond, this means that things like ((lambda (x) (if x ...))...) won't work, because the x isn't properly visible in the if. I'm not entirely sure how I'm going to fix this...but I'll figure it out soon!

Postscritpt: I guess I'm on Planet Factor now? Thanks Slava!

Update: As per Slava's comment, locals in cond now work in git, so everything is working now! Yay!

Blogging from Emacs

As a brief diversion from the Factor stuff I've been working on, I wanted to just share a bit of information that someone out there may find useful.

g-util.el and various frustrations

Since I first saw it, I wanted to use the gblogger-* functions for emacs that g-util.el gives. However, I ended up having a bunch of problems that I was unable to fix and soon gave up. However, yesterday I decided to have another go at it and managed to get it working! It turned out to only require two changes, but they didn't seem to be well-documented anywhere...So, without further ado, here's how to get the g-util stuff working in emacs (your mileage may vary, of course):

1. Make sure the shell is set to bash. I initially had the shell set to zsh, which resulted in parsing errors
2. It seems that w3m-buffer doesn't work as the g-html-handler. I set it to switch-to-buffer, which seems to work

And that's it! Enjoy blogging via Emacs!

Factor and Emacs!

As previously mentioned, I've been working on an implementation of Lisp in Factor. Although it's been tough, progress is being made. However, I'm taking a short break from that to work on something just as fun - writing a (better) factor-mode for emacs.

While Factor does come with a factor mode (in factor.el), it has some warts that I don't really like. For one, it doesn't support customization (via customize-group). I'm also not too fond of it's font-lock (that is, syntax highlighting) support, so I'm working on fixing that too. Once I get that working, I'm also going to add some factor-specific movement functions, and prehaps the ability to look up definitions of words and the like. I'd really like to have eldoc-like display of the stack effect of the factor word at point. That would probably entail having a nice inferior-factor mode too, which should be entertaining.

That's it for now!

So yeah, just a quick summary of what I'm up to now. Emacs is just so much fun, not only to write code in, but to write code for! Next time I write, I'll hopefully have made more progress on both factor-mode and extra/lisp.

Factor and Lisp, part three

Since last time I wrote about the state of my lisp-in-factor implementation, not much actual forward progress has been made. Instead, I was embroiled in a struggle to figure out how I was to properly implement the passing of arguments to lisp functions, specifically in the case of function calls.

The problem:

First, some background on my lisp converter: It essentially works by converting lisp forms to factor quotations in the following manner: (foo bar baz) ⇒ [ bar baz ] T{ lisp-symbol f "foo" } funcall (By the way, the T{ ... } is a literal tuple, which you may know as an object. So, T{ lisp-symbol f "foo" } is the literal (i.e. you could type that directly into a Factor listener and it would recognize it as a lisp-symbol object) for a lisp-symbol object whose delegate is f (which they usually are, since delegates are deprecated) and whose name field is "foo") In other words, we get a quotation containing the arguments to the function, which funcall will look up and then call appropriately. Certain primitive forms (e.g. quote, lambda, etc) are built-in primitives that are translated at parse time, but everything else will follow this general form.

Okay, that works...but what if we have something like this? (list 1 2 (list 3 (list 4) 5)) Hm...well, if we translate it the same why we did above, it ends up looking like this: [ 1 2 [ 3 [ 4 ] T{ lisp-symbol f "list" } funcall ] T{ lisp-symbol f "list" } funcall 5 ] T{ lisp-symbol f "list" } funcall Now, this is a problem...the argument list for that outer list is a literal quotation, which includes inner quotations, lisp-symbols and funcall words. This means that the arguments to list, rather than numbers and other lists (which is clearly what is intended here) will be quotations, symbols, and words...which is not what we want at all.

So, how to fix this? Well...I tried a few, fairly hackish methods, such as doing a pretty hideous reduce over the quotation inside funcall, to find occurences of the funcall word, then execute funcall on the last two elements of the quotation...blah, gross. I had been thinking...what I basically want to do is call the argument-list quotation, but somehow keep the result of calling it in a quotation. I toyed with the idea of making everything in the argument list into quotations, then doing a [ call ] map over the argument list (that is, evaluate each element of the quotation, putting the result into another list). However, doing this would require some fairly significant refactoring, and would make the rest of the code much less elegant, as the argument list shown above, rather than [ bar baz ], it would have to be [ [ bar ] [ baz ] ], or, even worse, [ [ 1 ] [ 2 ] [ [ 3 ] [ [ 4 ] [ ... ] ] ] ]...you get the idea. Very gross. However, while on #concatenative the other day, I asked slava about the possibility of writing a word that word that would take a quotation, call it, then put everything that calling the word put on the stack into a quotation...

Will that work?

It turns out that a superior version of the word I was looking for already exists: with-datastack. This word takes an array, which acts as the stack, a quotation, which it calls, and returns the array/stack, which now has the results of calling the quotation. For example: { 1 2 3 } [ + ] with-datastack . would output { 1 5 }. This can now be used to easily rectify my problems! In funcall, before looking up the variable name and calling the resultant quotation, it does 1array [ call ] with-datastack >quotation to the quotation which contains the argument list. Simple, easy, and clear!

What next?

Although I've said it before, it looks like I'm nearly done the factor side of this...All I have left to do is fix up my implementation of macros and provide facilites for loading and interpreting/compiling lisp files and I'll be just about done! At that point, I'll start bootstraping the lisp implementation in lisp itself, but the real "hard work" will be done. Wish me luck!

Factor and Lisp, part two

I've been continuing my work on extra/lisp, and I've finally realized the solution to my current problem, that of macros. The solution, in a nutshell, was to replace the arrays I'd been using to represent lisp s-expressions with actual linked lists. After Slava pointed this out to me, I began work on a simple implementation of the lists. However, my work had not gone far, when the good doublec suggested that, as there was a great deal of overlap, I move my extra/lisp/conses to extra/lists and merge it with extra/lazy-lists (now known as extra/lists/lazy). This took a while, as I also ended up refactoring lazy-lists to use new style accessors and then had to go through all the vocabs that used lazy-lists (which actually wasn't that many) to figure out which needed to be USING: lists, lists.lazy or both.

And then, things get interesting!

After this, things began to get odd...A test starting failing in extra/lists/lazy. It seemed that >>nil? (which, if you're not familiar with factor is a "new-style" accessor: You'd use it like obj new-nil?-val >>nil?, and would leave the modified obj on the stack - elegant, n'es pas? (the converse of that would be nil?>>, which would work like obj nil?>>, and would leave obj's value of nil? on the stack)) was being called on the wrong type of object: That accessor is supposed to be called on a memoized-cons, but was being called on a lazy-filter. This seemed patently impossible to me, as the only place that the word was called was in a method of memoized-cons, meaning that the only possible value that could be on the stack was a memoized-cons.

n After puzzling this one over for a while, and vainly trying to use Factor's normally very usefully walker to determine what was happening, I brought the matter to the attention of the gurus of #concatenative. At the recommendation of doublec, I tried inserting print statements in the word, to figure out where the object was changing from a memoized-cons to a lazy-filter.

The results left me even more puzzled than before.

My descent into madness intensifies

Initially, I tried just printing out the object that >>nil? was dispatching on: As expected, I saw that, while the first dozen times it was called, it was receiving the proper memoized-cons object, it was indeed ending up with a lazy-filter. Okay...that shouldn't be happening, but I already know that it is, so that's at least somewhat expected. What happened next was, however, quite unexpected.

"Well," I thought. "I can see that it's a lazy-filter here...but it must be a memoized-cons when it enters...I know, I'll put print statements throughout the word, so I can see just where it changes!"

Now, below is the word sans debugging print statements:

M: memoized-cons nil? ( memoized-cons -- bool )
    dup nil?>> not-memoized? [
        dup original>> nil? [ >>nil? drop ] keep
    ] [
        nil?>>
    ] if ;

(if you don't understand this, go to the Factor home page, and start learning)

After adding various debugging print statements, it looked like this:

M: memoized-cons nil? ( memoized-cons -- bool )
    "Entering nil?: " write dup class . dup nil?>> not-memoized? [
        "True branch: " write dup class . dup original>> nil? [ "Now at >>nil?" write over class . >>nil? drop ] keep
    ] [
        "False branch: " write dup class . nil?>>
    ] if "\n" write;

So now, after reloading lists.lazy and running the test case, I get a bunch of output like this:

Entering nil?: memoized-cons
True branch: memoized-cons
Now at >>nil?: memoized-cons

Entering nil?: memoized-cons
False branch: memoized-cons

Entering nil?: memoized-cons
True branch: memoized-cons
Now at >>nil?: memoized-cons
  

So, ready for the weird stuff? Here're the last two calls...

Entering nil?: memoized-cons
False branch: memoized-cons

Now at >>nil?: lazy-filter
  

Um...what? The print statements at the begining of the word, and on the conditional branch that was taken aren't being called. It seems that it's somehow jumping into the middle of the word, without calling anything prior to this one section.

What Next?

Well, I have no idea why this happening. doublec hypothesizes that I've exposed a bug in method dispatch. I'm not sure what else I can do, if that is indeed the case...I guess I'll wait until Slava returns and see if he can figure it out.

In the meantime, I guess I'll keep working on extra/lisp, which was the point of this whole exercise, was it not?

Update:The next day, I went through the code some more and realized I was looking at the wrong place: The bug was actually in the code for lazy-filters, and had to do with the transititon to new-style accessors. The skip word needed a drop at the end, since the new >>foo has the stack effect ( obj newval -- obj ), while the old version it replaced, set-obj-fo had the effect ( newval obj -- ).

Factor + Lisp = It Doesn't Get Much Better

So, I've started work on my first real Factor project: Writing a implementation of Lisp in Factor (which is referred to as "extra/lisp", as that is where it lives in the Factor repo). I'm having some weird problems with local variables and transformations, but it's going pretty well, and I am having tons of fun with it.

So, how does it work?

The structure of the Lisp compiler is fairly simple: First, it parses the Lisp code from a string into a Factor data structure using Chris Double's (a.k.a. doublec) awesome peg.ebnf, which allows you to just write out the EBNF for your grammar. The really neat thing about this is that, rather than your typical parser generator, this creates a "packrat" parser, which supports both direct and indirect left recursion. Very cool!

After the string is parsed, I then run a simple recursive transform on the code. Such is the beauty of Lisp, that I only need about five special cases (currently lambda, quote, if,begin, and cond - and yes, I know I don't need both cond and if, but it's easier this way). In general, the transformation turns (foo bar baz) into [ bar baz foo ] , albeit with some funny tricks to deal with variadic functions.

Current Status

Well, right now, it's failing when calling lambdas. Apparently, in the lambda-quotation [| x | x ], the two x's are not eq?. Not quite sure why, but I'm working on it. Once I get that working, all I really have left to do is do macros (which shouldn't be too hard, as I can just piggy-back off Factor's MACRO: (or so I hope)), and then I can start bootstraping it in Lisp. Should be fun!

Addendum

By the way, if you're interested, you can take a look at what I've done so far at git://factorcode.org/git/jamesnvc, or take a look in the main Factor repo (under "extra/lisp") as Slava pulls it in.