8.25.2006
9:03 AM

I've just spent a bit of time working with Syntax, a Ruby module made for doing syntax highlighting. I figured that since Io syntax was so simple, it would take little effort to make it work for me. Well, hey, I was right. Here's a little example of the results:

List docSlot("splat(*names)", "Assign one value from the list to each given symbol, starting from index 0.")
List splat := method(
    names := call message arguments map(arg, arg name)
    if(names size < size,
        Exception raise("Too many values to unpack")
    )
    if (names size > size,
        Exception raise("Need more than #io{size} values to unpack" interpolate)
    )
    0 to(size - 1) foreach(i,
        call sender setSlot(names at(i), at(i))
    )
    self
)

List => := List getSlot("splat")

fibo := method(n,
    a := 0
    b := 1
    
    0 to(n - 1) foreach(i,
        list(b, a + b) => (a, b)
    )
    a
)

fibo(10) println // 55

And here's the implementation:

require 'syntax'
require 'syntax/convertors/html'

class IoTokenizer < Syntax::Tokenizer
    IO_IMPORTANT = %w{
        and
        Block block
        call clone
        do
        getSlot
        List list
        Map message method
        newSlot not Number
        Object or
        print println
        self Sequence setSlot
        to
        with write writeln
    }
    
    def scan_string
        start_region :string
        loop do
            chunk = scan_until(/"/)
            if chunk[-2] and chunk[-2].chr == "\\"
                append chunk
            else
                append chunk[0..-2]
                break
            end
        end
        end_region :string
    end
    
    def step
        if comment = scan(/(#|\/\/).*#{EOL}/)
            start_group :comment, comment
        elsif comment = scan(/(\/\*).*?(\*\/)/)
            start_group :comment, comment
        elsif scan(/"""/)
            start_group :punctuation, '"""'
            start_group :string, scan_until(/"""/)[0..-4]
            start_group :punctuation, '"""'
        elsif scan(/"/)
            start_group :punctuation, "\""
            scan_string
            start_group :punctuation, "\""
        elsif digits = scan(/\d+(\.\d+)?(e\d+)?/)
            start_group :digits, digits
        elsif punc = scan(/[\(\)\[\]\{\}]|=|:=|;/)
            start_group :punctuation, punc
        elsif word = scan(/[A-Z][a-zA-Z_:\.]+/)
            if IO_IMPORTANT.include? word
                start_group :"important upcase", word
            else
                start_group :upcase, word
            end
        elsif word = scan(/[a-zA-Z_:\.]+/)
            if IO_IMPORTANT.include? word
                start_group :important, word
            else
                start_group :normal, word
            end
        elsif operator = scan(/[!@$%^&*+|\-]+/)
            start_group :operator, operator
        elsif
            start_group :normal, getch
        end
    end
end

Syntax::SYNTAX["io"] = IoTokenizer