Introduction to Lemur

Strings

Examples:

# Defining a string
mystr = "Daniel"

# Concatenating
mystr += " Bigham"

# Using variables within strings
myvar = "17"
mystr = "The value is: $myvar"

# Using expressions within strings
mystr = "$val1 + $val2 = #{ val1 + val2 }"

# Length
mystr.length

# Split and join
myarray = mystr.split( "," )
mystr = myarray.join( "," )

# Some other operations
mystr.left( 5 )
mystr.right( 1 )
mystr.reverse
pos = mystr.search( "substring" )

Subroutines

Subroutines are defined using the sub keyword, and closed using the end keyword. Parameters can be typed or untyped.

sub min(x,y)

    if x <= y
        return x
    else
        return y
    end

end

println min(4,2)

# Same thing, but enforcing parameter types
sub min( int x, int y )
...

Optional parameters are surrounded with square brackets:

sub min(x,y,[z])

    if z.specified
        return min( min(x,y), z )
    else
        if x <= y
            return x
        else
            return y
        end
    end

end

println min(4,2,1)

Subroutines can be called in such a way that parameters are explicitly named. This is useful if there are two or more optional parameters and you don't want to specify all of them, or if you want to simply make code clearer by naming certain arguments.

For example, to find the first instance of "a" in a string, starting from position 5:

pos = mystr.search( "a", start:5 )

Another example:

mystr.substr( start:5, len:10 )
mystr.substr( start:5, end:15 )

Subroutines can also be called like this:

min:
x: 3
y: 4

Arrays

Defining an array:

people = ( "Willy", "Nilly", "Silly" )

Without the need for all of the double quotes and commas:

people = (( Willy Nilly Silly ))

Accessing an array:

people[0]

Lemur understands the singular forms of most words:

person[0]

ie.	person[0] and people[0] are interchangeable

Hashes

Defining a hash:

ages =
    "Willy" => 22
    "Nilly" => 28
    "Silly" => 39

Note that the indentation of the items is important.

Accessing a hash:

# Equivalent
ages['Willy']
age['Willy']

Regular expressions

Like Perl, Lemur supports the =~ operator:

if mystring =~ /(\d\d\d\d-\d\d-\d\d)/

println [match 1]

end

Rather than using $1 to access the first match, we write [match 1].

Iterating

Iterating over an array:

# Standard form
for each item in myarray
    println item
end

# Using indentation rather than closing with end
for each item in myarray:
    println item

# Using Lemur's knowledge of singular/plural forms
people = (( Silly Willy Nilly ))
for each person:
    println person

# Using curly braces on the same line
for each person { println person }

# Manually defining a singular form
geese = (( Silly Willy Nilly )
singular of geese is goose
for each goose:
    ...

# Using the -item syntax. Useful if a singular form doesn't exist.
arr = (( 1 2 3 ))
for each arr-item:
    println arr-item

# Using the -number syntax to get the 1-based array index
for each person:
    println "Person $person-number: $person"

# Using the -index syntax to get the 0-based array index
for each person:
    println "$person[person-index] = $person"

Iterating over a hash:

for each key in myhash.keys:
println "$key => $myhash[$key]"

The old fashioned for loop:

for ( i = 0 ; i <= myarray.length ; i++ )

println "$i: $myarray[$i]"

end

Where clause

SQL uses the where clause to limit the results of query. Lemur also supports the where clause, but with arrays:

# Selecting an array subset with the where clause
names = (( Silly Willy Nilly Bob Sue ))
subset = names where name.length <= 3

# Iterating with the where clause
for each person where person.age > 50:
println person.name

# Selecting a single 'column'
ages = people.ages
ages = (people where person.city = "Waterloo").ages

Joins

SQL uses joins to select data from two or more tables. Lemur also supports the idea of joins:

struct Person:
    string name
    string city

struct City:
    string name
    string province

people = ( ( "Ronald", "Waterloo" ),
           ( "Alfred", "Vancouver" ) ) as Person

cities = ( ( "Waterloo", "Ontario" ),
           ( "Vancouver, "British Columbia" ) as City

# Similar to an SQL join
for each ( person, city ) where person.city = city.name:
    println "${person} lives in the province of ${city.province}"

The execution time of this join can be improved significantly if the cities are indexed by their name prior to the join:

# Index the cities
cities.index( :name )

Ordered iteration

for each person where city = "Waterloo", ordered by age:
println "$person.name is $person.age years old"

Group operations

Examples of group operations:
sum, min/minimum, max/maximum, avg/average, median

# Average of an array
values = (( 1 23 445 ))
avg = values.average

# Average of a class member
struct People:
string name
double salary

people.salaries.average

Working with dates/times

mydate = [Jan 1, 2007]
mydate = [1/1/2003]
mydate = [2008-03-03]
mydate += [3 days]
mystr = "1/30/2012"
mydate = mystr.parse( type:date )
println mydate.format( "MM/DD/YYYY" )

File I/O

File operations that automatically open/close a file:

# Reading a file
myvar = [/tmp/test.txt]

# Writing a file
[/tmp/test.txt] = myvar

# Appending to a file
[/tmp/test.txt].append( myvar )

File operations for manually opening/closing a file:

myfile = [/tmp/test.txt].open( "w" )
myfile.println "Hello, world!"
myfile.close

Other filesystem operations:

# All files and subdirectories
[/tmp].ls

# Another way to do it
[/tmp/*]

# Matching a regular expression
[/tmp].ls /.*A/

# All files
[/tmp].files

# All directories
[/tmp].directories

# Delete a file
[/tmp/test.txt].delete

# Move a file
[/tmp/test.txt].move [/tmp/subdir]

# Move everything
[/tmp/*].move [/tmp/subdir]

# Move all files matching a regular expression
[/home/dbigham].files( /[A-Z]{3}.tmp/ ).move( [/tmp] )

Web I/O

# Downloading a web page
mystr = [http://www.danielbigham.ca/]

# Uploading to an FTP server
[ftp://ftp.danielbigham.ca].upload:
    file: "/tmp/test.txt"
    user: "daniel"
    password: "3984393"

# Listening on a TCP port

tcp = new Tcp
tcp.listen( port:80 )

sub tcp.connectionRequest
   tcp.accept
end

sub tcp.dataReceived
    println "Data received: $tcp.buffer"
end

# Creating a TCP connection

tcp = new Tcp
tcp.connect:
    host: "www.google.com"
    port: 80

sub tcp.connected
    println "Connected"
    tcp.send( "GET /blahblah.html HTTP/1.0..." )
end

sub tcp.dataRceived
   println "Data received: $tcp.buffer"
end

# Sending an email

email = new Email
email.attachments.add( new Email.Attachment("/tmp/test.txt") )
email.send:
    recipient: "daniel.bigham@gmail.com"
    subject: "Subject"
    body: "Body"

Generating HTML

Just like ASP, you can embed code within an HTML document:

<html>
<body>
The output from the following code will be inserted into the page:
<% = for each person { println "$person<br>" } >
</body>
</html>

Another approach is to embed HTML in Lemur code. To do this, use the '<' symbol to start output and the '>' symbol to end output. For example:

myarr = (( This is fun ))

<<html>
<body>>

myarr.insert( 2, "really" )

<This text will appear on the web page, but
the less than and greater than signs won't
be there since they are what delimits the
text. We can also use the ASP syntax here
to embed code/values within this text.
Here we go: <% = myarr.join(" ") ><br><br>>

<<b>This is bold text</b>
</body>
</html>>

Parsing command line arguments

# One argument, 'file', which is required
[usage: file]

# Reading this argument
println "Argument: " + [arg:file]

# Two arguments, 'source' and 'dest', which are required
[usage: source dest]

# An option, -v
[usage: [-v]]

# Reading this option
if [arg:v]:
    print "The -v option was specified"

# An option, -v, with a long form --verbose
[usage: [-v --verbose]]

# An option, -f, with a long form --file, which has a required argument, 'value'
[usage: [-f --file value]]

# Reading this option's 'value' argument
if [arg:f]:
    print "The -f option was specified: " + [arg:f.value]

# Some commands take an ulimited number of arguments.
# For example, the 'cp' command would be specified with:
[usage: source[] dest]

# Reading these arguments
for each item in [arg:source]:
    println "Copy $item to $[arg:dest]"

Command line interaction

# Call the 'diff' command and capture all output into an array of lines
output = [cmd: diff /tmp/a.txt /tmp/b.txt]

# Using program variables in a command
file1 = "/tmp/a.txt"
file1 = "/tmp/b.txt"
output = [cmd: diff $file1 $file2]

# Passing lines of input to a command via a variable
[cmd: $input | wc -l]

# Using an environment variable in a command
output = [cmd: echo $[env:PATH]]

# Getting an environment variable
println [env:PATH]

Parsing data files

struct PersonRecord:
    string firstName[20]
    string lastName[30]
    string age[5]

# Parsing a comma delimited file
people = [/tmp/test.txt].parse( type:PersonRecord, format:delimited, delimiter:"," )

# Writing a comma delimited file
[/tmp/test.txt] = people.toString( format:delimited, delimiter:"," )

# Parsing a fixed width file
people = [/tmp/test.txt].parse( type:PersonRecord, format:fixed-width )

# Writing a fixed width file
[/tmp/test.txt] = people.toString( type:PersonRecord, format:fixed-width )

Configuration files

Lemur has a standard method for reading and writing configuration files:

# Read from a config file
config = new Configuration( "/home/dbigham/test.cfg", type:xml )
println "LogFile parameter: " + config.LogFile
println "Number of users: " + config.Users.count

# Write to a config file
config.Users[2].PhoneNumber = "519-393-3929"
config.save

# Another style of config file
# Example contents:
# MySetting: MyValue
config = new Configuration( "/home/dbigham/test.cfg", type:simple )
println config.MySetting

# Support for quotes
# Example contents:
# MySetting: "My Value"
config = new Configuration( "/home/dbigham/test.cfg", type:simple, obey-double-quotes:true )
println config.MySetting

Logging

Lemur has a standard method for reading and writing log files:

log = new Log( "/home/dbigham/test.log" )
log "This will be logged"

# Being more specific
log = new Log:
file: "/home/dbigham/test.log"
pattern: "YYYY-MM-DD HH:MM:SS: "

log "This will be logged"

Error/exception handling

# The 'error:' keyword can be used to catch errors
do
    println "Testing..."
    val = 20 / 0

    error:
    println "ERROR: $error.description ($error.stackTrace)"

    cleanup:
    println "Do any necessary cleanup here"
end

Throwing an exception:

if index < 0:
error "$index is an invalid index", "ERR9393"

"$index is an invalid index" can be accessed via error.description
"ERR9393" can be accessed via error.id (optional argument)

Example: grep

Problem:

Write a simple implementation of the grep command. The program will take two arguments: pattern and file.

Solution:

[usage: pattern file]

for each line in [[arg:file]]

if line =~ /$[arg:pattern]/:
println this-line

end

Example: Display a database table on a web page

db = [mysql://localhost/demo]

rs = db.execute( "select * from mytable" )

<<html>
<body>
<table>
<tr>>

for each field in rs.fields:
    <<td><% = field.name ></td>>

<</tr>>

for each record in rs.records

    <<tr>>

    for each field in rs.fields

        <<td><% = record[field] ></td>>

    end

    <</tr>>

end

<</table>
</body>
</html>>

Example: Inserting webform values into a database

class Person

    string firstName

    string lastName

    date birthday

    string phoneNumber:
        parse: /\d\d\d-\d\d\d-\d\d\d\d( *x\d{1,5})?/

end

person = request.parse( type:Person, trim:true )
db = [mysql://localhost/test]
db.insert( person )

This is a somewhat ruby-like way of using conventions to reduce the amount of work you need to do:

The parse routine can be called on a hash such as the request hash

We pass the Person type to the parse routine

The parse routine tries to map its values to those values in the Person class. It tries the obvious first, and failing that, it contains sophisticated logic for resolving the mappings.

The trim:true argument means that any whitespace at the beginning or end of the webform values will be removed

The parse routine makes use of the parse attribute of the phoneNumber property to make sure the value is valid

When we call the insert routine, the name of the class we're passing, Person, is automatically mapped to the People table in the database. Again, sophisticated mapping logic is used if the obvious mapping doesn't work.

There is no need to escape single quotes as their would be if you tried to create an SQL insert statement to insert values

Example: Advanced string operations: Diff, patch, md5sum

Lemur strings support 'diff', 'patch' and 'md5sum' functionality:

# Diff and patch
diff = mystr.diff( mystr2 )
mystr.patch( diff )

# Md5sum on a variable
println mystr.md5sum

# Md5sum on a file (file->string->md5sum)
println [/tmp/test.txt].md5sum

Example: Searching

Lemur tries to make searching for things easy.

# Find the first occurrence of a substring
pos = mystr.find( "substr" )

# Search using a regular expression; start from a non-zero position
pos = mystr.find( /[A-Z][a-z]*/, pos:32 )

# Search using a regular expression; within a substring
pos = mystr.find( /[A-Z][a-z]*/, substr:32..49 )

# Process all matches
for each match in mystr.search( "substr" ):
println "Match: $match.pos"

# NOTE: The 'find' and 'search' methods do the same things
# except that 'find' returns only the first match, while
# 'search' returns all matches unless it is given a 'matches'
# argument value

# Search words (Respects punctutation, quotation marks, etc)
mystr.search( /pattern/, mode:words )

# Recursively search a directory for all file names that match a pattern
[/tmp].search( /pattern/, recursive:true )

# Recursively search a directory for all matches within the files
[/tmp].search( /pattern/, mode:contents, recursive:true )

# Search the web
results = [http://www.google.com].search( "daniel bigham", matches:4 )
for each result:
println "$result.title: $result.url"

Example: Parsing HTML into the DOM

Lemur includes an HTML parser that will parse an HTML document into the DOM.

dom = [http://www.danielbigham.ca].parse( type:html )

# Traverse the DOM
println dom.body.div['menu'].span[0].a.href

# Get the tag with a specified ID
println dom['id'].href

# Process all links
for each link in dom.links:
print link.url

Example: Food database

Read in a comma delimited file that stores nutritional information. Have the user type in a list of foods, and display a running total of calories, saturated fat, saturated fat percentage, and sodium.

class Food

    string name
    int calories
    int fat
    int saturatedFat
    int transFat
    int carbs
    int fibre
    int sugar
    int protein
    int sodium

end

foods = [foods.dat].parse( type:Food, format:delimited )

foods.index( :name )

while getln -> input

    if foods.find( input, field:name, case:insensitive ) -> food

        eatenFoods.add( food )

        printTotals( eatenFoods )

    else

       println "Not found: $input"

    end

end

sub printTotals(foods)

    satFatPercentage =
      ( foods.saturatedFat.sum * 9 ) /
      ( foods.fat.sum * 9 + foods.carbs.sum * 4 + food.protein.sum * 4 )

    println "Cal: #{ foods.calories.sum }, "
            "Sat-fat: #{ foods.saturatedFat.sum } "
              "(" + satFatPercentage.round() + "%), "
            "Sodium: #{ foods.sodium.sum }"

end