daniel bigham.ca

Exercise 24: Basic context
October 14, 2008

Summary

The goal of this exercise is to implement basic support for the contextual words he, she, his, her, it, and its.

Context must be supported within a statement and across statements. For example:

Within a statement: "Daniel likes his bike."

Across statements: "Daniel is 27. He likes running."

If an entity is_a instance and is_a male, then it qualifies as a he.

If an entity is_a instance and is_a female, then it qualifies as a she.

If an entity is_a instance and is neither a male or a female, it qualifies as an it.

A very simple strategy will be used whereby any token which is interpreted to be a he, she, or it will become the contextual resolution for its respective class until it is replaced by the next match.

The entities context.he, context.she, and context.it will hold this information once a statement or question has finished processing.

The is_a instance relationship will be used to specify whether an entity is an actual instance of an object, rather than being a concept/class of object.

A short form notation will be introduced such that if an entity is defined as, for example, i:DanielBigham, an implicit relationship DanielBigham is_a instance will be created. All objects that should fulfill context must have this relationship.

Add basic support for multiple sentences. For the time being, simply split on ". ".

Tasks

Add support for the i: syntax and apply it where necessary so that all of the appropriate entities have the is_a relationship.

Add an IsInstance property to IEntity.

Add basic support for multiple sentences. For the time being, simply split on ". " and "? ".

Design and implement a Context class which will be used for resolving contextual words within the transformation process.

Add a ContextType enumeration with He, She and It as members.

Add a hash named localContext to the Context class that maps a ContextType to an entity. This will be used within the transformation process where we don't want to update context.he, context.she, or context.it since any time we consider an interpretation we might end up abandoning it.

Add properties He, She and It to the Context class, which return the current context for each respective type.

If the context isn't found in the localContext hash, get it from the context.he, context.she, and context.it entities. (Thus, the Context object will need a reference to the brain state)

Implement a ProcessToken class, which will take a token as its input and update the local context as needed.

Implement a Clone method.

Whenever a token is processed during the transformation search, call context.ProcessToken.

Add a Context property to the Fragment class, and set it with a clone of the current context each time a fragment is added during the transformation search.

Add a new transformation output type called TransOutput_ContextToken and thus a new token type called ContextToken. The syntax should be:

he -> context(he)
she -> context(she)
it -> context(it)

Update context.he, context.she, and context.it after a statement/question is successfully parsed.

Web UI

Click here

Disappointment with voice recognition
October 13, 2008

My initial playing around with Microsoft's Speech API was quite successful... its speech recognition accuracy seemed very solid, even if I took my headset off and rested it beside the monitor, or put in on the ground. (Wow!) That was until I tried using it in dictation mode, which increases its vocabulary from a few words to a few thousand words. Suddenly it was recognizing a short sentence correctly with < 5% accuracy.

I figured by reducing its vocabulary to say 150 words I'd get the accuracy back, but I couldn't find any easy way to do that. As a work around, I created a custom grammar that supports statements consisting of 1, 2, 3, 4, 5, or 6 words, each word needing to come from a list of 150 possibilities. Even then its accuracy was remarkably poor. For example, here is what it recognizes "My name is Daniel" as on five different attempts:

"My name is Daniel" was recognized as:

my name is Daniel

I name is Daniel

my name is Daniel

nineteen's Daniel

"I live in Waterloo" was recognized as:

"mon"

"man"

"I man name Waterloo"

"Bigham eighteen mon"

"mon"

"I live name one million"

...

In other words, about 50% in the best cases, and < 10% in many other cases. I would have hoped that the accuracy would have been at least 95% with such a small vocabulary.

Exercise 23: Grace
October 13, 2008

Summary

Construct a Windows application in C# that runs in the tray and constantly listens for audio input. If the word "Grace" is recognized, then accept audio commands for the next two seconds. Or, if the word "Grace" is detected as the first word in a recognized string, then act on the audio command that is found in the latter part of the string. In addition:

Use a grammar.

Log all activity to a log file using log4net

Respond to the command "What time is it?", or "What is the time?", or "What's the time?", or "What's the current time?" by reading the time using the Expressivo Jennifer voice. The time should be spoken as, for example, "four oh nine".

Test the program in a kitchen or living room setup with a laptop where the laptop is across the room from the speaker.

older >>