The Assistive Intelligence Blog

The Task Communication Language (TCL) System

2006-07-12T16:24:00.000-04:00

For those of you who have watched the demonstration video posted earlier and are interested in the innerworkings of the system; I will now briefly describe the process by which the human and agent can communicate as well as provide a listing that accompanies the reasoning over that communication within the system.

I will be throwing out a lot of terminology that I unfortunately do not have the time to define. I am working on a couple of journal articles which will do a better job of describing this system with some brevity. I ask you to either look up the terminology on the web, or hold out for one of these articles.

The system works by first parsing incoming text into a structured internal representation. In the demonstration, this is done ahead of time by hand. However, it is hoped that a parser can be constructed to do this dynamically in real time. The internal representation is based on the meaning-action concept, a marriage between the illocutionary force idea behind the speech-act, the interconnected protocol-like idea of the dialogue act, and the notion of the shared conceptulization of individual concepts expressible through natural language.

This layer, called the shared medium is composed of two components. The concept, or the shared conceptulization of a concept, and an operator, or the illocutionary force placed upon individual concepts.

The system then builds a dialogue manager upon this shared medium by incorporating the idea of rational communication, along with the structured foundation of protocol engineering and methodology. This dialogue manager utilizes a history, a shared concept graph, and obligations in order to process incoming messages and rationalize about them.

This rationalization is performed by a set of pre-defined rules. Only one rule may fire for each top-level operator, and once the rule fires the operator is removed. The rule may produce more top-level operators which will then trigger their own rules and so on and so forth. Rules are prioritized so that one rule may superceed another. In addition, they are dependent on a set of conditions that allow or inhibit the rule from firing.

Operators take four forms. The interaction operator, or the main operators of the meaning-action concepts. The agent-operators, or operators which represent reasoning inside the agent. The helper-operator which assists in maniuplating and checking the structure of meaning-action concepts as well as shared concept graph. And finally, the macro-operator which helps to figure out which rules should fire or which operators should result from specific circumstances.

A listing of the demonstration reasoning can be found in appendix C of my dissertation.

Again, I want to throw something out there for people to be able to look at, and unfortunately, I don't have the time to go into explanations at this point. But hopefully, this will do for now.

Publications Available

2006-07-08T23:41:00.000-04:00

Here are some available publications that relate to the demonstration:

The first is very old, but covers some of the introductory work in extending the capabilities of command-and-control.

John R. Lee, and Andrew B. Williams. “Behavior development through task oriented discourse.” Computer Animation and Virtual Worlds. Wiley Interscience, Vol. 15, Is. 3-4, pp 327-337, June 2004. [PDF]

The second and third give an idea of the overall system, but not very many of the details.

John R. Lee and Andrew B. Williams. "Towards a Theoretical Framework for the Integration of Dialogue Models into Human-Agent Interaction." To Appear AAAI Fall 2005 Symposium on Mixed Initiative Planning, Washington D.C. November 2005 [PDF]

John R. Lee and Andrew B. Williams. "Human Robot Interaction through TCL." HRI 2006 Young Researchers Workshop, Salt Lake City, Utah, March 2006 [PDF]

More to come...

Demonstration Available

2006-06-09T15:37:00.000-04:00

I am privileged to make available a demonstration that represents the last year of my life. Although, as far as media goes, it is rather boring; nevertheless, what it represents, as far as advancements in communication with intelligent agents, is revolutionary.

It is a demonstration of the intelligent control of an assistive agent within a real-time resource management simulation (or real-time strategy video game.) It is important to realize that the text being entered has been parsed into a specialized language. This means what you see typed has to be encoded by hand, by a human, in order for the agent to understand it.

From there, the agent has to rationalize about what is being said and tie it into context using models of negotiation, mutual planning, information seeking dialogue and so forth. The agent interprets the input sequence and responds accordingly.

What you will see is not only a human being able to give orders to an agent, but that the agent can react to those orders in a number of ways. At times, there are arguments, negotiations, mutual planning, discussions, interruptions, and more.

When you watch this demonstration, focus on what must go on inside the agent's mind that allows it to respond. What allows it to communicate back to the user in such a powerful and expressive way?

After you have watched the demonstration, please leave me comments about your thoughts and reactions, as well as questions about the technology and its implications for the future.

[Note: If the file causes permission denied, then the quota has been exceeded, try a mirror]

Download Recommended
44.9MB / 12:15 / 779kbps / 640x480 / WMV - Windows Media / (Captioned)

Mirror Recommended
44.9MB / 12:15 / 779kbps / 640x480 / WMV - Windows Media / (Captioned)

Mirror 2 Recommended
44.9MB / 12:15 / 779kbps / 640x480 / WMV - Windows Media / (Captioned)

Download Small
11.4MB / 12:15 / 158kbps / 320x240 / WMV - Windows Media / (Captioned)

Download Full
82.0MB / 12:21 / 1465kbps / 1024x768 / WMV - Windows Media 9

Mirror Full
82.0MB / 12:21 / 1465kbps / 1024x768 / WMV - Windows Media 9

Futurologists and Robotics, but what about Agents?

2006-02-18T05:10:00.000-05:00

Several futurists predict that robotics is set on course to change our lives. They way we work, live and play. And this change will come within the next decade!

Although history is littered with such 'futuristic visions' of the 21st Century that never came true, many people still put a big stake in futurologists and their predictions about the next decade.

In particular, David Smith claims that robots will "make many middle management and clerical jobs redundant and make people's lives easier by doing menial tasks. They will also become companions."

Although I agree that assistive technologies will be extremely big in the next decade, I feel that robotics are not yet where they need to be. Although locomotion, grasping, speech and vision recognition has come a long way, and companies like IBM are promising great advances in the near future, they are still not yet ready to be placed in the home or workplace.

Let me try my hand at futurology and predict a brief reign of the intelligent agent before the rise of the machines. The majority of the advances in artificial intelligence and personal assistant technology will first reside within computers, consumer devices and appliances. This is where most consumers will first interact with them and the majority of their initial development will take place. Once the robotics are ready, they will quickly find themselves embodied inside, able to move and more actively interact within the world.

Articles:

BBC NEWS | The business of future gazing
Financial Mail | March of the Machines

Death of Aibo: The Abandonment of The Robotic Age

2006-02-18T04:47:00.000-05:00

In what is by now old news, Sony has announced that it will abandon the AIBO robotic dog, along with its robot development team. Is this the burst of what was to be the beginning of the promised age of robotics?

The AIBO's have proved their worth from greatly advancing robotic soccer to increasing the allure of computer science to women and minorities, as well as increase the happiness of terminally ill patients and the elderly. Open source communities have developed many applications and systems for the dog, allowing a great movement in the areas of hobbiest robot software and robotics in education at all levels.

The only answer from Sony? They wanted to focus as a company and the Aibo wasn't in the direction they wanted to go. Sony is shifting back once again into attempting to dominate the entertainment industry, where most of the non military research and development bread and butter resides.

The entertainment industry has lead huge advances in computer technology, simulation, real-time rendering, ubiquitious and realtime media, portability and mobility, and more. If anyone has been successful in robotics for every day life, it has been Aibo because of their entertainment value.

Now that the Aibo is dead, will another company step up? Other than the few that are trying to create simple kids toys, this remains yet to be seen.

News Articles:

Sony to quit manufacturing AIBO, QRIO
What happened to the Robot Age?
For Sony's Robotic Aibo, It's the Last Year of the Dog.
Rest in Peace, Sony Aibo
Man Bites Robotic Dog
Man Kills Robot
Muttricide and Profits at Sony
Aibo put to sleep
Sony scraps four-legged robot pet Aibo
Aibo Collector on End of Robot Dog's Production (NPR-Audio)
Robotic toys are going the way of the dinosaurs

The Beginning

2006-02-08T08:00:00.000-05:00

A journey of a thousand miles starts with a single step.

Every generation seems to have a dream of machines that will serve mankind. We are continually promised in media and society that this dream will be reached sooner rather than later, but it seems like we are no closer today than we were when the dream first began.

We have had the promise of personal assistants such as HAL from 2001, A Space Odessy, or the computer system in Star Trek, The Next Generation; or even personal service robots such as Rosie from the Jetsons. But why don't we have them today? I believe that the technology is possible, and have founded the company, Assistive Intelligence Inc. to work on such technologies.

My journey... is to solve some of the hardest problems of personal assistive technologies.

This blog will serve as a social commentary on the technological innovations of our time, particularly those innovations that impact personal assistant technologies such as HAL or Rosie.