Franco (2000-2001)
A unifying theme for me has been the nature of information access.
From lexical semantics for search engines (PhD), through
Information Extraction for the health intelligence (DSTO) to, in
1998, deciding that the best way to refine a query is with a
conversational interface.
AskJeves is the right idea, but
we have very little idea of how to do a conversation that negotiates
the common ground.
After attending ICSLP'98 and having a few months at the Edinburgh
HCRC language technology group,
I returned to Defence Science and Technology Organisation (DSTO,
Australia) to a nicely funded programme exploring Embodied
Conversational Agents (ECA) as virtual assistants for Command and
Control.
Franco, an ECA in a defence 'data cave'.
|
The aim was to have the ECA act as a means of
accessing a range of data sources, including several databases. Rather
than seeing the problem as one of mapping the spoken word into an SQL query,
our plan was to have Franco know about data sources and negotiate
a shared understanding between the database designer and the user's view of the
problem. This would require world knowledge and in order to characterize
exactly what knowledge Franco would need, we set up a Wizard of Oz experiment
with one of the office staff, referred to here as KT. The outcome of
these experiments were:
- KT made plenty of mistakes - seen as an IE problem, KT's hearing had a 15 percent "fact" error rate
- KT didn't need to know much about the world. Although she knows that cars are vehicles, she did not use such information in her conversation
- What KT did spend effort on was being polite. Indeed at the time, politeness seemed to be everything.
That work is written up here: [Wallis et al
2001] and includes a description of using an Eliza like system to
engage the human and information extraction techniques to provide the
"understanding". It also describes the notion of using CTA
and CRC
cards to document a Wizard's thinking and using BDI as a model for her "folk psychological" reasoning.
The challenge is to collect enough of the right kind of knowledge
about language to provide content for ECA conversation. It takes a
certain kind of personality to sit down and write enough rules to
create something like ALICE.
Moving to Melbourne University, I had a
student re-run the KT experiments and attempt to implement an
(telephone based) conversational interface using JACK.
Hui's conclusion was that it didn't work. The major problem was
that we were treating our Wizard as an "subject expert" and
interviewing her to find out why she (thinks she) did what she
did. The trouble is that we are all expert language users
and the interviewer end up asking apparently dumb questions.
The solution (several years later) would appear to be to embrace that
expertise and use ethnomethods.