A brief introduction to Maluuba’s Frames dataset. It is designed to help drive research that enables truly conversational agents that can support decision-making in complex settings. The dataset contains natural and complex dialogues with users considering different options, comparing packages, and progressively building rich descriptions through conversation.
Data Collection
For each dialogue, a user was paired up with an available wizard and received a new task.
Task 1
Task Description :
Find a vacation between September 1st and September 8th to Havana from Stuttgart for under $700. Dates are not flexible. If not available, then end the conversation.
Why ?
The setting is simple and the user has a good idea of what she wants. Therefore, the agent only needs to help the user find suitable packages and book one. This situation is the first one that a conversational agent for travel booking should handle.
Task 2
Task Description :
Find a vacation between August 15th and August 25th for 2 adults and 1 kid. You leave from Atlanta. You’re travelling on a budget and you’d like to spend at most 2000. If you can’t find anything, then try ending before August 22nd and increasing your budget by $200.
Why ?
In this case, the user does not know the destination in advance. Besides, the last part encourages the user to make changes to her request if she cannot find a trip corresponding to her dates and budget. Therefore, here, we model both the exploration behaviour of the user as well as the possibility to change the initial constraints. This task is a bit more complicated than the previous one but represents a situation which is likely to occur.
Task 3
Task Description :
You either want to go to New York, Tokyo, Berlin, or Paris from Montreal. You want to travel sometime between August 23rd and September 1st. Ask for information about each package. Compare the packages and pick the one you like best.
Why ?
For this task, the user has to compare options for different cities. Here, we model the case where a user looks at specific destinations and tries to find the best trip. This requires extensive exploration of the database and a memory of the different options which have been discussed.
Frame
A new frame is created every time the user modifies a previously set value (for instance, the user changes the destination). Each package proposed by the wizard also creates a new frame. Frame changes represent the amount of exploration done by the user. The average number of frame changes per dialogue is 3.58, for an average of 6.71 frames created. This shows the variety of behaviours encountered in this dataset: some users will pick the first offer that looks good whereas others will consider different options before picking one.
A Frame Example
| Speaker | Act | Frame |
|---|---|---|
| User: I’d like to book a trip to boston from London on Saturday, August 13, 2016 for 8 adults. I have a tight budget of 1700. | inform(intent=book),inform(dst_city=boston,or_city=London, str_date = Saturday, August 13, 2016, n_adults = 8, budget = 1700)” | Frame_id=1 |
| Wizard: Hi.I checked a few options for you, and unfortunately, we do not currently have any trips that meet this criteria.Would you like to book an alternate travel option? | no_result,suggest(dst_city) | Frame_id=1 |
| User: Yes, how about going to Detroit from London on August 13, 2016 for 5 adults. For this trip, my budget would be 1900 | inform(dst_city = Detroit, n_adults = 5, budget = 1900, ref = [1{or_city = London, str = August 13, 2016}]) | Frame_id=2 |
| Wizard: I checked the availability for those dates and there were no trips available. Would you like to select some alternate dates? | no_result(str_date, end_date), suggest(str_date, end_date) | Frame_id=2 |
Frame Tracking
The main observation is that decision-making is tightly linked to memory. In effect, to choose a trip, users and wizards talked about different possibilities, compared them and went back-and-forth between cities, dates, or vacation packages.
Current systems are memory-less. They implement slot-filling for search as a sequential process where the user is asked for constraints one after the other until a database query can be formulated. Only one set of constraints is kept in memory. For instance, in the illustration below Figure 1 , when the user mentions Montreal, it overwrites Toronto as destination city. However, behaviours observed in Frames imply that slot values should not be overwritten. One use-case is comparisons: it is common that users ask to compare different items and in this case, different sets of constraints are involved (for instance, different destinations). Frame tracking consists of keeping in memory all the different sets of constraints mentioned by the user. It is a generalization of the state tracking task to a setting where not only the current frame is memorized.
Adding this kind of conversational memory is key to building agents which do not simply serve as a natural language interface for searching a database but instead accompany users in their exploration and help them find the best item.
_Figure 1 : Most dialogue systems implement goal-oriented conversations as a sequential, slot-filling process. Each dialogue state is either augmented with new information (left) or overwritten (right)._
_Figure 2 : Solving frame tracking would enable dialogue systems to memorize all the information provided by the user and allow comparisons between items._
More Complicated Examples
| User | Wizard |
|---|---|
| Hi im looking for a nice destination that i could go to from Columbus. | I have 2 choices, Vancouver or Toronto, both located in Canada. |
| What options are there for both | I have a few. Do you have a budget in mind? |
| 3700 is my budget at the moment | I have the Obsidian Gem Inn in Vancouver which is a 4-star hotel at 1934.64 or the Hotel Richard in Toronto which is a 3-star hotel at 2393.00. Both are for a 14 day stay. |
| Toronto seems like a better place this time of year, when would the flights be? | I have a direct flight to Toronto departing on August 15th and returning on August 28th. Would you like me to book that for you? |
| would by chance have one leaving on the 19th | I have a flight available departing on August 24th and returning on September 6th |
| let’s go ahead and book this | Perfect. Thank you. |
| User | Wizard |
|---|---|
| Hi wozbot! | To start, just give me some information on where you’d like to travel, your budget, your point of departure, or any other travel info you can think of. |
| Ok great! I’m looking for a trip from August 19th to September 3rd, and I’m thinking of going to Mexico City. I would be departing from Busan. | I’ve got some options available for you. Will you be travelling alone? |
| Yes! For the first time. | Ok! I have 5 days at a 3 star hotel available for 3259.79USD. What do you think? |
| Great! Do you know if there is free wifi and breakfast? | There is free parking and free wifi, but no free breakfast. I also have a free breakfast hotel available for 4002.28USD which leaves Busan at 7:00 am - Saturday, August 27th. |
| The second option is how many days? | The second option is also 5 days. |
| I’ll keep the second Mexico City option in mind. I have some other places that I’d like to consider. | Ok! Where would you like to go? |
| What about Santo Domingo for the same dates? | I can get you a hotel in Santo Domingo for 2446.15USD. |
| 5 days? | 9 days actually. |
| With free breakfast? | Free breakfast, wifi, and parking! |
| That sounds great to me. I have two other cities that I’d like to consider. Let’s start with Toronto. What can you get me for the same dates with free breakfast? | I have a 7 day stay in Toronto for 2732.81USD. |
| I like the sound of that. What are the departure and arrival dates for this one? | Wednesday, August 17th to Monday, August 22nd. |
| Excellent. Can you see what’s available in Athens? | Would you like to hear only options with free breakfast in Athens? |
| Yes please! | Unfortunately I have nothing available in Athens with free breakfast within your date range. However, I can get you a hotel with free parking and wifi for 3998.37USD. |
| Not interested. Let’s go back to the Toronto package. How many stars? | For the package priced at 2732.81USD in Toronto the hotel has a 4 star rating. |
| Does it have a spa? | There is no spa. |
| I will still book the Toronto package. I can’t wait! | Ok! I will book that package for you. Enjoy! |
| Great thank you so much wozbot! |