Webology, Volume 2, Number 2, August, 2005 |
| |
|
Yazdan Mansourian
Received August 6, 2005; Accepted August 12, 2005
This is an interview which was carried out at the University of Sheffield on Thursday 14th April 2005. The interview is mainly about the current trends and issues of web search research area and is divided into two parts. In the first section Dr. Spink explains her ideas about the area as a senior researcher who has published many papers and research reports on users' web search behaviours. In the second section Dr. Spink answers a few general questions to explain her views as one of the millions of web users who search this ocean of information everyday.
The interviewer, Yazdan Mansourian, is currently a PhD student in the department of information studies at the University of Sheffield under the supervision of Professor Nigel Ford and Ms. Sheila Webber.
To reflect the originality of the interview what is presented here is a full transcription of this interview as it exactly was without any further revision. Accordingly, the text does not seem like in a well-organized article design and is presented in the dialog format. In this interview Y.M. stands for Yazdan Mansourian and A.S. for Amanda Spink.
Y.M.: for the first question I would like to ask a general question. You have been researching in this area for quite a long time and you have published your findings in many publications. Is it possible to point out a few of these findings as the most significant achievements of your research?
A.S.: well, I think there are probably two areas, the two things that we ended up looking at. First of all, was "trends". I mean, looking at trends over time. So, that has been one of the achievements which is being able to say something about the trends in web search at the time which is still evolving. And the other area is actually modelling the user behaviours and identifying aspects of web search that hadn't been identified before. So, actually coming up with a more realistic model of the behaviours of people who do engaging on the Web, because most of the research that has been done before was being with information retrieval system interaction and alive with professional librarians not millions of people. So, I think we have been able to identify a lot of trends and also aspects that system designers; in particular; and interface designers need to support in terms of user behaviour.
YM: what trends in over time do you think you have noticed with the changes in approaches to searching the Internet?
AS: well, one of the big things we noticed was there wasn't a lot of change particularly in people's querying behaviours that there are a lot of people still doing short queries. And the second trend was probably the shift in the topics. So, probably in the mid to late 90s a lot of searches were about entertainment and when the 90s getting to the 2000 it was shifted towards more to business searching which also reflected the changing content of the Web as more business information; particularly in the United States; more business information is going on the Web.
YM: could it also reflect that fact that a lot of people who were in the teens in the 90s and they are in the 20s now and that is now and would be able to involve to the business?
AS: I don't know of that. I rarely relate these things to demographic shifts because you are dealing with a lot of different people of different age groups and it is not really possible to identify the age group of the people who submitted the query.
YM: what are the most applicable findings in terms of implementation in designing web-based search tools to improve information accessibility on the Web environment?
AS: well, I think it is probably three example areas. The first one was supporting people querying construction and that is an area where there has been a lot of work done in information retrieval systems, but trying to get people to do more complex searches, more interactive searches. You can see the Web search engines are doing that now trying to support people doing query formulation. So, that is one of the areas that we have identified. Another area that we have found was people doing successive searches over time. So, they do search on a topic and they log off and they come back and do another search and that was stimulating a lot of work on search histories. I am going to an interface search workshop at the University of Maryland in June. It is all about web searching interfaces. So, there is an interest there and particularly had a support to the users more longitudinal behaviours. And I think the other area that going to be important is the multitasking. Because as I said that system design people who are now looking at that as an issue. So, that is probably three areas.
YM: in your opinion, what are the most important unanswered questions in the Web Search Research? What do we really need to know about people search patterns on the Web that previous studies have not answered yet?
AS: well, I think the trends issue is ongoing. I mean you have to keep looking at the trends and that is going to be ongoing and I don't think that we have really good user models. We don't have really good theoretical models or even computer models of the user interaction. So, I think it is really wide open area.
YM: as you know, Recall and Precision are two measuring concepts to evaluate the effectiveness of traditional information retrieval (IR) systems, how applicable are these concepts on the Web? If we can not measure Recall and Precision on the Web, how can we measure the efficiency of a web search session? Is it a quantifiable concept? Can we really quantify it?
AS: well, I think Recall and Precision are extremely limited and I wrote a paper on that publishing in IPM 2001 or 2002 and we developed a new web search measure you might want to look at that. It is related to precision and limitation of precision. It is called "A user-centred approach to evaluating human interaction with Web search engines: an exploratory study" [Information Processing and Management, Volume 38, Issue 3, 401-426]. I mean the problem with Precision and Recall is because these measures have derived by researchers. So, if you go out in the street now and ask someone how does web searching and ask measure Precision and Recall they don't know what you are talking about. So, I don't think that any IR evolutionary literature has anything to do with the real users at all. It is useless.
YM: and also it relates to a very well controlled collection of data.
AS: but also measuring Precision doesn't necessarily relate to the effectiveness of the search.
YM: Transaction Log Analysis has been your main data collection tool in your studies, what are the main advantages and challenges in using this method?
AS: well, we have done other data collection methods as well. The advantage of using transaction log analysis is that you are actually looking at a very large amount of data because if you have to collect it from each individual it is very difficult. So, you are looking at a large amount aggregated data. The challenge of analysing that data is knowing what to analyse because you also have to have people who know how to analyse large amount of data using statistical techniques plus you can also use qualitative data; I mean qualitative analysis; but to do it effectively you have to understand the data, you have to know what the different variables are and you have to have a good strategy to how you are going to analyse it. It has limitations about the demographic about the users but I think in any research area you do research in different levels. This is more aggregated level which gives you insight into certain things but not everything.
YM: your studies are mainly based on massive quantitative data of transaction between users and search engines; in your opinion how essential is qualitative research to illustrate the issues which remain unanswered through quantitative data? In fact, how important is qualitative research to provide us with deeper understandings about web search process?
AS: well, actually whenever we have done quantitative we also have done qualitative because if you look at classified queries and look at search sessions and look at reformulation that is all has been done qualitatively. So, the Web log analysis has been a mix of quantitative and qualitative not just quantitative. Also we have done things like surveys and evaluation studies. I think it is a mixture of both. It depends on what you are trying to find out. There is something that you can only do on a very large quantitative scale.
YM: web search research is an interdisciplinary area, what are the major involved fields? What is the closest one?
AS: well, computer science, library and information science, there are people in cognitive science looking at web search, there are people in human factors looking at it, there even people is sociology, I know someone who did a PhD in Princeton in sociology who did a study on web search. So, it is really becoming a huge area across many different disciplines.
YM: what are the distinctive features of the Web Search Research which makes it different with the related areas?
AS: well, I think what has happened is each discipline has an effect on it. So, the cognitive scientists are interested in web searching like to look at certain things, and information science people look at certain things. And human factors take slightly different perspective. It all kind of relates to what existing research they are doing. It is really just building on the current methods that they use; they all are interested in slightly different problems to do with web search. Computer scientists are not doing the same kind of research we are doing.
YM: your research is mainly based on English language; in your opinion how compatible would be your findings in other languages on the Web?
AS: well, we did some work with Alltheweb.com and part of their queries was either Norwegian or German. And they were not that different in that sense. I mean the language was different but the links of queries are not as much different, people made spelling mistakes, they did not using Boolean. So, it is hard to get hold of other language datasets, large datasets but my initial impression is they may not be that different.
YM: your focus in your studies is on the human aspect of interaction between web users and search facilities on the Web, and your findings explains their behaviour while searching the Web, how much web search tool designers are interested in to consider your findings in developing their products to improve search facilities on the Web?
AS: well, I think the search engines that we got the data from are very interested. The people like the guy that I talked about who design the interface are very interested, and the number of citation that we get to the work is quite substantial. One of our papers published in IPM now has got 120 citations. So, I think it is being taken noticed by the people who design the Web tools. And also actually if I was not here I was invited to go to Microsoft. Microsoft is having a launch of their web search engine and they have invited different people today to go there. And the number of companies that I get calling me and asking about things. There are a lot of web search companies that don't have people who understand user aspects.
YM: your research is mainly based on users' interaction with general search engines. Although common search engines are one of the most important search tools on the Web, there are other facilities that people use for searching the Web including subject gateways and specialised search engines. In your opinion, how important is to investigate users' interaction with such resources?
AS: well, I think it is important if you could get access to it. We did actually look at a couple of specialized search engines particularly Meta search engines; we looked at INQUIRUS which is designed by NEC. I haven't really look at those more specialised search engines because we usually have so much data from other general search engines and we hadn't time to look at other specialised ones. But obviously it is pretty important to do that.
YM: you know much about web searchers' search behaviours, in your opinion what is the main secret of the popularity of some search engines like Google?
AS: marketing. Google is successful because they have been very successful in marketing their product. Something that the Web search engines at the moment are trying to identify is how to market and Google has to pay attention to the competitors and marketing is becoming really important. It is like a product, it is a brand. We, researchers are thinking about users' behaviour and their interaction but when you talk to most people that is not what they are using search engines. They use search engines because it is like an encyclopaedia. They buy certain encyclopaedias because they have been marketed.
YM: in your view can you think of a search engine that you regard as being as effective as Google but less well marketed?
AS: AltaVista was quite good, Alltheweb.com was quite good and also you have got a range of Meta-search engines that offer a lot more range of retrieval than a single search engine like Google. And also there are not a lot of comparative different studies of search engines. So, it is hard to really say that one search engine is better than another at this point.
YM: how effectively the metaphor of "Information Foraging" which is based on Optimal Foraging Theory can explain web users' behaviour while looking for information on the Web?
AS: I think my impression of foraging is that information behaviour has different levels and at some particular points people might information forage and at other particular points they may information seek. So, if you just put everything down on the information foraging I think that is too limited. I don't think it can explain. There are a lot of holes in information foraging. If you really look at it, it is very limited. I think it is just one part of information seeking.
YM: when we use "Information Foraging" rather than information seeking, it seems the process of looking for information has some instinctual aspects as well as intellectual sides, what do you think about it?
AS: well, foraging is potentially problematic because you apply food analogies to people information behaviours and I think that is not totally justified. It is certainly not justified by the people who are doing information foraging research.
YM: what do you think about the term "the Invisible Web"? Can we consider some parts of the Web as invisible?
AS: well, if you mean the invisible you probably mean they are not covered by the search engines.
YM: yes,
AS: it gets back to search engine crawling policy. There are a lot of websites that they don't want to be crawled. There are a lot of companies' websites that they don't want to be crawled. I mean the big thing in the United States is how to get your site on the search engines and how to get your site higher up than search engines. And there is lot of websites on the Internet that they don't want to be available to web search engines. So, it might be either people don't want to be in search engines to cover their sites or the search engines don't regard this site as important because each web search engine has a different crawling policy. I think Google crawls less depth in a website than say AltaVista. So, there are different reasons that different websites don't want to be picked up or just be ignored by search engines because they are not regarded as important.
YM: how do you predict the future of the Web Search Research? What are the main lines of enquiry in near future?
AS: well, I think it is going to become a much more interdisciplinary even more interdisciplinary than as being now. A lot of disciplines are interested in different aspects of web search. As it becomes more ubiquitous in society, you have got sociologists interested in; it has political science effects, it has economic implications, it is like if you study the implementation of television, you were implementing a tool on the massive people and say you have to use this tool and you find out most people are not using it terribly effectively. So, there are social implications and economic implications and it becomes a problem. So, I think it is going to become more important.
YM: what would be the priority of researchers in the future?
AS: well it depends on what your priorities will be. Probably priorities would be, to first of all, one of the problems is we don't often know really what is really going on. Because priority for academics would be how to get access to data because you have got a web company knows what is going on in this search engine but this is a little data. So, what is going on in the aggregate? So, the priority probably would be to find out what is going on in the aggregate and look at the trends over time in terms of use of this tool. And also from computer science perspective it would be probably designing more effective tools.
YM: what is your general recommendation to young researchers who have just begun researching on the Web Search Research?
AS: I think what they should do is read the previous research and then identify what the new research areas are. Because young ones don't want to reinvent the wheel and one other thing you have to be careful about it is you have to look at cross different disciplines. I mean you do not look at just your own discipline because you may find that somebody else has already been looking at what you want to look. You have to have a pretty interdisciplinary understanding of it.
YM: what is your general feeling about searching the Web? How do you find it in general?
AS: well, I suppose the problem is I have been using IR systems for more than 20 years and I have a research perspective on it. So, I don't think I approach these things like an ordinary person. My general feeling is they; web search engines; are just pain in the neck. They are just very inadequate for helping people in their information. So, they are frustrating.
YM: how can we improve them?
AS: to improve them! Yeah actually this is a very interesting question because I get asked a lot. I think one of the problems is when we are going to improve them we have to improve interface and improve the systems. What the research is also showing just because you have improved the system doesn't mean the people will search any better. Because also it has to be improvement on the users, because I don't think that all answers are only on the technology side.
YM: because one might be improvement in your eyes might make things more complex or confusing for users.
AS: people don't use features. Every search engine has an idea about what it going to improve web searching and often people don't use those features. Also, the prior studies showed people don't understand how web search engines work. So, they don't understand how to use these systems to get the best out of them. And unless people realize that it doesn't really matter what you do really in terms of algorithms and interfaces.
YM: well, you demonstrated that most users just enter one word and look at just one screen.
AS: well, partly that is because they get a little text box like this and partly because search engines that marketed as quick and easy and do not tell people this is more complex to get more out of it. Roger Summit who designed Dialog retrieval system said that IR systems are problem solving tools, so people have to think them as problem solving tool and move to complex behaviours. Our research showed not dramatic changes in people behaviours over time. So, I think that we are in this society and we think all the answers have to come from the computer scientists.
YM: how long have you been searching the Web?
AS: I think since 93 or 94, since Mosaic I think.
YM: do you have any favourite search engines?
AS: I wouldn't say that I have a favourite search engine; I try to use different search engines. Recently, I have been using A9 which is Amazon.com search engine, which has slightly different search features than some other search engines.
YM: how useful it was?
AS: well, one of the problems is no one search engine covers the whole of the Web. So, if you want something expansive you better use a Meta search engine that picks up from multiple search engines because there is often very little overlap between search engines. So, you can not assume that you are getting a broad coverage from one search engine.
YM: what kind of information do you usually look for on the Web?
AS: probably a broad range of things from academic to shopping, to travel, broad range I think.
YM: how satisfactory are search results for you in general?
AS: that is hard to measure because you don't know what you are missing out of it. It is hard to judge.
YM: because you don't know what you have missed?
AS: yeah,
YM: do you use advanced search facilities of search engines or like the majority of web users just use basic search?
AS: yeah, I try to use advance search features.
YM: can you give an example of a search that you have done recently and you found particularly unsatisfactory?
AS: well, sometimes I search my own stuff. So, you search for your own stuff on one search engine and you come up with a long list of things and then you go to another search engine and you come up with another ranking. Ranking is very unsatisfactory.
YM: do you always manage to find what you want on the Web or do you ever not find what you have been looking for?
AS: I don't know. It depends. Sometimes you do and sometimes you don't.
YM: is there anything else that you would like to add?
AS: I think the whole area of web searching is interdisciplinary, it is hard to get data, there is no particular journal that is totally devoted to the web users' studies at the moment but it is getting more important and the Web search engines are really pushing it but they are interested in user research but they still overwhelmingly interested in algorithm and interfaces.
Yazdan Mansourian would like to express his sincere appreciation to Dr. Amanda Spink because of participation in the discussion and also many thanks to Professor Nigel Ford who arranged the interview session.