BASICS OF ON-LINE MEASUREMENT

Peter Diem

Internet - The Third Electronic Medium

When President Clinton, in his 1997 State of the Union Address, encouraged his fellow citizens to "build the second generation of the Internet so that our leading universities and national laboratories can communicate in speeds 1,000 times faster than today, to develop new medical treatments, new sources of energy, new ways of working together" this may have appeared to be the typical exaggeration of a political leader. But it wasn’t. Quantum leaps are not uncommon in the development of communications technology. Take this example: Today it requires 4 hours and 15 minutes to download a 6 minute video (55 Megabytes) over the normal copper telephone wire. Tomorrow an advanced cable modem will do this in 44 seconds. This is already a factor of three hundred. And there is more to come. The Internet as a new medium for the distribution of audiovisual content stands obviously only at its beginning.

Three years after the Internet began its triumphant march from the ivory towers of the universities into the living rooms of Western societies, the television industry has woken up, rubbing its sleepy eyes to look at its new on-line companion who has just outgrown his baby knitwear. With around one third of homes in Western Europe equipped with personal computers and up to a tenth of households connected to on-line services, the PC finally gets some attention from television and radio strategists. Will the personal computer draw viewers away from television? Will the PC and TV merge into one all-purpose multi-media set ? Or will there be no measurable change in media use at all?

At any rate, over the last two years practically all television companies (and also the EBU) have created their own Web pages. While most public service stations have set up conventionally styled corporate home pages with general information, mission statements, programme tables, and some teletext, private commercial operators are concentrating more on programme trailers and advertizing banners. Increasingly, however, attempts are being made to send moving pictures and live sound down the line. One of the pioneers in this field is Ultimate TV, a worldwide electronic guide to television, which already offers the feature of "webstreaming", a technique to watch TV via the Internet in real time. After having downloaded a "real video player" as a special "plug-in" to the user’s browser, one can receive a number of programmes offered by American TV Networks. For more information about this refer to: http://www.ultimatetv.com/

Although the picture is relatively small and tends to stutter ("net congestion"), there can be no doubt that on-line TV is kept from entering the market only by the lack of band-width along the telephone network.

But what of the future?

The fact that the growth of cable subscription has been stalled by the ready-to-buy satellite dish has lead cable operators to look for an attractive incentive to win new customers. One such incentive is the cable modem - a device which gives cable system subscribers special access to the Internet. This new 10 Megabit per second gateway into Cyberspace has several decisive advantages over conventional surfing via a modem and telephone line:

  1. Access to on-line services via the cable system’s own server is 300 TIMES FASTER than via conventional 28.8 Mb/sec. telephone-line modems
  2. Text/audio/video stored on the cable server can be retrieved WITHIN A FEW SECONDS
  3. The downloading of content offered on the cable server helps to avoid delay due to EXTERNAL NET CONGESTION (mostly caused by U.S. peak-hour data traffic)
  4. Access to the Internet and other on-line services is UNLIMITED IN TIME
  5. At the same time the family’s telephone line remains FREE
  6. Surfing via cable IS NOT PUT ON THE TELEPHONE BILL

At the moment in several countries such as the USA, the Netherlands, and Austria, field tests are being conducted with the cable modem. For a monthly fee of US$ 50.- in the big cable systems of Vienna and Lower Austria, some 200 households are testing the possibilities of speedy and unlimited access to on-line services. These tests are monitored by a market research project commissioned jointly by the local cable firm and the ORF. In the Netherlands, the cable system of Veenendaal was selectecd for a comparable pilot project. There, a virtual Shopping Mall and access to the Internet via cable modems are being tested. At present, 20 households are part of the project - it is hoped to step the panel up to 750 families. But why should public service broadcasters become involved in on-line at all?

The answer becomes clear when one reflects more closely on the strategic implications of the cable modem and other technologies opening up practical ways to fast AND free surfing:

Especially national public service broadcasters sitting on vast amounts of topical news copy, sound clips and video footage - more and more of it produced in digital form anyway - are the best suited (and most sought after) content providers in their respective markets.

By working in a "trimedial" manner, i.e., producing their daily radio and TV material with an eye on its eventual on-line publication, they can provide cable distributors with news, weather, sports, documentation etc. This material being sped to cable heads by 2.4 Gb/sec. glass fibre lines, cable modem owners can download, store and print text, picture, and sound information at any time - without delay and without additional cost. Very soon compression software, streaming programmes, and local processor chips will be powerful enough to handle even very long video sequences - the road to serious muliti-media network distribution is open.

Note: the above-mentioned facts are not part of the many utopian projects proposed by media gurus, but will be realized in the very near future. Unlike applications so far associated with the Internet (research, e-mail, downloading software, adult pages and the like) the cable modem - and soon the direct-to-home satellite line ("Broadcast Internet") - will be a true outlet for radio and television programmes. Thus in the years ahead, not the flashy corporate homepage but the regular flow of digital radio, television, and gaming output from the principal national content provider into up-market cable homes will be the real challenge. These programmes will also be interactive, offering chat forums and other forms of up-loaded audience reaction. On-line will be a new, profitable field of business - especially for public service broadcasters whose mission is to serve the public with comprehensive and objective information and quality entertainment. In the future for broadcasters the distinction between inside communication ("Intranet") and outside communication ("Internet") will cease to exist. Editors will work simultaneously for three media, and their output will no longer be limited by time or by space. With a view to these possibilities, the ORF has set up a special on-line unit comprising 8 editors who will start to provide a regular ORF Internet service by April 1997. In the long run, there are several possible target groups for digitalized programme output on top of conventional broadcasting:

  1. Households having access to on-line services via a copper-line or ISDN modem
  2. Households subscribing to a cable modem
  3. Households equipped with set-top boxes able to receive Internet broadcast via satellite
  4. Buyers of high-capacity CD-ROM’s (DVD).

Finally, there is also ADSL, a special technology that uses today’s copper telephone wire for fast distribution of compressed content, allowing MPEG files to travel at up to 2 Mb/sec.

Conclusion: The development of on-line into a "third electronic medium" is of great strategic importance for (public service) radio and TV. All relevant aspects of this new technology - from cable modems to Internet broadcast by satellite ("Intercast") - should therefore be carefully studied both by broadcasters and the EBU. After all, the use of the Internet as a new point-to-point medium has also economic aspects such as the sale of programming to third-party distributors and the possibility of carrying advertising. In this respect the use of the Internet is also a great challenge to media researchers: content providers and site owners will soon call on them for precise consumption figures, ratings and demographics. The basics of on-line measurement are therefore laid out in the following article.

Download Time for Music Video (5 min 48 sec./55MB):

Transmission by

Performance

Time required

Telephone Line

28.8 Kb/sec.

4,24 hrs.

ISDN Line

64.0 Kb/sec.

2,08 hrs.

Cable Modem/parallel

500.0 Kb/sec.

15 min.

Cable Modem/Ethernet

10.0 Mb/sec.

44 sec.

Introduction

The use of computers and on-line services in business, media, science, and education is rapidly growing. The penetration of the PC into the private homes of Western industrial societies is in several countries approaching the 40% margin. Here are some topical figures:

PC penetration (% of private homes):

Netherlands

38,7

USA

38,0

Switzerland/Germ.

35,5

Denmark

35,0

Sweden

32,0

Austria

32,0

Great Britain

29,4

Germany/W+E

26,0

The introduction of modems (and soon cable modems) is picking up speed too.German-speaking Switzerland reports a modem rate of 10,9%, for the Netherlands it is 9,7%, while Austria has 8% households equipped with modems.

(Source: Survey by ORF Media Research/GEAR, February 1997).

At present, more than 30 % of Austrian families have at least one computer in their home. 8% of Austrian households own a modem, and 3 % have access to on-line services. 40% of Austrians 12+ use a computer at least occasionally - be it at work, at school or at home. "Daily reach" of PC-use (i.e. having used a PC "yesterday") is 10% of adults - of whom 7% use the machine at work and 3% at home. Use of on-line services on an average day is reported by about 1.5% of Austrians - a percentage equal to about 100.000 persons 12+

(Source: Fessel-GfK, Radiotest 1996, n= 24.000).

On-line services (the word "on-line" is used here as a general term for a number of popular PC-based data transfer applications such as e-mail, WWW, ftp, educational nets or commercial links like Compuserve or America On-line) are beginning to play a very important role in intellectual, economic and social life.
Measurement of their use, therefore, is becoming a major challenge for media researchers: while conventional research is able to report on print, radio, and television usage to the content of clients, universally accepted standards for quantitative and qualitative measurement of the "third electronic medium" (loosely called "Internet" or "Web") are still in the making. This paper intends to contribute towards internationally accepted principles of on-line measurement.

Several Distinct Forms of Measurement

Server-Centric Measurement

Due to its technical basis and point-to-point distribution, origin, direction, time, and intensity of on-line traffic can be logged and calculated by special programmes installed on the servers which provide the channels for such traffic. Thus Web site owners and advertizers can be supplied by the service providers on the basis of detailed logfiles with many types of quantitative data. This service is called "Webtracking" or "clickstream measurement". Parallel to the development of the World Wide Web, a number of principles and "currencies" have already been worked out by which to report on the use of pages, sites, banner ads, target ads etc. While the terminology has by no means been standardized yet, there is sufficient consensus within the industry about the meaning of "clicks" and "hits", "pageviews", "impressions" or "adviews", "click-throughs" or "adclicks", "visits", and "sessions".

A number of specialized firms provide advertizers with more or less sophisticated visitor measurement, analysis and presentation software.

Several glossaries explaining the terminology of server-centric measurement can be found on the World Wide Web (examples below).

"Click-through rates" or "ad click rates" express the proportion of transactions that have lead the visitor of a page from a mere "exposure" to a banner ad onto the hot-linked advertizing copy behind that ad. It is obvious that "Webtracking" allows for a more precise control of factual advertizing contacts than, e.g., recall studies for print advertizing.

To guarantee as much objectivity as possible, auditing firms have begun to offer their services by third-party evaluation of transaction counts. However, this does not mean that the system of clickstreaming does not have severe drawbacks. Among other things it may tempt providers to publish formidable amounts of data as a token of success.

Take as a deterring example the tables which the BBC puts on the Web regularly: http://www.bbc.co.uk/stats/www.bbc.9701.html

While clickstream measurement faithfully depicts hits and clicks along a time-axis (showing daily after-lunch peaks, distribution across weekdays, seasonal traffic)there are a number of grave technical/methodological disadvantages to server-centric Web measurement:

To understand the problems of server-centric interactive media audience measurement and to become acquainted with some ambitious attempts to solve them, it is advised to refer to the following papers:

  1. CASIE Guiding Principles of Interactive Media Audience Measurement (10 pp.)
    http://www.commercepark.com/AAAA/casie/gp/guiding_principles.html
  2. Measuring the Use of the Internet: The Future of New Media (12 pp.)By Barry Kiefl, Director of Research, Canadian Broadcasting Corporation
    http://www.cbc.ca/research/int.html
  3. New Metrics for New Media: Toward the Development of Web Measurement Standards. By Thomas B. Novak and Donna L. Hoffman (29 pp., bibliogr.)
    http://www2000.ogsm.vanderbilt.edu
  4. Glossary of NetCount Terms (4 pp.)
    http://www.netcount.com/f_glossary.html
  5. Nielsen I/PRO I/COUNT: Web Measurement System (2 pp.)
    http://www.nielsenmedia.com/ipro/icount.html

On-line Surveys

The number of projects to measure the use of the Internet by putting questionnaires onto a site has become a deluge. Advertizers, trade organizations, media, high school and university students - interested parties all over the world are attempting to find out who does what with the Web and its content by this method.

The fundamental problem with such research is the fact that the sample used is "self-recruiting" - a procedure which is almost a guarantee for demographic bias. Short of random or quota selection of interviewees, on-line studies will end up with a selection of persons with a special motivation to fill in questionnaires and submit their e-mail address.

Thus in many on-line research projects, aficionados of the Web, students and other young users are heavily over-represented. More often than not, respondents to such surveys are being recruited with a view to acquire future customers. For serious and detailed on-line surveys refer to the following papers:

  1. Georgia Tech Graphic, Visualization, & Usability Center (GUV) - 6th User Survey, conducted in October/November 1996, by which 59,400 unique responses were collected from over 15,000 unique respondents (12 pp.):
    http://www.cc.gatech.edu/gvu/user_surveys/survey-10-1996/
  2. Third MIDS Internet Demographic Survey (Fall 1995, n=1,000), 12 pp.:
    http://www.mids.org/ids3/ids3sum.603

The PC Meter - Precise User-Centric Measurement

The most ambitious and most effective way to measure computer use and on-line traffic has been designed by The NPD Group, Inc., Port Washington, N.Y., who in 1996 introduced the PC Meter, a passive tracking system which uses special software installed in a representative panel of no less than 10.000 computer-owning homes in the USA. This software is able to log even the shortest action taking place in the household’s PC - be it the use of operating system, word processor, computer game, e-mail, browser, Internet or any other application. It is obvious that besides monitoring the US hard- and software market, the PC Meter’s noblest task is to measure the movement of American "screenagers" and "Cybercitizens" through the World Wide Web.

In contrast to the two above-mentioned research methods, the PC Meter renders full demographics of all users in the panel. Furthermore, it is free of subjective bias and incomplete recollection - both of these are major problems of all kinds of studies relying on personal recall. It is no exaggeration to call the PC meter one of the most important developments in media research since the introduction of the electronic measurement of TV use.

The PC Meter may also be given the credit of setting a basic standard for a realistic estimate of the visitors to a Web site/ Web page. Mainly for practical reasons - in order to bring up percentage ratings from the decimals - NPD has defined "REACH" to be the "PERCENTAGE OF ON-LINERS WHO HAVE ACCESSED A SITE AT LEAST ONCE OVER THE PAST FOUR WEEKS". Together with REACH (as the single currency for rating) so defined, parameters such as FREQUENCY OF ACCESS, MINUTES SPENT SURFING, MINUTES PER VIEW, PAGES PER OCCASION etc. are provided to the subscribers of the service.

Mid-1996 sweeps published by NPD show that among the leading two dozens of sites (down to a REACH of roughly 5%) there were many browsers (i.e. programmes opening the way into the Internet), research engines (i.e. programmes scanning the Web for specific contents), and major service providers as well as some universities. Commercial offers that could qualify for the top group were but a few - among which Playboy and Disney.

Is there anything the PC Meter is NOT able to do? Well, at least in its present form, NPD’s software is designed as a purely passive device. No provision is made for what audience researchers call "appreciation" - the qualitative rating of a page along a scale, say, from 0 (=very bad) to 10 (=very good). Furthermore, like any household panel, the PC Meter will disregard on-line traffic originating outside of private homes - such as for business, within organizations ("Intranet"), in libraries, classrooms, and, last but not least, "Cybercafé" surfing, i.e. operating computers provided for visitors of special Internet coffee-shops. Finally, NPD can so far only measure MS-DOS machines while software for Macs is in preparation.

The main challenge of NPD, however, like in all meter measurement systems, is cost. While there is hardly any technical problem that cannot be mastered, a company needs a lot of other income to make up for the enormous efforts and considerable investments which are required to recruit, build, maintain, operate and evaluate a panel of the size necessary to arrive at a reliable sample of on-line users (or "Netizens" as Americans like to call them). In this connection, one must take into account the high mobility of younger, upmarket American PC-owners which may cause a lot of panel churn.

The high cost of a PC Meter service is the main reason why so far no similar project has been realized in Europe. One of the few companies to envisage transferring NPD’s know-how to Europe is GfK Nuremberg, one of the leading German media research firms, which operates a state-of-the-art TV panel of 4,500 households. Unlike in the United States, where local telephone calls are free of charge or cheap, in all European countries local telephone fees are quite high. This is one of the reasons why in spite of high PC penetration, the use of the Web in Europe is still rather low.

The comparatively slow growth of the on-line market in Germany and elsewhere in Europe makes companies such as GfK cautious about early investment. So it might take quite a while until the PC Meter will reach the Old World. The Group of European Audience Researchers (GEAR) - which the author has the honour of chairing until May 1997 Erik Nordahl Svendsen of Danmarks Radio will take over - will at any rate follow closely all relevant developments. For further information please refer to NPD’s PC Meter homepage: http://www.npd.com/pcmeter.htm

Other User-Centric Measurement

It is obvious that audience and advertizing researchers will not sit back and wait until the PC Meter has become available also in Europe. There are a number of possibilities to measure PC and on-line use also by more conventional methods. Service providers, site owners, advertizers, and interested media need reliable data to follow the development of the budding on-line market as well as to optimize their page design and copy in order to arrive at credible rate-cards.

Basically, FACE-TO-FACE, DIARY, AND TELEPHONE RESEARCH (and maybe combinations thereof) are all possible approaches to quantitative and qualitative measurement of computer penetration, PC, and on-line use. FOCUS GROUPS AND SMALL ON-LINE PANELS offer themselves for in-depth pre- and post-testing of sites. From the point of view of cost efficiency and speed, the COMPUTER-ASSISTED TELEPHONE SURVEY (CATI) has the best chances for being employed in pilot research.

The advantage of the personal (face-to-face) interview lies in its possibility to present larger quantities of test material in combination with optical aids. The diary has the advantage of allowing for more detail and to reduce recall weakness, but it is expensive to administer.

In the following, on the basis of an Austrian pilot study conducted by telephone, some of the methodological problems arising from computer market research will be discussed

(Source: Integral, November/December 1996, n=2.000).

Universe 1: Persons with PC at Home

As has been indicated above, the computer began its victorious crusade in offices and universities, continued into factories and workplaces and has now set out to conquer the homes of, especially, but no longer only, the younger people of the industrialized world. From there it proceeds into cafés, hotels and maybe still other premises.

But let’s be careful - what after all IS a PC, what IS a Personal Computer?

When asked about PC ownership in the home without further specification, close to32 % of Austrians 14+ answer in the affirmative. Upon closer scrutiny by means of a second question, viz., to describe the type of the respective PC, it turns out that about 3 percentage points have to be deducted: 2% of respondents will include smaller homecomputers (Ataris, Amigas), 1% included videogames such as Sega or Nintendo in their answer. On the other hand, 3% report private access to TWO PCs and 1% have THREE OR MORE Personal Computers at their disposal at home.

Another indication that the number of Personal Computers available in private households may be somewhat overrated lies in the fact that only 24 % of Austrians report having a printer at home. Although it is clear that it is in no way necessary to have a printer at one’s disposal in order to operate a PC off- or on-line, the difference of about 8 points (equal to one quarter!) is rather high. What are the reasons for NOT having a printer? Is it youth, is it income, is it education or professional status? Is it a combination of these? There is even some evidence that printers are more common in remote regions where computers are required to process more "snail mail" than in urban neighbourhoods. At any rate, any "serious" PC-home will tend to include a printer in its hardware collection. So anywhere between a quarter and a third of the Austrian population - that is the size of the universe of persons having access to a Personal Computer at home.

While around 15% of respondents in Austria say they want to purchase a PC, roughly 10 % of families intend to obtain modem/on-line services within the next year. These figures are relatively high compared with similar technology rolled out over the last years - it can be taken as a sign of comparatively fast development of this segment in the near future.

Universe 2: Access to a PC/Working with it

"Access to a PC" or "working with a PC" can be measured in the following way:

Question: "Do you personally work with a computer - be it at work, at school, at university or at home" (these locations may also be asked separately).

if yes, "Do you work with the PC daily, several times per week, a few times per month, less often/hardly ever."

In Austria, just about 40% of adults report working with a PC at least a few times per month. In the continuous telephone survey "Radiotest", beginning with January 1997, PC-use is surveyed by a "matrix" of questions consisting of the following cells:

Universes 3 and 4: Modem Available, On-line Services Available

Asking for the presence of a modem in the respondent’s household is a less ambiguous question than the one concerning the PC. Possession of a modem in a PC-owning home will in most cases be known and correctly stated - not least due to the fact that the family telephone will have been occupied at times by someone surfing or downloading.

In Austria 6 -7 % of private homes report having a modem, which means that roughly one out of four computer households will have an on-line connection.

About 3% of homes report being provided with Internet or some other on-line service. This in turn means that only one out of ten private PC owners is connected to Cyberspace.

Use of the Internet or of Other On-line Services

From the above-mentioned hardware penetration figures it will be clear that "DAILY REACH" of the Web and other on-line services will still be relatively low.

There are two practical methods of measuring the contact of a population with on-line services on the average day:

  1. The DIRECT QUESTION about Internet/on-line use "yesterday". In Austria this results in 1,5 % affirmative answers.
  2. The FREQUENCY QUESTION as outlined on page # (6) renders 2% of "daily" users.

As one can see there is practically no difference between the results of these two questions. Another fact is also interesting: about ONE THIRD of those who do have the possibility at home to use Internet or other on-line services do NOT make use of that possibility.

Intensity of Use of Internet/On-line

There are two ways of measuring the INTENSITY with which on-line services are being used:

The first one was referred to above - it is by the FREQUENCY QUESTION . By using the frequency scale it is possible to classify users as either HEAVY or LIGHT surfers. HEAVY SURVERS use the services daily or several times per week, while LIGHT USERS have contact with on-line services less often. (Note that these questions do not distinguish between persons using on-line merely for e-mail and persons using it also for surfing in the World Wide Web).

The second method is with the help of a KNOWLEDGE SCALE. Persons who can NAME THE BROWSER they use, KNOW THE NAME OF THEIR or are FAMILIAR WITH SEARCH ENGINES can be classified as the "intensive segment" of on-liners.

Incidentally, the market shares of Netscape Navigator and Internet Explorer in Austria are roughly 45 to 30 % among "Cybercitizens".

Other Demographic Aspects of PC and On-line Users

As we have learned from serious research elsewhere, an even spread of computer access across demographic groups will still take some time. At present, computer use is mainly a matter of the younger age brackets, with the student population somewhat over-represented. By the end of 1996, 47 % of Austrian men were working with computers as compared to 28% of women. But the number of female users goes up steadily.

There is a close relation of PC use with income, education and professional status.
The differences are even more noticeable in the segment of on-line users, where persons 50+ are still hard to find. Daily use of the Internet is highest among managers (21%), followed by middle-income, urbanized 25-39 year olds (20%).

The most active age bracket on the Net are those in their twenties (males 5.6%, females 1.4% daily use). Average daily use of the Internet is also high among professionals, executives and owners of businesses - almost 5% of these report to have used some on-line service "yesterday". The same figure applies for college graduates. Students report using on-line on an average day at a rate of around 3%.

The demographic group with the highest number of Internet contacts comprises only 2.3 % of Austrians: it is the regular audience of Blue Danube Radio, a public-service station broadcasting in English from Vienna and sharing its frequency at night with a programme for teenagers. A daily on-line use of 8.9% within this rather sophisticated group shows that a good command of English is one of the prerequisites for profiting from, and having fun on, the Web. It is interesting to note that a group of similar size (and obviously also of similar life-style) is the segment of Austrians who do not own a TV set: in this group daily use of on-line is 4.9%.

To sum up:

Daily exposure to the Internet or to other on-line services in Austria (and certainly in most of Europe - think also of local telephone costs !) is more than three times above its 1.5% national average in the young, male, educated upmarket segment, where PC ownership generally exceeds 50% of households. 15 to 20% of this group say that they intend to join the Internet community within the next one or two years.

Measuring Reach

While "NET REACH" of a site is defined in our pilot research as "the percentage having visited a particular site AT ALL", "REACH" (based on the model of the PC Meter) is defined as "the percentage having visited a site WITHIN THE LAST FOUR WEEKS". Both are calculated on the basis of PERSONS HAVING ACCESS to Internet/on-line services. It is well understood that the question used is a "closed" one, with site/page names read to the respondents in a rotated sequence. Under these circumstances there is no doubt about the fact that -

a) the number of sites/pages which can be submitted to respondents is limited by the maximum practical duration of a telephone interview (in Austria about 20 minutes) and by the willingness of interviewees to follow prolonged questioning,

b) recall thus measured is a relatively "soft" figure with a number of variables possibly intervening - such as page quality, off-Web advertizing pressure, personal interest, social prestige etc. But so long as there is no PC meter in Europe, we will have to live with that. (We also have a year or two remaining before the Swiss-made radio meter "Radiocontrol" will have entered the market to give radio measurement a new basis.),

c) other measurement techniques (such as face-to-face surveys with optical presentation of site images, paper or e-mail diaries) will start to compete with telephone research in order to probe deeper into the problems of measuring on-line use and Internet activity.

What were the results of the Austrian pilot studies?

First, it was a big relief for the researchers to observe that (net) reach figures for sites/pages differed substantially from one another. Much like in the PC Meter results, there were very few two-digit ratings. These were achieved mainly by media such as TV, leading newspapers or magazines (in this section we did not ask for browsers or search engines). The majority of other sites, however, were reported to have a 1% to 4% reach.

Qualitative Measurement

From our experience with APPRECIATION in TV meter measurement and ACCEPTANCE INDICES in other qualitative research, we applied the 0 to 10 SCALE (which is widely used in Europe) to the judgement of sites/(home)pages. In spite of the fact that the low number of cases ruled out completely reliable figures, we noticed that the results again spread widely. There were quite a number of pages which received indices well over 8.0, while only a few sites were judged below 6.5 (which according to our long-time observations is the "QUALITY MARGIN" on that particular scale).

Bearing in mind the limitation again of intervening "image variables", we nevertheless feel entitled to go on in the forthcoming months with qualitative measurement along the 0-10 scale in our monitoring. With reach figures expected to be still rather low in the near future, this will help site managers and advertizers at least to estimate site quality. We hope to accompany our research by other qualitative approaches, like, e.g., group discussions, copy tests, eye-tracking etc. to optimize page layout, building an "electronic quality panel" from telephone research, i.e. asking a representative number of on-line households whether they would be willing to participate in surveys on specific subjects. It is not decided yet in which way such "on-line panelists" will be surveyed in practice. It will be either by e-mail - probably the most elegant way - or by diskette, telephone or diary - or a combination thereof.

The Next Steps

Serious Internet and on-line research is difficult to start for a number of reasons:

  1. So far, low penetration of modem/on-line access requires relatively high samples
  2. The number of providers, site-owners, pages, banners etc. is exploding
  3. There is only one meter service available, the U.S. based PC Meter by NPD
  4. For methodological reasons reliable user-centric studies are quite expensive
  5. The two "cheap" solutions, viz., server-centric clickstream measurement and on-screen self-recruiting questionnaires remain popular and very competitive.

To overcome these difficulties, research will have to be:

This requires speedy harmonization of methodology on a European or, preferably, Atlantic, basis- i.e. including the US and Canada. In Austria, ORF, the public-service broadcaster and principal content provider in the country, has decided to participate in the setting up of a continuous national Internet/on-line survey. The project has been entrusted to INTEGRAL, a middle-sized research institute with a decade of experience surveying electronic media by computer-assisted telephone interview.

The new survey will be called Austrian Internet Monitor (AIM) and will be conducted as a multi-client research project. On a monthly basis of 1.000 interviews trend reports will be published quarterly. AIM has already started in January 1997. It will comprise all aspects of recall research mentioned in this article.

President Clinton, in his "State of the Union Address" of 4 February 1997, challenged the American information industry with the following sentences:

"We must bring the power of the Information Age into all our schools. Last year, I challenged America to connect every classroom and library to the Internet by the year 2000, so that, for the first time in our history, children in the most isolated rural towns, the most comfortable suburbs, the poorest inner city schools, will have the same access to the same universe of knowledge ...
Our effort to connect every classroom is just the beginning. Now, we should connect every hospital to the Internet, so that doctors can instantly share data about their patients with the best specialists in the field.
We must build the second generation of the Internet so that our leading universities and national laboratories can communicate in speeds 1,000 times faster than today...
As the Internet becomes out new town square, a computer in every home -- a teacher of all subjects, a connection to all cultures-- this will no longer be a dream, but a necessity. And over the next decade, that must be our goal."

Our goal, as audience researchers, will be to MEASURE.

Vienna, 3 March 1997
Comments welcome to:

diem@orf.at
pdiem@t0.or.at
www.t0.or.at/~pdiem
Quelle: http://www.t0.or.at/%7epdiem/online2.htm