Testing process for Advertising effectiveness & Method – BMS Notes

Ads for any media, including radio, television, print (magazine, newspaper, or direct mail), outdoor billboards (on highways, buses, or trains), and the Internet, can be optimised through research. Various techniques would be used to properly collect the required data.

The two main categories of advertising research are viz. both the pre- and post-test phases. Pretesting is the process of testing an advertisement before it is broadcast, with the goal of increasing the chances of creating the most effective ads possible by giving the chance to identify and fix any weaknesses or defects. Following the media run of the advertising, post-testing is conducted. Because the commercials are tested in actual settings, this is also the most realistic option despite being more costly and intricate.

Alternatively, research on advertisements can be divided into two categories: syndicated and customised. To meet the needs of a particular client, customised research is carried out. The research’s findings are exclusive to that customer. A single research study carried out by a research firm and its findings made available for purchase to several businesses is known as syndicated research.

Pre-testing:

Pre-testing, sometimes referred to as copy testing, is a type of customised research that forecasts an advertisement’s in-market performance prior to its airing by examining audience attention spans, motivation, entertainment, and communication levels in addition to dissecting the ad’s flow of attention and flow of emotion. (Young) Ads that are still in their early stages, like animatics or ripomatics, can also benefit from pre-testing. Pretesting can also be used to find weak points in an advertisement to boost performance, edit 60-to-30 or 30-to-15 seconds more skillfully, choose images from the spot for the print ad of an integrated campaign, extract the most important moments for use in ad tracking, and pinpoint branding moments.

Pre-testing is therefore done in order to:

Determine if the advertisement “says” what it was meant to.

Determine how likely it is that the reader will respond.

The following are some examples of frequently used pre-tests:

Pre-tests for advertisements in print media:

Consumer Jury Examination:

A small percentage of customers band together and vote on one or two advertisements out of several that are being evaluated. The jury members rate the advertisements and answer questions such as which one was the most visually striking, which one convinced you to buy the goods, which one did you notice first, and so on.

There are two ways to administer this test: the Paired Comparison test and the Order of Merit Rating. The jury members rank the advertising according to their preferences in the Order of Merit grading test. At the end, there’s a general agreement on the greatest ad copy. However, the best might be the best of the worst.

Two ad copies are compared one to one at a time in the paired comparison test. Every advertisement is contrasted with all the others. Cards are used to record sources. They are condensed. The highest score goes to the winner. Compared to order of merit, this method is simpler. There is good accuracy up to ten copies, after which it declines. The number of comparisons that must be made using the following formula:

(n-l) / 2

The number of adverts to be rested is denoted by n.

Portfolio Evaluation:

A folio contains several fake copies in addition to the actual ads. Next, the folio is seen by the consumer-sample. Next, the customer is questioned regarding what he saw in each advertisement. The best advertisement is the one that plays back the least. After that, though, it’s important to check to see if the selected advertisement is real or fake. If a dummy is discovered, the real one is enhanced in the same way.

Test of a Mock Magazine:

Test advertising are sent to an experimental group to read in a genuine magazine, as opposed to the above method of retaining the advertisements in a folio. The identical magazine is given to the control group, but there are no test advertisements in it. To evaluate the efficacy of I test advertisements, a recall test is later administered.

Asking Direct Questions:

A consumer jury is assembled, and direct questions are used to examine the entirety of the advertisement or individual components. In order to evaluate the attention, read-through, affective, and behavioural strengths of the advertisement, a simple question may be included, or a complex questionnaire may be created. The copy receives points for each component. Every advertisement is graded from best to worst.

Studies on Perceptual Meaning (PMS):

This approach exposes the respondent to test advertisements for a finite amount of time. An instrument that could be used in this exam is the tachistoscope. The respondent is put through a memory test regarding the product, brand illustration, and primary copy after viewing the advertisement.

Pre-tests for advertisements in broadcast media:

Broadcast media may also utilise all of the above mentioned strategies. Additionally, there are specific techniques available for pre-testing radio and TV advertisements in broadcast media. The methods employed are:

In Tests of Home Projection:

To view the test ads, a movie projector screen is erected in the customer’s home. Both before and after being exposed to the commercials, he is questioned. The questions centre on the advertisement and the changes it makes after being seen. It is possible to evaluate the commercials’ strong and weak points.

Tests for trailers:

There are two client categories taken into account. They are invited to shop at a physical store, department store, mall, etc., and are offered discount coupons to buy the brand they are considering. Prospects are invited to showcase their merchandise. As of right now, one group sees the test advertisements while the other does not. Both groups’ coupon redemption rates are calculated, which may provide insight into how successful the test advertisements were.

Test of the Theater:

A questionnaire is given to a group of persons who might make up a captive audience for an entertainment programme. They later receive the complimentary tickets for the show where the test advertisements are broadcast. After seeing these, students are required to complete an additional survey. It evaluates the brand, the product, and the subject.

Test Live Telecast:

Either live telecasting or narrow casting is used to air the commercials. These are not the normal commercials; these are test ads. Interviews are conducted later to find out the viewers’ responses.

Additional Pre-Testing Methods

Test of Sales:

A modest ad campaign consisting of one or more advertising is run prior to the nationwide release of a product advertisement. To do this, two or more test centres are chosen. The advertisements run for a set amount of time, say one to four months, after which the sales responses are recorded. For FMCG products and advertisements that try to persuade consumers to make a purchase right away, it is a very helpful and successful metric.

Tests for Direct Mail:

Different test ads are delivered to a group of prospects that are randomly picked from the mailing list. Next, the orders are compared to each lot in order to gauge the reaction.

Examination of the Physiology:

In this exam, the respondents’ physiological responses are valued more highly than their verbal responses. There are three main tools for doing this:

Camera of Eye Movement:

It tracks the eye’s movements throughout test ad layouts. In order to assess the areas of interest and attention, the eye’s path as well as any pauses are documented.

Galvanometer:

It uses the palm to evaluate the skin’s reaction to stimuli such as sweating. Increased sweating causes the resistance to drop and the stream to flow more quickly. Tension is created. The advertisement is more effective the larger it gets. The method works best for less sensitive advertisements.

Pupilometric or perceptoscope devices:

They document variations in pupil dilation. Dilatation is a sign of attentive reading. Retraction demonstrates the respondent’s distaste for the text that is being read. It assesses visually appealing and intriguing stimuli. James Polk and Eekhard Hess are the developers. To document dilatation, a picture of the left eye is taken.

Some experts also refer to pre-testing as copy-testing. Copy testing is a specialised area of marketing research that involves pre-airing analysis of television commercials. Pre-testing, though sometimes referred to as copy testing, is seen to be the most precise, contemporary term (Young) for the assessment of an advertisement’s performance based on the examination of input obtained from the intended audience. Every test will either determine whether the advertisement is strong enough to air in accordance with business action criteria or will point out areas where editing can enhance the advertisement’s effectiveness. (Young)

All of the tests that we observed were pretests. Here’s another way to categorise pretests or copy tests. The past century of copy testing has been interwoven with four overarching themes.

Measures on Report Cards:

The search for a reliable, single-number statistic that may accurately reflect the total effectiveness of the advertising creative is the first theme. Many report card metrics have been developed as a result of this search. These controls are meant to sift through commercial executions and assist management in deciding which advertisements to run. (Young). Day-After Recall (DAR), the most common copy testing metric used in the 1950s and 60s, was meant to gauge how well an advertisement “broke through” to the customer and lodged a message from the brand in their long-term memory. After Procter and Gamble implemented this metric, it became a standard practise in research.

However, nothing about these exams was particularly clever. Experiments were carried out in the 1970s, 80s, and 90s to verify a connection between the recall score and real sales. Procter & Gamble, for instance, examined 100 split-cable tests spanning ten years and discovered no correlation between sales and remember ratings. (Young) Wharton University’s marketing virtuoso Leonard Lodish also carried out a more thorough analysis of test market data and was unable to discover a connection between sales and recall. Persuasion proved to be a more accurate predictor of sales than recall, according to Harold Ross of Mapes & Ross.

Measures for Diagnosis:

The development of diagnostic copy testing, which has optimization as its primary goal, is the second theme. Advertisers can find innovative ways to enhance executions by knowing the reasons behind high or low diagnostic measurements including attention, brand association, and motivation. (Young)

However, this approach was not flawless. Research firms have established many methodologies to ascertain the attention, brand linkage, and motivation report card measurements. For instance, Unilever “triple-tested” a database of ads using the three most popular methods for assessing branding (Ameritest, ASI, and Millward Brown). The results indicate that each method measures a separate aspect of branding that is unrelated to the other two. (Kerr & Young, Kastenholtz).

Nonverbal Indicators:

The third topic is the creation of non-verbal metrics in response to the perception held by many in the advertising industry that respondents would find it challenging to articulate or rate verbally many of a commercial’s impacts, such as its emotional impact. In fact, a lot of people think that the commercial’s impacts might be occurring at a level lower than consciousness. (Young) Researcher Chuck Young asserts that “our favourite music touches us in ways we cannot articulate … there is something in the exquisite sounds that we cannot verbalise” (Young, p.22)

Measures taken moment by moment:

The development of moment-by-moment measures to characterise the internal dynamic structure of the viewer’s experience of the commercial, as a diagnostic counterpoint to the various gestalt measures of commercial performance or projected impact, is the fourth theme, which is a variation on the first two. (Young)

Moment-by-moment methods began to be experimented with in the early 1980s as a result of a shift in analytical perspective from viewing a commercial as the fundamental unit of measurement to be scored in its entirety to viewing it as an organised flow of experience. The most widely used of them was the dial-a-meter response, which asked participants to indicate their judgement on what was on the 011 screen at that very moment by turning a metre, in degrees, toward one end of a scale or the other.

However, things were not that simple back then. If the data is not normalised to each person’s reaction time before calibrating the dial-a-meter, the total sample data will be dispersed over numerous measurement intervals. Second, because responder response times vary, dial-a-meters have an uncertainty range surrounding the precise instant being measured. There is a dearth of published research comparing dial-a-meter diagnostics to conventional metrics of overall ad performance like recall and persuasion.

Post-Testing:

Post-testing, also referred to as Ad tracking, can be syndicated or customised. In-market research tracking a brand’s performance, such as brand awareness, brand preference, product usage, and attitudes, can be conducted on a regular or ongoing basis through tracking studies. Advertising tracking can be carried out through online or phone interviews; the results of these methods yield essentially different measurements of consumers’ recollections of advertisements, recall versus recognition.

Why Conduct Post-Testing?

Generally speaking, the goal of ad tracking is to quantify the combined impact of the media weight or spending level, the efficacy of the media buy or targeting, and the calibre of the creative or advertising executions. Some more recent types of internet tracking focus on the relative performance of advertisements in comparison to rival commercials that are running concurrently, separating the concerns of the calibre of the media buy from the creative component’s quality. Marketing science statisticians utilise all types of tracking data as inputs for their Marketing Mix Models, which are used to calculate the return on investment for advertising (ROI).

While some ad monitoring studies are done online, others are done over the phone. Because the interviews utilise fundamentally distinct criteria—recall versus recognition—to elicit consumer recollections of advertising, the two methodologies provide significantly different assessments of advertising awareness.

In an online study, for instance, the participant may be shown a few striking, de-branded still photos from the TV commercial or a de-branded copy of a print or online advertisement, and they will then be asked to reply to three important questions:

Recognize this advertisement? (Metric of recognition)

Please enter the advertisement’s sponsor. (Self-reliant awareness test)

iii. Please select the sponsor of this advertisement from the list below. (A measure of aided awareness)

Using images in a telephone survey is not possible. For a campaign that features multiple ads with the same character or characters in the same scene with just minor modifications, spoken descriptions are exceedingly challenging to offer. The telephone isn’t thought to be a sufficiently adaptable approach to be utilised in any circumstance.

The following information could be obtained from a post-test:

Analyst of Decisions
awareness of a brand at the forefront of mind
Self-generated brand awareness
promoted brand awareness
Brand compatibility
Brand perception scores
Test of a brand
Recurring purchase
Regularity of usage
Purchase intention
perceptions of prices
Unaided awareness of advertising
increased public awareness of advertising
Recall of advertising messages without assistance
facilitated the recollection of advertising messages
helped with the commercial recall
Advertisement fatigue
Encourage usage and awareness
Features of market segments
media usage patterns
Lifestyle/Psychographics
Demographics
Various methods of post-testing:
The following are the most often utilised post-testing approaches among the many others:
Tests of penetration: Identification/recall
Tests of progress or sales effects.

In addition, measurements of attitudes, perceptions, and images can be used to evaluate the success of advertisements. Tests for penetration (recall) may be combined with the attitude assessment.

Tests for Penetration:

The specifics of this examination were initially disclosed by Daniel Starch in his book Principles of Advertising (Chicago- A W Shaw, 1923). Tests of Recognition/Readership/Viewership are another name for these assessments. Recall tests going back to 1923 are used to help them. Since then, the Daniel Starch Organization has carried them out in the United States. The magazine issues that the respondents say they have read are displayed to them in this instance. They are asked if they have read the advertisements and if they can identify them. Three categories are created from the results:

I Noted (N): An individual who solely recalls seeing the advertisement in the research issue

(ii) Observed-Correlated (A). a person who affirms to have seen or read portions of it in addition to recalling having seen it. He might even connect the advertisement to the brand or marketer.

(iii) Most Read (RM). the individual who has perused at least half of the advertisement’s textual content.

The reader groups mentioned above are stated as percentages. This technique can also be modified for broadcast advertising, which play commercial tape adverts. The following formula was provided by McGown (1979) to determine Readers per Dollar:

Readers Peer Dollar = Noted Primary Readership of X Magazin / Space Cost in Dollars

Nevertheless, there are errors in this procedure. Responses using this manner are always prone to reporting inaccuracies. They may willfully omit facts or purposefully embellish it. They occasionally make up responses to appease the interviewer and conceal the fact that they haven’t seen the ads.

However, there are a few techniques listed below that could improve measurement efficiency.

The tachistoscopic method involves showing the respondent advertisements in full or in part at a rapid pace and asking them to provide information based on those commercials.

The screen approach involves placing many screens over an advertisement, removing each one one at a time until recognition is achieved at different visibility levels.

iii. Researchers also frequently employ two more techniques: confusion contre approaches, which combine some unpublished and published commercials before measuring recall, and pre-publication control, which calls for a recognition survey of previously unpublished advertisements.

Test of Gallup-Robinson Impact:

A commercial research company called Gallup-Robison has developed standardised assisted recall tests to measure the impact of advertisements. When a respondent sees the cover of a magazine, they are asked if they have read the issue. If so, the person is asked to explain whatever they can recall seeing in that particular issue. After that, the person is handed a deck of cards with brand names that were mentioned in the issue and asked to list the ones they can still recall seeing there.

Assisted Recall vs. Recognition:

While the recognition approach requires respondents to first qualify as readers of a certain issue, the aided recall method keeps the test topic closed and requires the respondent to answer solely based on his recollection.

The aided-recall approach has stricter requirements than the other; as a result, many people with “less desirable” traits are effectively removed from the audience.

iii. According to studies conducted in the USA, the average advertisement score obtained using the recognition approach is six times higher than the average PNR score.

Readers of the aided-recall advertising tend to be younger, with lower levels of education, employment, and money.

The aided-recall approach yields much lower ratings, according to a Printed Advertising Rating Methods (PARM) study. These ratings are susceptible to methodological aspects like the length of time before the interview, the interviewers’ skill, and the sample type.

Tests of Unaided Recall:

This type of recall test deprives the responders of any hint as to how to recall the advertisement. This turns out to be more difficult than assisted recall, since respondents who can recall the brands on their own demonstrate a higher level of advertisement penetration.

Various Unaided Recall Types:

Recall Day-After (DAR):
The readers or viewers are questioned a day after the commercial runs.
Prime Time Total (TPT):
Watching time on television is the primary research focus here.

Test of Triple Association (TAT):

This assessment gauges the amount of brand knowledge a reader or viewer has gained from the commercial. The respondent is instructed to locate the brand name associated with a feature or benefit of a product. When someone is asked which toothpaste advertisement has salt, for instance, and their response is Colgate Active Salt, we know that the advertisement has achieved its learning goal.

Tests of Progress:

These tests are also known as sales effect tests because they evaluate the overall sales impact of the advertisement. Stated differently, the stages a buyer goes through before making a purchase are examined to see whether or not the advertisement had any influence. Even though it can be challenging to quantify the rise in sales brought about by advertising, we have the following tried-and-true techniques.

The Netapps Approach:

The acronym for Net-Ad-Produced-Purchases is Netapps. This technique was created by Staff Company and Daniel Starch. It requires a sample population, in which some have not seen or read the advertisement. Each group’s buyers and non-buyers of the brand under inquiry are identified, and the proportion of buyers who did so because of the advertisement is calculated.

Plan-to-Buy Test:

We inquire about the readers’ or viewers’ intention to purchase from them. If the answer is good, more research is conducted to determine the persuasive elements in the advertisement that led the customer to make the purchase.

Tests of Sales Results:

Here are a few sales results tests that calculate the extra sales brought in by the advertisements.

Previous sales are tracked before and after the advertisement, with the difference being attributed to the commercial’s impact.

The dealer’s inventory may be audited both before and after the advertisement.

Tests of Inquiry:

Certain consumer durables firms include coupons in their advertisement material. Customers are expected to fill out the coupons and return them to the company after using them. Thus, the advertisement copy is visible to the buyers while they complete the coupons. Therefore, an estimate of the advertisement’s readership can be determined based on the quantity of coupons received.

Tests of Attitude:

Marketers track how consumers’ attitudes have changed following the ad campaign to see whether there has been any shift in their perceptions of the brand they are looking at. Additionally, they believe that a favourable perception of their brand could result in more sales. Typically, scales such as the Likert, Thurstone, Differential, Guttman, and others are used to rate an attitude.

Therefore, advertising research, like all other aspects of market research, seeks to investigate a variety of true facts from the market. It makes an effort to gauge and assess how well the organisations are communicating. Numerous significant strategic communication decisions are contingent upon these assessments. Thus, it turns out to be a highly significant field since modern organisations understand that, in addition to sales numbers, brand image and goodwill are also crucial factors that heavily rely on advertising activities