I’ve been kicking around the idea of doing a post about research and measurement in PR for some time. Namely, I really want to ask why PR hasn’t done a better job making this core to the function when there are scads of resources on the subject.  Frankly, I’m not the expert on this subject as I’ve spent more of my career creating content than measuring how it impacts the audiences its intended for.  Despite having an advanced degree in this area, I’m guilty as charged.   Instead I reached out to my former colleague — Forrest W. Anderson who is not just a measurement expert but one of the sharpest communications strategists I’ve been around in my career.

Below is an un-edited Q&A I had with him this week.  My questions.  His answers.  My parenthetical interjections.

Q: What is the single most important thing people need to remember when looking to measure the impact of communications programs?

A: The single most important thing people need to remember when looking to measure the impact of a communications program is their definition of impact, which should come from the initial, measurable objectives of the program.  If you’re trying to change behavior (increase sales, reduce employee turnover, etc.) you should try to measure that change in behavior.  If you’re trying to change awareness and/or attitudes, you should measure changes in awareness and/or attitudes.  Most likely you would do this with a pre- and post-program survey.  If you’re trying to increase media coverage, then you might measure clips.  If you’re trying to increase positive media coverage, then you need not only to count clips but assess the tone of the content in those clips.

Q: What is the most common mistake you see companies making in buying measurement and research?

A: To me, the most common mistake I see companies make when they do invest in measurement and research is they focus more on evaluation (or measurement) and less on the research they should do to plan the program. The first step in any program should be to articulate a measurable objective.  The next step should be to do research on the target audiences and the business environment so you know what kinds of messages and concepts will appeal to the target audience, which media reach the target audience and what’s going on in the world that might affect the way your target audience will react to your intended messages.  The evaluation piece is fairly straightforward, if you’ve created a solid measurable objective for the program.

The measurable objective is a big stumbling point.  Without one, you cannot evaluate.  This is why so many companies that invest in media evaluation systems that use online data bases are disappointed after a year of using the service.  Neither the client organization nor the evaluation system vendor thinks out what the measurable objectives should be for any given program.  It frequently turns out, then, that there is a mismatch between what the tool measures and what the organization wants to achieve.

Q: Communications research has been around a long time, why haven’t PR people done a better job making it core to their programs and the industry?

A: I think there are a number of reasons.

• In the past it has been expensive relative to the investment in the program, so there was a question regarding whether you should spend the money trying to get more results or measuring what you achieved.  The online systems have made clip analysis less expensive than in the past, and we can also do surveys online for much less than in the past.
• Some communications professionals are afraid of what they will find out if they measure.  An agency, for example, might not want its client to learn that a program had not achieved the goals the client requested or the agency promised.  This is a very unprofessional point of view because there is no way anyone can improve as a professional if they do not measure the effectiveness of what they do, learn from it and try to do better.  The same situation exists for some internal communications departments, with organizational executives taking the client role and the communications departments acting like agencies.  Again this is too bad.  There is a fair amount of anecdotal and some scholarly evidence that communications departments that do evaluate are more highly thought of by the CEOs of their companies than are those that do not evaluate.
• Last, but certainly not least, I believe many people go into public relations to avoid having to deal with numbers and numerical analysis.

(So true. I’ve lost count how many PR people have said to me “I’m not good with numbers.” Cop out.  Reading cross tabs isn’t that hard. And who hasn’t had a client that was BS-ing his boss about results?)

Q: CEO’s often just want to pay for clippings and see their names in headlines, how do you get past this?

A: If a CEO is that shallow, you’re going to have a number of operational problems in the organization that probably will outweigh communications issues.  These will only be the tip of the ice berg.  That said, the best way I know to influence CEOs is with data.  If a senior communications person believes the organization should be doing something, he or she should look for data that supports their point of view and present it to the CEO.  For example, if our communications executive (CE) believes the main competitor is winning partly because of the good media coverage it is getting vs. the poor media coverage the CE’s company is getting, a quantitative report demonstrating this would be more likely to sway the CEO than just saying “I think we should do this.”

I once did a $200,000 communications audit for the U.S. subsidiary of a European owned company.  The whole purpose of the audit was to demonstrate to the European owner that the U.S. subsidiary needed to invest in public relations.  The study made the case and HQ supported a major increase in PR funding.

Q: Is social media helping or hurting research and measurement in communications?

A: I would say social media is confusing research and measurement in communications.  What it is helping is dialog. In the past, there were very few direct communications channels open between an organization and its stakeholders, so market research gave management insight into who comprised a stakeholder group, what they cared about, what they thought, etc.  However, with social media, organizations can actually communicate directly with stakeholders, assuming stakeholders wish to communicate with the organization.  This is great!

The danger comes when an organization begins to believe that a handful of active users of social media users represent the entire stakeholder group.  There can be a big difference between what a few vocal individuals think and what most the population thinks. So, I believe social media is a wonderful way to get some insight into stakeholder groups, but I also think we need to be very careful about extrapolating that insight to larger populations.  I do not believe social media is a replacement for research.

(I agree with that. And think the hype and sex appeal of social has done a lot to distract companies from focusing on the basics — like research and measurement. People are off building Facebook pages when they haven’t even studied their core audiences to see how they interact with existing PR content and programs.)

Q: Is agenda setting theory still valid?

A: Unless I misunderstand your question, the agenda setting theory is based on the idea that the media sets the news agenda by choosing which topics to cover.  Thus the news media exerts great influence over not only the topics its audience thinks about but also how the audience thinks about those topics.  I’m not sure I ever completely bought into this theory, because I believe good journalists tried to choose topics that were of interest to their audiences and did some research with their audiences to determine what these topics were.  So, the influencers were influenced by those they influenced.

Whether this last bit was true in the past or not, it is certainly true now.  Anyone with access to the Internet can publish now, and very many do.  Tools such as Twitter’s “Trending” will tell you which topics (or key words) are being discussed the most at any given time, and journalists can and do use those kinds of tools to choose the topics they cover.  I would say the influenced are influencing the influencers more than ever before.  However, this is just a theory.  I don’t have data on this.

Q: If you could build a strategic communications program for Facebook, what would it look like?

A: This sounds like what should be a paying gig.

I told you he was sharp.  It really should be a paying gig.

-Jose Mallabo