Evaluating Social Media: Step 4. Identify your audience

Organizations often take the easy way out and measure what’s easy to count, such as visitor numbers for their Web sites. But those numbers won’t tell you what those visitors did with the information or content they found. According to the authors of the guide, “you’re ultimately trying to move an audience; therefore, getting feedback from the right source is crucial for a good evaluation.”

The authors tell us to be specific when identifying the audience. Simply the “general public” or “young people” (a particular favorite among managers that jump in with social media efforts) is not acceptable. Narrow your audience to a well-defined group. But identifying your audience does not simply mean knowing, for example, the demographics of those who are most likely to visit a performance (white, 55-64, household income of…). It’s all about behaviors.

Marketing expert Doug Dome explained to me: “behavior and financial value of behavior are at the heart of integrated marketing communications.” The financial value of behavior is an important aspect for your commerce goals, but there is more value to behavior than purely money. And that other value is what we’re trying to evaluate here.

In our example, you will want to evaluate those online users who are most likely to influence others, those who will become your evangelists.

The authors of the book “Groundswell: Winning in a World Transformed by Social Technologies” created so-called Social Technographics, a classification of people according to how they use social technologies. Here are the different categories and its public’s behaviors:

Creators
Publish blogs, Web pages, stories, audio/music/video

Critics
Post ratings, reviews; comment on blogs/forums; contribute to/edit wikis

Collectors
RSS feeds, add tags to pages or photos, vote for web sites

Joiners
Maintain profile on social networking site; visit social networking sites

Spectators
Read blogs, watch video from others, listen to podcasts, read forums, ratings and reviews

Inactives

In social media, the creators and critics will be your audience. Those are the people that are most likely to create content that advances your mission, content that contributes to your goal and objectives. Their online behavior influences others. But keep in mind; you have to concentrate on the relationships, not the technologies. It’s about what they do, not necessarily about what they use.

In addition, the authors of the guide advise that your evaluation should include both the audience targeted for your activities as well as any other audience who may influence your goals and outcomes. How are the collectors, joiners and spectators contributing toward your goal?

It would, for example, be interesting to learn how they have made their purchasing decision. Were they influenced by the creators and critics, your target audience?

Tomorrow, we’ll look at step five: establishing your baseline.

(Source: Are We There Yet? A Communications Evaluation Guide)

Evaluating Social Media: Step 2. Define your goal

Even if you are evaluating tactical efforts, you will still need to define your goal. You ultimately are concerned about how well your message is disseminated, or you ultimately have a Facebook presence, for a strategic, overarching goal.

Furthermore, the tired “raising awareness of your organization” is not acceptable. The authors ask: “why do I want people to know about my organization?” Behavior change is the ultimate behavioral goal and as the authors write: “when people start behaving differently, you have reached your goal.”

The question is: what is the change your organization is trying to achieve over five to ten years?

The authors differentiate between policy goals and behavioral goals. Policy goals progress from awareness to building supporters and a constituency, and from creating public will to policy change. Putting that in classical music terms, we can see how a policy that will support classical music, passed by your state’s legislature, could be an example of a desired outcome. Or perhaps think about a policy regarding arts education and the role of your orchestra in that policy.

Behavioral goals progress from awareness to attitudes, from changing social norms to behavior change. Again putting that in classical music terms, we can see how a change in attitude or social norms toward classical music, and ultimately a change in behavior, could be an example of a desired outcome.

Depending on the stage of the road you are in and the state of the environment and your audience, you could aim for any of the goals above, from awareness (as long as you ask yourself why) to change.

Now I certainly like to think that your mission statement is a good guide to inform your goal. As I wrote before: you should start thinking about how social media can help your organization’s core mission of providing classical music to audiences in your community and around the world.

Whether your goal is a policy goal or behavioral goal, the key themes that become apparent from the guide seem to be building a network of supporters and advocates to drive policy or behavior change.

Perhaps in other words, not just creating ticket buyers (although that is certainly part of it), but rather evangelists that will advocate for classical music; people who will advance your mission of bringing classical music to communities and audiences the world over.

So let’s define our goal as: “To build a network of supporters (patron evangelists, partner organizations, and bloggers) who can work together to advocate for an increase in classical music participation.”

One important last note on defining a goal: your communications plan is only part of the effort toward this goal. There are also program initiatives and supporting activities. These, perhaps more so than your communications plan, drive your road toward the goal; they are the meat and bones of a classical music organization. They include artistic programming, fundraising, education programs, and much more. Although you would again work from an integrated perspective, this series is not here to evaluate those elements.

On Monday, we’ll look at step three: stating your objective.

(Source: Are We There Yet? A Communications Evaluation Guide)

Evaluating Social Media: Step 1. Determine what you will evaluate

We have already determined that we’re evaluating social media efforts. But the question remains whether you are measuring a strategic initiative or a tactical effort.

A strategic initiative might include aiming for behavior change (e.g. new communities to participate in classical music; one-time ticket buyers to return); a repositioning effort (e.g. going from an organization that purely presents performances to an organization that stands in the middle of the community); or brand awareness (e.g. earning the reputation of an innovative arts organization; earning the reputation of an arts organization for the community).

A tactical effort might include evaluating how well your message is being disseminated; quality of communication with your audience (e.g. satisfaction of you social media engagement or quality of blog responses); a social media campaign; or simply one aspect such as your podcasts, videos or Facebook presence.

Our evaluation of social media efforts is therefore an evaluation of a tactical effort (social media). But throughout the subsequent blog posts, as you will soon read, we will be looking at how it is placed within a larger strategic context.

Let me now take just one minute to highlight an important point: the difference in evaluating a communications plan as opposed to evaluating a marketing plan. While I have indicated that I believe it is important to approach your communications plan from an integrated marketing communications perspective, in general terms, a communications plan is focused on policy or behavior change, whereas a marketing plan is focused on sales and commerce. Your social media campaign will likely touch on both and I will try to incorporate a brief comparison between the two concepts in the different steps. However, keep in mind that we’re working with a communications evaluation guide.

Your influence on policy or behavior change is a more abstract matter to measure than an increase in ticket sales or recording sales. But that doesn’t mean it can’t be done.

Tomorrow, we’ll look at step two: defining your goal.

(Source: Are We There Yet? A Communications Evaluation Guide)

Take A Friend To the Orchestra: Flip the Funnel

This week, Drew McManus’ Take A Friend To the Orchestra 2010 kicked off on his Web site Adaptistration. I was invited to write a guest contribution this year and was immediately inspired by a book I was reading: Joseph Jaffe’s Flip the Funnel: How to Use Existing Customers to Gain New Ones.

So, head over to Adaptistration and read my contribution: Flip the Funnel: moving from today’s toward tomorrow’s customer experience. Don’t forget to leave a comment to let us know your thoughts!

TAFTO 2010: Marc van Bree

Evaluating social media for classical music organizations

I have been a fan of the communications evaluation guide Are We There Yet? by The Communications Network ever since I learned about it through Issuelab. Designed for philanthropic and nonprofit organizations, the report guides managers through evaluating their communications efforts. The authors warn that it is not a communications planning tool, but I believe a greater understanding of the proposed evaluation process will lead to a better, more focused communications plan.

So naturally for me—combining classical music and social media—the question was: how can orchestras or classical music organizations evaluate their social media efforts using this guide?

Over the next week, I will go through the guide, step by step, to see how it might apply to classical music organizations and their social media efforts. I will take a fairly broad, general approach and perhaps make certain assumptions and create certain hypothetical scenarios that could be typical for classical music or arts organizations around the country.

The steps, according to the guide, include (the steps will become links to each respective article as they are posted):

Step 1. Determine what you will evaluate
Step 2. Define your goal
Step 3. State your objectives
Step 4. Identify your audience
Step 5. Establish your baseline
Step 6. Pose your evaluation questions
Step 7. Draft your measurements
Step 8. Select your evaluation techniques
Step 9. Estimate your budget
Final Word

Although I will be looking at social media in particular, I believe it is important to see these efforts as part of an integrated marketing communications approach and as part of a larger strategy toward an overarching goal.

Even if I don’t go into those elements beyond social media, including offline advertising, promotions or public relations efforts or programmatic supports, keep in mind that the social media efforts should always be seen as part of an integrated approach toward achieving a mission statement-inspired goal. Your offline efforts should always complement your social media efforts, and vice versa.

I hope the walk through will be helpful and insightful. Perhaps you have suggestions or improvements along the way. If so, don’t hesitate to let me know. Likewise, leave a comment if you have an evaluation story to share or if you have any questions on your own evaluation efforts. I’m looking forward to hearing from you.

Happy evaluating! Tomorrow, we’ll get started with step one.

New Year’s resolution: organizational culture and change

Greg Sandow recently composed a list of the top ten changes in classical music in the past decade. It’s an interesting, albeit incomplete, list with plenty of agreeable and perhaps some disagreeable items. What stuck out for me was item number three: a new attitude toward classical music:

Mainstream classical music institutions have tried many innovations — new, informal ways of giving concerts, musicians talking to the audience, large videoscreens, and much, much more.

Some of these innovations have been smart, some have been silly, and many of them haven’t been well thought out. Rarely have mainstream classical institutions developed real strategic plans for change. More often (as I’ve observed firsthand), they’ve tried things out impulsively, only to discover, a year or two down the road, that their innovations don’t make much difference. In particular, they don’t attract a large new audience.

That’s because a single innovation isn’t enough. But all of them, taken as a whole, have changed the mood of classical music, bringing it closer to contemporary life.

A new attitude toward classical music is crucial. But I would like to frame it from a marketing and strategy perspective. The market has changed, now institutions and administrators must change. Let’s introduce two concepts: product-driven vs. market-driven organizations, and strategy in strong-culture organizations.

The latter concept comes from the book Strategy: Core Concepts, Analytical Tools, Readings by Thompson, Gamble and Strickland. A book I have used before. According to the authors, strong-culture organizations have well-defined corporate characters with deeply rooted values and behavioral norms. Clearly, orchestras fit this description.

But first, orchestras are by their very nature product-centered organizations, as opposed to market-centered organizations. The product (by that, I mean the production of art in any form) is the raison d’être, whereas for corporate institutions, the market (or maximized profit) is their reason for existence.

It is easy to be a product-centered organization when you hold a near monopoly and your product is in demand. But as we have seen before (PDF), the arts and culture have significantly proliferated in the last decades. According to the National Endowment for the Arts (NEA), in the last 20 years, the number of performing arts groups has risen to an all-time high. On the other hand, as the NEA recently reported, fewer people attend the arts. In other words, more choices, but a shrinking market.

The danger in product-centered organizations is the assumption that the superiority of the product is the key to the success in the market and the dismissal of external information and sources. We all recognize a certain elitism in classical music institutions: an attitude of play it and they will come.

For years, decades even, institutions have dismissed reports that spoke about the decline of classical music. Greg Sandow calls it “the myth that the audience would always renew itself.” Let me bluntly generalize and state that orchestras are run in this culture of superiority, elitism and dismissal. There appear to be some cracks in the foundation of this culture, but only just recently, the League of American Orchestras recognized the dire situation of declining audiences.

The authors of Strategy: Core Concepts explain how a mismatch culture and strategy occurs in strong-culture organizations, such as orchestras. They “tend to occur when a company’s business environment undergoes significant change, prompting a drastic strategy revision that clashes with the entrenched culture. A strategy-culture clash can also occur in a strong-culture company whose business has gradually eroded…” That is already sounding familiar.

The authors cite the example of electric utility companies that were “long used to operating as slow-moving regulated monopolies with captive customers.” I think we can all recognize orchestras in this example.

Electric utility companies are now “confronting the emergence of a vigorously competitive market […] and growing freedom on the part of […] customers to choose their own energy supplier.” I think the comparison is canny.

Well then, what’s the answer? The authors recommend shifting “away from cultures predicated on risk avoidance, centralized control of decision making, and the politics of regulatory relationships and toward cultures aimed at entrepreneurial risk taking, product innovation, competitive thinking, greater attention to customer service, cost reduction, and competitive pricing.”

Should orchestras then change from a product-driven strategy to a market-driven strategy? Hold your horses, not so fast. In market-centered organizations, the product is born from, and evolves, based on the needs of the market. If this were applied to orchestras, they would all turn into Andre Rieu’s blasphemous Strauss orgy.

Thus, for orchestras, a market-driven strategy does not mean performing Beethoven’s Fifth in every concert because that’s what the market wants. The product should unwaveringly focus on artistic integrity and innovation. In marketing terms, the product is not up for grabs in a market-driven strategy.

But that still leaves the other three P’s in the marketing mix: price, place, and promotion. How can orchestras be a more market-driven organization in those aspects?

Greg Sandow’s list offers some insights on the changes that have happened in the last decade: Peter Gelb taking over at the Met and infusing its strategy with bold, risk-taking innovation; improved customer service; and artistic improvising.

But the greatest opportunity for innovation lies in Sandow’s eighth place item: classical music goes online. Think about the opportunities in place (streaming, social media, new sale outlets, partnerships etc.), promotion (search engine marketing, social media, partnerships etc.), price (micro-payments, donations, subscriptions etc.), and yes, even product (adaptations, mash-ups, collaborations etc.).

That is exactly why we need to get rid of the product-driven culture of superiority, elitism and dismissal. The Internet culture, and particularly social media, is the antithesis of those forces. Shift away from risk avoidance and centralized decision making and move toward innovation, competitive thinking, customer service, and competitive pricing.

Number one on Greg Sandow’s list is a movement for classical music change. “I know they’re out there,” writes Sandow:

So the next step might be to draw this movement together. Maybe we do need an organization. I’m available to help to build it. Or maybe we just need some new but more informal ways to meet each other, and connect. I’m available to help make that happen, too.

But that might be our new year’s resolution. Let’s make our movement stronger. Let’s connect, and make some noise.

Orchestra Survey: Monitoring and Measuring

None of the orchestras indicated they have implemented or established metrics for measuring social media activities. However, the majority of the orchestras (73%) recognized the need to measure social media activities and the remaining 27% were currently planning metrics for measuring social media.

This is one of the findings I found most interesting. In addition, the majority do find the need to measure and two-thirds of the managers rate importance of measuring social media at least a four (out of five). This, I believe, truly indicates that orchestras are at the brink of a strategic use of social media.

But they don’t quite seem to be ready to answer the tough questions. I can’t stress enough to read the wonderful report Are We There Yet? to get yourself ready for those tough questions. In a comment on yesterday’s post, I detailed an example for goals, objectives and milestones that should make measurement clearer.

Back to the survey, it seems contradictory that the question on what exactly managers measure yielded some results. Just over half of the managers claim to measure ticket sales and incoming traffic from blogs and social networking (even though they indicated earlier that they have no established metrics). Perhaps they are tracking these numbers, but have not connected them to a social media goal yet. And I wonder how many managers really do measure ticket sales that stem from social media efforts, or if they rather measure general online ticket sales.

Tracking these measurements then seems to fit more under social media monitoring, rather than measuring. Google Alerts is the main tool of choice for managers to monitor social media. It would have been interesting to see how many use the “comprehensive” alert versus the “news” alert. Obviously, you can’t monitor social media with just a “news” alert. Frequency of monitoring was not surveyed, but it is perhaps an important indicator of how involved the monitoring effort really is.

Just as the engagement question before, the relatively narrow framing doesn’t show the full scope of measuring and monitoring. Qualitative research on the topic, including interviews, would be a welcome addition to find out how managers keep on top of what’s being said in the social media sphere.

For more on social media monitoring (or listening), read Beth Kanter’s wiki on the topic with some great instructional materials.

How do you interpret the results? And how would you go about monitoring and measuring your social media efforts?

Orchestra Survey: Activities, Engagement and Goals

The survey demonstrates a high participation rate among orchestras in the different social media tools. All orchestras in the survey are on Facebook and 80% of the orchestras have a Twitter account. But this just shows that orchestras have set up a presence, not what they are doing with the presence.

That’s where the engagement question can answer some questions. Eighty percent of the orchestras respond to questions and comments on social networking sites. A little more than half (53%) of the orchestras adapt press and marketing materials for social media and 40% actively pitch bloggers, and 20% maintain a separate mailing or pitch list for new media outlets and authors.

There is an obvious problem with such narrow framing: it doesn’t show the full scope of engagement. Furthermore, the interpretation of the question leaves room for different answers.

I was surprised to find that a little more than half claim to adapt materials for social media. Then again, condensing a press release into a tweet and posting a link could be construed as adapting materials. And perhaps instead of “responding to questions and comments,” the question should have included “regularly” or some sense of frequency and a segmentation of platforms (blog comment, Facebook, Twitter etc.).

And what about pitching bloggers? What kind of bloggers are being pitched? Are they just the arts critics from traditional media who maintain a blog? Or do they include blogs like Sequenza21and Opera Chic, or local blogs like Gothamist and Chicagoist, or even local classical music enthusiasts who blog?

Most telling was the question about responses to negative comments. Nearly half of the orchestras said they did respond externally. All orchestras are on Facebook, yet only half respond to possible negative comments. Perhaps this finding tells the truest story about engagement.

The reason I highlight these points, is because I feel the survey findings might reflect an appearance of frequent engagement through social media. This doesn’t completely match up with my own observations, even though there are the obvious exceptions that show both quantity and quality.

I received a comment via e-mail:

What kind of content are these orgs trying to share over social media? I think videos, podcasts and blogs are the main types of content that orgs should be sharing when they get into social media. Perhaps that is part of the problem. Orchestras aren’t creating the kind of content that is popular to interact with, easily shareable with others, etc. Thoughts?

I think this touches on a very important point. Orchestras are just dipping their feet into social media. It seems a lot of organizations have joined to “keep up with the Joneses.” So yes, they’re active on Facebook and Twitter, spending time on social media, and yes, they find it moderately important, but there’s no know-how on communicating in a social media environment, and there’s no strategy or policy behind it. This knowledge and a well-defined goal, along with well-established metrics, is what’s lacking at most organizations.

And talking about goals: driving Web traffic and increasing awareness of programming and the organization are the most important social media goals, according to managers. Although opinions are split on increasing ticket sales, it is only ranked sixth out of eight.

Drew McManus writes:

Certainly, organizations shouldn’t exclusively use social media as an outlet to shill and sell tickets yet there is no way to reasonably separate revenue development from other mission related goals. Simply put, revenue performance and the institution’s mission are not mutually exclusive.

I would suspect that if managers were pressed on why they would want to increase Web traffic, or why they would want to increase awareness, it would all come back to ticket sales.

Then again, not necessarily. I believe social media can be a great tool to advance the orchestras mission of bringing classical music to the community. Although tremendously important for the bottom line, ticket sales is still only one of the components in that mission. That’s where the distinction between goals and objectives is important. Your mission is your goal; ticket sales is an objective toward that goal.

Update: Elliot Harmon at the techsoup blog finds it odd that orchestras focus on Web traffic as “it seems a clumsy proxy for the less transparent goals of awareness and education.”

What do you think?

Orchestras and Social Media Survey: Key Findings and Full Report

Last Friday, I introduced you to the Orchestras and Social Media Survey. Today, the full report is available for download.

You can download the report from my Web site at mcmvanbree.com

In short, the survey found that social media activities, familiarity and usage seem to be widespread among orchestras. Managers find social media important and organizations are generally enthusiastic. However, the efforts are far from organized and strategic. It seems many orchestras are dipping their feet in the social media pool, but do not have the policies, budgets, and metrics in place to effectively use the tools at their disposal, even if they do recognize the need for checks and balances.

The rest of this week will be dedicated to in-depth follow up posts about the different findings, and I will pose some discussion questions. I encourage you to contribute.


Orchestras and Social Media Survey: Preface

Nonprofit blogger Beth Kanter recently highlighted two new research studies about nonprofits and the adoption of social media released by Weber Shandwick and Philanthropy Action. These studies yielded some interesting results.

The Weber Shandwick study of 200 nonprofit executives revealed that an overwhelming majority (85%) will use more social media in the next two years. There is extensive experimentation with social media in the nonprofit sector, but only half (51%) are active users. Most nonprofits (52%) do not currently have the infrastructure, staff and expertise necessary to take full advantage of social media’s potential, and ultimately, for most nonprofit executives (79%), the true value of social media has yet to be determined for their organizations.

The Philanthropy Action study questioned the use of social networking for mid-size nonprofits and concluded that ―in terms of fundraising and attracting volunteers, metrics that most nonprofit boards and executive directors highly value, the available evidence suggests that social media is not very effective.

How do orchestras stack up in this environment? What kind of resources do managers commit and how active are orchestras in social media? Do orchestra managers share the opinion that fundraising and attracting volunteers are highly valuable metrics, or do they have different goals and objectives?

Orchestra survey

There has never been a comprehensive, industry-wide look at if and how orchestras are using social media. In order to get a clear and complete picture of what is happening with orchestras and social media around the country, I sent around a survey to 53 orchestra marketing, communications and web managers in October and November, 2009. I received 15 responses and my gratitude goes out to each of the orchestra managers that responded.

The survey was designed to collect details about the state of social media, ranging from budget size and time allotment to attitudes and goals. It was not meant to single out organizations, or show what is good or what is bad. All responses have been tallied to provide a comprehensive, collective report that aims to aid orchestra managers in years to come.

Blog series

Next week, I will kick off the Orchestras and Social Media Survey 2009 series on my blog by publishing the full report, followed by in-depth posts about the different findings.

I am doing this to provide an opportunity for feedback and discussion. I believe, and even hope, the survey results will raise more questions than answers. I am interested in hearing your thoughts on what it all means and how orchestras can use the results to inform a better, more strategic approach to social media.

So if you have any questions or comments about the survey, don’t hesitate to contact me at dutchperspective (at) mcmvanbree.com, and stay tuned to the blog to join the discussion about the survey next week.

Check back for the full report on Monday. (Full report can be found here)