Wednesday, 25 April 2012

Measuring Performance - Unknown to Known

Whether a manager of people, products or projects you need to know if you are succeeding in you goals. In my last post, In praise of data - Why should we analyse?, I mentioned the three areas that I focus on when deciding my analysis priorities; planning, measuring performance and targeting activity. In order to deal with the second of those, measuring performance, I have decided that I need to channel the living spirit of that old sage Donald Rumsfeld. His profound wisdom can help us dissect what it is we need to be thinking about when looking at the state of our business.

It is essential that you measure the things that matter. You need to consider what it is that marks out success, risk and failure in your business.

If you work in a small or new fundraising operation you need to decide where to start with reporting and setting targets. You may be looking to replace existing systems by integrating data into a new CRM. What questions do you need to ask the vendor to ensure that your new CRM meets your reporting requirements? Defining what is important to measure is a more difficult question to answer than it may appear at first. This is why it is really important to keep it simple and to start with the basics.

So lets remind ourselves of Donald’s words. Examining your needs from “known knowns” to “unknown unknowns” will give you a solid insight into what is important to your business and what you can do to minimise any risk of straying from those priorities.

Known knowns

These are the things which you have already identified as significant where you have some idea of scale. It is wise to consolidate what you already know.

It is true that there is a difference between counting and measuring but it is a good staring point to think of how big your constituency is, how many donors you have, how much money you have raised. You may already have some idea of the age breakdown and location of your donors. I would also suggest it a priority to know how many good addresses, phone numbers and email addresses you hold.

Known unknowns

So counting is fine it tells us what we have, gives us a sense of scale, but it is not so great for directing strategy. You need to turn those counts into measurements by looking at change over time. I suspect you already know what your known unknowns are. Thinking about these will help you settle on your key measures.

You may already know that you need to record change but you don’t understand yet the impact that change will have on your business. The trick is to know if the numbers are rising or falling.

Break your donors into three groups. Those that are new to your organisation, those that are renewing (gave last year as well as this) and those that are regained (gave at some point in the past and gave again this year).

Measure these per year and measure these per constituent. How many of your new donors last year progressed to become renewed donors this year?

The more important metric in this case is the attrition rate. You will no doubt uncover that you have trouble retaining first time donors. What we are really talking about here is defining a risk which will lead to decisions on what can be done to negate that risk. 

Breaking down your donors into groups allows you to think about different retention tactics for each.

By focusing attention on turning known unknowns into known knowns, fundraising managers can improve the effectiveness of their donor acquisition and retention programmes.

Unknown unknowns

The unknown unknowns are something else and really at the heart of data mining generally. Recording data effectively and devoting time and resource to investigate relationships within will tease your unknown unknowns out. You have to be sensitive to what the data is telling you and the changing business need.

When thinking about donor participation levels it was new and interesting for me to uncover just how connected age is to giving levels. I discovered that 20% of constituents over the age of 43 had donated at some point while only 1% of constituents under the age of 43 had ever made a donation. Discovering basic facts like this can lead to a change in tactics for communicating with and asking younger constituents. Adding context gives you the insight you need to inform strategy. Where will I spend the rest of my budget in order to maximise my return?

Measuring constituents by time zone is a newly identified known unknown brought in by a new business need to call overseas. Email opens and clicks also now need to be measured. This is brought about by the shift in focus from paper communication to electronic.

Unknown unknowns will jump out all the time. You just have to pay attention.


So when establishing a programme of measuring start with a short list. Remember not to over complicate. It is ok to start by counting what you have. You can then move your process forward by looking at change over time and change by the various constituent types. Remember to not only look at year on year change but look at trends over time. What is your 3 year rolling average?

I’d be interested to hear from others that have started measuring or have established metrics in place. What do you consider to be most important?

Of course there are many ways to represent performance measurements in insightful ways and I’ll say something more about that another time.



No comments:

Post a Comment