• Trial advocacy training materials

    • Legal professional development

Improving the Quality of Your Feedback Forms

Knowing how to collect and use feedback is an essential part of a PD professional’s tool kit. Every time you deliver a program you make choices about speakers, topics, delivery formats, materials and other elements that you hope will contribute to a great attendee experience. Effective PD people are always innovating so you need reliable evaluation methods to accurately identify those elements that deliver and those that disappoint.

The best evaluation forms do not take long to complete, measure behaviour or knowledge as well as attitudes, and answer three key questions – what worked, what didn’t work, and what can be improved next time. This post is about how to design feedback forms to get the kind of information you need to improve programs. For suggestions about how to get more responses to a well-designed feedback form, see last week’s post, Practical Tips for Improving Feedback Response Rates.

Here are 10 basic principles for designing helpful feedback forms:

1. Keep the feedback form short. It should be a maximum of one page and take no more than five minutes to complete.

2. Only ask attendees about what you can change or improve, for example, if the program has to be delivered in a particular location and you can’t do anything to change or improve it, asking about it wastes attendees’ time and loses an opportunity to collect meaningful information.

3. Tie evaluation to program objectives and desired outcomes. What are you trying to assess – content, teaching methodology, delivery format, price point, networking opportunities? Ask attendees about what you want to evaluate.

4. Ask attendees about learning as well as attitudes. Questions about attitudes measure perceptions and feelings, for example rate the quality of the program or presenter. Questions about learning measure behaviour and knowledge, for example, did you understand the program content or materials? Do you feel you can apply what you’ve learned?

5. Mix closed-ended multiple choice questions (easy and quick to answer, and enable comparison of different programs but are not particularly informative for improving a particular program) with open-ended questions that ask for comments (require more time to answer but are capable of generating specific, informative feedback).

6. When using closed-ended multiple choice questions, choose numbered responses (rate the quality of the program materials on a scale of 1 to 5, 5 being excellent and 1 being poor) or labelled responses ( I learned new information that will help me in my day to day practice –  1 strongly agree to 5 strongly disagree.) Clearly define the numerical rating for each response and make sure that the scale is visible, and that the questions fit the responses.

7. For closed-ended multiple choice questions, offer five responses to avoid stilted results that can happen with too few choices or paralyzing participants with too many choices. For more on this topic see my future posts on Quick Tips for Feedback Compilation and Analysis, and Top Box s v. Average Scores.

8. Make open-ended questions as specific and focused as possible so attendees don’t feel obligated to write an essay. For example, what are the three most important things you learned today? What was your favorite part of the program? What content was not covered in the program that you would like to see included next time? Provide adequate space for attendees to share their suggestions.

9. Reserve two response options (yes/no) for questions regarding future actions ie. will you recommend the program to a colleague, or, would you attend an advanced version of this program? Include a small space for additional comments for maybe’s and qualifications.

10. Include a question about the action attendees are willing to take, for example recommending the program to others and whether the attendee will return next year. Both of these are very useful indicators.

Properly conceived and executed, feedback forms are powerful tools for change and innovation. Make it your mission to design user friendly forms that capture the best and worst parts of your programs, and then use the information you receive to make your programs better.

Rethinking the Business Card

What’s a business card for? When my firm recently underwent a website and logo makeover, I decided to tackle my business card at the same time. As I compared three equally uninspiring designs I asked myself “what’s a business card all about anyway?” Is it something that you give to someone simply so they will have your contact information? Or is it, like a website or a blog or a presentation, an extension of you – an insight into who you are, what matters to you and what motivates you? Maybe a business card is about relationship building, an opportunity to persuade someone to want to get to know you. Of course it is!

So, as I began to think about how to best use this new palm-sized canvass, I did a little research and found some helpful suggestions in an interview with Ann Handley of MarketingProfs.com and co-author of Content Rules. Here are my three favourite tips from Handley:

1. Lose the kitchen sink approach. Edit your contact information ruthlessly. It’s my favourite tip because I want to be that cool but I’m not so I couldn’t really do it. For me, editing my contact information ended up meaning including my cell, email and web and mailing address. All good – and I think essential – ways to connect with me. I left out the fax and linked in but is that really editing?

2. Be visual. Think about adding images or graphics that spark curiosity or communication. I decided to add a 5 second elevator pitch about what I do – “Hello. I help organizations use professional development to evolve grow and lead.” My designer is great at her job so she made it look pretty.

3. Think of your card as a call to action. Consider producing special runs of customized cards for a specific event, for example, see you at this year’s after party or don’t miss my presentation in track 3. I haven’t done this yet but I’m intrigued by the possibilities.

For all the good advice Handley had to offer, I think the most challenging part of a business card is the flip. I can’t remember the last time I saw someone accept a business card and not flip it over to see what was on the back. Some people believe the back of a business card should be blank for note-taking – the old sales training line about recording when and where you met the person and what you talked about. I’m not sure I agree with that. For me, when they turn it over, I want my card to start a conversation. I hope I achieved that; I’ll find out soon.

To read Handley’s full interview, see the August 2012 print edition of  Entrepreneur with the headline: Allow Me to Introduce Myself.

Practical Tips for Increasing Feedback Response Rates

A lot of clients I work with still favour end-of-program paper evaluation forms. Call me old fashioned but I like paper forms too. Among other advantages, hand-out surveys, as they are sometimes called, let you personally communicate the importance of providing feedback, allow you to capitalize on the fact that participants are physically available, provide an instant check on your assessment of the program, and work for big and small budgets alike.

Assuming that you have crafted quality questions (see next week’s post, Improving the Quality of your Feedback Forms), response rates for hand-out surveys tend to average in the 40 to 50% range for most CLE providers (usually closer to 40%). I think those numbers are too low. My experience is that the smaller the program, the higher the response rate. So, for example, in a hands-on workshop limited to 12 participants, it’s not unusual to see a 91% response rate (or 11 forms). As the number of attendees grows, the response rate declines so that but the time you’re in the 500 plus category, you’re maxing out at 50%. But those are big attendance numbers typically reserved for signature events. Most CLE programs in Canada are in the 50 to 120 participants range. For that size audience, your goal should be a response rate of close to 70%.

Why? The better the response rate, the more weight you can give the results. And if you don’t have feedback from one third or more of your attendees, there’s a chance their responses could change the results. So how do you encourage more attendees to respond? A good starting point is to remove barriers to completing and returning the form. Here are some practical suggestions:

  • Make the form easy to identify and find (print it on distinctive paper)
  • Give respondents the option of keeping their comments anonymous or providing their name and contact information for follow-up
  • Make the form easily readable (font size matters) and if possible, visually appealing
  • Make the form easy to fill in (aim for a mix of open-ended and multi-response ie. “1 to 5” or “poor to excellent” rating-style questions)
  • Don’t waste attendees’ time with questions that don’t matter (“will you come back?” matters)
  • Allow enough time to complete the form (for example, in a one day program, hand it out when attendees check in)
  • Tell attendees why their feedback is important and what you will do with it
  • Remind attendees throughout the day to complete the form and for certain, remind them at the last break or before the last presentation
  • Have extra forms on hand for attendees who lose, leave behind, or take notes on their form
  • Provide pens for attendees who don’t have one
  • Provide a surface to write on
  • If there is no surface to write on, consider printing the form on card stock so it is easier to write on in an awkward position
  • Consider motivating attendees to return the form with some kind of extra push (it shouldn’t be the reason for completing the form, and it doesn’t always work but I have seen response rates soar when completed forms were tied to receiving an answer key or certificate)
  • Make it easy for attendees to turn in the form on their way out (provide a box at every exit or even better, have people standing there, asking for and collecting forms)
  • Add a fax number and/or mailing address to the bottom of the form for people who say they will “send it in” (they almost never do but still…)
  • The day after the program circulate the form (or an online version of it) by email and remind attendees who did not return it why you value their response
  • Thank attendees for completing the form

And one last gem. Get to know your attendees. Attendees who you know by name, or who know you or your team, will be more likely to turn in feedback forms than attendees with whom you have no relationship.

Form filling is not fun (even when an attendee really enjoyed the program). It’s important for providers to remember that, and dedicate a little bit of time and resources to increasing response rates.

An Editing Checklist: 10 Words to Punch up Your Marketing Copy

If you work for an association or open enrolment CLE provider, at some point or another, copy writing has probably been part of your job. I’ve seen enough program fliers and brochures to know that good layout and fancy graphics may help grab a reader’s attention, but words are what drive registrations. Why? Because words build relationships. In your personal life, you choose words that let people know you care about them. When marketing programs, you choose words that let readers know you understand their needs. Start with first principles – focus on the reader’s problem and your ability to offer a solution. But what about the actual words, you ask? Are some words better than others?

For years copywriters have talked about the most powerful marketing words in the English language. Every year, some announcement about the “10 most persuasive” words in sales gets picked up and recycled by the media. Here’s the rub: the research is reportedly a myth. Beginning in the 1970’s, the now infamous list of “power” words was attributed to Yale University researchers. From then on, the story snowballed with the list of words changing from time to time, and being variously credited to Duke University, the University of California, and other reputable sources. But Yale apparently never did a study, nor did any other university to which the research has been attributed. The shame, you think! Everyday copywriters and marketing legends alike have sworn by the list for decades. They must feel pretty foolish, right? Well, maybe, except for the fact that copywriters keep testing the words and the imaginary Yale list still reigns supreme.

So will using proven marketing “power” words help you achieve better results when promoting CPD programs? Yes. I’ve seen small changes of only a few words increase response rates. But cherry-picking “power” words without an appreciation for and attention to the fundamentals of well written copy is like using a band aid to fix a severed limb. Instead, I suggest you treat the list more as an editing tool. Once you have your first draft of marketing copy, pull out the list and see where you can add “power” words – not in a contrived way, but naturally. As I noted at the start, the list has changed over the years, and I’ve intentionally omitted some consistent “power” words that, in my view, aren’t relevant in PD copy. Here are my top 10:

  • You/your…relevant to you/safeguard your practice
  • Immediate/immediately…strategies you can put to work immediately
  • Easy/simple….easy to learn/simple, straight forward techniques
  • New…content since last presented
  • Discover…the steps used by
  • How to…draft and negotiate
  • Learn…the traits that will make you
  • Results…get better results
  • Guaranteed…to make you a better project manager
  • Proven…solutions/strategies/techniques

Think I’m crazy? Read this sentence from ALI-CLE that arrived in my inbox yesterday:

“Attend this program to learn proven legal writing techniques that you can immediately apply….to keep your reader’s attention and get better results!”

How sweet is that? If you want to fill a program, promote it so your members or subscribers will take notice. If you get stuck, try using this checklist to punch things up. It’s no magic bullet but it may help. Happy copy writing!