Over the years I have been asked to change the way the Podcast Awards process work. Everyone has an opinion until it comes to determining an execution strategy on how to make it happen. My goal with the Podcast Awards has always been to execute it with transparency, so that every show that is nominated has a chance of winning.
Podcasters have always also said that their shows have extremely high engagemente with their audiences. That is why the Podcast Awards are the “People’s Choice Podcast Awards”. As we lay out in the rules on the site the process is simple and broken down into three steps.
1st is the nomination period of 15 days: Shows engage their audiences to nominate them in the category they feel they are most aligned to. Fans get to only nominate their slate of podcast one time to prevent ballot stuffing. So we want the podcasters to make sure they tell their audiences that they get a single chance to nominate a full slate of 22 shows.
2nd is the Review: Being my show and family of shows are not eligible to win. I recruit 44 listeners from my audience to do the review process which takes on average 4 hours per category. Each of those reviewers review 2 categories each. This results in each category being reviewed by 4 separate listeners. So exactly what do we provide them. They get the full list of shows that where nominated in the specific categories they are assigned. We ask them to review the 30 top nominated shows in each category.
They are provided a confidential non-releasable grading sheet that has the following elements.
- Verification Podcast fits in Category Nominated
- Verification that the Podcast has 10 Shows Produced
- Verification the the Podcaster has been creating shows on or before January 1st of the awards year
- Verification that the Podcaster has a visible RSS 2.0 feed with Enclosures on his home page!
Once validated the sheet then has 4 sub sections that have a variety of elements associated with the section and a weighted grading scale.
- Number of Nominations 40% (Pure Number of Unique Validated Nominations)
- Quality of Website Design 15% (15 Elements)
- Quality of Sound 15% (5 grading Elements)
- Quality of Podcast Delivery and Show Format 10% (4 grading Elements)
- Relevance of Content 20% (4 grading elements)
This review and scoring is then averaged across the 4 grading sheets, per category to come up with a top 10 in each category. We have seen in the past where shows with much lower nomination numbers made the top 10 slate by beating other shows by scoring very high on the grading elements.
3rd is the Daily Voting. This is the most controversial part of the process. I fall back to engagement. If you have an engaged audience, smaller shows will beat bigger shows hands down. So in this stage we see what shows have the most engagement with their listeners.
Is the process perfect, no process is perfect, but it is the most transparent process I have found to date. We eliminate personal biases by having a big enough pool of listeners to review the nominated shows, and by cross examining grading sheets we are able to ensure the slate nominated is a sound slate of shows. The rest is up to the listeners of the shows nominated.
Shows often help shows by sending votes their way if they where not nominated. This is where having alliances with other shows help in a big way. Podcasters often forget to work with other podcasters and many times shows with alliances with other shows will beat bigger shows easily..