For example, let's say you scan a high school cafeteria and choose one table to represent the population. And you just happen to pick the nerd table. You might come to the erroneous conclusion that most Americans wear glasses or study during lunch. Is it likely? Maybe not. But is it POSSIBLE that all the people who are watching Perfect Couples are the ones without Nielsen hook-ups? Or that the reason Two and a Half Men is still on the air is because like thirty people in America are the only ones watching it? Yes. That is possible.
So this year I am making a proposition to anyone who is a current Nielsen household. Don't watch the Super Bowl at home, or on any TV with a Nielsen hook-up. For just one year, let's try a little experiment. Go to a friends' house. Have a party. We know that all over America people will be watching the game. So please do watch it, but don't let Nielsen know you are. Let them think you aren't. Perhaps when Monday comes around and everyone is discussing the game, the commercials and the half-time show, but the ratings are almost non-existent, the flaws in the system will be made obvious. If nothing else, I think it's a great joke to play on them.
If you like the idea, pass it on!
That is only a flaw in the system if you can organize the families of skew the statistics that chose them. Statistics do not lie, the clue is to really know the assumptions that the statistics are based on. These are often hidden or misrepresented.
ReplyDeletestatistics may not lie, but they dont necessarily mean what you may want them to. What the Nielsen ratings tell you is that this group of people watched these things. You may decide to draw conclusions based on that, but they may not necessarily be accurate. And while probability might be that they are representative of a larger whole, the possibility exists that they do not. That's all I'm saying, and why I generally don't trust statistics. You can make numbers mean whatever you want them to mean. It's all about presentation.
ReplyDeleteI fully agree but if the study is properly designed, it does not lie. The question is exactly how was the study constructed.
ReplyDeleteThis is why the opinion of the healthcare reform act is becoming more negative while approval of pieces of the bill are becoming more positive. You need to know the actual questions asked.
In the case of the Nielson ratings you need to know how the households were chosen. If you can obtain that information it is possible to judge the validity of satistics.
But even given that the households are chosen for specific demographics or what have you, I still think it's a specious idea. While you would obtain data from a wide and varied sample of different types of people, that fact alone doesn't mean the data translates to everyone across that same demographic. Just because somebody is a white 24 year old man in New Jersey with no health insurance doesn't mean that every other white 24 year old New Jersey male with no health insurance watches Mad Men.
ReplyDeleteThat's all I'm saying. When it comes to TV, when the fate of series depends on ratings (and thus, people's JOBS depend on ratings), either use an enormous sample size (or the whole country) or don't waste your time.
Actually they are not chosen to represent a particular demographic. They are chosen to be representative, so that if there are 1000 24 year old white males in NJ and 200 like this but 100 hate it, the 10 Neilson households are tend to have 2 who like it and 1 who hates it. It is not predefined but the number of households choosen makes that outcome really (95%) likely.
ReplyDeleteYou are confusing how the data is reported (18-24 year old males in NJ) with how the families are chosen (large enough randomly selected group to represent the entire population).
It was just an example.
ReplyDeleteWe may believe that the 10 Neilsen households represent a real response that way, but the TRUTH is, we have no way of gauging whether there are 200 who like it and 100 who hate it. We reverse assume that the Neilsen sample has a probable likelihood that means that.
I understand the logic that tells them their large selection is representative of the entire population, I'm simply not convinced that it always actually is. Even if its 95% likely, that still leaves a 5% unlikely and that's something I think deserves investigation.
With the vast majority of television viewers in this country using cable, there MUST be some way of getting this data from EVERYONE direct from the cable company, thus giving us a much better idea of what people are REALLY watching (at the user's consent of course).
Yes 5% of the time you will get an erroneous result. It's like flipping a coin 10 times and getting 10 heads in a row, real but not likely to happen twice.
ReplyDeleteI think your real issue with the ratings is what is done with them. A few passionate fans may actually be more valuable to the advertisers but that is not measured (it is how things like star trek got rescued). In reality most advertisers want to play it safe with the conventional wisdom of big numbers or bet again.