Monday, September 25, 2006

Poll Magic

As I have stated in the past, the art of polling has fallen into the same disrepute as statistics. And I have some bones to pick with the two polls that have been released recently.

#1. If you do not release your methodology, it is hard to take your poll seriously. There is absolutely no reason to do this other than trying to hide something. If you are trying to hide something, there has to be bias in your poll. It is as simple as that. If your job as a pollster is to ferret out what is happening in the electorate from an objective sample, you will want to release your methodology to stamp your reliability. Neither of these polls have done that.

Both third party polls taken during the primaries DID do this.

#2. Obviously the Greenburgh-Quinlin-Rosner poll was sloppy and can almost be tossed aside in terms of value. I would bet a dozen Krispy Cremes (and those are hard to get now) that this poll by far oversampled Democrats. We do not know why this poll was commissioned, or what feedback was being targeted by the Giffords campaign that led to these results. It could have been Pima County residents for all we know. Again without the methodology, the results are almost as useful as an online poll.

Also, look at the favorablility ratings for Graf. 82% of respondents know who he is, only 32% think favorably of him, and yet 35% will vote for him? Who are these people that have no idea who Graf is, or actually have an unfavorable opinion of Graf, but will vote for him anyway? Also notice that there are 11% of respondents who know Randy Graf but have no opinion on him. This seems very unlikely. It appears to me that people were being pushed to answer certain questions, perhaps not in a specific way, but nevertheless it diminishes the sampling credibility of the poll. An "I don't know" is perfectly valid as a choice.

And finally, anyone looking at this poll with an analytical bone in their body would immediately know that Gabby is not 19 points up. That was just silly.

#3- I believe the Star poll to be more sound fundamentally, but definitely not beyond question. Again, no breakdown is listed. What I can tell you right off the bat is that women were oversampled in comparison with men. It is also telling that in the Star poll released during the primaries, they DID give the breakdown and defined "likely voter."

Here is the thing that also makes me question the poll. If you buy that 45.8% of the voters view that Border control is the number 1 issue for this race, and that Gabby beats Graf on this issue, that means that Graf has to be beating Gabby on other issues such as the war and health care in order to make the numbers stand up. This is very counter to the conventional wisdom going both directions which could happen, but is unlikely. I also do not buy that Gabby is outdistancing Randy in the outlying counties, unless the sample size of these counties is statistically insignificant (which would explain a lot.)

The over sampling of women and the probable bad sampling of the outlying counties would suggest to me a rushed, inexact poll that the Star knew was flawed, but released anyway minus the internals. "600 likely voters" is probably the truth, but you also need to account for party registration, location, et al. I doubt that this was done.

So while this poll is certainly better than the Giffords poll, it does not, nor cannot tell the entire story.

I would argue that the last valid, verifiable, accurate poll that was taken was the Star poll just before the primary. And that poll is getting moldy.

I am sure that there are others who disagree. But take notice that whenever the poll methodology tightens up, so does Gifford's lead.

19 comments:

sirocco said...

Framer,

I generally agree with you. I think the Star/KVOA numbers are much more reliable that the Zimmerman poll. I am not sure I agree about under-sampling of rural areas or over-sampling of women, but we won't know unless they release figures.

However, this statement of yours:

"Here is the thing that also makes me question the poll. If you buy that 45.8% of the voters view that Border control is the number 1 issue for this race, and that Gabby beats Graf on this issue, that means that Graf has to be beating Gabby on other issues such as the war and health care in order to make the numbers stand up."

... is simply incorrect. It's entirely possible for, say, Giffords to have something like a 45-40 edge on border issues compared to Graf, with 15% undecided, and Graf's supporters make up most of the 40% (Giffords supporters making up the majority of her 45).

It's also entirely possible for voters to favor Graf's positon on the border, but favor Giffords over all (the reverse can also happen, of course).

If anything, the polls claim that more voters favor Giffords on border issues meshes well with the results of the primaries, which I have been arguing showed more support for her position on the matter than Randy's. The results are entirely consistent.

Framer said...

Sirocco,

Damn you for making me do math!

If we assume a 50-50 split in men vs. women polling, based on the percentages given, men vs. women the total poll results would be:

47.6 Giffords
36.5 Graf

This cuts the percentage by 1.5 points. I would argue further that more men will vote than women in the actual election, so the results are probably closer to just below 10% for Giffords if all the other ducks are in a row, which they probably aren't.

Notice there is far more growth room in male undecideds as well.

Additionally, If Graf truly is within 0-5 points with Gabby on Health Care and the Iraq War, then this has to be a tremendous boost. All he would have to do is clarify his his views on illegal immigration which are not near as extreme as you or anyone have been led to believe and further clarify Gabby's record on the issue which is pretty close to non-existant.

I still do not believe that Gabby is leading in the outlying counties. It would take more than this flawed poll to convince me of that. And you must admit that the absence of a breakdown is pretty glaring.

I do believe that the errors were probably due to sloppiness and not partisanship or a desire to deceive. The Giffords poll on the other hand. . .

sirocco said...

Framer,

You keep referring to the Star/KVOA poll as flawed without any actual knowledge that it is, in fact, flawed.

I am perfectly ok with throwing out the Zimmerman poll if want. However, given the methodology of the last Star poll was reasonably sound, I see no reason whatsoever to make an assumption that this one is as well -- although I, like you, would like to see the actual data behind it.

Having said that, I should note you can only get Randy's results down to -10% or so by making a series of assumptions:

1. The poll is flawed, particularly in the rural regions

2. The polling was split 50-50 by gender.

3. More men than women (by some significant number) will vote in the election.

4. Graf is within 0-5 points of Giffords on Iraq/Health Care (neither of which is close to true, IMO). It's not necessary to make this assumption to account for Graf's % in any of the polls.

This is a lot of ifs ...

The most notable _consistency_ between the three polls (counting the pre-primary Star poll as well) is the percentage of voters who say they plan to vote for Graf. I would argue a large part of this number is made up of people for whom:

a. The border is the single determining factor in who they plan to vote for (single-issue voters), and

b. They prefer Graf's more hard-line approach.

This is the core of voters Graf was always going to have, and this number has remained essentially unchanged across all polls. It's hard to argue Graf is picking up anything based on those figures.


In considering the question of rural support, I would assume Graf gets all the base R's and hard-line border people, Giffords gets almost all D's, a lot of moderate R's from certain areas like Green Valley, and some (probably largely Hispanic) of any stripe who are adamantly opposed to Graf's border stance.

If I's break Giffords way (all polls currently indicate this, to a greater or lesser degree), it's conceivable she could hold a small edge even in "rural" areas. Of course, I am assuming they are counting "rural" as "anything not Tucson" ... but who knows,



On another topic ... I heard Graf and Giffords have agreed to a series of six debates, including one in Sierra Vista, starting mid-Oct. (including two in one day, I think).

I actually am surprised Giffords would agree to this many debates. No matter how you slice the numbers she is the clear front-runner at this point, and typically front-runners want to have fewer debates, not more.

Her campaign staff must be feeling pretty confident she can win on the issues (which would be consistent with polling which shows she's even ahead overall on border issues -- another small, indirect argument to support that claim).

Framer said...

Sirocco,

I never claimed that Graf was in the lead at this point, nor close to it. I just can't believe that the internals of the first Star poll were laid out and they "forgot" to do this with the second poll. The only reason you do not do this is because they are flawed and the pollster knows it. The Star does not have the time or funding, however, for a redo.

The oversampling of women was demonstrable. Randy Graf came within 130 votes of beating the entire Democratic side with Primary votes in Pinal County (I would give Giffords a wash or, at best, a slight lead in the other two outlying counties, but not enough to make up for Pinal.) I will need more than this poll to show that Gabby is ahead.

You do have to understand that I have a huge skeptacism of polls since long before this race. Took a class once from a very good pollster in college that led me to this, especially polls done by local publications. I want to see at least three REAL polls (the Star is marginablly acceptable, the Gabby poll is not) before I start drinking from their wisdom.

If the race is around 10% (which we are not all that far apart on) then it is certainly far from over at this point Especially as Gabby has agreed to six debates. I'm not going to say this is a mistake, because it is the right thing to do for the electorate. It does, however, expose her to a great deal more risk, as many will be expecting Graf to kill at least three live puppies during the first five minutes and will be pleasantly suprised when he turns out to be very humble and articulate.

I do tip my hat to Giffords for agreeing to six debates. That was classy.

Just as you notice that Graf is hoovering aroung his previous primary numbers, I am noticing that Giffords is settling close to the Kerry numbers of two years ago. Do you think that Graf can convince any of those Republican undecideds that Nancy Pelosi as Speaker of the House would be a bad thing?

At least it will give us something to keep blogging about until Basketball season.

Randall Holdridge said...

This is all completely silly. Graf is running for Congress as an authoritarian, rules-of-golf police-statist in one of the last remaining of the Western free life-style districts in the Arizona.

The West is still alive here, and golf courses, well f..k them.

This is Barry Goldwater, Morris Udall country. You may be able to spoil it ultimately, and jerk us all into your narrow-minded developers' line -- but not quite yet.

Do you have any clear idea how despiable you seem to free men?

Framer said...

Randall,

What?!!!

You really need to take a break from politics for a while. You're rattling off stuff only you understand now.

You are, of course always welcome, but people are just going to start ignoring you if you keep this up.

Don't be the guy in the corner of the room talking to yourself. You're better than that.

Anonymous said...

It was a mistake for Gabby to release her numbers because her numbers will never reach 19% again. That will give the impression that she is slipping.

Randall Holdridge said...

Framer,

I don't want to be either ridiculous or ignored, but maybe you could spell out why I'm so wrong in my understanding of our region?

I'll admit I was trying to be facetiously humorous on the golf rules reference to Graf on the Daily Show, and maybe such a reference is a natural flop on this site, but I'm not kidding about southern Arizona.

I said it was Barry Goldwater, Morris Udall country, and I'll stick by that assertion. You think I'm over engaged; we'll talk after November 7.

Randall Holdridge said...

Suddenly I'm perplexed about you, Framer, the supposed conservative, and I wonder if you ever read "The Last Picture Show" by Larry McMurty, or saw the picture show by the same name, directed by Peter Bogdanovich (1971) -- nominated for 8 Oscars and starring Jeff Bridges, Timothy Bottoms, Cloris Leachman, Cybill Shepherd, and Ben Johnson.

You think I'm the guy in the corner of the room talking to himself? You suppose a Green Valley golf pro is the avatar of a new southern Arizona, despite his having been defeated just two-years ago by 14% against a ranch-boy who was the only openly gay Republican in high office?

In your mind, this probably overrides the fact that Mo Udall was John McCain's mentor and friend, and that McCain visited Mo to read the paper aloud every week in Mo's last oblivious years in a Washington hospital.

No doubt it overcomes the fact that Jim McNulty followed Mo to the seat in Congress, and that his son is a chair of Gabrielle Gifford's camapign, and that Mo's brother Stewart, the former Interior Secretary, and that Congressmen Mark and Tom Udall are actively on board for Giffords.

But you think, Framer, that I'm deluded in asserting a separate southern Arizona identity?

Your candidate has the backing of the Tucson Chamber of Commerce, whose spokesman believes that veal comes from swine, and imagines that there are only two pig raisers in Arizona.

Your candidate, who is Mr. Guns, supposes that gun rights stalwarts don't care about any other of their Bill of Rights except the 2nd, when after all everyone in this country knows they can pile up and store secretly any arms they can buy.

He's Mr. Anti-Abortion and Mr. Anti-Science in a world both better educated and much closer to the biological (evolutionary) realities of life than the most urban constituencies. Your candidate not only ignores the influence of a great land-grant university on our region, he is ignorant of a farmboy's most basic barnyard observation.

Oh, Framer, you think I'm over the top. Long-term, maybe so. But for the moment, golf courses and the vast reach of SE Arizona aren't the same thing. It wouldn't surprise me if you weren't glad we haven't given up our identity. So back off, Pal.

Arizona need not just yet belong to Texans.

sirocco said...

Framer,

I am not sure what I said to indicate you had claimed Graf was "winning". Clearly you've said nothing of the sort.

However, even to get Graf to being within 10% of Giffords, a fairly long set of assumptions have to be made ...

I suspect at this point Giffords actual lead is in the 12-14% range. I agree 19% from Zimmerman is either a badly flawed poll or a statistical outlier on the high side.

Anyhow, aren't we supposed to get another Weekly poll here shortly? That'll give us some more meat to chew on!

John said...

Randall,

Did it ever occur to you that the political orientation of an area can change, especially with a large influx of population? Again you site all sorts of incongruous facts but your overall theme wanders. Yes, Jim McNulty briefly represented the district for two years until Kolbe beat him. Back in 1984 Kolbe did not run as a soft on the border, gay congressman. I remember because I was living in Tucson at the time and followed the campaign closely.

Can the area really be that conservative if Al Melvin just got elected? As to the right to life and firearms issues maybe you should check to voting records of Tim Bee, Jonathan Paton, Marian McClure, and Jennifer Burns. Even Toni Hellon was a stalwart on the right to keep and bear arms.

Yes, Randy used to be a golf pro. Is there a point to that message? Ronald Reagan used to be a B movie actor. From what I understand Gabby ran her family business into the ground and then sold it off to a large corporation.

sirocco said...

John,

Not going to comment on your debate with Randall, but as to your final sentnce, you understand incorrectly.

Gabby was brought in to head the business when it was _already_ in grave trouble, and her father's health was quite poor. She was asked from the start to get it back on a reasonable footing so the company _could_ be sold, which she managed to do with a couple years of hard work.

John said...

Still does not sound like a glowing success story to me. She fixed it up enough to sell, is that what you are saying? Her ads to not give that impression.

sirocco said...

She did what she was asked to do -- took a business in trouble, got it back on it's feet so it was stable, and then sold it.

Her father (who had primarily run the business before) was looking to retire and she wasn't interested in keeping the business in perpetuity ... so what more would you have her do?

Was it successful along the lines of Microsoft? Of course not. Was her tenure successful in achiving what was desired and more? Yeah, it was. Turning around a failing business is etremely difficult, and she succeeded at it.

The Committee said...

El Campo was not doing that poorly when Gabby took over. She made a numer of impressive decisions early on to make the business more women-friendly. Unfortunately, the business did not do well and seemed to suffer from a change in management style resulting in high turnover of key employees. Several stores had to be closed. Facing bankruptcy, the family managed to sell what was left before losing it all.

Not a success story. But then, how was she trained in the business, or any business for that matter? Gabby is bright, but why would anyone expect her to know how to run a business of that magnitude?

I would submit that running a successful business would be an asset, but not necessary for serving in public office.

sirocco said...

TC,

I beg to differ ... I am quite well aware of what the situation was when Gabby came to the business and why she was asked to come take it over in the first place.

The business was, in fact, in terrible shape when she came in. Her father's health and the business health had become kind of a viscious cycle, each making the other worse.

Whatever the financial standing of the business (I don't believe it was ever as close to bankruptcy as you imply, and it certainly was not at the time it was sold), it had improved at the end of her period of stewardship. This included selling several under-performing stores, which is normal in any business which is re-consolidating.

I know the family turned down several offers when things were at their worst, and sold it for a higher one once the situation had stabilized.

As I said before, was it a momentous success? No, it wasn't. Was it successful in achieving what the family hoped for, and maybe a little more? Yeah, it was.

John said...

sirocco,

I stipulate to your version of the story, mostly because I do not know otherwise. But Garry’s ad says she was a successful businesswoman.

Taking over already running business and closing down non-performing stores is not my definition of a successful businesswoman. She did not start or build up a business but cut one down in preparation to be sold. Her ad gives a false impression. She sounds like a real Gordon Gekko.

The Committee said...

I stand corrected on the events. My information was obtained at the time second hand through business contacts closer to the situation than I.

sirocco said...

John,

I guess it depends on what you consider a success. Given the state of the business at the time she took it over, and the state of the business at the time the family sold it, I don't think there is any question her time there was successful.