Today I got an email from eBay asking whether I'd mind filling out a survey for them. The survey was one of those that asked questions designed, quote, to improve the user experience when buying, unquote. Generally, with these, I tend to fill them out, providing I have the time, and by that I mean it'd better take much less than 10 minutes or I will just abandon it. Of course, if the survey is on something I feel strongly about, I'll stick it out — at least most times.
This particular survey was about presentation of items for sale and how that influences my decision. I was asked to imagine I wanted to buy a Canon Powershot camera and was presented with a fake list of such items (here's the current list). I was then asked to choose one of the items and answer various questions about that choice.
Right off the bat, the scenario was meaningless. The reason is that I wouldn't just look for "Canon Powershot". I would be searching for a particular model (presumably I'd read reviews and was now looking to score an example of the best). The list had every single kind of Canon Powershot, just like the list I linked to. There are older, much cheaper cameras in that list, and there are newer (maybe new) and more expensive cameras. My primary choice therefore would have nothing to do with the presentation of the items, but everything to do with the particular model I putatively was looking for. The survey, in my mind, was poorly designed, at least for me. So I randomly chose one in the middle of the list as the one I wanted to buy. (It also didn't help that the list was presented in "Best Match" order, one of the most irritating sort sequences ever devised by man.)
It was only on the next survey screen that I realized that every item in the fake list had a fake page behind it. The survey was assuming that I was going to read the detail pages, all 25 or so of them, before making my decision. Er, not. One, the list is too vague, too diffuse, too unordered, to map to the way I use eBay, and two, there's no friggin' way I'm going to read the made-up descriptions, feedback rankings, shipping policies, etc, etc, yadda, yadda, for 25 or so items in order to fill out a survey. It's like playing blackjack or roulette with funny money: you have no skin in the game and so you don't concentrate, get bored, and do silly things like bet everything on 00.
So, bye bye, survey.
There was another kind of survey that I struggled though a couple of years ago, this time for Adobe. Man, this one was like wading through treacle. It asked you to reflect on several scenarios for pricing the next version of Creative Suite. I thought it was only for one scenario though, so I tried my best ("No, I think $500 is too much for upgrading from X to Y, but $400 is reasonable. If X contained Illustrator and Photoshop, I'd be willing to pay such-and-such."). The setup was just as complex as the eBay one since it posited all these different package possibilities and pricing charts for new purchases versus upgrades and you had to keep all this in mind and be reasonable about your choices and to make sure that you didn't make the Premium option cheaper than the Standard option, and so on. I completed it, went on to the next screen, and suddenly realized that there was more than one such set of scenarios to wade through. It was like completing a level in some game and then being presented with the next, but without the expectation of some fun times ahead blowing away the aliens. I can't remember how many levels there were (at least three) but I sweated through it, and then it asked a bunch of questions about my reasoning. All the way back to the first scenario/level. It was the crème de la crème of survey torture and I needed a stiff drink afterward to calm my shattered nerves.
And then there's the survey like I filled in last night as it happened — sheer coincidence, in case you're thinking I just spend my spare time filling in surveys — where I answered X to a question on page 2 only to be presented with a question on page 5 that assumed I'd answered not-X. And, no the survey wouldn't allow you to go back. Cretins.
Now playing:
Collin, Marc - Les Kidnappeurs (Main Theme)
(from Café del Mar, Vol. 6)
Loading links to posts on similar topics...
3 Responses
#1 Terri Morton said...
08-Oct-09 8:13 PMWhat was the incentive for the respondents to complete the survey? Gift card? Free merchandise? Just a "feel good"?
#2 julian m bucknall said...
08-Oct-09 8:51 PMTerri: the first two were feel-good surveys: "you're helping us improve our product; you'll eventually benefit from the results".
The third super-duper "FAIL" one actually paid $10 for me to complete it -- paid towards the service the survey was all about. Too early to see if they paid up (they said it could take up to 2 weeks to appear).
Cheers, Julian
#3 Patrick said...
11-Oct-09 2:11 PMLast week, sourceforge repeated their first (failed) attempt to let open-source-developers fill out a survey. The questions where okay-ish, but after filling in page two (which took me about 20 minutes in-between other work, so the page was open for about an hour) and submitting, the session timed out. Dang! Back button gave me a time-out page, merrily advising me to login again and start over..... FAIL!
Leave a response
Note: some MarkDown is allowed, but HTML is not. Expand to show what's available.
_emphasis_**strong**[text](url)`IEnumerable`* an item1. an item> Now is the time...Preview of response