posted on March 26, 2016 04:54 PM by
u/completely-ineffable
6
u/LWSurvey2016AMA5 pointsat 1459057076.000000
I wrote the LessWrong Survey this year, ask me anything.
Hi /r/sneerclub. This years LessWrong survey is pretty intensely
criticism and improvement focused as well as being explicitly targeted
at people who are dissatisfied with the current LessWrong, so you’re
welcome to take it as long as you’re not ballot stuffing/brigading.
I'm just going through this and some of your questions are atrociously constructed. Your politics question is straightforwardly offensive to people like myself who identify as further left than social democratic, and yet have only the option of communist(!) which is shockingly described as "complete state control of many facets of life"! And this has nothing to do with quibbles about my special category not being included, it is first, a totally inappropriate category as compared to the others on the list, and a flatly ignorant description of the category itself
Yes, my apologies about that one. It wasn't written by me/is a carryover from previous surveys and I felt forced to continue with it for data control purposes. (The survey isn't usually written by me, so I tried to minimize the amount of editorial discretion I exercised over questions that appear on previous surveys. If I were running this survey again and felt I could comfortably do that I'd definitely look into just entirely redoing quite a few of the questions which I don't feel are particularly well done either.)
If I was writing the survey from scratch I would never include an option written like that.
As a point of comparison this is the 2014 survey: https://docs.google.com/forms/d/1h4IisKq7p8CRRVT_UXMSiKW6RE5U5nl1PLT_MvpbX2I/viewform
That's fair, but as far as I'm concerned all of the previous data from that question is junk data as soon as you have left of center respondents. Not to be rude, as I came off a bit abrasively there, but I also felt that the *majority* of the substantive philosophical and political questions were ridden with other false choices. This is especially hard to avoid in surveys I know, but there was a considerable bias towards,for example, lesswrongian orthodoxy (as in the communism question). This fact alone makes identifying coherent, popular, but unorthodox positions much more difficult. I'm afraid I think you really could have taken much more care with this aspect of the survey. On the other hand, good job on the self-criticism stuff, that was all fairly clear and well constructed
>That's fair, but as far as I'm concerned all of the previous data from that question is junk data as soon as you have left of center respondents.
That seems like a pretty fair point.
>This fact alone makes identifying coherent, popular, but unorthodox positions much more difficult. I'm afraid I think you really could have taken much more care with this aspect of the survey.
The philosophy questions were also carryovers largely. I get plenty of complaints about them and I can't really object to any of them besides that.
>On the other hand, good job on the self-criticism stuff, that was all fairly clear and well constructed
Yeah I got to design that part and I actually think it was one of the weaker areas of the survey, in terms of anticipating the full range of things people will put in. On the other hand I plan to publish the public-opt-in write ins as some standalone more readable format. If the ones I've seen so far skimming the results manually are anything to go by they should be pretty entertaining food for thought.
>The philosophy questions were also carryovers largely. I get plenty of complaints about them and I can't really object to any of them besides that.
Then change them! But seriously I know that's tough with the previous data set and all, fair enough
Well, it's tough but certainly not impossible. The basic point is you need to decide when to cut off backwards compatibility and then redo sections from scratch. I definitely want to put more love into those once we do.
Well, there's several components to that decision and I'll go ahead and address them all separately.
>Why is the first set of questions after basic demographic information about sex and relationships?
Great question, the reasons are:
A. Implicit bias. This was how the 2014 survey was laid out and I started writing this survey by taking that survey as a base. I kind of didn't think about it that much.
B. Deliberate decision on the basis of data control. I figured that keeping the sections mostly in the same order would help keep the environment impacting the results similar enough so it can be meaningfully compared.
Those are the boring answers though, why didn't I go ahead and change the ordering? Especially when I know that survey respondents are going to drop off on a curve the longer the survey goes on so it would make sense to ask the most important questions up front.
C. Special interest in the data from the standpoint of demographic information. Some of the questions in this section are basically extended demographic information, it wouldn't be inappropriate to have subtitled the section "Demographic Information Part Two". Whether you're married or not for example is a pretty huge piece of information about where you're at with your life and what kind of concerns you're dealing with. Same with the questions about number of children, etc.
D. Special interest in the data from the standpoint of community wellbeing. Kind of for the same reason we ask about mental health, it's important to understand how people are doing collectively in terms of quality of life, outcomes, etc. (We actually moved the mental health section up closer this year because we want to know what kind of things we should be doing in terms of accessibility.) If you take the results seriously they also suggest that LessWrong is something approaching a quarter LGBTQ, which is interesting to keep track of in and of itself.
I wrote the LessWrong Survey this year, ask me anything.
Hi /r/sneerclub. This years LessWrong survey is pretty intensely criticism and improvement focused as well as being explicitly targeted at people who are dissatisfied with the current LessWrong, so you’re welcome to take it as long as you’re not ballot stuffing/brigading.