r/SneerClub archives
newest
bestest
longest
Apparently, no one in academica cares if the results they get are correct, nor do their jobs depend on discovering verificatable theories. (https://i.redd.it/siqmd364h83b1.png)
128

Meanwhile, EY only gets paid if he’s right about the acausal robot god, right? Right?

damn. i hate when im reading through old stuff on reddit and in the middle of a sparkling, scintillating discussion i find someone has written over all her old comments with nonsense, fragmenting the discussion permanently. what hilarious, moving, romantic, haunting things could she have said? just to wash it all away, in this digital era of permanency? wow. that takes courage. i bet she was really cute, too

I mean he might. A lot of the AGI Doom stuff he does is premised on what is essentially "magically knowing if things are true".
Ironically, since we're talking about Yud, that's the idea you might get from a certain kind of education. Uninspired primary and secondary science education tends toward a model wherein students do an "experiment" to test an already known outcome and their marks depend only on how accurately they got the right answer and explained their experimental error. I guess there's some merit in that for the self-persuasion of students who are deeply skeptical about gravity or germ theory or whatever (not sure how history teachers are supposed to win them over), but it exemplifies a completely wrong idea of how science works and why you'd even do an experiment.
Hey, I had that experience! I measured the acceleration of a cart, and discovered that the acceleration times the mass of my cart was greater than the force applied. Either my track was tilted, or Newton's second law is incorrect. I was marked down because my experimental results didn't validate Newton's law. I was annoyed, because that's the exact opposite of how science is supposed to work.
Yes, sounds like it. He thinks we publish “theories” Theories

Null results are hard to get published, especially in higher impact journals, and publications, especially higher impact publications directly build towards tenure and better grant funding.
Ironically, the types of reforms Eliezer has vaguely gestured to before (that have been better proposed by more informed people) would probably change this by making it better incentivized to publish null results.

This also once again demonstrates his Efficient Market Hypothesis type thinking where throwing money in the form of markets at a problem should automatically fix it.

This also demonstrates a 5th grade science fair understanding of science, where everything breaks down into neat hypotheses with clean yes or no answers and not messy spectrums of theories each with many overlapping claims and few with completely distinct predictions.

He's also missing an element of risk aversion. If the funding model for science is that you make your money by being right about stuff, you're going to lose a lot of people who are (not unreasonably) worried that they will not be right often enough to make a reasonable income. The problem (as I, a non-academic, perceive it) is that you cannot make enough money in academia, particularly as an early-career academic, to support yourself. Adding some cash prizes for the top end doesn't seem like it fixes that.
Exactly, academia relies on the labor of severely underpaid PhDs and moderately underpaid (at least relative to their level of education) PostDocs. Professors already spend a lot of time writing grants and appealing for funding after having spent years being underpaid as PhDs/PostDocs, his proposal would add an additional burden of (possibly literal if prediction markets were used) gambling on hypotheses as well as the bureaucratic step of trying to convince the funding agency you deserve credit for being right.
And also underpaid grad students. Support grad student unionization y'all!
Right I meant to write PhD students
>This also demonstrates a 5th grade science fair understanding of science, where everything breaks down into neat hypotheses with clean yes or no answers and not messy spectrums of theories each with many overlapping claims and few with completely distinct predictions. That's what immediately stood out to me. If I make a moderately accurate but very cheap and fast model, it might be better in many practical applications than the state of the art, but that doesn't fall into a "actually true" vs untrue binary. ​ Even high school science adds nuance to the idea of scientific understanding. 10th grade chemistry often includes the history of atomic models from Dalton to Thomson... to the quantum model. The quantum model is still accepted as correct today, but we needed those increasingly less wrong guesses to get there (doubly ironic considering his blog title).
I don't know if he truly sees it this way. He is a published academic so his very binary way of looking at things shows a bit of a deficit in his understanding of anything that isn't computer-oriented. I'd do some horrible things if someone told me I could get a solid yes answer for anything I decided to work on.
He isn’t actually published in peer reviewed journals, he just uploaded his work to preprint servers and presented at MIRI hosted conferences.
Oh! I honestly know very little about his credentials but that would explain why I couldn't find them. He seems like he'd really want people to know about them
He has tried (once?) to get one of him game-theory/decision theory papers peer reviewed and published, but the peer reviewer pointed out he completely ignored relevant previous work by other people, and that his decision theory completely neglected how to deal with the cases it was attempting to make counterfactual… full description here: https://www.umsu.de/blog/2018/688
"academia isn't incentivised to care" which is why someone took his whole paper apart. He's a bit of an odd guy.
Yeah, because academia hasn’t paid attention or awarded status to his work, he needs a rationalization for this, he can’t just accept his work isn’t that good or that he hadn’t put in the effort to engage with a academia. The focus on “incentivizing” is part of a broader rationalist obsession where they think markets solve everything and the Efficient Market Hypothesis is literally true.
But he's a "researcher" he his work is just as valid of a contribution because he sees the future something something AI doomsaying I'm seeing that pattern and my field of psychology has a bit of blame there. Too much pushing of behavioral theories about rewards and punishments to where people think you can get individuals to be literally trained like dogs.

And of course there exist no perverse incentives outside of academia for not caring about the truth of the theories you’re espousing.

I thought these guys had moved away from such simplistic incentive structure theories of social behaviour with their “the 20s had better ‘social technology’ than we do today” bs. But I guess there’s never any actual progress in these kinds of discourse communities, just endless churn.

Elizer, you understand that you can advance in your academic career by publishing important and accurate results, that you can damage your career by doing the opposite of that, and that your career may impact your compensation? Furthermore, you understand that, by virtue of these scientists being in academia they have chosen to not prioritize money and have other reasons for doing things?

Fuck’s save you autodickdact

> autodickdact Someone who figures out all on their own how to go fuck themselves?
Only if we're lucky.
The rest of us had to go to school and get degrees in self-fucking because we aren’t special
> Furthermore, you understand that, by virtue of these scientists being in academia they have chosen to not prioritize money and have other reasons for doing things? I don't really think this is a great argument. Money not mattering more than the difference between working in research or at Jane Street (or whatever the high-salary private sector alternative for you is) doesn't mean people won't gravitate to a research field that comes with higher salaries. Financial incentives aren't the only thing that matters, but they do matter on the margin. That said, the big financial incentives here is the gap between the private sector and research, not so much that we haven't structured research incentives correctly. Especially because "this doesn't work" is also a useful result. I sort of doubt that you'd improve research quality more by offering cash prizes for positive results (or whatever it is Yud is imagining) than by making it so that people who would've gone to Jane Street went into research instead.
> I sort of doubt that you'd improve research quality more by offering cash prizes for positive results You'd worsen it. It would just incentivise faking good results, in the worst way, to the worst people.
All of this is fair.
does anyone have the link to when he said he was offered an AI safety position but the money wasn't good enough?
How does any of this explain the existence of Marc Tessier Lavigne or all the others like him? Famous scientist gets caught falsifying data in flashy journals, results are disproved, blames postdocs and becomes president of Stanford.
I mean is the problem there really that Marc didn't get $10k in cash when he put out his initial results? I don't think Yud is wrong that there are problems with academia, I just don't think his solutions seem likely to be particularly effective.
If anything, giving people a bonus for "successful" results just encourages them to falsify stuff rather than admit your research is a dead end and move on to something else.
I’m sure Yud is an asshole and his “solution” would make everything much worse. But this comment is hopelessly naive, scientists don’t get to be in academia unless they prioritize (grant) money over everything else, including whether or not their theories have any basis in reality.
Every STEM academic I know could be making more money in corporate work, but has chosen to stay in academia because of the ability to work on projects that would not fly in a profit driven environment, and grant money is very different from personal profit through a start up or salary. I'm not saying academics are selflessly pursuing truth, I'm saying they either can't or won't operate operate in corporate environments for social or obsessive reasons. Literally, every physics grad of the last twenty years could make more coding than working on physics, yet there's still people doing physics. Take home pay for the most well compensated physicist is probably much lower than that of a random software development, barring pop culture successful authors. His suggestion to introduce financial incentives for certain research is deeply dumb and part of his market fetish.
Improve your career to the tune of probably 15k a year over a decade in the UK for example, he's really not that wrong on this (academic incentives are all wrong at the moment, e.g. Terry Tao has to do his own conference/money paperwork!)
Yes, but Yudowksy's take is a willfully ignorant person's understanding of issues. What does "actually true" even mean for most sciences? ​ If I create a new macroeconomic model that is slightly less accurate, but 90% cheaper, do I qualify for a "actually true" reward? Or, the reverse, slightly more accurate but infeasibly expensive for most uses model? ​ Would Newtonian physics qualify? It was state of the art and helped advance humanity for a long time before we moved past it, and even today air planes don't worry about quantum level interactions. ​ If I discover gene A increases cancer risk, but it turns out that it is the interaction between gene A and B that does so, and I was missing the second part, does that count? Does it matter if the later researcher cites and builds upon my work? ​ I'm not disputing objective truth, but defining true / good vs. bad research is not as easy as it sounds.
The answer is that it's fuzzy and should be decided fuzzily. If you invented Newtonian physics it should be obvious that you should be rewarded. A lot of academic research in STEM + Economics is basically non-serious in attitude (e.g. papers about a software that don't bother publishing their code). I think financial incentives would be a very good way to keep it real - money is just a tool, and a very effective one: there is some research on the compensation of traders that reveals that the vast majority would much rather have a bonus today than one next year tied to the firm's performance, a direct reward Vs waiting for career progression feels similar.
I do think Yud's idea for fixing this is probably "pseudo-privatized prizes", which strikes me as not a great idea. I do not think (though I am not in academia, so perhaps I am wrong) the problem is "we know where the most promising research can be done, but we aren't doing it for some reason" so much as "it is hard to know what research is promising and there is a lot of bureaucracy that slows things down". Incentives do matter, but they are not the only thing that matters. The structure around those incentives also matters.
Also this runs into the trust problem. Especially if it's privatized, then what stops Big Sheep from saying "$10,000 to any scientist who confirms and reproduces these studies showing that cotton is a leading carcinogen, $20,000 to anyone who can prove wool underwear makes your dick bigger" or whatever. It's incredibly easy for experimenter effects and methodology choices to mess with results and create a (bad) study that proves nearly anything even without actively trying to subvert the process. This is the whole reason why replication is important, which you may recognize as the entire goddamn problem. Creating incentives that specifically reward getting a certain result is just going to make the underlying problems (re: science is hard) worse.
> you can advance in your academic career by publishing important and accurate results, Eh, you can also advance your career by knowing the right people, engaging in relentless self-promotion, burnishing results to make them more impressive or novel, and parlying all this into shoe-in-publications at unreasonably impactful journals (I'm thinking of particular people here).
I think that all of those incentives are at least the same, arguably worse, with a prize based incentive structure.

My brother in Christ, have you ever heard of grant applications?

How would you even determine if the theory in your research is true? And if it is later false would you need to turn back the money? Not sure this is all a great plan to fix the problems in academia.

This also will make university administrators very rich, and incentives short bullshit papers which are easily proven over fundamental harder work.

E: hmm, perhaps he recently realized that string theory is not as valid a theory as he though and this is how he is lashing back?

Spoken like someone who has never tried to publish in a reputable journal

“The problem with academic science is that the rewards aren’t naive, gimmicky and gameable!”

Maybe I just have a particularly bad case of post-PhD burn-out, but I am somewhat more sympathetic to this take than I usually am to Big Yud.

Glossing over the philosophically thorny issue about what it even means for a scientific theory to be “true” (and that probably shouldn’t be glossed over, tbf), anyone who has spent time in academic science has heard broadly similar critiques being made by working scientists.

Most of the metrics by which scientific careers are judged are largely independent of whether the work you’ve done is good, let along “true” in any objective sense. Anyone who has ever had a paper rejected b/c Reviewer 2 felt like engaging in bullshit political gatekeeping knows what I’m taking about. Granting committees can be the same way. There are dozens of thinkpieces out there written by scientists describing how the current system incentivizes shoddy, thrown-together papers at best and pushes people towards outright fraud at the worst.

Of course, the the fact that these critiques exist and can be read proves that Yud isn’t really saying anything particularly novel or insightful here, but when your baseline of Hot Takes is “acausal robot god justifies nuking China” or “Aella should let me [DATA EXPUNGED] in her [DATA REALLY EXPUNGED]”…you take what you can get.

I think we should learn from another nuclear standoff: "Do not congratulate." Giving the man credit when he pumps out a a take that is not horribly far from reasonable just rewards his strategy.
yeah, but I'll never pass up a chance to grouse about how bullshit academia is.
I would argue that the problem with academic research is that incentives are \*too\* sharp, not that they're missing. It's such a cutthroat, sink-or-swim environment with so few winners that no one can afford to fall behind, so they churn out papers and game the metrics. I'm sure I'm romanticising it but there really was an historical moment, not that long ago, where being an academic was basically a decent reliable job that paid below the odds, but in exchange you got to live a life of the mind and pursue research for its own sake. That's a pretty great way to attract people who value quality in research, and get out the way of them keeping standards up.
> Anyone who has ever had a paper rejected by Reviewer 2 Bonus points if another lab publishes your idea, but tweaked a tiny bit for plausible deniability, very shortly after.

Yes it would be much better if everyone got paid a bunch of money for all their theories that were true. They definitely wouldn’t make up results.

What academia definitely needs is more winner-take-all competitions with vague simplistic objectives that might allow people to game the system.

verifiable

which is problematic epistemologically but at least it’s a word

Nobody cares except for the journals who’s name gets dragged down because they published false results, the scientist who published them and is now going to have a much harder time finding funding, the people who funded this useless science, the people employing the researcher who are now getting their name dragged through the mud for having lax standards, students involved with the research who now need to answer for why their name appeared on fake science…. You know, nobody involved cares at all, because there’s no “correctness bonus”

Yud would be a lot more compelling if he would just strip his arguments down to their truest fundamental essence: struggling to suck his own dick.

Good to know that the man who thought he was singularly equipped to save the entire universe is unfamiliar with the concept of salaries.

Good to know that the guy who's spent his adult life founding nonprofit organizations is unfamiliar with the concept of pursuing non-monetary rewards.

How is this dude so popular? This has to be the dumbest take ever conjured up. “These people don’t get paid extra to do their literal job correctly, so their results must be wrong (and mine are obviously right).”

He’s not that popular

There are organizations in academia that will very much pay you and fund your research if “the theory is actually true”

money is the only thing that matters, apparently. The esteem of peers, or a place in history doesn’t count because it can’t be measured in dollars. What a pathetic little man

He is partially correct though roughly not at all in terms of the implication I think he is making.

Much of academia would look much worse if not for a considerably high density of “true believers” who pursue some intellectual project out of intellectual curiosity, egalitarian motives etc.

Arguably it is getting worse as these critical true believers are in fact penalised, as their ideology makes them resistant to engaging in trend chasing, game playing, ass kissing, producing superficially acceptable or impressive but actually flawed work etc.

These true believers will actually put in a lot of hard work if they trust the institution they are working for, but that also is less the case as university management tend to have very different culture, ideology etc.

Poisson thought Fresnel’s wave theory could be dismissed with a thought experiment. Poisson showed Fresnel’s theory predicts a bright spot in the shadow of a circular disk - Poisson stopped there, thinking he had refuted Fresnel since everyone knows the shadow has no bright spot. Arago did the experiment and found the bright spot.

Of course he is unaware of this thing in academia called “tech transfer” where professors and their school split profits from commercializing inventions. Quite a few professors have become filthy rich from this, for example the guy who invented Gatorade, the chemotherapy drug Taxol, etc

Said by no one working an academia. If your stuff is true and works, it makes it easier to get more grants, in STEM stuff gets patented, and the publications are open to review by anyone else that reads them.

Meanwhile, this guy gets venture funding for sounding right. It is always projection.

As much as I like to dunk on him, he’s not wrong there. Academia’s incentives are rotten, I left Academia after a few postdocs because my marketing skills were more important for my career than actual results.

It’s incentives are bad but fixing it is a lot harder than Eliezer makes out, and throwing an extra 10k here and there isn’t a good solution.
What is really needed: prediction markets.
On the blockchain. Finally, a Web3 use case!
Oh definitely, throwing money at it won't change anything especially when the whole system isn't designed to figure out what's "true" (truer ?) or not.
I would argue that things in academia are as bad as they are exactly BECAUSE "good research" is directly financially incentivised in the way that big yud claims it isn't (ie don't get enough impact factor and goodbye grants)
There's space between "Yud is right that there is a problem" and "Yud has the right solution to the problem". In general, people are great at identifying when there is a problem, okay at identifying what the problem is, and bad at coming up with solutions to problems.
Oh I totally agree. The last thing we'd need is Yud deciding what is true.

That’s not true, cutting edge researchers who have a track record of being right get promoted to Assistant professor (to get tenure) to full professor to a named chair. Each come with a pay bump. Also, they become attractive to other departments, so they start getting paid more not to leave.

Yud: To be clear, I’m not saying that nobody in the system cares

OP: Apparently, no one in academica cares

bruh

Yud isn’t directly saying it, but if you’re familiar with his… *economic incentive* frame for understanding the world you’d understand he’s basically saying almost nobody cares.
I'm not as familiar with him, but it seems like he's saying that nobody is *incentivized to care by the system.* And in this case it's a very short-term, myopic view of it. But there's lots of cases where people do care about things that they're not incentivized to. I'd argue most public school teachers care about what they do far more than they're incentivized. It just means that the caring has to come from an intrinsic source rather than the system.
> it seems like he's saying that nobody is incentivized to care by the system Which isn't necessarily correct either; I was an academic, and I'd argue that "being right" (at least in interesting ways) is directly incentivized by getting papers accepted in Nature/Science/big-journal-of-your-choice, which in turn leads to bigger research grants and job titles.

the word ‘verificatable’ irritates me

Stopped clock moment for Yud

I don’t know if it’s twitter, the inevitable rot that comes from repeating the same thing over and over for decades, or simply old age/bad health(like he says), but wow the quality of his reasoning has taken a very sharp nosedive hasn’t it. He used to be much better at the game of constructing a logical edifice given some initial premises, even if the premises were mostly incorrect, even if if the edifice was many times more logical-sounding than actually logical, even at the face of the fact that trying to predict the future behavior of complex systems by ‘arguing from first principles’ is a fool’s errand anyway. If you *wished* to play the game, it wouldn’t be so easy to point out exactly what the fault in the reasoning/analogy would be.

These days it’s like there’s a 25% chance that he will start typing a “hot take”, forget where he was going with it before he hits “send”, and immediately trying to backpedal. Offering a counterargument used to be harder than ‘well i would definitely hire ants to dig holes for me if only i could get them to listen’ or literally ‘well so are you’.

Keeping their job, that is their incentive to be correct. Maybe eliezer can get away with not telling the truth but in most jobs you arn’t correct, you get fired!

Well then academia isn't "most jobs", because you certainly don't get fired if your theory is wrong there. Although it's not as bad as it sounds: testing the wrong theories to destruction is a useful way of narrowing down the right ones, so wrong academics aren't _completly_ useless, although they are less useful than correct ones.

True is whatever the party says