posted on May 17, 2023 07:25 AM by
u/JasonPandiras
59
u/breckenridgeback74 pointsat 1684321555.000000
Weird fetishism about prediction markets from rats aside, “the lower
probability option sometimes wins” isn’t really a criticism?
It’s a probabilistic prediction. If it’s a good one, a thing it says
will happen ~25% of the time should happen 25% of the time. If 25%
outcomes happened 0% of the time, it wouldn’t be a good prediction. If
you want to evaluate a probabilistic prediction, you need to look at all
of them, not one of them.
yeah, the us presidential voting “markets” literally track polling most of the time because that’s the information people have.
the idea that someone knows some SIGNIFICANT insider information about a campaign, and will use it to make a renegade “investment” on some market site, and that everyone will take this nEW iNfoRmATiOn seriously (despite the same action being taken by hundreds of idiots for no particular reason at all), is kinda silly.
for-profit regulated consumer corporations (the most predictable economic actors in the world) still surprise investors regularly.
In the late 90s, early 00s era, when people were not paying as much attention to polling and the major media were much more invested in punditry and horse-race reporting, a prediction market which mostly functioned as an aggregator of polling would do quite well. After Nate Silver broke into the media and blew up the silly punditry (see 08 where he did well, 12 where pundits really had egg on their face in comparison), everybody's making predictions based on poll aggregation, essentially. The question is then what the best way of doing that is, and frankly nobody takes prediction markets all that seriously as a way to do that. I believe they are an input to the 538 model and Silver looked at them a little, but it's just one thing among many, not this magic superpredictor.
yeah, they’re not completely useless but on any given day the prediction market is usually going to reflect a combination of:
1. yesterday’s poll results
2. bias/hype
3. noise
It might have some value if, for example, a prediction market was taking bets on CERN experiment results and the participants were all the world’s particle physics scientists.
But that’s not how they seem to operate in practice.
this was the original idea of prediction markets but it turns out educated smart honest people don’t really need magical crystal balls to tell what each other are thinking.
and if there is a cONsPiRAcY going on, what dumbfuck is going to use it to make a bet on some shady betting site? at best you make a pittance by gaming the market a little, and at worst you get murdered by your coconspirators. GREAT!!!
they’re great for getting stupid people to slowly waste their money tho, which is not all bad.
One can get polls that say just about anything these days as well, in that situation wouldn’t one expect almost all political prediction markets to trend towards 50% and thereby be essentially uselss?
the market will presumably “punish” the polls that don’t correspond to reality.
instead of polls, just consider the usual market: “one can make a company to do just about anything these days as well,” but that doesn’t mean anyone is going to buy your stuff, especially if it is not fit for purpose.
a political prediction market CAN play a role assigning credit to which polls are better. it’s just that this is SO DISAPPOINTING compared to the “tEh mARkTe iS sUPeRhuMAn oPTiMaL ALwAys aND FREEEEEEEDOMMMMMM!! replace all human organizations with markets immediately!” boosterism.
i mean, yeah, maybe political prediction markets are an optimal “filter” for polls in some sense. but that’s a significantly less grandiose claim than the nuts are making.
> yeah, the us presidential voting “markets” literally track polling most of the time because that’s the information people have.
What does it mean for a market to track polling?
well then prediction markets can’t work at all.
futures pricing don’t necessarily translate into a real-world outcome. they’re measuring two different things. 😂
They are good because they aggregate the news in the optimal way, or you can go make some money until they are optimal again. How am I supposed to know how much each news story matters? With prediction markets you can just see how much the probability moved.
what does “aggregate the news in the optimal way” even mean? you really can’t decide what news matters to you? and you need other people to exercise THEIR profit incentive, in order for YOU to get information? roflmao okay, uh… no offense, but… why bother living? 😂
Oh sure I can make an alright guess but it would be nice to be able to make better ones. If I want to know how much inflation is going to be over the next year, I could try to look at a bunch of news stories and what it historically has done or I could just look at current 1 year TIPS spreads.
In addition, it is more easily verifiable to other people. If you aren’t getting automatic COLA then it is easier to get a raise if you’re citing inflation being 6% over the next year based on TIPS than it would be to say you’re getting inflationary vibes.
I don’t think prediction markets are some necessary thing for societal development or that they are going to be a massive boon to the economy, but they’re a pretty sweet thing to have like google maps versus a map.
Oh I thought this was a discussion about having a relatively unregulated prediction market in the US rather than our current system of semi legal betting with caps and fake money systems. Stuff like metaculus is slightly useful but nothing like having billion+ liquidity markets for stuff.
> Oh I thought this was a discussion about having a relatively unregulated prediction market in the US rather than our current system of semi legal betting with caps and fake money systems. Stuff like metaculus is slightly useful but nothing like having billion+ liquidity markets for stuff.
But I mean this is kind of the big issue with the rhetoric about prediction markets, they're all puff and not real. There's a lot of skin in the game on TIPS (and note the market also *has something to do with* the future inflation rate), we're involved in actual trades, that's never going to be the same as betting on who wins the presidential election even if you really do have some money on the line.
Prediction markets aren't meant to forecast random events though, their whole thing is that if there's money involved then the efficient market hypothesis will yield results superior to common sense and media narratives, for a variety of alleged reasons.
Erdogan beating the polls seems like exactly the thing prediction markets should be good at predicting, yet here we are.
>yet here we are.
The prediction markets gave him a 20% chance of winning.
>[prediction markets seem like a useless intermediary if all we're doing is following the news, though.](https://www.reddit.com/r/SneerClub/comments/13jun7z/superforcasters_be_like_best_i_can_do_is_state/jkhwv0o/)
What probability did the leading newspapers give him?
But wouldn’t that make them largely useless in any case in which there aren’t any prior outcomes to compare? Or in instances where the field changes significantly along with prediction techniques over extended periods?
Political comtests for instance. One might do better simply choosing the most historically reliable pollster than going to a market where the results might reflect an aggregate of different polls as interpreted by a number of different users. Wouldn’t that just lead to an extra level of complexity that might make the probabilities too vague to be meaningless?
Also has there been any research into the reach of prediction markets beyond the anglonet? Something tells me that a prediction market in which the investors are primarily non Turks might not reflect the potential outcomes of Turkish elections very accurately.
I’m genuinely curious as I am not very well versed in statistics and prediction markets in general.
This post removed in protest. Visit /r/Save3rdPartyApps/ for more, or look up [Power Delete Suite](https://github.com/j0be/PowerDeleteSuite) to delete your own content too.
That's true, this example offers no probabilistic evidence against the prediction validity model of prediction markets.
There are more valid ways of confronting the idea, but I have to say I still like this post as an ok-tier sneer :X, that's irrational anti-rationalism here :D
But it also points out something fundamental about what probability means. For repeatable experiments that statistically behave as a stochastic process would, we can talk about probability as an inherent property of the system. But for non-repeatable experiments, probability doesn't have the same meaning as it does to a roulette wheel (when observed with "ordinary" observation methods); rather it's a measure ascribed to the betting behaviour. Of course, we could statistically analyse the betting behaviour itself, but given the non-repeatable nature of the experiments, what makes for a good statistical fit is also not obvious and requires a subjective interpretation. It is still obvisouly true that sometimes people lose bets, but that's not quite the same as "sometimes the lower-probability event occurs" in situations where probability has a less subjective meaning. Rather, it means "sometimes people don't have the pertinent information."
I think it also runs into some of the quantification biases that the rationalists are prone to. For non-repeatable events, what's the difference between predicting a 25% chance of a thing happening and a 40% chance? How can you tell which prediction was more accurate if the event only happens (or fails to happen) once? Probabilistic descriptions are valuable for examining different states of repeated events in consistent systems, but unless you can repeat the events the probabilities don't actually communicate anything about the system. It also assumes that a "generic predictor" is a thing a person can be regardless of the actual systems being predicted. It's yet another attempt to adopt the aesthetic of rigorous inquiry and rational evaluation but without a meaningful source for the additional data they're claiming to add.
This post removed in protest. Visit /r/Save3rdPartyApps/ for more, or look up [Power Delete Suite](https://github.com/j0be/PowerDeleteSuite) to delete your own content too.
Yes (and assuming the universe is deterministic, they *are* the same at a low level), it's just that the meaning of probability as something that is subjective is much more apparent in the non-repeatable case because it's not testable. In the repeatable case it's easy to say, even in retrospect when the outcome is known, whether or not the bet was reasonable; it's not easy to do in the non-repeatable case.
If prediction markets fail on big events (they also failed on the Trump election, for example), I'd argue it can overshadow the more numerous yet trivial cases when it makes correct predictions. (I'll admit the experts mostly failed with Trump too though)
---
The ancient Greek Oracle at Delphi was able to maintain its reputation for accurate prophecy because the priests in charge of it screened petitioners and would only let them ask the oracle after their question had been rephrased and approved by the priests -- and then when the oracle made her prophecy those very same priests would tell the petitioner how to interpret it.
I think there's a lesson in there.
---
PS. A random thing I just remembered: it hasn't been commented on much, but I think it's noteworthy that FTX also ran a prediction market. It failed anyways.
> (they also failed on the Trump election, for example)
They gave Trump a ~20% chance on the eve of the election, which was down a bit from the previous week since the last few polls were quite good for Clinton. 538 did have them beat (they gave Trump a [28% chance](https://projects.fivethirtyeight.com/2016-election-forecast/)), but pretty much no one thought he was a favorite outside of his cult.
Yes, Silver was fine -- he said the exact way Clinton could lose, pretty much. Also we didn't have good polling post-Comey, which probably did have a big enough impact. Going into the election, everybody was ragging on 538 because there were people putting Trump at 5% and clearly Silver just wanted Trump to win by giving him such a relatively high probability.
yea,,, I don't think giving something 20% odds and it happening is a prediction failure unless you do that a bunch of times and it happens significantly more or less often than 20%
I guess the takeaway is that prediction markets would be good at events that repeat very often and therefore the probability gets tested multiple times? But that just ends up repeating the stock market to a given extent, right? At best, it could expand the concept out to some non-business cases.
You have to remember that the end game for these people is using prediction markets to literally replace the government (Robin Hanson called this hypothetical system "futarchy"), I think the rationalists' fetishization of prediction markets has more to do with a libertarian ideological stance about the role of markets in society than anything about the actual accuracy of prediction markets.
Definitely, and you can see it in the absence of any predictions that are more sensible, aka the repeating ones. So actual accuracy has never been their goal, only the appearance of it.
I know that Taleb is a tad silly, but he did write an interesting
paper on why “superforecasting” to produce a single-point estimate is a
load of wank: https://arxiv.org/abs/2007.16096
Weird fetishism about prediction markets from rats aside, “the lower probability option sometimes wins” isn’t really a criticism?
It’s a probabilistic prediction. If it’s a good one, a thing it says will happen ~25% of the time should happen 25% of the time. If 25% outcomes happened 0% of the time, it wouldn’t be a good prediction. If you want to evaluate a probabilistic prediction, you need to look at all of them, not one of them.
prediction markets offer perfect information about prediction markets
we need to rename them superduperforecasters
50-50 is the only probability.
I know that Taleb is a tad silly, but he did write an interesting paper on why “superforecasting” to produce a single-point estimate is a load of wank: https://arxiv.org/abs/2007.16096
“The weatherman was wrong” energy.