Nobody outside the company has been able to confirm whether the impressive benchmark performance of OpenAI’s o3 model represents a significant leap in actual utility or just a significant gap in the value of those benchmarks. However, they have released information showing that the most ostensibly-powerful model costs orders of magnitude more. The lede is in that first graph, which shows that for whatever performance gain o3 costs over ~$10 per request with the headline-grabbing version costing ~$1500 per request.

I hope they’ve been able to identify a market willing to pay out the ass for performance that, even if it somehow isn’t over hyped, is roughly equivalent to an average college graduate.

You could argue that another moral of Parfit’s hitchhiker is that being a purely selfish agent is bad, and humans aren’t purely selfish so it’s not applicable to the real world anyway, but in Yudkowsky’s philosophy—and decision theory academia—you want a general solution to the problem of rational choice where you can take any utility function and win by its lights regardless of which convoluted setup philosophers drop you into.

I’m impressed that someone writing on LW managed to encapsulate my biggest objection to their entire process this coherently. This is an entire model of thinking that tries to elevate decontextualization and debate-team nonsense into the peak of intellectual discourse. It’s a manner of thinking that couldn’t have been better designed to hide the assumptions underlying repugnant conclusions if indeed it had been specifically designed for that purpose.

Oh no I’m in this sketch and I don’t like it. Or at least, I would be. The secret is to acknowledge your lack of background knowledge or basic grounding in what you’re talking about and then blunder forward based on vibes and values, trusting that if you’re too far off base on the details you’ll piss off someone (sorry skillissuer) enough to correct you.

I think that’s something to keep an eye on. The existence of the AI doom cult does not preclude there being good-faith regulations that can significantly reduce these people’s ability and incentives to do harm. Indeed the technology is so expensive and ineffective that if we can find a “reasonable compromise” plan to curb the most blatant kinds of abuse and exploitation we could easily see the whole misbegotten enterprise wither on the vine.

“…according to my machine learning model we actually have a strong fit in favor of shooting at CEOs. There’s a 66% chance that each shot will either jam or fail to hit anything fatal, which creates a strong Bayesian prior in favor, or at least merits collecting further data to scale our models”

“What do you mean I’ve defined the problem in order to get the desired result? Machine learning process said we’re good. Why do you hate the future?”

It’s definitely linked in with the problem we have with LLMs where they detect the context surrounding a common puzzle rather than actually doing any logical analysis. In the image case I’d be very curious to see the control experiment where you ask “which of these two lines is bigger?” and then feed it a photograph of a dog rather than two lines of any length. I’m reminded of how it was (is?)easy to trick chatGPT into nonsensical solutions to any situation involving crossing a river because it pattern-matched to the chicken/fox/grain puzzle rather than considering the actual facts being presented.

Also now that I type it out I think there’s a framing issue with that entire illusion since the question presumes that one of the two is bigger. But that’s neither here nor there.

I think his criticism of the economics and business sense is pretty reasonable, even though he is definitely being pretty credulous about the capabilities of the underlying tech. One of the fun side effects of the diminishing returns in raw scaling is that the competitors are rapidly catching up with the capabilities of ChatGPT, which is going to be bad news for Saltman and the gang. What goes unaddressed is the bigger underlying problem; these systems don’t actually do what they’re being advertised for and burn an unsustainable and unconscionable amount of money (and actual resources in case anyone forgot) to do it. That’s going to be the difference between OpenAI falling apart and being overtaken by another company with better monetization or the entire tech sector facing a recession, and I’m pretty sure the latter is more likely.

Apparently a well-known event where authors gave away a ton of free ebooks led to people getting their Kindles locked down and reset due to a fraud detector falsely flagging it as suspicious activity.

I found one single article about it

But this is a representative Reddit thread

Unfortunately actually working in bio/med didn’t go well despite training for it aggressively and working her ass off given that she graduated at the perfect time to compete for entry-level positions with recently laid-off people with 5+ years of experience. Between that and chronic illness in the family will all the associated experience with the failings of our medical system I’m actually pretty sympathetic to the biohackers from a purely ideological perspective, but these people are just begging for a disaster.

Beyond reading through and enthusiastically agreeing with everything you had, she did say that if you’re working on anything consumable or injectable using 3D printed parts at all is going to be a red flag. Your first two pieces of equipment should be an autoclave and a fume hood, at which point you’re better off working with all glass for durability and not melting reasons. Making your work space actually sterile and sufficiently free of contaminants to do any of this jnt be first place is also going to be a pain, require a lot of tape and curtains and the like, and probably not work as well as you’d want.

Also even working in a proper university lab with a fume hood and climate controls you still get sufficiently different results that the mk1 eyeball is utterly insufficient for identification. You’ll learn worse than nothing.

I read through this with my wife who actually did a whole course of organic chemistry, pre-med, etc. Her reaction to your criticisms was largely “of course you would have to do that and that’d be a pain in the ass but it’s definitely doable.” And I feel like that’s probably true but at the same time as a reasonably smart dude this is the first time I’ve heard the majority of these words.

It feels like they’re reacting to the same tendency in tech culture that I’ve complained about before where specialized knowledge and domain expertise are treated like arcane revelations from beyond. It’s not that your average person is incapable of understanding or going through the relevant processes; Cs ,as the saying goes, get degrees and I’m sure many of the labs actually doing this work at scale are staffed largely by those C students. But it’s also worth acknowledging that there is a level of investment in time and resources required to actually do it and this kind of “you have nothing to lose but your pharmacological chains” rhetoric is dramatically underselling how difficult those processes are and how bad the consequences can be if you screw it up. Anyone who wants to try should first read The Iliad, the Oedipus cycle, and at least one other epic-sized meditation on hubris. And the once you’ve forged ahead read the relevant books on actually following the process for the love of God.