Yep. Perhaps humanity would be better off if we instituted and enforced the notion of "Thou shalt not make a machine in the likeness of a human mind." It's worth thinking about; just because we can build AI systems doesn't mean we should.

I would remind you that the fictional society that did this was also described as a feudal monarchy in which a tiny class of nobles exploits the economic output of trillions of people reduced to serfs by controlling all means of interstellar trade. Oh, and they also have legal slavery.

The implication in the book is that the exploitation enabled by AI in their history was somehow even worse.

During the Butlerian Jihad, the people involved in it didn't know that ten thousand years of slavery would be what they are choosing, so we don't know how bad it actually was.

The way Jihad presented in the oblique references in books, anyway, it sounds a great deal like a fanatical fundamentalist religious movement to me. Those are not known for their fine reasoning.

Honest question, is an oppressed human society better than no human society at all? Isn’t there at least a hope for humans to free themselves from human oppressors?

I'll remind you that a fictional product that provided longevity and woo woo powers of prescience and was only available on one planet in the entire universe was the key context that enabled such a centralized society. Dropping thinking machines and their masters like a bad habit was only part of the cause, as it eliminated one means of safe interstellar travel in that reality. I can take ideas from this plot without bringing in the whole ridiculous mess if I like, just like I can walk and chew bubble gum at the same time, thank you very much.

“B-b-b-but muh pop culture trope”

Thank you for commenting this. I'm tired of hearing about Dune like its some kind of utopia

Do you seriously think that's that people mean when they quote that?

Of course they don't, but if you choose to use a work of fiction as an illustration, it comes as a coherent package (assuming the author did a good job at worldbuilding) - you can't just take the bits you like and ignore the bits you don't and still have something coherent.

I think it would be depressing in its own way if this technology just ended up being regulated out of existence.

Yes. In the best possible world AI could enable a much better future, but currently we are on a quite bad trajectory.

Yes, and mass excitement

Don't you mean mass investment?

> We should be happy that they’re stealing people’s work?

No. We should credit that work is being done that otherwise may not be. I'm not defending OpenAI. I'm defending AI in general, something both OpenAI and Buckmaster (and his co-author) used.

You literally said they _stole_ the work. Why should we give credit to that?

> You literally said they _stole_ the work

No, I did not. If someone says either X or Y happened, they didn't say X happened.

My exact statement: "Either by OpenAI on its own, OpenAI stealing another's work and finishing it or a researcher using AI to iterate more rapidly than a human alone could" [1]. (Either X, Y [a and b] or Z. An Oxford comma could have helped.)

[1] https://news.ycombinator.com/item?id=49671523

No one owns mathematics. It's not covered by IP law.

It's immoral to steal someone's research. Quite sad that you don't believe that.

Yes, let's lie and call something stealing, then pontificate about morality.

At worst what OpenAI did was inconsiderate, but not referencing prior work is a minor academic sin. This is especially the case when AI is not playing the vanity game that seems to so inflame some mathematicians.

> Yes, let's lie and call something stealing, then pontificate about morality.

We could also acknowledge that everything that could justifiably be considered stealing is not necessarily covered by current statute.

It was inconsiderate and it was also stolen, just like everything else OpenAI has stolen to train their models.

The problem with technology is that the people who have it conquer and kill all the people who don't, essentially ensuring a destructive race to the bottom. You would need a mutual international alliance of nations who not only renounce the technology but also agree to invade anyone who attempts to develop it. Our closest precedent is nuclear weapons, but rather than renouncing them, the US's position is that "only we should have them" and that other people acquiring the same tech they have are valid targets for invasion, and they will obviously apply that notion to any new technology that can be weaponised as well.

Google which countries have nuclear weapons. Hint: It’s not just the US, it’s also its allies and frenemies who’ve not been invaded. It’s only nutball countries like Iran that get invaded when on the cusp of acquiring it and for good reason.

Try googling which country was the only country to use nukes. Hint: it's another nutball country.

There is strong evidence that Iran was not as close to a nuclear weapon as you say.

We will find out the truth eventually, like we found out that the WMDs in Iraq claim was a complete lie.

The consequences of that invasion debunks the claim it was done "for good reason". It seems to have been done without use of reason at all.

Ah yes, such nutball countries who have apparently been on the cusp for 30 years. Wild definition of cusp for sure.

The technology is fine, great even. It’s the rabid greed of venture capital and sociopathic CEOs that try to integrate themselves into every aspect of our lives for profit, that’s the problem.

“For 19.99 a month you too can be a world class mathematician!” Meanwhile they are extinct.

I feel the opposite. Human history is depressing overall. I imagine AI will eventually lead to less scarcity and better lives overall.

I am sure it will lead to more material wealth, but I can’t help but feel pessimistic about the political economy of making millions of people redundant. What leverage will knowledge workers have in the new world?