This is the real problem. We're looking at a future where those who control AI have an insurmountable advantage in everything. They can control the amount of intelligence the masses have access to -- for their own safety, of course -- and they will never, ever be able to close the gap.

There's lots of competition in AI. Where do you see the 'insurmountable advantage in everything'?

They said "We're looking at a future". And if that future ends up developing the way these companies want it to then we're looking at extremely dystopian future where AI is essential to all work and people have to buy their "intelligence" from an oligopoly of large providers who have total control over the price and capabilities whilst simultaneously having unfiltered read/write access to people's stream-of-consciousness - their work life, their personal problems, their political opinions. This level of access and power is unprecedented in our society.

Open AI models are just a few months behind closed models, and the gap is actually getting more narrow.

Yes. And for most applications it's more important that the open models get better in absolute terms and perhaps that they are competitive on a per-Watt basis.

Two concerns:

- I think competitive open models are just an artifact of the AI race we're witnessing right now. What's the incentive for a company to spend billions researching, developing, and training a model, only to release it for free? Leading to the next point.

- Even if open models are good enough to be competitive, how are we going to run them? Doing so locally is next to impossible and I don't see that changing. The capabilities of models that you can run locally will always get better, of course, but the level of quality that is considered essential for work will always stay pinned at "near-frontier". Datacenters will always have better optimization and economies of scale, the industry will consolidate over time and eventually we'll end up with a handful of companies that operate the hardware serving 95% of all inference needs.

I'm not convinced that we can reach the fantasy world that they're trying to sell without killing the entire planet, but if we somehow do I don't see how we can avoid the world turning into a dystopian hellscape. We would need extremely radical interventions to avoid that scenario, such that these models and the hardware to run them would be owned and governed democratically, i.e. the end of capitalism.

So far we haven't seen much of that consolidation.

And a lot of people are using trailing edge models just fine already.

> [...] but the level of quality that is considered essential for work will always stay pinned at "near-frontier".

Why? When we'll finally all write our software in Lean and prove it correct and prove it fast, it won't matter that a slightly more clever model could have found a slightly nicer proof or whatever.

Just like today people happily use Python for many programs, even though rewriting in C might give you a performance boost. Good enough is often good enough.

Say they do. What then? Will people buy from them? What if we decide not to?

At a certain point, you don't have a choice. Before China got into the game, the only way to avoid giving Luxotica money if you wanted a pair of glasses was to essentially not buy glasses. This is the same for many industries -- consolidation behind the scenes.

e.g. Zenni has sold $7 glasses for like 20+ years. They appeared shortly after Luxotica started buying retailers. If there's a problem, it's that advertising reduces consumer information (basically economic jamming) and distorts markets.

Huh? I've been able to buy reasonably priced glasses for all my life. (However, I've never lived in the US nor China.)

What is reasonably priced? Lenses are lenses so one vould debate the real cost, but the frames are incredibly overpriced for a piece of metal or plastic with two joints and those components that land on your nose.

Frames could literally cost 1 dollar (only happened after china entered this market), but good luck finding ones like this with good lenses.

You need to buy from one supplier who intentionally offers cheapest frames for 50+ dollars and those frames look like crap. The ones that look better (even if same plastic) cost 500+ dollars - and all due to price gouging.

For lenses I am not sure, but suspect something similar.

I grew up in Germany, but I got frames in UK and Singapore and Australia, too.

Fielmann in Germany has been offering very reasonably priced glasses and frames for ages.

Note that there are no trillionaires anymore - spacex stock went down and musk „lost” a lot of money so rejoice, poor must be much better off now that we do not have any trillionaire.

The trailing edge of AI is catching up fast. There's plenty of open source (and even more open weight) AI models and they are getting better and better.

If that's your only objection: in a few years you can prove Rieman's hypothesis on your smartphone, no need for any trillionaires to give you permission. Does that make any change to your argument, or did it not actually matter?

The problem with your opinions is that you tend to state certain very quesitonable ideas with 100% confidence. Right now, we don't know if AI will be able to solve bigger problems, or if they'll be so efficients that you no longer need a whole datacenter for running them. We don't even know how much of creative work they are able to do. We cannot make decisions that could destroy decades of progress just because of hype.

We don't know how good models will get, but the pattern of open weight models keeping up on a relatively short delay has been holding pretty well. And even among the proprietary models there's healthy competition. The concentration of resources is pretty well counteracted by these factors.

> Right now, we don't know if AI will be able to solve bigger problems, or if they'll be so efficients that you no longer need a whole datacenter for running them.

For the latter: I assume that having a whole data centre will always be an advantage. I am saying that for a fixed target, like proving the Rieman hypothesis from scratch, the required hardware will shrink.

And, yes, the Rieman hypothesis hasn't been proven yet. So to take your fears into account, replace my example with something they've already done, like constructing a solution to the Navier-Stokes-problem.

> I am saying that for a fixed target, like proving the Rieman hypothesis from scratch, the required hardware will shrink.

Yes, but to what point the hardware will shrink? There are several orders of magnitude of difference in what in your mind AI will become and what more conservative people believe. You take your view as granted...

Exponential growth in hardware's capabilities, like we are used to, means that several orders of magnitude are a matter of a relatively short time.