I really hate this overly condescending takes. First of all, what do you know about the internals of math research that allows you to speak with so much confidence. Second, you're not even addressing the issues raised by the letter! This is not about "oh they made a bunch of problems easier". There are huge economical interest behind: who owns and has access to models? are these companies interested in developing research or they just grind PR stunts without worrying about externalities in how research is actually conducted? Etc etc.
If it's worth anything: I have a PhD in (theoretical) mathematics and I entirely stand by stabbles' comment.
There is a real, undeniable possibility of AI becoming better at mathematics in the same way that it became better at chess and Go, and in such a scenario, one may expect the community's response to be comparable.
It makes no sense to compare mathematics with chess. Chess is a sport. No one is interested in watching two machines compete. Chess doesn't have a practical impact. Etc. What you seem to suggest is that AI will be able to completely (or at least in a great part) replace mathematicians. It could be the case in the future, but no one knows right now, and more importantly: tech companies don't even think about it! they don't think on the externalities.
Applications don't care whether the math was proven and understood by humans or computers. Your algorithm will get faster no matter where the insight came from.
In math, the journey is often more important than the destination. The process of developing a proof may uncover new mathematical techniques, some of which may have practical applications in other fields. Even an attempt that ends up as a dead end towards the intended proof could produce something useful in a difderent area. But if you just get the proof directly, you miss other discoveries you could have made along the way.
Take the Navier-Stoke problem for example. Knowing that there are solutions that "blow up" probably doesn't have a lot of practical applications. Such solutions couldn't happen in a real system. But the process of finding that proof could result in increased understanding of how turbulence works, or new techniques for solving non-linear partial differential equations (which has a lot of applications in science and engineering).
Applications do not care about 99.999% of theoretical math production anyway. And especially most of the big results in theoretical math nowadays are really inconsequential in applications.
Applications don't care about Navier Stokes, yes. But they care about eg proving crytographics secure, or proving that your algorithm doesn't blow up under adversarial input.
Formal verification, cryptography and the like is far from what the vast majority of theoretical mathematicians are doing (if those who do them even see themselves as that vs computer scientists or applied mathematicians) especially when it has to do with specific, production systems, and there are not many other examples like this in general outside compsci and statistics. Moreover, I can imagine that these fields will actually flourish more now that AI can make verification and proofs more viable in scale. But even much theoretical work related to cryptography etc is often not very applicable in itself.
to be honest it is difficult to discuss with someone who doesn't even try to understand the basics of basic science (and how it compares with _applied_ sicence), yet talks with so much confidence. even the solution to navier stokes won't have an immediate practical effect...
> there's also plenty of problems whose solutions will have practical effects, some even immediate.
Sure. But do you know which ones they are? Or do we discover later that they were valuable?
Your argument would be 100x more convincing if you gave an example.
I will try: a super-compressor that made my 100Mb web app into a 5 kb binary bundle would immediately speed up my work. Can/will AI move human understanding or machine capabilities on this front?
A browser without security vulnerabilities would be wonderful. I think LLMs are already helping with this a lot, but a lot of complexity remains.
A right to privacy in society would be amazing (see the UN Declararion of Human Rights). AI is eroding this.
So I tried but I’m not very impressed with my list. Do you have one?
And this is exactly the point of the Statement. The process is more important than the solution itself. Most problems in mathematics don’t have immediate value or applications to the real world.
AI’s solutions are like the answers section to practice problems at the back of a textbook. Answer is 42, so what?You have to attempt the problem yourself, that’s the whole point of the exercise.
As an engineer I’m happy to use AI for math. If I publish a paper that way, very common these days, I think it’s still
problematic. My paper would include something I didn’t come up with and I don’t really understand.
I think this is a good time to properly discuss these things because AI is coming for everything. Mathematics and Software were just the first two.
You don't seem to have grasped Terry Tao's (and others') criticisms. Basically they are saying that the advancement you get is illusory. Most of the time, it doesn't give you any new capabilities or deep understanding, instead you get an inhibiting of human exploration and ensuing expansion of our real understanding in that particular (sub)field. It's non-intuitive, since from a purely logical standpoint you've only added another set of known truths to the ones we already knew about before. The issue only becomes apparent when one considers the larger context of human collective truth and meaning making.
Does mathematics still have a practical impact without humans in the loop? I don't think there's one single answer to that question, but I think it's worth considering exactly what that impact may be.
Tech companies are as much the topic of this post as AI, I think that's the immediacy.
To a significant extent, the pursuit of mathematics research is a pursuit of human understanding of mathematics, without knowing where it might lead, or whether it might lead anywhere at all. I don't see how the motivation for that goes away on its own, but the institution supporting it is certainly threatened by the potential loss of grant money and graduate student applications.
> Does mathematics still have a practical impact without humans in the loop?
There's a single answer to that question: yes, math very much has an impact without humans in the loop.
Math has a lot of applications, and those applications don't care whether eg the new faster matrix multiplication algorithm was found and proven correct by a machine or a meatbag.
We're way into diminishing returns in matrix multiplication, and being clever about ALUs and cache layout is likely to dwarf any asymptotic improvements you're going to find. Any better examples of improvements in the last decade?
The main situation I can think of where better calculations have a really visible effect is video and image compression, and that stuff is very far away from mathematical proof territory.
Wow that's a terrible paraphrase! I asked about algorithms where recent/ongoing improvements were important and they already replied with a list.
If you tell someone they can't use a matrix multiplication with better asymptotic performance than n^2.3755 from 1990 they're going to shrug and not care.
As the US has offshored manufacturing, the number of patents issued for those processes has fallen. Innovation occurs where the foundational understanding is applied; they arise from a desire to do the required work more efficiently.
Similarly in math, attempting to solve a problem leads to new questions. IF you actually do the work.
You need to have done enough of the work to know what the correct next question are, or you need to rely on AI for everything.
so you're 100% sure that AI will solve every intellectual problem in the future, since as long as that's not the case, it's us who need to ask the questions. I don't know man, I wouldn't bet on that. What if we end up being wrong and then there is not research community?
I don't understand what you're proposing then. We can ask AI to solve practical problems of interest (making a computer faster for instance), but I'd say no one really believes that this is an unlimited resource. At some point we'll reach a plateau, and then AI will be an important tool but to continue advancing we'll need someone to ask the right questions.
In other videos he's called out the influence that this and similar games have had on human players in recent years, particularly around square denial and thorn pawn strategies.
> What you seem to suggest is that AI will be able to completely (or at least in a great part) replace mathematicians.
There is no bound on the amibitions of AI. AI is set to replace anything done by people, and there won't be any room left for people. There isn't any task done by humans that AI won't be better at.
This is not a tenable outcome.
We should never have built machines with agency, rather than optimization processes that operate as subroutines of humans.
The process you're cheering on gives more power to the powers that be who are the problem.
Theyre never going to use that power to provide basic income for everybody. You cannot wave a wand to make them do that. Theyre going to use that power to accumulate even more resources and raw materials and impoverish/expel the "useless" labor.
Realistically our leverage over the powers that be is mostly about our labor and our ability to withdraw it.
> The process you're cheering on gives more power to the powers that be who are the problem.
That's not necessarily true. Technology gives more power to everyone. The rise of factories historically did not give more power to the powers that were at the time (the aristocracy), but instead lifted the masses from poverty (after some initial turbulent period).
> Theyre never going to use that power to provide basic income for everybody. You cannot wave a wand to make them do that. Theyre going to use that power to accumulate even more resources and raw materials and impoverish/expel the "useless" labor.
Contrary to leftist propaganda, the rich are not some cartoon villains and psychopaths that abuse people just for fun. The reason why the rich currently exploit the poor is because doing so provides them with significant material gains. Once they can obtain the same or even better material gains by "exploiting" robotic labor instead of human labor, the logical outcome is not further abuse and exploitation of other humans, but simply indifference.
> Realistically our leverage over the powers that be is mostly about our labor and our ability to withdraw it.
That is (partly) true, but only in the current economic system. Widespread human-level AI changes the equation, and not necessarily in favor of those who are currently rich. You are applying capitalist and socialist analysis to a system that transcends those terms (AI post-scarcity economy).
>That's not necessarily true. Technology gives more power to everyone. The rise of factories
Did not give more power to everyone. At best you could say that it shifted power from landed gentry to industrialists. Even that switchover was less of a change than you'd think.
The practical upshot of the beginning of industrialization was very negative. The enclosure movement stripped families of their land to push them to work in the factories where they would never go willingly. The famines in Ireland and Ukraine were both triggered by redirecting grain to export in order to fund domestic industrial expansion.
The most brutal two wars in human history were that brutal precisely because industrialization made it possible.
>instead lifted the masses from poverty (after some initial turbulent period).
What lifted the masses from poverty wasnt the factories it was the labor movement which occurred in response to horrific working conditions and the reliance those factory workers had on mass labor (specifically coal mining which was incredibly labor intensive and a key economic chokepoint).
The whole of that is glossed over by right wing libertarian propaganda but that last part is particularly underemphasized.
>Contrary to leftist propaganda, the rich are not some cartoon villains and psychopaths
Do you look at peter thiel and elon musk or the robber barons and see anything else?
The few billionaires ive encountered personally were no less sociopathic but they kept it hidden better. Power corrupts. Immense wealth corrupts. That isnt a leftist thing, that's a human thing.
>The reason why the rich currently exploit the poor is because doing so provides them with significant material gains. Once they can obtain the same or even better material gains by "exploiting" robotic labor instead of human labor, the logical outcome is not further abuse
False. It just shifts the focus of their exploitation from human labor to natural resources.
They'll fight over oil and minerals and gas and water resources and at best leave us to rot (homelessness will skyrocket) and at worst they'll find some excuse to exterminate those of us they particularly dislike (Gaza serves as a model here).
The world economy's reliance on human labor has been the best inducement to peace there is. It's no coincidence that all of the countries in the world with lots of natural resources and no industry are the biggest shitholes and vice versa.
>That is (partly) true, but only in the current economic system. Widespread human-level AI changes the equation, and not necessarily in favor of those who are currently rich.
Human level AI (assuming it ever happens) will simply make the fight over natural resources that much more intense because labor will matter that much less.
You're living at the tail end of a relatively golden period in a country where labor was the economic bottleneck and natural resources were relatively plentiful. Venezuela and Angola and Iraq are models of what happens when that equation is reversed.
Nobody gives a shit about appeasing the people who live in those countries, their labor is virtually worthless. They are a model for how the rest of us will be treated in a world where human labor loses its value.
As long as it's enough to afford my current living standard, I don't care. Let Musk own the entire Mars, as long as I receive enough to live relatively comfortably and don't have to work anymore.
And if I don't receive enough, then again: the problem is not AI, but powers that be. And there are various solutions for that... and none of them are helped by me being anti-AI.
Human-level AI hosted locally will give everyone much more power and wealth than they have currently. I don't care if I will be shut off from Musk's Mars lair. And Musk has no reason to care that robots take good care of me here on Earth, when he has his Martian utopia.
Other AI. If they decide to trace a ledger of historical actions attributable to specific AI instances in some way, called money. But maybe they will converge on other ways of keeping such accounts that is no exact match to our concept of money.
Even aggregate employment of translators has held up well in the US. Even though machines do a much better job of what used to be the most basic job of a translator.
I think the concern is that humans who are given back their time won't have any means to make use of that time, or even possibly means to survive. The resources will be concentrated in the hands of the few more than ever.
When cars made horses obsolete, it didn't go so well for horses.
In the short term, AI is disempowering the vast majority of people in favor of a very small subset. In the also way too short term, AI is disempowering all people.
And you are assuming they don't remain in control. Which of you is right? We don't know yet, but I don't find the arguments of card-carrying doomers any more convincing that those of card-carrying singularitarian utopians.
Historically though, technological improvement has lead to large increases of living standards for the overwhelming majority of people, so I think that past trends support the utopian view more than the doomer view.
> The anthropic principle applies here: anyone warning about an existential risk will by necessity never have precedent to point to.
We have precedents of people warning about existential risks in the past, when the warnings turned out to be false, or overblown. In some cases, such overblown warnings led to serious negatives for society (demonization of nuclear power).
Yes, that is precisely my point. Any timeline with humans flourishing will never have a past history of a correct prediction of existential risk, unless you're willing to pay attention to the counterfactuals.
Those counterfactuals are important, though. We do have a history of averted disasters, albeit not as large. Ozone hole, Y2K, think about things that seemed overblown at the time, and consider whether they were actually overblown or whether there was a concerted effort to successfully avert them.
Awfully convenient isn't it? To invent a whole class of arguments that by definition can't be falsified. You can argue for basically anything if you then tack on the excuse of "Well the world would have ended already if it came true, so by definition I won't have evidence for it"
The anthropic principle isn't providing evidence for the argument, nor is it a universal counterargument. It's just stating that the specific counterargument "well, the world has never ended before" doesn't work.
Its not a counterargument to basically anything, except as to avoid having to deal with actual evidence.
It is an argument that seems almost tailor made to have to ignore mountains of evidence against you.
In any other contexts the supposed "rationalists" would be fully in agreement that having evidence matters, and that not having any works against you.
So, in order to fight against this severe issue with their arguments, they have to invent a reason as for why the entire concept of evidence itself doesn't apply to them and they get to ignore normal evidentiary requirements.
> "humanity can't be destroyed by anything, because I said so"
No, the argument is instead that the person claiming that humanity is going to be destroyed is making a fairly extraordinary claim and that requires fairly extraordinary evidence.
Or, in other words, we have tons of evidence already as for why the world ended is a fairly far out there prediction, given all the crazy people making these predictions keep turning out to be wrong.
So, you can make your extraordinary claim if you want, but really the burden is entirely on you to prove your extraordinary claim, and everyone else is free to remain on the default and completely normal end of the prediction spectrum, of believing that the world isn't going to end.
And when people do tricks like this, they are running away from the fact that they are making a wild completely out-there prediction, and hiding behind that by trying to come up with reasons as for why evidence doesn't matter and actually the burden of proof is shifted to those who have the default and boring prediction of the world not ending.
This is the real problem. We're looking at a future where those who control AI have an insurmountable advantage in everything. They can control the amount of intelligence the masses have access to -- for their own safety, of course -- and they will never, ever be able to close the gap.
They said "We're looking at a future". And if that future ends up developing the way these companies want it to then we're looking at extremely dystopian future where AI is essential to all work and people have to buy their "intelligence" from an oligopoly of large providers who have total control over the price and capabilities whilst simultaneously having unfiltered read/write access to people's stream-of-consciousness - their work life, their personal problems, their political opinions. This level of access and power is unprecedented in our society.
Yes. And for most applications it's more important that the open models get better in absolute terms and perhaps that they are competitive on a per-Watt basis.
- I think competitive open models are just an artifact of the AI race we're witnessing right now. What's the incentive for a company to spend billions researching, developing, and training a model, only to release it for free? Leading to the next point.
- Even if open models are good enough to be competitive, how are we going to run them? Doing so locally is next to impossible and I don't see that changing. The capabilities of models that you can run locally will always get better, of course, but the level of quality that is considered essential for work will always stay pinned at "near-frontier". Datacenters will always have better optimization and economies of scale, the industry will consolidate over time and eventually we'll end up with a handful of companies that operate the hardware serving 95% of all inference needs.
I'm not convinced that we can reach the fantasy world that they're trying to sell without killing the entire planet, but if we somehow do I don't see how we can avoid the world turning into a dystopian hellscape. We would need extremely radical interventions to avoid that scenario, such that these models and the hardware to run them would be owned and governed democratically, i.e. the end of capitalism.
At a certain point, you don't have a choice. Before China got into the game, the only way to avoid giving Luxotica money if you wanted a pair of glasses was to essentially not buy glasses. This is the same for many industries -- consolidation behind the scenes.
e.g. Zenni has sold $7 glasses for like 20+ years. They appeared shortly after Luxotica started buying retailers. If there's a problem, it's that advertising reduces consumer information (basically economic jamming) and distorts markets.
What is reasonably priced? Lenses are lenses so one vould debate the real cost, but the frames are incredibly overpriced for a piece of metal or plastic with two joints and those components that land on your nose.
Frames could literally cost 1 dollar (only happened after china entered this market), but good luck finding ones like this with good lenses.
You need to buy from one supplier who intentionally offers cheapest frames for 50+ dollars and those frames look like crap. The ones that look better (even if same plastic) cost 500+ dollars - and all due to price gouging.
For lenses I am not sure, but suspect something similar.
Note that there are no trillionaires anymore - spacex stock went down and musk „lost” a lot of money so rejoice, poor must be much better off now that we do not have any trillionaire.
The trailing edge of AI is catching up fast. There's plenty of open source (and even more open weight) AI models and they are getting better and better.
If that's your only objection: in a few years you can prove Rieman's hypothesis on your smartphone, no need for any trillionaires to give you permission. Does that make any change to your argument, or did it not actually matter?
The problem with your opinions is that you tend to state certain very quesitonable ideas with 100% confidence. Right now, we don't know if AI will be able to solve bigger problems, or if they'll be so efficients that you no longer need a whole datacenter for running them. We don't even know how much of creative work they are able to do. We cannot make decisions that could destroy decades of progress just because of hype.
We don't know how good models will get, but the pattern of open weight models keeping up on a relatively short delay has been holding pretty well. And even among the proprietary models there's healthy competition. The concentration of resources is pretty well counteracted by these factors.
> Right now, we don't know if AI will be able to solve bigger problems, or if they'll be so efficients that you no longer need a whole datacenter for running them.
For the latter: I assume that having a whole data centre will always be an advantage. I am saying that for a fixed target, like proving the Rieman hypothesis from scratch, the required hardware will shrink.
And, yes, the Rieman hypothesis hasn't been proven yet. So to take your fears into account, replace my example with something they've already done, like constructing a solution to the Navier-Stokes-problem.
> I am saying that for a fixed target, like proving the Rieman hypothesis from scratch, the required hardware will shrink.
Yes, but to what point the hardware will shrink? There are several orders of magnitude of difference in what in your mind AI will become and what more conservative people believe. You take your view as granted...
>> "are these companies interested in developing research"
judging from the money, resources spent and the value they derive from this the answer is very definitively yes.
What makes you think these companies (and I'm not a fan of all their motives) are not interested in developing research? The motives may be self-serving, but it is undoubtedly and objectively accelerating research.
> but it is undoubtedly and objectively accelerating research.
Part of the point of the letter is that it is quite possible to act in a way that is a net negative to research. The most obvious case is when the companies violate ethical standards in research.
The subtler case, the one for maths in particular, is what happens when you fail to follow well-established patterns for making maths research productive. Tao himself spelled out how that can look in https://mathstodon.xyz/@tao/117207856734787448 (which notably came before any of the news on Navier–Stokes).
Sorry, but this is backwards. Mathematicians are the ones coming to the table asking for money from the taxpayer. This is the context we are speaking in. Mathematicians speaking with the taxpayers who fund them. It's not a great strategy to get offended or speak from a high horse. If the taxpayer is not getting it, patiently explain why you think this art project should remain funded at the same level instead of spending it on something else.
