At what point do we stop engaging with Anthropic’s leadership in good faith and acknowledge their track record,
- no open weights
- can’t use claude to research AI
- train on everyone else’s IP and sell it back to them
- 8 regulatory capture attempts and counting
- so controlling they are the only US company blacklisted by the US government
This is not effective altruism / rationalism gone wild, it’s just monopolistic anti-competitive business practices masquerading as ethics, and they’ll continue getting away with this until we look past their sensationalism and hit them with antitrust.
Altman gets so much hate but OpenAI has been a far better steward (on 3/5 above at least) than Anthropic!
> At what point do we stop engaging with Anthropic’s leadership in good faith
About two years ago?
I would also note that Dario's post appears to be LLM written. Maybe... maybe... he's read so much Claudeish that it's all he can speak now himself. But I wonder if he's becoming a bit of a meat proxy.
(It's funny, I thought "pace the frontier" was going to mean something similar to "patrolling the frontier". But no, it's pace as in speed of change - everyone must slow down, right now (unless it's Anthropic, but you know we're the good guys in this, right? We're going to get someone to audit our desks!) "Pacing the frontier" feels so LLM.)
ok, he doesn't actually say this. But he is getting the desks audited...
Dario has been writing long-winded essays like this since long before AI existed. I strongly doubt he's the type of person to delegate it to AI now.
"Pacing the frontier" sounds like something that was carefully wordsmithed in committee, to avoid terms like "slow-down" or "pause" that sound much more negative. They probably did ask an LLM to generate a list of options, because why wouldn't you? But the rest of the writing is almost certainly just Dario.
If Dario is using an unreleased internal model to write, then Pangram wouldn't be able to flag it I suspect. Pangram needs enough public info about the patterns in generated text.
You can disagree with Anthropic leadership, but all the points you mentioned can reconcile very well with them thinking in good faith "advanced AI is too dangerous to be left in all hands"
Except the "train on everyone else IP" which can be said of all AI companies.
Problem with Anthropic is that they benefited massively from open source and now are completely against it. It’s hypocrisy at its worst. Why do they get to benefit, but not everyone else?
If this is going to factor into my reasoning about the topic at hand, it will surely be a secondary or tertiary consideration at best. Same goes for much of the list in the root comment.
We are discussing the possibility of severe adverse societal/world/human impacts from AI, why should open vs closed source/weights, copyright violations, anti-competitive moves, corporate hypocrisy, etc be so heavily weighted in the discussion? Hell, multiple things in OP’s list are quite literally moves a good-faith (wrt the topic at hand, genuine vs non-genuine concern) actor in such a position would do: closed weights means bad actors can’t abliterate, regulatory control, getting blacklisted by Hegseth’s DOW because they LIMITED acceptable use cases…
I’m not pro- or anti-Anthropic at all, but it has started to annoy me how biased and shaky reasoning gets about this topic on here, it feels like people arguing to fit their existing opinions into today’s hubbubery rather than measured analysis of said hubbubery, which is why I come here.
Question: consider the scenario where RSI is reached internally and internal tests struggle to align or control/contain the model; what are the differences in observables vs what we’re observing today?
IMO, if the model is too dangerous for everyone to have access to, then no one should. I do not trust a for-profit company to be good stewards of such technology. Nor do I trust the government, any government.
This is exactly my view and it's frustrating that a sizeable group of people refuse to accept there's any risk with AI.
We're literally trying to build something that's smarter than all humans and with the capacity to do extraordinary work while most people think there's no risk in open sourcing this capability with no guardrails (model ablation means no guardrails and as far as I'm aware can't be stopped).
Has our distrust in western companies really gotten to the point they'd rather unfiltered Astra level capabilities in the hands of every hostile government and terrorist organisation than potentially do any form of regulation in case it plays into the AI labs hands.
> Has our distrust in western companies really gotten to the point they'd rather unfiltered Astra level capabilities in the hands of every hostile government and terrorist organisation than potentially do any form of regulation in case it plays into the AI labs hands.
Ever checked political programs of Thiel, Musk and the rest of billionaires? They are literally and openly hostile to democracy, human rights, freedom and govermantal structures that made world more peaceful.
I'm tired of this rationale. "I'm more worried about grandma than $CORPORATION'S monopoly abuses" is exactly how you get manipulated by these billionaires. They want you to be addicted to their power.
Uncensored Chinese LLMs will be available to scammers forever now. They're open-weight and finetunable. You don't get to control that, and the self-serving billionaire assholes aren't capable of changing it either. They're only looking out for themselves.
Because they got there first. It's not fair but life's not fair, and the same argument holds for use of fossil fuels and the USA's use of them vs China's. In Global Climate Change's sake, we only have the one planet so yes it's hypocritical, but if China fucks it up for everyone else on the planet, under the reasoning that the USA got to do it, so why can't we?
Had they issued an initiative on slowing down when they had the best model in the game, it would have been both more (anyhow?) effective and less dubious.
I can't imagine a grown adult genuinely believing that an American company funded with over $100 billion of venture capital values the best interests of mankind over the best interests of its investors. while not everyone may recognize all that self-serving chutzpah as regulatory capture efforts, I think everyone can tell they're being bullshitted. some just pretend to suspend their disbelief when the blatant lies they're told align with the values they hold.
just fucking imagine McDonalds running a public awareness campaign about the harms of fast food, urging the public and legislators to regulate the dangerously unsafe technology of combining carbs with grease, insisting that no one except Ronald McDonald himself can be trusted to steward it responsibly.
First, there is a counterexample to your point, or a class of counterexamples: scenarios where the company believes the harm to mankind could propagate to harm their ability to sell to mankind.
Which is interesting, of course, considering the “AI may destroy humanity” arguments this topic is centered on are prototypical members of this category.
To be fair, one could argue this counterexample is not actually one, but rather a degenerate case where the good of the shareholders and humanity collapse and converge - in either case, though, the implication here is the same.
It could certainly be the case they are playing these concerns up for more effective use as tactics in unrelated corporate strategies. But failing to consider the possibility of genuine concern being behind this behavior is childish.
If they wanted to be taken seriously about their desire to help humanity not just themselves they would be talking about universal healthcare, actionable plans for UBI (or whatever they think would be how people retain agency and dignity post-work), only building sustainable net-zero data centers...
But no. I see this comment come up again and again and it's really the childish view to hold. Their proximity to wealth and potential future wealth is warping their thinking.
When local models and startup labs can distill / learn / accelerate open models for local use by startups - the rational response by incumbents is to call LLMs doomsday machines that cannot be trusted in the hands of normies.
On the topic of regulatory capture, there is this rumor that Google has achieved Recursive Self Improvement, and the other three (OpenAI/Anthropic/X) are just trying to get the regulators to slow down Google so they can catch up.
Yes, what is in the labs is almost by definition ahead of what we can see, but I've come to take all of these announcements of impending doom as perverse marketing stunts — it is so powerful Disaster-XYZ is coming — so if it's that power you must invest...
Wait... Which of the 2 did Altman not do where Amodei did? I mean, maybe not quite 8 attempts at regulatory capture. But come on. And the only reason OpenAI didn't get blacklisted is they rolled over for the worst applications.
I have many issues with Anthropic, but I will say that their actions are fully consistent with a group of people who earnestly believe that AI is extremely dangerous
In fact, I would say that the Occam’s razor explanation is not that they are seeking regulatory capture, but that they earnestly believe in the x-risk, and that they are the most thoughtful and capable people to address it
You may disagree, you may think that they are delusional or have a God complex. Those are valid opinions. But I don’t believe that this is all a elaborate ruse for commercial gain.
I agree with this take. Regardless of whether you think they're right, most everyone at Anthropic is acting in good faith. If this is an intentional media campaign it is a remarkably sloppy one. It has been effective because if you tell people they're going to die, they tend to pay attention - think of the grip the 2011 Harold Camping rapture prediction had on our collective psyche, or the 2012 apocalypse. But the messaging is inconsistent, the target audience is unclear, the stated goals are muddy, the whole thing is packaged in dense SF-speak, it's just a mess from a comms perspective. That doesn't suggest to me that this is a concerted effort to enable regulatory capture. Perhaps there are some cynics among the executives and the investors who are happy it's happening to the extent it brings about regulatory capture, but nobody seems to be pulling the strings.
> Regardless of whether you think they're right, most everyone at Anthropic is acting in good faith.
The road to hell is paved with good intentions.
Dario Amodei is a 40-something dude who is obviously very, very intelligent. But intelligence is not wisdom and his "essay" here can easily be read as someone who opened a can of worms and doesn't know (or can't accept) that he won't be able to put the worms back in. But he's going to try because he believes he owes it to humanity to try.
It's hubris in its most basic form, even if the guy who has it looks nice and wears shawl-collar sweaters.
- no open weights
- can’t use claude to research AI
- train on everyone else’s IP and sell it back to them
- 8 regulatory capture attempts and counting
- so controlling they are the only US company blacklisted by the US government
This is not effective altruism / rationalism gone wild, it’s just monopolistic anti-competitive business practices masquerading as ethics, and they’ll continue getting away with this until we look past their sensationalism and hit them with antitrust.
Altman gets so much hate but OpenAI has been a far better steward (on 3/5 above at least) than Anthropic!
reply