recalling the exact phrasing, several senior people at anthropic made public statements agreeing a 10% chance of causing extinction in less than 10 years.
10% is uninsurable, priced in with ordinary treatment of risk it suggests that anthropic should be worth zero today. creating that risk would put every executive in jail.
on top of that it would demand under existing laws of conflict, a military campaign to destroy anthropic. that is not optional, it is demanded now to save lives.
hard to make comparisons but we mourned and rembered 9/11 recently. a 10% risk of hundreds of millions dead in 10 years would make anthropic a thousands of times greater threat than al qaeda. many countries would assassinate dario amodei and the leadership of anthropic now, within weeks or months.
actually just on the vague risk of having a nuclear weapon in 10 years, the USA killed ayatollah khamenei, his daughter, his son-in-law, his daughter-in-law and his 14 month old granddaughter. then, they killed over 120 children ages 6-12 by accidentally bombing a school.
in the sense that i would analyse a company, at least, the claim is false. it's not true that ai has a 10% chance of causing human extinction within 10 years.
they are making false claims about the technology they sell.
i have a fairly inflexible approach to that. sure, exaggerate but outright lies about the nature of the product don't work for me.
Is that a serious question? Many people would consider Potala palace, the Milan Duomo, Neuschwanstein, etc more beautiful than anything designed by Zaha Hadid and Gehry. Myself included, even as someone who likes their work. Limiting ourselves only to the category of modern event centers, I'd still put the Harbin Grand Theater ahead of it.
The article literally talks about polling people. The disparity between lay people and architects is the point of the piece.
Architectural taste doesn't align with the taste of everyone else. Why should that be the case? Why shouldn't architects design buildings people enjoy?
> Why shouldn't architects design buildings people enjoy?
In many fields (architecture, art, cooking all come to mind) there seems to be an outright resistance to doing things that people actually enjoy. My hypothesis is that practitioners in those fields have spent so much time studying them, they have become bored of the "normal" stuff which people actually enjoy. In an effort to satisfy themselves, they optimize for novelty, either not knowing or not caring that doing so is making their work something which can only really be appreciated by people as desperate for novelty and steeped in the field as they are.
With food at least it isn't so bad, because there are a ton of not pretentious restaurants you can go to enjoy. Nobody can force you to go to the pretentious restaurants run by chefs bored of normal food. But with architecture, a building is something everyone in that city is forced to experience, whether they like it or not. It would be nice if architects were therefore more mindful that they should try to appeal to a broad range of tastes, not just architecture nerds who are bored of what came before.
There's also the issue that the names they're actually disproportionally told and taught about are people who "innovated". Who pushed a marginally new style.
It becomes change and following avant garde for the sake of it and a group of people trying to be different in all the same boring ways.
> There's also the issue that the names they're actually disproportionally told and taught about are people who "innovated". Who pushed a marginally new style.
Is Palladio not taught? Christopher Alexander (of A Pattern Language)? The classical order of columns?
Brent Hull (for one) has done a whole bunch of stuff with his restoration work on trying to teach people about proper proportion:
No one will care if you build 1000th replica of a nice building. It's just not interesting to anyone and actually becomes boring very fast. People may live in it but no one will come from afar to visit it because of its beauty. It will become another one in a row of the same.
If you build a striking, exceptional building, it can become a big success (like Gehry's in Bilbao which is a success story for the whole city), so people who commission buildings sometimes select for that.
And also there is a sizable group of people who actually enjoy interesting buildings, like people who enjoy haute couture or haute cuisine etc., so it's not like there is no market for that.
that one looks too Calatrava-ish (specifically Oceanografic), I'd prefer the Dongdaemun Design Plaza if we are talking Zaha Hadid - especially once you see it in person.
The Register of course take what Kahn says (where he is arguing that no new laws need to be created to regulate AI) and taking extending it to say "breakout the handcuffs".
Kahn doesn't even imply that. Instead he's making a well reasoned point about using existing tools (as many in this discussion point out existing laws could be applied to inadequately controlled cyber incidents)
I think they should make a range of climbing shoes to reinforce credibility on this market. Adidas has 5.10 and some pretty serious climbers (Janja Garnbret!) on their books. The competition (LaSportiva, Scarpa) is ripe for disruption.
The numbers in the article are forecasts but let’s take them as real. That’s $10bn of revenue, the majority from enterprise customers, let’s say 75% from enterprise API usage: $7.5 billion. If the margin on inference is 80% that means of the $7.5bn in enterprise revenue they’re spending $1.5bn on compute. Yet we know that they actually spend over $5bn per month on compute, which includes the $1.25bn per month to SpaceX.
If $7.5bn is their enterprise revenue and it costs just $1.5bn to generate, that leaves $3.5bn in compute costs to account for. Dario previously said that training costs less than inference so training can’t explain it.
If subscriptions aren’t the majority of usage and aren’t subsidized, where is the money going? Anthropic don’t spend money on data centre build out so that can’t be it either.
> If the margin on inference is 80% that means of the $7.5bn in enterprise revenue they’re spending $1.5bn on compute.
I don't think you can reverse this out like this because the 80% rate is before payments to "distribution partners, including Amazon". I think that payment includes the hosting cost for that those model but it's unclear.
> Dario previously said that training costs less than inference
Do you have a source for that?
Are you sure you aren't conflating the statements Dario has made that training costs less than they make on inference (over the life cycle of a model)?
> I don't think you can reverse this out like this because the 80% rate is before payments to "distribution partners, including Amazon". I think that payment includes the hosting cost for that those model but it's unclear.
The "hosting cost" is paid for by Anthropic and is the largest cost. The money Anthropic pay to Amazon for delivering Anthropic models via Bedrock is separate, independent of compute costs, best thought of as commission.
The forecasted / guessed / estimated 80% number is based what customers pay per token minus the projected compute costs, i.e: the people who believe that Anthropic has 80% margins on tokens believe that Anthropic spend $0.20 on inference compute for every $1 of per-token billed-via-the-api revenue.
We know that there are hundreds of thousands of fixed-price subscriptions being used to their absolute maximum, with many people bragging about how many subscriptions they run in parallel. These tokens are not included in the 80% margins, they are acknowledged to be "subsidized". People like @theo on Twitter post almost daily about how much they're milking Anthropic and OpenAI with leaderboards.
Both Anthropic and OpenAI (more so OpenAI) do "resets" where they increase the limits available to people on their fixed price plans. We know that there are people paying $1,000 per month for multiple subscriptions to generate tokens that would cost $50,000 via the API. Even if Anthropic's margins are 80% on compute for per-token billing, that's still $10,000 of cost to Anthropic generating just $1,000 in revenue. Multiply that by tens of thousands or maybe even hundreds of thousands of subscriptions.
Anthropic and OpenAI have raised over $100 billion each and continue to raise. If they're making 80% or even 50% margins on $10 billion in revenue per month they would not need to raise, they would be shouting for the roof tops about how profitable they are, they wouldn't be delaying their IPOs, yet they're only profitable by non-GAAP metrics like WeWork's classic "Community-adjusted EBITDA" or in this case "per-token-adjusted EBITDA" or whatever they will call it in their IPOs.
Yes, they're selling tokens via the API for more than they cost, they are profitable on per-token billed inference, it has positive margins, but those profits are obliterated when you account for all the inference they're paying for out of pocket on fixed price subscriptions, upon which they keep increasing limits because they desperately need to show growth further harming their profitability (consuming all of the money they make from their API).
If Anthropic and OpenAI needed to be profitable tomorrow, they could be, they could kill off all their fixed price subscription plans and charge only for usage via the API, they'd print money, but they'd lose mindshare because nobody except for enterprises can afford to pay the true cost, all the regular people would switch to cost effective good-enough models, and then within months, the enterprises would start to switch too because no longer would their employees be claude-pilled.
Anthropic and OpenAI cannot turn off subsidization, thus, their margins on per-token API billing are not important in any discussion about their long term financial wellbeing. Just look at the large scale customers like Harvey (~15 trillion tokens per month, ~$50m+ in spend) who are, sensibly, investing in building their own specialized models that are cheap to run so they can cut their spend by 90%. That's profitable revenue for Anthropic / OpenAI today, but completely gone soon.
"This week, Noah Smith and Erik Torenberg are joined by Dario Amodei, CEO and Co-founder of Anthropic. Dario talks about the economics of AI development, the comparative advantage of AI companies like Anthropic, AI safety, and his stance on California's SB 1047 bill. They also discuss the impacts of AI on global power dynamics, competition between the US and China, and inequality in an AI-powered world."
At around 12 minutes in:
"I think actually even if such a model is released one thing you know that's a this analogy to to open- Source software is that these big models they're actually very expensive to run on inference the majority of the cost is is inference not necessarily the training of the model so if you have only you know I don't know 10 20% 30% better way to do inference that can kind of negate the effect so the economics are kind of strange yes there's this giant fixed cost that you have to amortise but then there's also the per unit cost of inference and small differences in that can actually again assuming the thing is deployed widely enough make a very big difference so I don't know quite how that's going to play out"
The scales have changed since then with inference costs falling and more being spent on training but the fundamentals are the same. Inference is expensive, in part, because peak usage dictates capacity whereas capacity can dictate training. Anthropic must pay billions of dollars per month to be able to handle peak inference, hence their efforts to try and shape usage by offering discounts / flexible limits at different times of the day. They can train when capacity permits.
> The money Anthropic pay to Amazon for delivering Anthropic models via Bedrock is separate, independent of compute costs, best thought of as commission.
My point is that you can't reverse out the maths like you did without knowing how much this is.
> These tokens are not included in the 80% margins, they are acknowledged to be "subsidized".
No, this isn't correct. Even including these they are claiming 80% margins.
There's nothing at all that indicates subscriptions aren't included in this - it's a simple statement of their running margin.
> these big models they're actually very expensive to run on inference the majority of the cost is is inference not necessarily the training of the model
I don't think you can take this statement to claim that currently they spend more on inference than on training. I think he's saying over the lifetime of a model maybe inference ends up costing more unless they keep finding "better way to do inference that can kind of negate the effect".
> If Anthropic and OpenAI needed to be profitable tomorrow, they could be, they could kill off all their fixed price subscription plans and charge only for usage via the API, they'd print money
My point is that their subscription costs are a lot less than you think because of this statement by Anthropic that they have 80% margins including these subscriptions.
Your version of reality cannot be real because the numbers do not make sense. Anthropic's supposed revenue run rate for 2026 puts December 2026's forecasted revenue at $10 billion. They're only "profitable" according to a non-GAAP measure that excludes all of their costs, they are not cash flow positive, they are not bringing in more money than they are spending.
If their margins are 80%, that means on $10 billion in revenue they're spending just $2 billion. Anthropic's own announcements put their spending at much, much higher, such as the $1.25 billion per month they are paying to SpaceX for compute, and the ~$3.5 billion they're paying to Google each month, and the billions to Amazon each month too.
> We’ve signed an agreement with SpaceX to use all of the compute capacity at their Colossus 1 data center. This gives us access to more than 300 megawatts of new capacity (over 220,000 NVIDIA GPUs) within the month. This additional capacity will directly improve capacity for Claude Pro and Claude Max subscribers.
There's no world in which Anthropic has 80% margins. At their current expenditure on compute it would require at least $20 billion in revenue to be mathematically possible. The 80% figure on compute margins that is widely discussed is based on analysis by SemiAnalysis and refers only to their per-token compute margin (which is calculated comparing hardware costs + electricity costs to what they charge via the API).
The estimated training costs for models like Opus and Astra are ~$1 billion and they're not training multiple frontier models in parallel every month. Training costs cannot explain where billions of dollars per month are disappearing if they have 80% margins. And that's before even considering all the money they're raising and spending. Anthropic raised tens of billions just a few months ago, OpenAI even more.
Vendor financing[1] has a long and successful history. Here's a good WSJ article from 2001 about the practice and risks[2], written after the DotCom crash in March 2000.
There is nothing illicit or illegal in anyway about what NVidia is doing. It's reasonable business practice, and people on HN are simply ignorant to think otherwise.
NVidia is very aware of the risks it entails, but has the money to cover those risks.
A big problem in discussions about Nvidia is that people can't distinguish between:
1. The equity investments Nvidia has made in its customers.
2. The guarantees/backstops it has extended to some of its customers.
3. Vendor financing.
The vendor financing is the least interesting of the bunch. Nvidia has already disclosed that when it provides vendor financing, the average customer pays in less than 60 days. These are not long-term financing arrangements and virtually every big company sells on these type of terms (net-30, net-60, etc.).
The equity investments and guarantees are where there is room for legitimate debate.
> equity investments and guarantees are where there is room for legitimate debate
The guarantees dwarf the equity investments. If there is a shenanigan, it's going to be there.
A problem: the line between the guarantees and traditional vendor financing is blurry–one could argue use commitments are no different from repurchase commitments.
I should have been more precise with my use of "vendor financing" in saying "trade credit."
I agree that the guarantees are where the risk is.
First, under US GAAP accounting rules (ASC 606), these are absolutely not repurchase agreements. The customer takes title to the asset (the chips) and Nvidia does not have a contingent obligation to repurchase the asset. Providing a contingent guarantee to purchase services is not a repurchase under the accounting rules, but of course you can have legal accounting and still have a problem.
As I've made public market investments in and traded in this space, I've done some math on the guarantees and what came out was this: relative to Nvidia's current earnings, the guarantees amount to approximately a quarter of a year's revenue at the guarantee cap.
That's my own analysis and I'd encourage anyone who cares to run the numbers themselves. It's easy enough as Nvidia is publicly traded.
The thing that differentiates Nvidia from previous vendor financing examples like Lucent in the 1990s/early 2000s is that its margins are huge. So what I see, based on the current numbers, is that in the really ugly scenario, Nvidia has to live with depressed earnings, no stock buybacks and a weaker (but still comparatively strong) balance sheet for a number of years. This is a stock problem, not a solvency issue.
The wildcard is if Nvidia keeps extending big guarantees, or starts using debt to do so, to the point where the commitments are expanding faster than its cash flow. At that point, the risk obviously compounds accordingly.
Enron was illegal but also a sham. The point is we have no evidence Nvidia's financing is a sham. It could be. And if it is, it's a huge problem. But the shammiest parts of what Enron did do not apply to Nvidia, which makes the comparison a bit like saying OP is Hannibal Lecter but legal while glossing over the fact that OP never murdered anyone but once drank red wine.
Their implication ("A la Enron", "they can buy good lawyers to keep it completely legal") that there is something illicit or wrong in what NVidia is doing.
The OP clearly is implying that it should be illegal for some reason. This is wrong - not only is it nothing like Enron (!?) but it's a great way for both NVidia and the companies building on them to build what they want.
Would love to learn more about some techniques that "everybody" uses to do this well. So far, everything I've seen that meaningfully advances the frontier has been high-touch (involving human experts in one way or another).
It's fairly easy to describe a task that is slightly harder than an existing one.
For example if frontier models are able to one-shot a database query across 20 columns and 10 tables add one additional relationship then test. Keep doing this until the pass-rate drops below acceptable and now you have your new frontier eval.
My thought experiment was along the lines of "Let's say I'm Anthropic and I want to significantly improve my frontier model's performance on, say, theoretical physics research. How do I build a fully autonomous process capable of constructing an eval that's somewhat outside the current capability in some useful direction (decided by the autonomous process itself)?"
> A strict liability crime is something of an oxymoron. Crimes always require intent, the mens rea element.
This is wrong.
In criminal and civil law, strict liability is a standard of liability under which a person is legally responsible for the consequences flowing from an activity even in the absence of fault or criminal intent on the part of the defendant.
The Chernobyl disaster wasn't caused by a single person, as you note in your comment "proposed and exercised by people without knowledge or experience."
> OpenAI would have you believe this result shows how powerful their mystery better-than-Astra model is, but the reality here is that this model needed 10,000 agents, $20M of compute, and the assistance of a whole team of people at OpenAI, to replicate (then exceed) the work that just took two people, with some academic grants as an AI spending budget to achieve (a few $100K - listed below).
I think you have to work pretty hard to minimize what OpenAI achieved here like this.
The Navier-Stokes equations have been around since 1850. The smoothness problem has been well known for over a hundred years and has only gained importance. It's been a Millennium Problem since 2000.
Levent Alpöge and Tristan Buckmaster did great work to solve the related Euler problem, but didn't solve the Navier-Stokes smoothness problem.
The Navier-Stokes smoothness problem has previously had significant resources working on it. Computational fluid dynamics is one of the most important tools in modern engineering and is closely related.
You speak of 10,000 agents as though it is somehow extreme, and yet within the past month I've had a single task that used over 100 agents on a mere Anthropic team plan. I think two orders of magnitude more compute to solve one of the greatest unsolved physics problems[1] is nothing.
I don't excuse Brubeck behavior because of this, but that doesn't minimize the achievement here.
[1] Wikipedia quote: In particular, solutions of the Navier–Stokes equations often include turbulence, which remains one of the greatest unsolved problems in physics, despite its immense importance in science and engineering. https://en.wikipedia.org/wiki/Navier%E2%80%93Stokes_existenc...
This logic doesn't follow at all.
If their argument is that there is 10% chance of extinction then they also believe there is a 90% chance it won't.
reply