Will the GenAI scaling craze see a group spend >$1B to train a single large-scale model in the next 12 months?
6
36
110
Oct 2
58%
chance

Taken from the State of AI Report 2023.

  1. The GenAI scaling craze sees a group spend >$1B to train a single large-scale model.

This question will be resolved based on the resolution of the 2024 report.

Get Ṁ600 play money
Sort by:

Surely llama 3 405B qualifies?

@RemNi The market is about price, not parameter count. Unless you’re claiming that we know that llama 3 cost >$1B?

@BionicD0LPH1N

1 h100 = 40k dollars

350 000 x 40 000 = 14 000 000 000

Assuming training llama 3 is 1 / 20th of the compute they will use that cluster for, and then considering all other costs beyond just the GPUs, yeah easy 1B dollars

@RemNi Thanks for the info, though I don’t think that’s convincing enough for resolution. I think it might make sense to wait for an official number or an estimate by Epoch AI before resolving.