OpenAI launched GPT-2 in 2019, reportedly costing $256 per hour to train. However, it’s been five years since then, and we’re already at GPT-4o. Advancements in hardware, software, and data mean that training the same model will take less time and less money, as Andrej Karpathy , the developer behind the project to reproduce GPT-2 in llm.c, has proven.
The primary driver of cost savings







