Behind the AI Magic

AI might seem like magic, but it comes with a very real price tag. From training sophisticated models to maintaining their day-to-day operations, the costs can add up quickly.

 

Understanding the costs involves looking at data limits and the computing power for training and running AI:

Data limits

Data limits refer to the vast amounts of information AI needs to learn and make decisions. This data isn’t just large in volume but also needs to be managed and stored, which can be costly.

Computing power to train AI

Computing power to train AI is the intense processing power required to teach AI how to perform tasks.

Computing power to run AI

Computing power to run AI is the ongoing energy AI requires to operate daily.

These elements contribute significantly to the overall cost of developing and maintaining AI systems.

Let’s dive into these costs to understand better what goes into powering AI.

 

AI: A City That Never Sleeps

Think of a relatively small-sized AI system as a city. This city needs constant energy (data) and infrastructure (computing power) to keep it running smoothly and efficiently.

Data Costs: The City’s Power Supply

Data is the electricity that powers our AI city. Just as a city needs power to light up buildings, run transport, and keep homes warm, AI needs data to learn and make decisions. This power isn’t free, and the cost can be quite substantial considering the vast amounts of data AI consumes.

Let’s estimate the City’s Monthly Power Bill
Suppose each gigabyte of data is like a kilowatt-hour of electricity. If storing and processing one gigabyte costs $0.10 and our AI city uses 100,000 gigabytes a month, how much does it spend on its power bill?

 

Answer
$10,000

 

Compute Costs to Train: Constructing the City

Building this AI city isn’t a one-time effort. It involves massive computational resources, similar to constructing skyscrapers, roads, and parks. This construction phase is expensive because it requires high power usage for a prolonged period.

Building the Infrastructure
If one hour of compute time (like one hour of construction work) costs $0.50, and building our AI city (training the AI) takes 10,000 hours, how much does construction cost?

 

Answer
$5,000

 

Compute Costs to Run: Daily Operations of the City

Once the city is built, it needs to be maintained. This includes everything from street lights to water systems, all running on the initial data electricity. Each task the AI performs is like a street light being turned on, using a bit more of the city’s power.

Operating Costs
Imagine each interaction with AI (each question answered) is like turning on a lightbulb. If one lightbulb uses a tiny bit of energy, costing 0.001 cents, and the city has 1,000,000 lightbulbs lit per day, what is the daily cost of keeping the city operational?

 

Answer
$1,000

 

Now that you’ve played the role of the city planner for our AI city try working out its annual cost using what you’ve calculated for data, construction, and daily operations. You’ll see how quickly the costs accumulate and gain a deeper appreciation for the complex infrastructure behind everyday AI interactions.

 

Answer
$192,000 a year ($16,000 per month) is a lot for a small-scale AI system. In fact, it’s estimated that LLMs like ChatGPT are spending up to $700,000 a day just to keep things running.

 

Exploring the costs of our AI city shows us that completely replacing human jobs does not make sense. Even though AI can do some jobs well, it’s tough and costly for machines to mimic human abilities. This means that even in jobs where AI is useful, there’s still a strong need for human skills.

Working with AI
Instead of taking over all our jobs, AI is more likely to work alongside us, helping to make our work easier and more effective without replacing us.

Categorized in:

Blog,

Last Update: November 21, 2024