Water used is a terrible metric. Water is not consumed in the process; many cooling systems are closed loop and even open loop systems just release water into the environment where it will be renewed.

What matters is strain on water infrastructure, but that is wildly variable and can't be compared apples to apples. Like a water cooled data center drawing from an aquifer is a massively different beast than one drawing from a river. Likewise a data center with a dedicated tap is much different than one drawing from the standard municipal system.

If datacenters are set up in places that can not support them, that's on the operators of the datacenters themselves, not the users of the compute power. And if datacenters are set up in locations where the resources they need are present but they create an excessive burden on the infrastructure upkeep, that's on the municipality for not appropriately charging them for their usage and/or approving plans without checking their capabilities.

I think the comparison that should be made here is the actual alternative - what is the cost in kw/hr of me sitting in front of my powered-on and not-sleeping laptop while I spend the next 10 minutes penning an email?

If the machine can make that happen in a second, and it takes me 600x longer to do that while consuming power that entire time, I suspect that the AI carries an overall advantage when you actually include the full story.

I'll file this under "lazy tech journalists didn't bother to do the math past the first, most obvious step".

Does anyone else have a hard time accepting these calculations? I don’t doubt the serious environmental costs of AI but some of the claims in this infographic seem far-fetched. Inference costs should be much lower than training costs. And, if a 100-word email with GPT-4 requires 0.14 kWh of energy, power AI users and developers must be consuming 100x as much. Also, what about running models like Llama-3 locally? Would love to see someone with more expertise either debunk or confirm the troubling claims in this article. It feels like someone accidentally shifted a decimal point over a few places to the right.

According to the paper (https://arxiv.org/pdf/2304.03271), the vast majority of water usage is not from cooling but indirectly from energy usage. The paper in turn gets its electricity water intensity from: https://www.wsp.com/-/media/insights/us/documents/wp_guidanc... (reference [20]) - which notes:

> Even though hydropower water withdrawal and consumption intensities are usually orders of magnitude larger than other types and likely to skew overall regional averages, it is important to include hydropower in the factors to show not only the power sector’s dependency on water but also its vulnerability to water shortages.

Some people have pointed out that using water for cooling does not destroy it - it'll all rain back down. I think it would've still been fair to consider how much processed/drinking water was being evaporated, since it'd need processing again, but I can't really see the justification for the article's framing when the figure is measuring water that would've just flowed into the sea had the hydroelectric dam not been there.

In all these calculations you have to wonder what "using water" even means. Add to that that most water cooled systems are closed loops, i.e. no water escapes

Is this true in a marginal cost sense? I was under the impression most of the environmental impact occurred during the training stage, and that it was significantly less costly post training?

With reports like this discussing training cost, is the usage reported per-model? Or is it aggregated across the entire creative model development/exploration process?

At the risk of doing original research, one thing I don’t see a lot of discussion on is that AI companies don’t train one model at a time. Typical engineers will have maybe 5-10 mid-size models training at once. Large automated hyperparameter grid searches might need ensembles of hundreds or thousands of training runs to compare loss curves etc... Most of these will turn out to be duds of course. Only one model gets released, and that one’s energy efficiency is (presumably) what’s reported.

So we might have to multiply the training numbers by the number of employees doing active research, times the number of models they like to keep in flight at any given time.

Just because AI is new, it doesn't mean it's less valuable than the older things using up resources. AI is far more valuable than, say, television.

If the goal is to reduce overall usage, what to stop should be determined by value, not chronologically/LIFO.

Why water? It's not lost, it's endlessly renewable, specially since it's mostly just evaporated.

Energy or carbon released is much more interesting.

Do they mean evaporated?

That's 2.45 kJ per gram starting at 20C or 1.2MJ per half liter water bottle or around, roughly 340 Wh per email.

So, at an average price of let's say, a dollar per kWh, that's 34 cents of energy spent on the response.

Something is off in the cost (unless I messed up in the math which is likely).

While using water is one interesting point of the actual paper(from which this article is written), I would say, to a lay person, water volume makes more sense than arbitrary values like KiloJoules or KiloWatts, because I can imagine what a 20mL water would look like, but can't imagine what 20mJ or 20mW looks like.

I wonder what the environmental costs of most American homes, stores and offices using A/C at full blast 24/7/365.

I argue it's impossible to measure the total environmental impact of anything, and it's a waste of time to even try.

Let's break this down. There is water used at inference time and training time. The humans working on the project consume water. The building in which they did the project uses water. The whole supply chain for the computers? Good luck measuring the water usage in there

This may seem pedantic, but I promise there is a point to this.

Measuring environmental impact is like trying to understand a neural network.

If you want to discourage water usage, the only way is to tax the marginal water used at any step in the supply chain. You don't need to know the total water usage for the last step of the supply chain, in this case, an LLM

https://archive.is/CAhVY

Water. Does. Not. Disappear.

The water circulates through a cooling system. It soaks up heat from the servers and goes outside and cools down, then goes back again. You are not "using up" the water. This is article is nonsense.

I don't think a single 100-word email using GPT-4 alone can cost 140wh, even considering how expensive GPT-4 is to run.

If it takes 20 seconds for the model to compose the letter, that means: (3600/20)*140=25200W, a 25200W piece of hardware is used just to compose your email and no other request, this seems wrong by several orders of magnitude.

[dead]