GenAI: Technological Masterpiece or Ecological Disaster?

On this page
Generative AI is both: a technical achievement that has changed how many of us work, and a fast-growing consumer of electricity. The International Energy Agency (IEA) estimates that data centres, AI and crypto could use more than 1,000 TWh of electricity in 2026, roughly as much as Japan. Which way GenAI goes depends on efficiency, and research already shows large gains: in one study, a 1.58-bit model used 41.2 times less energy than a comparable 16-bit model.
This article looks at the energy numbers first, then at the techniques that make GenAI cheaper to run: specialised hardware, Mixture of Experts, knowledge distillation, Mixture-of-Depths, layer pruning, BitNet quantization, RAG and load shifting.
Update (November 2026): The energy figures below come from the IEA's Electricity 2024 report. In Key Questions on Energy and AI (2026), the IEA puts data centre consumption at 485 TWh in 2025 and expects it to roughly double to 950 TWh by 2030. The 2024 forecast also counted crypto mining, so the two sets of numbers are not directly comparable.
How GenAI went mainstream
The origins of generative AI go back several decades, but for a long time these techniques were accessible only to small circles of experts. That changed in the fall of 2022, when OpenAI launched ChatGPT and started a new commercial sector. With GPT-3 and ChatGPT, OpenAI made a model with deep semantic understanding, multilingual capabilities and high accuracy widely available as a "ready to use" product. Image generation moved just as fast: diffusion models like DALL-E, Midjourney and Stable Diffusion deliver astonishing results.
In less than two years, ChatGPT and similar tools have become household names (ChatGPT: 1.8 billion visits in March 2024), and they have already changed the way we work in some areas. These technologies are here to stay.
The trend toward ever larger models
The push for better and more accurate results has started a race for more powerful models, trained on ever larger amounts of data. GPT-3 (2020) had 175 billion parameters and was trained on 300 billion tokens, drawn from a dataset of about 0.5 trillion tokens. For GPT-4, the training data is estimated at around 13 trillion tokens, for an MoE model with (rumored) 16x111 billion parameters. OpenAI doesn't publish official figures. Even for models with known numbers, the dimensions have grown considerably: Llama 2 used 2 trillion tokens and has up to 70 billion parameters, and PaLM 2 used 3.6 trillion tokens and has 340 billion parameters.
It's impressive how much progress has been made in such a short time, in both text and image generation. But this race also has its downsides.
The carbon footprint of GenAI
Large GenAI models need immense computing resources, both during training and during inference (the actual use of the model). According to the International Energy Agency (IEA), data centres were responsible for about 2% of global electricity usage in 2022. Of this, 40% goes to computing, 20% to associated IT equipment and another 40% to cooling alone.
The IEA forecast is even more dramatic: "Electricity consumption from data centres, artificial intelligence (AI) and the cryptocurrency sector could double by 2026. Data centres are significant drivers of growth in electricity demand in many regions. After globally consuming an estimated 460 terawatt-hours (TWh) in 2022, data centres' total electricity consumption could reach more than 1 000 TWh in 2026. This demand is roughly equivalent to the electricity consumption of Japan."
The report also notes: "Future trends of the data centre sector are complex to navigate, as technological advancements and digital services evolve rapidly. Depending on the pace of deployment, range of efficiency improvements as well as artificial intelligence and cryptocurrency trends, we expect global electricity consumption of data centres, cryptocurrencies and artificial intelligence to range between 620-1050 TWh in 2026, with our base case for demand at just over 800 TWh – up from 460 TWh in 2022. This corresponds to an additional 160 TWh up to 590 TWh of electricity demand in 2026 compared to 2022, roughly equivalent to adding at least one Sweden or at most one Germany."
IEA energy consumption forecast
In a report published in October 2023, Alex de Vries (Ph.D. candidate at VU Amsterdam and founder of the digital-sustainability blog Digiconomist) analyzed trends in AI energy use. He predicted that AI technology could be on track to consume as much electricity per year as the entire country of Ireland (29.3 terawatt-hours per year).
De Vries also concludes that if current trends in AI capacity and adoption continue, NVIDIA will ship 1.5 million AI server units per year by 2027. Running at full capacity, these 1.5 million servers would consume at least 85.4 terawatt-hours of electricity annually, more than many small countries use in a year.
Technological masterpiece or ecological disaster?
So does GenAI lead us towards an easier workday, or does it push us towards an energy crisis at a time of climate change and energy saving measures?
We can't answer that in this article, and we don't intend to. What we can say is that GenAI will stay part of our lives, and demand for it will keep rising, as the forecasts above show. As the IEA report puts it, "Future trends of the data center sector are complex to navigate, as technological advancements and digital services evolve rapidly." Efficiency will decide which path we take. So the rest of this article looks at the efficiency question from a technological perspective: what does this young field already have to offer?
Specialised AI hardware: TPUs and LPUs
Most of the computing power for AI today comes from GPUs. Their architecture offers much better throughput than ordinary CPUs, but because GPUs were designed to be flexible, they have bottlenecks that throttle their performance for large, specialised workloads. Hardware built specifically for AI can use its computing power much more efficiently, as Google shows with its Tensor Processing Units (TPU).
"A TPU is an application-specific integrated circuit (ASIC) designed by Google for neural networks. TPUs possess specialized features, such as the matrix multiply unit (MXU) and proprietary interconnect topology that make them ideal for accelerating AI training and inference." "Cloud TPUs are optimized for training large and complex deep learning models that feature many matrix calculations, for instance building large language models (LLMs). Cloud TPUs also have SparseCores, which are dataflow processors that accelerate models relying on embeddings" Google TPU
Groq goes even further and develops hardware specifically for LLMs.
"An LPU™ Inference Engine, with LPU standing for Language Processing Unit™, is a new type of processing system invented by Groq to handle computationally intensive applications with a sequential component to them such as LLMs. LPU Inference Engines are designed to overcome the two bottlenecks for LLMs – the amount of compute and memory bandwidth. An LPU system has as much or more compute as a Graphics Processor (GPU) and reduces the amount of time per word calculated, allowing faster generation of text sequences. With no external memory bandwidth bottlenecks an LPU Inference Engine delivers orders of magnitude better performance than Graphics Processor."
Mixture of Experts: expert models instead of giant monoliths
To improve the accuracy and quality of model outputs, model builders trained ever larger models on ever larger datasets. The result is massive monolithic models that store everything they learned in one huge network. Every use of such a model requires matrix computations across the entire network.
Mixture of Experts (MoE) is an alternative approach. An MoE model consists of a number of expert models and a gating mechanism. Instead of one large model, several smaller specialised models are trained. Each expert is tailored to specific areas, and the gating mechanism activates it only when it's needed for the input. The router (the gating mechanism), which is itself a neural network, is trained together with the expert models. MoE models can also get huge (GPT-4 is estimated at 16x111B parameters), but only a few smaller expert models need to be activated and computed for each input, not one massive model.
Mixture of Experts: a router sends the input to expert models
Knowledge distillation
Large language models are now good enough to automate many steps that used to need labor-intensive manual work by humans. This also makes it possible to transfer much of the behavior and quality of very large models to much smaller and simpler models. This technique is called "Knowledge Distillation".
In knowledge distillation, a student model is trained on a dataset, and the teacher model creates soft targets for it. Soft targets are probability distributions across the output categories. They serve as training labels for the student model, alongside the one-hot encoded actual labels. Training minimizes the difference between the teacher's soft targets and the student model's predictions.
The much smaller student models can "mimic" their larger teacher models and deliver similar performance with far less computational effort.
Mixture-of-Depths: skipping transformer blocks per token
Transformer models consist of a series of transformer blocks that are processed one after another. Each block is made up of attention layers, normalization layers and feed-forward networks. The depth of this stack lets the model recognize more nuanced relationships between words. In a traditional transformer, compute is distributed evenly: every token passes through every transformer block. But not every token is equally complex or relevant for the final result.
Decoder-only Transformer architecture
For this reason, the team at Google DeepMind developed a conditional computation technique called Mixture-of-Depths (MoD). MoD uses a learned routing mechanism to decide whether a token passes through the current block or is routed around it. The MoD Transformer gets a computation budget, and only the top-k tokens in a block are processed.
Visualisation of MoD principle
This limits the required computing power, and it lets you optimize the Transformer in one of two directions: better accuracy with the same compute (FLOPS) and even higher throughput, or the same accuracy with less compute.
The Google DeepMind team tried different settings and approaches for the MoD routing. The best results in the study came from processing only every eighth token (12.5%) in every second block.
MoD (deciding whether to skip a block) is very similar to Mixture of Experts (deciding which expert model to use), so the DeepMind paper also evaluated a combination of the two. In the combined approach, called MoDE, the router decides between an expert model and skipping the block. This variant produced even more efficient models than pure MoD.
Layer pruning: removing the deeper layers of an LLM
The paper "The Unreasonable Ineffectiveness of the Deeper Layers" shows that you can remove a large share of the layers of popular open-weights model families such as Llama 2 without significantly degrading output quality. To decide which layers to remove, the authors measured the angular distance between the input and output of each layer. This metric shows how similar the input and output are, and thus how much a layer changes its input. The layers with the least influence were removed.
The study found that the deepest layers generally have the least influence, but the last layer or last few layers have a very strong impact. The authors tested the models right after pruning and after a "healing" step, a Parameter Efficient Fine-Tuning (PEFT) with QLoRA. In both cases, the models performed very consistently up to a certain fraction of removed layers, and "healed" models were even more stable up to that point. Beyond it, the performance of all models dropped quickly towards random behavior.
The paper states: "Importantly, we see a characteristic flat region of robust performance followed by a sharp transition to random accuracy at a pruning fraction around 45%-55% for models in the Llama-2 family, 35% for Mistral 7B, 25% for Phi-2, and 20% for models from the Qwen family. This implies that the essential knowledge required to achieve a model's top score isn't removed by significant layer removal – even though the fraction can be quite large(!) – until eventually that knowledge is lost at a critical model-dependent threshold.
Contrasting the curves with and without healing, we see that finetuning offers a modest improvement by better preserving the unpruned performance and pushing the phase transition to random guessing to slightly larger pruning fractions."
On efficiency, the authors also note: "In particular, the released version of Llama-2-70B spans 140 GB of memory and consumes approximately 3×10^10 FLOPs per token. With 4-bit quantization and a layer-pruning fraction of 50%, the model fits in approximately 17.5 GB of memory and requires roughly 1.5×10^10 FLOPs per token: quantization from 16-bit bfloats to 4-bit QLoRA precision reduces model memory by a factor of 4, but keeps FLOPs more or less the same, since calculations are performed in 16-bit precision; layer pruning will additionally reduce both memory and FLOPs by an amount equal to the layer-pruning fraction.These memory and compute requirements enable open-weight state-of-the-art models to be run and even finetuned efficiently on consumer-level GPUs without any CPU off-loading and with only minor performance trade-offs."
Layer pruning shows that current model generations use resources very inefficiently, and that their requirements can be reduced a lot, especially for inference.
Quantization from the ground up: BitNet and BitNet b1.58
Microsoft took a different approach: make models efficient from the ground up. To reduce a model's memory and compute requirements, its weights (parameters) are often quantized, meaning their resolution is reduced. This is usually done after training, which is called post-training quantization. It costs a little accuracy, but it's simple and shrinks the model proportionally.
Instead of quantizing after training, Microsoft's study reduced the resolution of the parameters to the absolute minimum from the start. The researchers built the feed-forward networks with only binary (1-bit) parameters, which can only take the values 1 or -1. As a result, features from the preceding layer can have either a positive or a negative effect on the activation of the next nodes. This model is called BitNet. In a second approach, they used ternary parameters with three possible states: -1, 0 and 1. That corresponds to 1.58 bits, which is why this model is called 'BitNet b1.58'. The extra zero state lets the model filter out features from the preceding layer. We cover the method in more depth in BitNet: LLM quantization at its extreme.
The results are remarkable. With the usual 16-bit floating-point values replaced by just these three states, the matrix calculations need no complex multiplications, only integer additions of the input values (see figure).
'BitNet b1.58' Pareto Improvement
The study compared BitNet b1.58 with a reproduced FP16 LLaMA LLM. Despite the drastic reduction in precision, BitNet b1.58 achieved almost identical results to the LLaMA model in various benchmarks.
The quantization only applies to the feed-forward networks (FFNs), not to other parts of the Transformer, so the attention layers and other components still run at higher precision. The memory savings therefore grow with model size. For a model with 3B parameters, the required memory drops by a factor of 3.55. For a 70B model, memory drops by a factor of 7.16 and the model is 4.1 times faster. Throughput reaches 2977 tokens/s, an 8.9-fold increase.
Less computation means a smaller memory footprint and higher throughput, and it also cuts the energy required. According to the study, the 70B BitNet b1.58 model reduces end-to-end energy consumption by 41.2 times compared to the LLaMA 70B model.
'BitNet b1.58' energy consumption 'BitNet b1.58' resource reduction and speed improvements
The gains are big enough that, in terms of resources, BitNet models compete with LLaMA models that have far fewer parameters. The paper states:
- 13B BitNet b1.58 is more efficient, in terms of latency, memory usage and energy consumption, than 3B FP16 LLM.
- 30B BitNet b1.58 is more efficient, in terms of latency, memory usage and energy consumption, than 7B FP16 LLM.
- 70B BitNet b1.58 is more efficient, in terms of latency, memory usage and energy consumption, than 13B FP16 LLM.
Using foundation models with RAG instead of training new ones
RAG (Retrieval-Augmented Generation) systems also make AI applications more efficient. RAG lets you 'integrate' proprietary information into LLMs with standard models, without fine-tuning them or training a model from scratch. Advanced retrieval techniques improve output quality while keeping the input context for the LLM small.
A RAG system also doesn't depend on one specific LLM, so you can swap the LLM at any time. Any efficiency improvement in newer models then flows directly into the system. The same idea, picking the most efficient model for each task, is behind LLM routing.
Load shifting: running AI where renewable energy is available
Some solutions that make AI more sustainable come from outside the AI field.
Paul Walsh (Data Scientist at Accenture), for example, described Accenture's approach of load shifting in a LinkedIn post:
"One way to address sustainability in AI is to consider demand side interventions, which are initiatives that affect how and when electricity is used by consumers. This includes load shifting, whereby demand for electricity is rescheduled to a time when energy requirements can be more easily met by renewable resources. Essentially demand can be managed to meet renewable supply, to maximize the amount of renewable energy used in the mix. Indeed, by procuring renewable energy for neural network training, emissions can be reduced by a factor of between 30 and 40, compared to energy sourced from a fully fossil fueled grid. Such load shifting is highly feasible for many AI technologies, such as ML training, as they are not latency bound; developers can utilise compute resources in geographically dispersed regions with minimal little impact on the performance of production systems. This is also demonstrated by the work we have done in Accenture, whereby we can seamlessly move compute intensive workloads across widely dispersed geographic data centers to maximize the use of renewable energy with little to no impact on system performance. In essence, we are bringing computation to where renewable energy is available, rather than bringing renewable energy to the computation, exploiting the fact that transmitting electricity long distances is less efficient than sending code and data as photons over optical networks. This approach brings additional benefits in terms of driving demand to where renewable energy supplies are available. One of the main drawbacks of renewable energy is that it is intermittent, that is solar energy only works when the sun shines and wind renewables only work when weather conditions are right. While storage systems can be used to store energy, these systems are expensive and generally have limited capacity. Moreover, when supply exceeds demand many renewable energy sources are curtailed as demand is not available locally to meet supply. Load shifting solves the issue of intermittency and curtailment by bringing heavy compute loads to where supply can meet demand. This also drives demand for renewables, helping to finance and grow this critically important sector and help meet current and future demands in a sustainable way.
Load Shifting
Load shifting is a concept that has been applied to demand side management of energy, for example by scheduling the use of smart appliances to align their operation with the supply of renewable energy. Load shifting in this context refers to moving the compute load to a new region or new time of day where carbon intensity is lower. To address this, we strive to power AI by renewable energy when and where it's available by shifting heavy compute loads. The idea is to identify regions that utilize renewable energy at a particular time of the day and create a system to automate the transfer of compute load for machine learning models to the most suitable region while monitoring the actual GPU/CPU usage. We do this by identifying cloud providers and geographic regions that support renewable energy around different time zones and then implement decision logic to automatically spin up the required computational infrastructure across the globe and automatically move compute loads to take advantage of the most optimal data center, all the while tracking energy usage of the models."
Temporal and geographic load shifting of LLM training
Conclusion
GenAI's capabilities come at a real energy cost. The IEA expects data centres, AI and crypto to use between 620 and 1,050 TWh of electricity in 2026, up from 460 TWh in 2022, and de Vries puts the AI share alone on track to match a country like Ireland.
The research covered here shows how much of that cost is avoidable. Specialised hardware like TPUs and LPUs runs AI workloads more efficiently than GPUs. Mixture of Experts and Mixture-of-Depths compute only part of the network for each token. Distillation moves the abilities of large models into small ones. Llama 2 models kept robust performance until roughly 45-55% of their layers were removed, and BitNet b1.58 70B used 41.2 times less energy than an FP16 LLaMA 70B model. Load shifting, finally, moves compute to wherever renewable energy is available.
Whether GenAI ends up a masterpiece or a disaster depends on how quickly these techniques reach the models people actually use. For your own applications, the practical levers are smaller or quantized models, RAG instead of retraining, and running heavy jobs where and when the power is green. Mixture of Experts and quantization together are why a 30B coding model like GLM-4.7-Flash runs locally on a single 16GB GPU.
Further reading
- BitNet: LLM Quantization at its Extreme
- KAN: Kolmogorov-Arnold Networks as an alternative to MLPs
- 11 strategies to reduce LLM costs
Interested in building high-quality AI agent systems?
We prepared a comprehensive guide based on cutting-edge research for how to build robust, reliable AI agent systems that actually work in production. This guide covers:
- Understanding the 14 systematic failure modes in multi-agent systems
- Evidence-based best practices for agent design
- Structured communication protocols and verification mechanisms



