I don't doubt that ISO models are not optimal, but the annual cost of power outages to US consumers is above 100 billion dollars last time I looked already. Electric system reliability is a place where you have to be somewhat conservative as costs rise really non-linear with outage intensity. [1]
And ISOs/RTOs/TSOs operate in complex political environments and have to ensure reliability in adverserial markets. They are often constraint in what they can/can't do.
What I will say is that their modelling should absolutely be open and transparent, and typically it's not. That would make necessary discussions around modelling assumptions, potential improvements and political constraints much much easier and more fruitful.
[1] That said, my intuition is that current outages are probably not due to supply insufficiency, but due to transmission system failures.
The article's main complaint is that generating capacity has been overbuilt by maybe 10%. That's not a bad thing. There are worse things than extra generating capacity.
I’ve made a large mistake that ended up being a positive for the company I was working for (profit wise) and I took it as seriously as if it had gone in the opposite direction. It was a simple thing but boy did the anxiety of worry destroy me. Can’t imagine what the worry would have been like if it was in the opposite direction
PJM Interconnection LLC (PJM) is a regional transmission organization (RTO) in the United States. It is part of the Eastern Interconnection grid operating an electric transmission system serving all or parts of Delaware, Illinois, Indiana, Kentucky, Maryland, Michigan, New Jersey, North Carolina, Ohio, Pennsylvania, Tennessee, Virginia, West Virginia, and the District of Columbia. PJM is the largest power grid operator in the United States, serving 67 million customers from Chicago to New Jersey. (Wikipedia)
Unfortunately, the article misses the forest for the tree: it points the shortcomings of one particular electricity market model, but never acknowledge that the whole endeavor is doomed to fail: the way electricity works physically (coupled with the way societies have been built around it being readily available) makes it fundamentally impossible to manage through market forces. And as a result, every attempt results in an apparently nonsensical failure (see also how the European electricity market's "copper plate" model leads to the insanely high re-dispach costs).
Not all markets, and not even all European markets, assume a single trading zone (i.e. a copper plate network). Re-dispatch costs are high because policy choices have created a physical reality that drives high constraints.
That could have been prevented by simply not permitting connections in certain parts of the network. Yes, theoretically it could also have been prevented by forcing generators to directly bear re-dispatch costs which would have led developers to build a different set of generating assets. This would have been a different set of market mechanisms.
However, if generation decisions had been made centrally, they would have been made by the same system operators, regulators, and governments that mandated the construction of the current system and they would have therefore built the same system more or less. Do you really think that if e.g. Germany or the GB network had been centrally planned by their respective government mandated regulators and system operators, they wouldn't have put all those windfarms exactly where the market-led model we actually have has put them?
> -dispatch costs are high because policy choices have created a physical reality that drives high constraints.
You don't seem to understand what's driving the so-called re-dispatch costs in Germany so let me explain: when there's power consumption in the South, it is paid at the market price, when the "supplier" is located in the North (typically wind) and there's not enough network capacity to transfer the power to the consumer, the transport authority sends an order to the supplier not to produce the electricity, and instead pays a thermal power plant in the South (at a higher price than market price, by definition) to provide it instead.
That's not a problem of physical realities driving constraints, that's the problem of a market that uses a very simplified model that simply can't deal with the complexity of the underlying phenomenon (we don't even have enough computing power to accurately model the electricity transport at national scale given how challenging the equations are, and even the lineralized version that are being used in practice are already very compute-intensive, it's entirely futile to expect to build a market that can efficiently reflect them).
Huh, did you get this example from an LLM, or is it the bog-standard way of explaining it? I asked ChatGPT to explain what a "copper plate" is because I was lazy, and it used 1:1 the exact same example as you did (Germany,wind generation in the north paying a thermal plant in the south)
Nope, it's just the textbook example and that's why the LLM says the same thing (it's as if you were surprised someone explained distributed systems using Byzantine generals and the LLMs did the same).
> Do you really think that if e.g. Germany or the GB network had been centrally planned by their respective government mandated regulators and system operators, they wouldn't have put all those windfarms exactly where the market-led model we actually have has put them?
This is a huge "it depends", because we are then comparing a partly-political system with a fully-political system. The privatized model builds at the optimal locations for getting paid under the prevailing rules; the state one is much more prone to getting distracted by lobby groups. See the German over-dependence on coal and the UK conservative ban on onshore wind farms.
I will be interested to see how the deployment of affordable battery storage changes the economics. I've also seen a striking presentation about how much renewables the UK plans to deploy (roughly double!) by 2030. More on the strategy: https://www.gov.uk/government/publications/clean-power-2030-...
"Clean Power means that by 2030, Great Britain will generate enough clean power to meet our total annual electricity demand, backed up by unabated gas supply to be used only when essential."
>the way electricity works physically (coupled with the way societies have been built around it being readily available) makes it fundamentally impossible to manage through market force
This reminds me of the 2013 finding that showed an earlier 2010 study praising austerity as a recovery mechanism was flawed because of an Excel calculation error and lack of diligence.
>It just may be that these things should be reviewed a little closer by people who are obsessive about correctness
No amount of review is enough. As anyone who's worked in quant science will attest, a model that hasn't repeatedly made correct out-of-sample predictions is most likely worthless, as it's far far easier to overfit a model to data than to predict the future.
> flaws in the London epidemiological model about how to respond to covid.
That one was fun. I got into an argument with a scientist who was (IIRC) working with or for the specific professor that made that mess, and he blamed the entire software engineering/computer science industry for making C++ too hard to use, and said it should never have been released if mistakes like this could be made. And then accused me of gatekeeping when I said perhaps they should get an expert to look over this stuff before submitting results to governments to form a policy basis.
The attitude seemed to be "I am a scientist and am clearly very smart, and therefore anything you do must be trivial in comparison. If I can't pick up in seconds what takes you years to learn and more years to perfect, you've clearly done it all wrong and it's your fault."
The money has gone to pay power plants that can supply winter capacity, exactly as the market is designed to do.
The intention of the payments is to increase revenue for that kind of power generation capability to encourage more such plants be constructed.
The argument in the article is dubious to me. Of course the higher price isn’t leading to more generation today, that’s not the point, the point is to reward developers that build and built capacity CA needs in winter. The disagreement then becomes which model is correct about how much capacity is actually needed.. but the fact that a tiny move in demand moves the price so substantially seems to me to undermine the entire premise of the blog post, clearly supply is severely constrained?
But that's why they're arguing that repricing the entire generation fleet is bad compared to having a different auction for new capacity. You don't need the same incentives to keep an existing, profitable plant online as you do to invest in a new plant.
I can assure you the spreadsheets at firms building plants factor in forecasts of revenue for the life of the plant into investment decisions. If they don’t pay past the first few years, that directly translates to lower forecast lifetime plant value for new plants.
I’m not a quant, and I’ve worked energy trading desks long enough to know there is a lot I don’t understand.. but I don’t see how separating auctions by plant age does anything other than move numbers around while keeping the total bill the same. Plants still need the same lifetime revenue to make investment decisions pencil out; whether you front-load payments or spread them evenly, the total in current value needs to be the same.
Even if the total cost is the same, if our payments better align with the behavior we want to incentivize, we may gain greater utility from the spending.
So it can matter how we distribute that revenue as to whether or not the business responds in the desired way, eg, actually investing in new capacity by linking payments directly to new capacity.
What's to say that encouragement works? Does the 12B subsidy stipulate that more plants must be built? If not, keeping supply limited and price per Watt high may be more profitable.
I'm assuming nothing -- the party that burns 12B of taxpayer money on subsidies should show that it works. It is plausible that there are no new potential market participants as building power plants is expensive and the entrenched players don't want to compete with themselves.
1. It does not seem to be “taxpayer money” nor “subsidies”? It’s customers paying money for electricity.
2. If no new or existing market participants are willing to expand capacity, that would indicate that existing prices aren’t worth it for them to do so. Companies are willing to spend far more than the cost of building a new power plant for things like AI - and they’re often even building power generation on top of the AI build outs. So it has to be more than “It’s expensive”.
What you're touching on is an ancient debate :) Should this part of society be top-down planned or should it be market organized?
I know how I feel about this - I much prefer price signals here; it allows any developer that can meet the spec to build energy production - distributed decision making over centralized decree - and it allows the other side of the equation, consumers, to decide they'd rather not pay for this generation: Curtail demand rather than increase supply.
Like - maybe I don't want to pay energy prices this high, I can choose to insulate my house better or get a heat pump and thus reduce my electrical bill. The price signal lets everyone in the market choose how to act, rather than a central authority declaring we must build, say, more gas turbines and share the cost of that across rate payers.
But, there are people much, much smarter than I that completely disagree with this position. In the end I think the answer is how you feel about human nature, the capability and shortfalls of markets and the complexity of deciding how societies' resources should be allocated.
I don't doubt that ISO models are not optimal, but the annual cost of power outages to US consumers is above 100 billion dollars last time I looked already. Electric system reliability is a place where you have to be somewhat conservative as costs rise really non-linear with outage intensity. [1]
And ISOs/RTOs/TSOs operate in complex political environments and have to ensure reliability in adverserial markets. They are often constraint in what they can/can't do.
What I will say is that their modelling should absolutely be open and transparent, and typically it's not. That would make necessary discussions around modelling assumptions, potential improvements and political constraints much much easier and more fruitful.
[1] That said, my intuition is that current outages are probably not due to supply insufficiency, but due to transmission system failures.
The article's main complaint is that generating capacity has been overbuilt by maybe 10%. That's not a bad thing. There are worse things than extra generating capacity.
I’ve made a large mistake that ended up being a positive for the company I was working for (profit wise) and I took it as seriously as if it had gone in the opposite direction. It was a simple thing but boy did the anxiety of worry destroy me. Can’t imagine what the worry would have been like if it was in the opposite direction
.. at a cost of about $12bn.
PJM Interconnection LLC (PJM) is a regional transmission organization (RTO) in the United States. It is part of the Eastern Interconnection grid operating an electric transmission system serving all or parts of Delaware, Illinois, Indiana, Kentucky, Maryland, Michigan, New Jersey, North Carolina, Ohio, Pennsylvania, Tennessee, Virginia, West Virginia, and the District of Columbia. PJM is the largest power grid operator in the United States, serving 67 million customers from Chicago to New Jersey. (Wikipedia)
That was not clear in the article.
Unfortunately, the article misses the forest for the tree: it points the shortcomings of one particular electricity market model, but never acknowledge that the whole endeavor is doomed to fail: the way electricity works physically (coupled with the way societies have been built around it being readily available) makes it fundamentally impossible to manage through market forces. And as a result, every attempt results in an apparently nonsensical failure (see also how the European electricity market's "copper plate" model leads to the insanely high re-dispach costs).
Not all markets, and not even all European markets, assume a single trading zone (i.e. a copper plate network). Re-dispatch costs are high because policy choices have created a physical reality that drives high constraints.
That could have been prevented by simply not permitting connections in certain parts of the network. Yes, theoretically it could also have been prevented by forcing generators to directly bear re-dispatch costs which would have led developers to build a different set of generating assets. This would have been a different set of market mechanisms.
However, if generation decisions had been made centrally, they would have been made by the same system operators, regulators, and governments that mandated the construction of the current system and they would have therefore built the same system more or less. Do you really think that if e.g. Germany or the GB network had been centrally planned by their respective government mandated regulators and system operators, they wouldn't have put all those windfarms exactly where the market-led model we actually have has put them?
> -dispatch costs are high because policy choices have created a physical reality that drives high constraints.
You don't seem to understand what's driving the so-called re-dispatch costs in Germany so let me explain: when there's power consumption in the South, it is paid at the market price, when the "supplier" is located in the North (typically wind) and there's not enough network capacity to transfer the power to the consumer, the transport authority sends an order to the supplier not to produce the electricity, and instead pays a thermal power plant in the South (at a higher price than market price, by definition) to provide it instead.
That's not a problem of physical realities driving constraints, that's the problem of a market that uses a very simplified model that simply can't deal with the complexity of the underlying phenomenon (we don't even have enough computing power to accurately model the electricity transport at national scale given how challenging the equations are, and even the lineralized version that are being used in practice are already very compute-intensive, it's entirely futile to expect to build a market that can efficiently reflect them).
Huh, did you get this example from an LLM, or is it the bog-standard way of explaining it? I asked ChatGPT to explain what a "copper plate" is because I was lazy, and it used 1:1 the exact same example as you did (Germany,wind generation in the north paying a thermal plant in the south)
Nope, it's just the textbook example and that's why the LLM says the same thing (it's as if you were surprised someone explained distributed systems using Byzantine generals and the LLMs did the same).
> Do you really think that if e.g. Germany or the GB network had been centrally planned by their respective government mandated regulators and system operators, they wouldn't have put all those windfarms exactly where the market-led model we actually have has put them?
This is a huge "it depends", because we are then comparing a partly-political system with a fully-political system. The privatized model builds at the optimal locations for getting paid under the prevailing rules; the state one is much more prone to getting distracted by lobby groups. See the German over-dependence on coal and the UK conservative ban on onshore wind farms.
I will be interested to see how the deployment of affordable battery storage changes the economics. I've also seen a striking presentation about how much renewables the UK plans to deploy (roughly double!) by 2030. More on the strategy: https://www.gov.uk/government/publications/clean-power-2030-...
"Clean Power means that by 2030, Great Britain will generate enough clean power to meet our total annual electricity demand, backed up by unabated gas supply to be used only when essential."
(note details of wording)
>the way electricity works physically (coupled with the way societies have been built around it being readily available) makes it fundamentally impossible to manage through market force
This is anti-scientific woo. Both theoretically and empirically market-based electricity systems are much more efficient, that's why even China is adopting one: https://www.enerdata.net/publications/daily-energy-news/chin...
> This is anti-scientific woo.
This is hilarious coming from someone defending a concept that assumes neither thermodynamics nor Maxwell's laws exist.
> Both theoretically
When making preposterous hypothesis you can prove literally anything to be efficient.
> and empirically
Ah yes, like the PJM and the European market are “empirically efficient”.
This reminds me of the 2013 finding that showed an earlier 2010 study praising austerity as a recovery mechanism was flawed because of an Excel calculation error and lack of diligence.
https://inthesetimes.com/article/the-excel-error-heard-round...
It also reminds me of the flaws in the London epidemiological model about how to respond to covid.
It just may be that these things should be reviewed a little closer by people who are obsessive about correctness
>It just may be that these things should be reviewed a little closer by people who are obsessive about correctness
No amount of review is enough. As anyone who's worked in quant science will attest, a model that hasn't repeatedly made correct out-of-sample predictions is most likely worthless, as it's far far easier to overfit a model to data than to predict the future.
> flaws in the London epidemiological model about how to respond to covid.
That one was fun. I got into an argument with a scientist who was (IIRC) working with or for the specific professor that made that mess, and he blamed the entire software engineering/computer science industry for making C++ too hard to use, and said it should never have been released if mistakes like this could be made. And then accused me of gatekeeping when I said perhaps they should get an expert to look over this stuff before submitting results to governments to form a policy basis.
The attitude seemed to be "I am a scientist and am clearly very smart, and therefore anything you do must be trivial in comparison. If I can't pick up in seconds what takes you years to learn and more years to perfect, you've clearly done it all wrong and it's your fault."
What software did they build their model in?
Having been in the industry for 20 years, there is a good chance it was good ole Excel
Every "wasted $" ended up in some parasitical grifter's pocket.
Whilst these sorts of analyses are informative, they lack answers to the who profits? question.
The money has gone to pay power plants that can supply winter capacity, exactly as the market is designed to do.
The intention of the payments is to increase revenue for that kind of power generation capability to encourage more such plants be constructed.
The argument in the article is dubious to me. Of course the higher price isn’t leading to more generation today, that’s not the point, the point is to reward developers that build and built capacity CA needs in winter. The disagreement then becomes which model is correct about how much capacity is actually needed.. but the fact that a tiny move in demand moves the price so substantially seems to me to undermine the entire premise of the blog post, clearly supply is severely constrained?
But that's why they're arguing that repricing the entire generation fleet is bad compared to having a different auction for new capacity. You don't need the same incentives to keep an existing, profitable plant online as you do to invest in a new plant.
I can assure you the spreadsheets at firms building plants factor in forecasts of revenue for the life of the plant into investment decisions. If they don’t pay past the first few years, that directly translates to lower forecast lifetime plant value for new plants.
I’m not a quant, and I’ve worked energy trading desks long enough to know there is a lot I don’t understand.. but I don’t see how separating auctions by plant age does anything other than move numbers around while keeping the total bill the same. Plants still need the same lifetime revenue to make investment decisions pencil out; whether you front-load payments or spread them evenly, the total in current value needs to be the same.
Even if the total cost is the same, if our payments better align with the behavior we want to incentivize, we may gain greater utility from the spending.
So it can matter how we distribute that revenue as to whether or not the business responds in the desired way, eg, actually investing in new capacity by linking payments directly to new capacity.
What's to say that encouragement works? Does the 12B subsidy stipulate that more plants must be built? If not, keeping supply limited and price per Watt high may be more profitable.
Wouldn’t that assume there are no potential market participants who want more market share in a profitable market?
I'm assuming nothing -- the party that burns 12B of taxpayer money on subsidies should show that it works. It is plausible that there are no new potential market participants as building power plants is expensive and the entrenched players don't want to compete with themselves.
1. It does not seem to be “taxpayer money” nor “subsidies”? It’s customers paying money for electricity.
2. If no new or existing market participants are willing to expand capacity, that would indicate that existing prices aren’t worth it for them to do so. Companies are willing to spend far more than the cost of building a new power plant for things like AI - and they’re often even building power generation on top of the AI build outs. So it has to be more than “It’s expensive”.
What you're touching on is an ancient debate :) Should this part of society be top-down planned or should it be market organized?
I know how I feel about this - I much prefer price signals here; it allows any developer that can meet the spec to build energy production - distributed decision making over centralized decree - and it allows the other side of the equation, consumers, to decide they'd rather not pay for this generation: Curtail demand rather than increase supply.
Like - maybe I don't want to pay energy prices this high, I can choose to insulate my house better or get a heat pump and thus reduce my electrical bill. The price signal lets everyone in the market choose how to act, rather than a central authority declaring we must build, say, more gas turbines and share the cost of that across rate payers.
But, there are people much, much smarter than I that completely disagree with this position. In the end I think the answer is how you feel about human nature, the capability and shortfalls of markets and the complexity of deciding how societies' resources should be allocated.
> Should this part of society be top-down planned or should it be market organized?
To have a market, some organization needs to administer it. PJM operates transmission and facilitates a regional wholesale electric market.
https://en.wikipedia.org/wiki/PJM_Interconnection
https://en.wikipedia.org/wiki/Regional_transmission_organiza...