Comments on Jan 13 EIM Resource Sufficiency Evaluation Workshop

1. Please provide your organization’s general feedback on the Resource Sufficiency Evaluation (RSE):

The Bonneville Power Administration (Bonneville)[1] appreciates the opportunity to submit comments on CAISO’s EIM Resource Sufficiency Evaluation Workshop held on January 13, 2021. Bonneville thanks the CAISO for all of the detailed information it provided at the workshop, including how the Resource Sufficiency (RS) test performed for the CAISO BAA during the Summer 2020 heat wave events. Bonneville believes the CAISO should make the EIM Coordination during emergency conditions and enhancements to the EIM Resource Sufficiency test initiative a very high priority for implementation prior to Summer 2021.

The core objectives of the resource sufficiency (RS) evaluation are to promote reliability by ensuring each EIM Entity has sufficient resources to meet its obligations independent of the EIM and to ensure equity by preventing EIM Entities from leaning on the EIM and on other EIM Entities to meet their energy, capacity, and flexibility needs.  It appears that these objectives were not met during the heat wave events of Summer 2020. As discussed in the January 13^th workshop, the EIM RS test produced some counterintuitive results that showed the CAISO BAA passing the EIM RS test during intervals in which it was declaring Energy Emergency Alerts Stages 2 and 3. And during the intervals the CAISO BAA did not pass the RS test while in Energy Emergency Alerts Stages 2 and 3, the CAISO was importing significant amounts of EIM imports, arguably allowing the CAISO to lean on the EIM during those periods.

NERC defines the circumstance of an Energy Emergency Alert 2 as “the BA is no longer able to provide its expected energy requirements and is an energy deficient BA”.[2] And the circumstance of an Energy Emergency Alert 3 is “the energy deficient BA is unable to meet Contingency Reserve requirements”. As shown at the January 13^th workshop, there were several intervals during the evening peak hours (between hour ending 14 and hour ending 21) on August 14, 15, 17, and 18, in which CAISO passed the RS test while declaring the CAISO BAA was energy deficient per the NERC-defined Energy Emergency Alerts. These outcomes indicate that underlying components of the RS test are not properly evaluating the resources and obligations of the CAISO BAA and changes need to be made to ensure that the RS test is producing accurate results.

Bonneville also believes the failure consequence for the RS test should be re-examined. Leaning on the EIM to meet energy, capacity, and/or flexibility needs can not only create reliability issues in adjacent EIM Entity BAs, but it also has financial implications that can cause harm to adjacent EIM Entities and undermine incentives to ensure adequate forward procurement of resources through longer term resource and transmission planning processes.

[1] Bonneville is a federal power marketing administration within the U.S. Department of Energy that markets electric power from 31 federal hydroelectric projects and some non-federal projects in the Pacific Northwest with a nameplate capacity of 22,500 MW. Bonneville currently supplies around 30 percent of the power consumed in the Northwest. Bonneville also operates 15,000 miles of high voltage transmission that interconnects most of the other transmission systems in the Northwest with Canada and California. Bonneville is obligated by statute to serve Northwest municipalities, public utility districts, cooperatives and then other regional entities prior to selling power out of the region.

[2] See NERC Attachment 1-EOP-011-1

2. Provide your organization’s feedback on the Bid Range Capacity Test for the RSE:

Bonneville acknowledges the couple of issues CAISO has found in the calculation of the capacity test – resource Pmax Derate and Pmin Rerates are not included in resource capacity calculations and inaccurate Net Schedule Interchange values (NSI) are used for the CAISO BAA due to inclusion of the mirror resources - and we appreciate CAISO making the changes needed to address these issues. 

Bonneville strongly believes that supply – import or internal – counted towards meeting the RS tests must be identifiable at the time of the test and must be real and capable of performing at the shown level for the portion of the operating hour being tested. The RS test diminishes in value if the supply that is counted towards meeting the test is not real and/or is not capable of performing when needed. For example, even if the Pmax Derates and Pmin Rerates were included in the resource capacity calculations, the Final Analysis appears to indicate that there are additional capacity deficiencies beyond the derated values that need to be further evaluated and addressed.[1]  

While Bonneville believes more discussion is needed to work out the details of a more comprehensive capacity test, Bonneville notes its support for inclusion of operating reserves and non-EIM exports as obligations in an EIM Entity’s RS showing.

Bonneville believes the capacity test needs to accurately reflect passes and failures. The flexible ramping sufficiency test should not be relied upon to capture false passes in the capacity test. Bonneville requests the CAISO include in the capacity test all of the same relevant elements as the flexible ramping sufficiency test, including consideration of derates and having the correct NSI calculation. 

In addition to derates, the Final Analysis makes specific reference to an Outage/Temporal Constraint category of unused RA capacity – incremental to the CAISO’s categories for outages and derates – which accounts for approximately 800 MW of natural gas resources across the peak and net peak periods on August 14^th.[2]  Bonneville seeks to understand whether and how the capacity amounts associated with these temporal constraints should be recognized in the capacity test if they are not already.  These changes help to validate that supply counted towards meeting the test is identifiable at the time of the test and capable of performing at the shown level for the portion of the operating hour when it is needed.

[1] Reference Final Analysis, Appendix B, Figures B.16 through B.19 which appear to indicate that actual energy production from natural gas resources was approximately 1,000 MW below the real-time awards which should be inclusive of all Pmax derates known to the CAISO.  This 1,000 MW amount also exceeds the 494 MW contribution from the forced outage of the Blythe Energy Center on August 14th and the 248 MW lost to the erroneous dispatch to the Panoche Energy Center on August 15th.  These same figures also appear to indicate that actual energy production from hydro resources was between 500 MW and 1,500 MW below the real-time awards.  While the report notes that some of these resources could have been carrying ancillary service obligations at the peak on August 14th there is no explanation provided for August 15th.

[2] Reference Final Analysis, Appendix B, Figures B.4 through B.7 illustrate the unused RA capacity by resource type.

3. Provide your organization’s feedback on the Flexible Ramp Sufficiency Test (FRST) for the RSE:

Bonneville supports further examination of the interplay between EIM imports and the EIM flexible ramping sufficiency test. Bonneville agrees that the RS test should be measuring a BAA’s stand-alone capability to meet its own obligations. As such, prior-hour EIM imports should not be inappropriately providing credits towards meeting an EIM Entity’s next hour’s EIM flexible ramping test. In other words, prior hour EIM imports should not exceed an EIM Entity’s bid-range capacity that is offered and capable of performing for the next hour being evaluated, as this would be an indication that the EIM Entity is relying on the EIM to meet its own energy, capacity and/or flexibility needs.

Bonneville has consistently requested the CAISO prioritize improvements to the accuracy of the resource sufficiency test.[1]  One such example is that Bonneville has requested that CAISO incorporate generation forecasts into the computation of the flexible ramping requirement.  However, the CAISO continues to use a histogram approach for calculating the flexible ramping requirement. The CAISO has stated in previous stakeholder processes that this histogram approach does not account for the forecasted level of demand, wind, and solar output during a given market interval and can lead to inaccurate flexible ramping requirements.

Bonneville would like to better understand how the histogram approach performed during the Summer 2020 heat wave event and if CAISO’s actual demand was within or outside of the P95 uncertainty requirements that were calculated for these intervals. Bonneville requests the CAISO provide the following information regarding the intervals in which CAISO was declaring Energy Emergency Alerts on August 14, 15, 17, and 18:

• Forecasted demand
• Actual demand
• Forecasted VERs
• Actual VERs
• The P95 uncertainty requirements that were calculated for each interval.
• EIM imports and exports in FMM and RTD

In its 2018 Whitepaper titled “Energy Imbalance Market Resource Sufficiency Evaluation”, the CAISO stated that they would evaluate a potential alternative to the histogram methodology used to compute the RS flexible ramping requirements. The CAISO had run simulations of a probabilistic forecasting approach as part of the Day-Ahead Market Enhancements initiative[2] that showed a more accurate day-ahead flexible ramping product requirement. And the CAISO stated it planned to complete a similar simulation to recognize potential benefits between the fifteen-minute market and the 5-minute real-time dispatch. Bonneville urges the CAISO to re-commit to and prioritize this work through a public stakeholder process to begin Q2 of 2021.

[1] See the Bonneville Power Administration EIM Implementation Agreement, section 14(h)(3), and comments of the Bonneville Power Administration, CAISO Annual Policy Initiatives Roadmap Process August update to the 2020 revised draft Policy Initiatives Catalog, July 2, 2019.  http://www.caiso.com/Documents/BPASubmission-2020DraftPolicyInitiativesCatalog.pdf

[2] Reference Day Ahead Market Enhancements: Updates to Revised Straw Proposal, FRP Requirement (Slide 49 – 56). http://www.caiso.com/Documents/Agenda-Presentation-Day-AheadMarketEnhancements-Jun19-2018-Updated.pdf

4. Provide your organization’s feedback on the Balancing Test for the RSE:

Bonneville believes CAISO should be subject to the balancing test like every other EIM Entity. To pass the balancing test, each EIM Entity must have base schedules sufficient to meet its hourly demand forecast prior to the operating hour. CAISO should also be subject to a showing of equivalent “base schedules” sufficient to meet its hourly demand forecast prior to the operating hour.

5. Additional comments:

Bonneville believes that the consequences for failing the RS test should be both proactive and graduated consistent with operator actions that are similarly proactive and graduated in support of maintaining reliability.  To this end, Bonneville supports expanding the current EIM RS failure consequence to include new physical, financial and oversight consequences. The EIM RS failure consequence should ensure that entities who fail the RS test are not enabled to lean on the EIM and to ensure that the failure consequence provides sufficient incentives for appropriate forward procurement of capacity. Bonneville also recognizes that abrupt limiting of transfers can cause large swings and negatively impact adjacent BAAs and limiting of EIM transfers to a BAA that may be in an Energy Emergency Alert can create even more severe shortfalls that may put that BAA, and other adjacent BAAs, in load shedding or rolling blackout conditions. For these reasons, Bonneville supports allowing an EIM BAA to continue to receive EIM transfers when the EIM BAA fails the RS test, but with graduated physical, financial and oversight consequences.

Bonneville supports defining different failure consequences depending on which RS test was failed.  For example, deficiency transfer limits could be established for each failing BAA depending on which RS test was failed.

• Capacity Test failure: Bonneville believes more discussion is needed to determine what the right deficiency transfer limit should be and whether transfer limits or the penalty parameter can be adjusted for periods of sustained failure.
• Flexible Ramping test: Bonneville supports setting the Deficiency Transfer Limit to the imports from the previous interval as is done today.

Bonneville supports defining a financial consequence at which the Deficiency Transfer Limit would be relaxed, with inclusion of a protection measure that would ensure failing BAAs would deploy their own resources first before relying on EIM imports. The financial consequence must be high enough to incent EIM Entities to procure the energy, capacity, and flexibility it needs prior to the operating hour and not simply incent entities to fail the test and lean on the EIM because it is a cheaper alternative. Bonneville is also open to a graduated financial consequence approach, i.e. the greater the magnitude and/or the longer the failure continues the more severe the financial penalty. However as already stated, failing the RS test and leaning on the EIM should not become an economic alternative to the appropriate forward procurement that is necessary to be resource sufficient. Therefore a high financial consequence should be applied even for small RS failure. How the financial consequence is structured (i.e. penalty parameter vs. after-the-fact penalty) and how the revenue is allocated are details that require further vetting with stakeholders.

Bonneville also supports reporting test failures to the EIM Governing Body.  This oversight consequence could also be different depending on which test was failed.  It could also be graduated in the sense that repeated failures could face greater oversight, such as a description of the failure and actions to be taken to avoid failure in the future.

Bonneville appreciates the attention and consideration these issues have been given to date and urges the CAISO to prioritize this effort for reliable operations of Summer 2021.

1. Please provide your organization’s general feedback on the Resource Sufficiency Evaluation (RSE):

CalCCA is opposed to the Powerex proposal to apply a $2,000/MWh penalty after-the-fact to Energy Imbalance Market (EIM) transfers in intervals in which there are resource sufficiency test failures. (http://www.caiso.com/InitiativeDocuments/PowerexPresentation-MarketEnhancements-Summer2021ReadinessJan13,2021Workshop.pdf.pdf Slides 15 and 16). Applying such penalties would result in uplifts charged to load that has no means to observe or respond to any price signal in either the real-time market or the EIM. Further, the proposed penalty payment would be on top of market capacity and energy payments that already are compensating suppliers and providing price signals to loads. These price signals are provided in CAISO’s sequential market process (i.e., day ahead Integrated Forward Market (IFM), Hour Ahead Scheduling Process (HASP), Fifteen Minute Market (FMM) and Real-time Dispatch (RTD)). Before resorting to penalties, CAISO should determine whether market-based solutions can be implemented.

A fundamental challenge is that for an organized market to work, at some point parties have to get comfortable with allowing financial incentives to motivate parties to perform. This means that when there are differences in any of the modelled assumptions (e.g., loads, transmission constraints) or in any of the resource bids, the prices that sellers will be paid and buyers will pay at any location almost certainly will be different (sometimes much different). PowerEx’s Slide 10, which describes the economic consequences of EIM capacity leaning by comparing southwest region day ahead bi-lateral prices to 16-hr average EIM FMM LMPs and to 16-hr average EIM RTD LMPs, does not acknowledge the many factors that can lead to differences in the bi-lateral day ahead market prices and EIM (and CAISO) real-time market prices.  Powerex asserts that “EIM Entities purchase bilateral firm energy to meet their own RS requirements” which “inappropriately results in large losses for some EIM Entities.” In particular, on August 18, “500 MW of DA purchases re-exported in EIM would result in a $8 million loss.”  But Powerex fails to note that for every entity that bought $1400/MWh power in the bi-lateral day-ahead market, there was another entity that sold $1400/MWh power. Parties that were hedged aren’t harmed by buying and selling at the same price (high or low) in the same market, or by buying and selling excess at a lower price in a subsequent market. Parties that were long in the day ahead market enjoyed buying back at low prices in the real-time market, and vice versa. Implementing a $2,000/MWh penalty price after-the-fact, without reflecting the price in the CAISO dispatch, would not be an efficient solution to the problem Powerex is asserting. 

CalCCA believes that critical questions remain before moving forward with market design changes. Neither the CAISO resource sufficiency presentation (http://www.caiso.com/InitiativeDocuments/Presentation-MarketEnhancements-Summer2021ReadinessJan13,2021Workshop.pdf  nor the Powerex presentation addressed the questions CAISO posed on Slide 17 of its Jan 12 presentation. http://www.caiso.com/InitiativeDocuments/Presentation-MarketEnhancements-Summer2021Readiness-Jan122021Workshop.pdf ) CAISO asked these questions in the context of load and export scheduling, but they are relevant for resource sufficiency evaluations.  In particular:

• If scheduling priority is determined on the basis of whether firm transmission has been obtained in advance, should resource sufficiency evaluations have similar requirements?
• Would both CAISO and non-CAISO transmission need to be procured prior to market and, if so, what quality of transmission (e.g., last leg firm, source to sink firm, some monthly non-firm)?
•  If CAISO is going to require that transmission be procured prior to market, what should be the appropriate term and the required timing? Hourly, daily, monthly, annual, multi-year? Post day-ahead market, pre-day-ahead market, pre-month, pre-year?

Questions such as these need to be addressed prior to implementing market design changes to avoid unintended consequences.

2. Provide your organization’s feedback on the Bid Range Capacity Test for the RSE:

3. Provide your organization’s feedback on the Flexible Ramp Sufficiency Test (FRST) for the RSE:

4. Provide your organization’s feedback on the Balancing Test for the RSE:

5. Additional comments:

1. Please provide your organization’s general feedback on the Resource Sufficiency Evaluation (RSE):

Please see link to comments below:

http://www.caiso.com/Documents/DMMCommentsonMarketEnhancementsSummer2021Readiness-Jan12-13Workshops-Final.pdf

2. Provide your organization’s feedback on the Bid Range Capacity Test for the RSE:

Please see link to comments in Section 1.

3. Provide your organization’s feedback on the Flexible Ramp Sufficiency Test (FRST) for the RSE:

Please see link to comments in Section 1.

4. Provide your organization’s feedback on the Balancing Test for the RSE:

Please see link to comments in Section 1.

5. Additional comments:

Please see link to comments in Section 1.

1. Please provide your organization’s general feedback on the Resource Sufficiency Evaluation (RSE):

PG&E supports the goal of the Market Enhancements for Summer 2021 Readiness initiative: finding near-term solutions to ensure grid reliability in Summer 2021, in concert with the reliability proposals under consideration in the Resource Adequacy Enhancements initiative and by the CPUC.  For each topic in scope for this initiative, PG&E believes that CAISO should focus on discrete actions that meet the following criteria:

• Market rule or process changes that apply strictly during pre-defined emergency conditions
• Incremental changes that offer clear reliability benefits in Summer 2021
• Temporary changes that would sunset at the end of Summer 2021 to allow time for further examination before making any permanent changes

PG&E appreciates the CAISO’s January 13 workshop and review of the resource sufficiency evaluation (RSE) and encourages the CAISO to focus on discrete solutions that meet the above criteria or address the immediate needs of Summer 2021.  Powerex’s proposal, which would impose more restrictive penalties for failure of the RSE and impact the fundamental operation of the RSE, is contrary to these criteria and would only exacerbate reliability challenges during tight system conditions.   

PG&E believes further discussion of this topic should be focused on developing emergency protocols that will improve reliability during rare, system emergencies and allow the EIM to continue to enable real-time transfers of economic energy when they are most needed.  For example, PG&E would support a rule change that would be triggered by a Stage 1 Emergency that would suspend the current practice of imposing transfer limits on a resource deficient BA following a test failure for the duration of the emergency.  Such a step might only be limited to emergency conditions if multiple BAs failed the test during the same intervals, or a similar condition that protects against cascading reliability issues across multiple EIM balancing areas. 

PG&E welcomes further discussion of constructive solutions to enhance reliability during system emergencies in Summer 2021, along with appropriate protections against the possibility of cascading reliability problems.  Larger anti-leaning measures that do not improve reliability in summer 2021 should not be addressed in this expedited initiative.

2. Provide your organization’s feedback on the Bid Range Capacity Test for the RSE:

PG&E does not have suggestions to alter the Capacity Test at this time and would encourage the CAISO to only consider targeted changes that improve reliability during emergency conditions. (Please see section 1 for PG&E’s additional comments on this topic) 

3. Provide your organization’s feedback on the Flexible Ramp Sufficiency Test (FRST) for the RSE:

PG&E would like further clarification on whether the unsupported exports, as a result of the Pricing Inconsistency Market Enhancements (PIME), may have contributed to the CAISO BA’s failure of the flexible ramp sufficiency test on 8/14/20 and 8/15/2020.  The scheduling of exports during emergency conditions affects a BA’s ability to pass the RSE, and we believe it is important to document and better understand the interaction between export scheduling and the inputs and results of the RSE.  A thorough understanding of this interaction and the timeline of events for each respective process can be instructive in how to avoid future failures of the flexible ramp sufficiency test. 

4. Provide your organization’s feedback on the Balancing Test for the RSE:

PG&E does not have suggestions to alter the Balancing Test at this time and would encourage the CAISO to only consider targeted changes that improve reliability during emergency conditions. (Please see section 1 for PG&E’s additional comments on this topic)

5. Additional comments:

1. Please provide your organization’s general feedback on the Resource Sufficiency Evaluation (RSE):

PacifiCorp appreciates the CAISO’s efforts and diligence in the RSE information that was provided on January 13, 2021.  This information has been helpful in furthering PacifiCorp’s understanding of how the RS tests are applied to the EIM entity BAAs and the CAISO.  The current Business Practice Manual (BPM) on how the tests are applied to each BAA, more specifically to the CAISO, would benefit from additional clarity, potentially leveraging the material and information that was provided during the workshop.  The RSE has been in place, largely in its current form, since the inception of the EIM yet a significant error was revealed as a result of the Summer 2020 heat wave event.  Greater clarity on how the tests are applied in the BPM is needed to prevent this type of error re-occurring.  In addition, PacifiCorp also recommends that the CAISO institute a process to evaluate circumstances where an entity passes the RSE but was not actually resource sufficient based on real-time conditions.

The extreme events of this summer posed a situation that was likely not contemplated when the EIM was designed.  One of the principles of the EIM is that an entity must be resource sufficient, but at the same time, no BAA with available capacity would want to restrict the transfer of energy to an entity in need.  The shedding of retail load is a serious consequence for any BAA, whether it be the CAISO or PacifiCorp, and we must work together to ensure a reliable grid.  However, ensuring reliability requires better coordination within the EIM such that there are not undue effects to those entities that are providing necessary energy in the event of an energy emergency alert (EEA).  For example, if the CAISO BAA is in an EEA and it does not have sufficient capacity to meet its load and reserve obligations, but it passes the RSE, this likely puts PacifiCorp at risk to price volatility within the hour.  As an example, if PacifiCorp is exporting its flexible capacity in the fifteen-minute market and then uncertainty occurs in the real-time market, the PacifiCorp BAA is subject to higher prices for the reduced export to the EIM in the five-minute market due to scarcity. 

In general, there needs to be a better understanding and guidelines for the actions of an EIM entity, including the CAISO, when an EEA is declared.  Currently, there is an uncoordinated and disparate approach within the market when a BAA is in an EEA, which can result in market volatility and uncertainty that the energy awards are reliable. Whether that means there is a contingency flag type of operation, a pricing impact that more accurately reflects the scarcity conditions within the BAA that has declared the EEA, or other measures, PacifiCorp considers the creation and consistent implementation of standardized measures and clear procedures across the EIM imperative.   

PacifiCorp requests CAISO to provide operational guidelines on market operation during times of reserve deficiencies and EEA events that apply to all BAAs, including the CAISO. As these events may occur for multiple consecutive hours, there seem to be limited documentation and procedures that outline proper mitigation for market operations. For example, an EEA event may require suspension of market operations, however there is no formal procedure that states how entities should mitigate this event while minimizing impacts on the market and other EIM entities. These events may require a mitigation timeframe which could extend past the published advisory dispatches utilized during the market contingency function which may lead to significant shifts in dynamic flows and impacts to the entire EIM footprint. PacifiCorp believes that a formal procedure in these situations will help minimize market impacts to the EIM.

2. Provide your organization’s feedback on the Bid Range Capacity Test for the RSE:

The CAISO provided an overview of the capacity test and how it is applied to each BAA, including the CAISO.  They also, as stated above, revealed that there was an error in the test that allowed the CAISO to erroneously pass the capacity test this summer when the CAISO was in an EEA 2.  The CAISO also stated that its passing of the capacity test was irrelevant, because any interval in which it incorrectly passed the capacity test it failed the flex test.  The CAISO seems to be indicating that the flex test is the only test that is material with regard to resource sufficiency.  However, PacifiCorp believes that the capacity test was intended to also be material and ensure that a BAA had the capability to serve its load obligations, not just its ability to serve a change in load plus uncertainty reserves.  One inadequacy that the CAISO highlighted in the capacity test was that it did not consider operating capability or deliverability of a resource or bid.  For example, the CAISO stated that if a resource was off-line, but had a bid, its entire capacity would be counted for purposes of the capacity test.  This appears to be an obvious error in the test and something that the flexible ramp test does not compensate for because it will not catch any capacity deficiencies.  PacifiCorp recommends that the CAISO work towards correcting the capacity test and ensure that it is working as originally intended. 

Currently, the capacity test calculations do not account for EIM entities’ contingency reserves. However, during the summer events of 2020, the CAISO was short contingency reserves, but it still passed all RSE tests. PacifiCorp recommends that the CAISO consider how the contingency reserve obligation is included in the RSE, or separately, as part of the EEA coordination efforts. 

3. Provide your organization’s feedback on the Flexible Ramp Sufficiency Test (FRST) for the RSE:

PacifiCorp believes that the CAISO FRST test has generally worked as intended, however, improvements can likely be made to ensure that the resources that are counted for purposes of flexible capacity have the ability to actually perform.  In addition, it continues to be unclear how the EIM transfers are accounted for in the CAISO’s FRST and whether there is an inherent benefit to the importing BAA that has economically backed generation. 

Lastly, the consequences of failure for a BAA may need to be reviewed as a comprehensive review of last summer’s reliability events.  Allowing a BAA to continue to receive an import when it is not resource sufficient appears to place undue risk on the exporting BAA relative to price volatility and uncertainty that can occur within the hour.  As stated previously, PacifiCorp supports the ability of the market to continue to provide energy to a BAA that is not resource sufficient, but it cannot be because it was an economic choice that the BAA has made due to the fact that there are no pricing penalties for failure.  While PacifiCorp does not believe that any BAA made that choice this summer, what did occur implies that it is possible for a BAA to make that type of a decision and potentially pay less in the EIM than what it would have paid in the bilateral market while not being significantly harmed by a transfer freeze.   

4. Provide your organization’s feedback on the Balancing Test for the RSE:

While PacifiCorp understands why the CAISO does not apply the balancing test to itself, as compared to all EIM entities because it does not have base schedules, it is inequitable to apply penalties to EIM Entities that do not also apply to the CAISO.  During the summer 2020 events, the CAISO BAA was not “balanced” and did not have adequate capacity or flexibility to meet its load and reserve obligation.  However, a balancing penalty was not applied to the CAISO BAA during this period for the amount that it was insufficient.  This inequity between EIM Entity BAAs and the CAISO BAA should be rectified. 

PacifiCorp recommends that the CAISO immediately remove itself from receiving revenue under charge code 6046.   

5. Additional comments:

PacifiCorp appreciates the opportunity to comment on these items and the CAISO’s willingness to pursue this topic prior to the summer of 2021.

1. Please provide your organization’s general feedback on the Resource Sufficiency Evaluation (RSE):

The Public Generating Pool (PGP[1]) appreciates the opportunity to comment on EIM Resource Sufficiency Evaluation informational workshop.  CAISO staff’s presentation was helpful and informative on the details of the bid range capacity test and the flexible ramping sufficiency test, and Powerex’s presentation laid out a clear set of concerns as well as some proposed remedies.  However, it is not clear what the next steps are regarding improvements to these tests.

PGP stated in comments to the 2021 Final Policy Initiatives Roadmap the importance of ensuring that all components of the EIM Resource Sufficiency Evaluation (RSE) are applied transparently and consistently to both CAISO and non-CAISO EIM Entities.  Data from events of August 14^th – 19^th suggest that there were multiple hours when entities were in an Energy Emergency Alert while passing the EIM RSE, and it is encouraging that CAISO investigated these events and resolved issues related to the bid capacity test.  However, the presentation indicates that resolution of these issues would not have resulted in any additional RSE failures, so PGP believes that additional improvements are required to ensure that an entity’s EEA and RSE status are aligned.

The presentation also made clear that the RSE balancing and feasibility tests are not applied to CAISO, though there was little discussion as to why or whether or not that practice should continue. PGP believes that all components of the EIM Resource Sufficiency Evaluation (RSE) should be applied transparently and consistently to both CAISO and non-CAISO EIM Entities.

In addition, PGP agrees with Powerex’s general conclusion that the consequences for EIM RSE failure do not provide sufficient incentive for a market participant to resolve capacity shortfalls ahead of the operating hour.  PGP also believes that it is worth considering different consequences for failing the different RSE tests and that Powerex’s proposal is a good starting point for further conversations.  However, it is important that any solutions in this area are mindful of impacts to other initiatives (such as Scarcity Pricing).

For these reasons, PGP continues to recommend that CAISO specifically include EIM RSE Improvements in the scope of an existing initiative or create a new initiative.

[1] PGP represents eleven consumer-owned utilities in Washington and Oregon that own almost 8,000 MW of generation, approximately 7,000 MW of which is hydro and over 97% of which is carbon free. Four of the PGP members operate their own balancing authority areas (BAAs), while the remaining members have service territories within the Bonneville Power Administration’s (BPA) BAA. As a group, PGP members also purchase over 45 percent of BPA’s preference power.

2. Provide your organization’s feedback on the Bid Range Capacity Test for the RSE:

While it is encouraging that CAISO resolved two issues related to the bid capacity test, PGP believes that there are still underlying issues that need addressed.  In its presentation, Powerex stated that the Bid Range Capacity Test includes supply that is not physical and/or not capable of performing at the levels submitted by their bids.  While CAISO stated that the bid range capacity test was intentionally simplified, it is not clear if they acknowledged the issue that Powerex raised.  PGP would like CAISO to indicate whether or not they agree that the bid range capacity test counts resources that is not physical and/or not capable of performing at the levels submitted by their bids.  If there is agreement that this is the case, PGP believes that this test should be modified to only count supply that is physically available.

3. Provide your organization’s feedback on the Flexible Ramp Sufficiency Test (FRST) for the RSE:

CAISO stated that the purpose of the FRST is to “Assess whether there is sufficient ramping capability among all resources in the BAA to meet the forecasted demand change across intervals plus historical uncertainty.”  PGP notes that the historical uncertainty is determined with a histogram approach and that EIM Entities have requested to incorporate the use of forecasts in the determination of this uncertainty requirement.  PGP would like CAISO to indicate how the use of histogram methodology for determining the historical uncertainty requirement performed during August 14th-18th and whether or not the use of forecasts during this period would have resulted in a more accurate FRST requirement.

4. Provide your organization’s feedback on the Balancing Test for the RSE:

The presentation made clear that the RSE balancing and feasibility tests are not applied to CAISO, and CAISO indicated that the day-ahead RUC process results in no need to perform an EIM balancing test prior to the hour.  Does CAISO believe that there are no conditions where CAISO would not be balanced prior to the hour?  In general, PGP believes that all components of the EIM RSE should be applied transparently and consistently to both CAISO and non-CAISO EIM Entities.

5. Additional comments:

PGP believes this workshop is an example of a scenario where a governing body market expert would provide substantial value. While the presentations made by CAISO and Powerex were both helpful, PGP believes a governing body market expert would be uniquely situated to provide an objective assessment of how the RSE performed during the Summer 2020 heat wave and to frame and address outstanding issues and propose solutions.

1. Please provide your organization’s general feedback on the Resource Sufficiency Evaluation (RSE):

The Public Power Council (PPC) appreciates the information provided by CAISO staff and Powerex at the January 13, 2021 CAISO EIM Resource Sufficiency Evaluation Workshop and the opportunity to submit comments.  CAISO’s overview of the mechanics of the resource sufficiency evaluation and Powerex’s analysis of the performance of the evaluation during the August 2020 events provided helpful context to frame this topic.  The resource sufficiency evaluation is a key component of the EIM design meant to ensure each EIM entity can adequately balance their own supply and demand prior to participating in the EIM.  This test sets the ground rules and requirements for participation and is meant to prevent any EIM entity from leaning on other EIM entities’ capacity and flexibility. 

PPC is concerned the current resource sufficiency evaluation and failure consequences are not effective at preventing a EIM entity from leaning on other EIM entities.  The presentation given by Powerex showed that during the August 2020 heat wave the CAISO BAA was importing significant volumes through the EIM during periods in which it failed the resource sufficiency evaluation.  Additionally, the CAISO BAA was passing the resource sufficiency evaluation even during stage 2 energy emergencies.  The events highlighted in the presentation demonstrate that the resource sufficiency evaluation, as currently formulated, is allowing EIM entities to pass the evaluation despite clearly not having sufficient resources to balance their own supply and demand.  It also illustrates the current practice of freezing of imports is not a sufficient incentive to procure necessary capacity on a forward basis.  These failures of the resource sufficiency evaluation can have significant economic and reliability impacts on other EIM entities.  PPC believes modifications to the existing evaluation are necessary to ensure it meets the objective of preventing leaning.

PPC strongly supports a stakeholder process to address necessary modifications to the resource sufficiency evaluation.  Many of the suggestions made by Powerex are a good starting point for openly discussing improvements to the resource sufficiency evaluation.  PPC believes the right set of modifications can more accurately identify when a BAA is not sufficient and create stronger incentives to come into the test sufficient while maintaining the operational flexibility to use the EIM as a last resort during energy emergencies.  PPC looks forward to further discussion on specific modifications to the evaluation. 

The events of August 2020 and the materials from the workshop support the need for an outside market expert and additional transparency related to EIM entity resource sufficiency.  The workshop materials were a helpful start, but ongoing data availability is necessary for stakeholders to fully evaluate the performance of the sufficiency evaluation.  Despite the significant attention the events of August have received, very limited analysis on the performance of the EIM and impacts to EIM entities have been shared prior to this workshop.  While the EIM was not a root cause of the August load curtailments and therfore not a primary focus, CAISO staff, DMM and MSC analysis on the EIM’s functioning during this period has been very limited in publications subsequently released.  PPC believes an outside market expert, or “Governing Body Market Expert” as it is referred to in the GRC proposal could provide additional expertise from a neutral party.    

2. Provide your organization’s feedback on the Bid Range Capacity Test for the RSE:

3. Provide your organization’s feedback on the Flexible Ramp Sufficiency Test (FRST) for the RSE:

4. Provide your organization’s feedback on the Balancing Test for the RSE:

5. Additional comments:

1. Please provide your organization’s general feedback on the Resource Sufficiency Evaluation (RSE):

Please see attached comments.

2. Provide your organization’s feedback on the Bid Range Capacity Test for the RSE:

Please see attached comments.

3. Provide your organization’s feedback on the Flexible Ramp Sufficiency Test (FRST) for the RSE:

Please see attached comments.

4. Provide your organization’s feedback on the Balancing Test for the RSE:

Please see attached comments.

5. Additional comments:

1. Please provide your organization’s general feedback on the Resource Sufficiency Evaluation (RSE):

Shell Energy is concerned the RSE evaluation erroneously results in false pass screens, allowing EIM BAAs to lean on the EIM market for capacity needs.  CAISO’s continued passage of the RS screen while distressed during EEA1, EEA2 and EEA3 emergency conditions on 14 August 2020 is evidence that RS reforms are prudent as the generation resources evolve in energy markets. 

2. Provide your organization’s feedback on the Bid Range Capacity Test for the RSE:

Shell Energy is concerned that out-of-market operator intervention (e.g. load bias, exception dispatch) could affect the accuracy of the Bid Range Capacity test.  Any FMM load bias or manual dispatch of resources affecting the CAISO’s Net Scheduled Interchange could impact the screen.

3. Provide your organization’s feedback on the Flexible Ramp Sufficiency Test (FRST) for the RSE:

Shell Energy requests the CAISO provide additional data on how the flexible ramping product enhancements could affect the FRST.  Additionally, the CAISO has not adequately explained how the FRST diversity benefit is derived. 

Shell Energy has concerns the FRST overstates the CAISO’s diversity benefit under the screen resulting in false passes of the RSE screen.

4. Provide your organization’s feedback on the Balancing Test for the RSE:

Shell Energy does not understand why the CAISO is exempt from the Balancing Test as part of the RSE screen.  Additional discussion would be appreciated in future sessions.

5. Additional comments:

Shell Energy continues to propose the RS framework in CAISO is important—to signal which resources are needed on the grid to ensure reliability.  The structure of the EIM RS test has worked since 2014; however, the resource mix of the Western Interconnect is very different today. 

Limits to an EIM BAA’s ability to transfer energy following RS failure is prudent.  Shell Energy suggests, transfers be limited by the amount (expressed in MWs) of a RS capacity test failure or set transfers to 0 MW (in same direction as failure) for any hour an EIM BAA has failed a RS screen.  Continued transfers for the duration of any hours which the BAA passed is reasonable. 

Shell Energy supports the proposed solution by Powerex to disincentivize EIM “leaning” by deficient BAAs by creating a sufficient financial mechanism to encourage compliance.  More discussion is needed but any penal structure must discourage misappropriation of capacity attributes to failing EIM BAAs.

1. Please provide your organization’s general feedback on the Resource Sufficiency Evaluation (RSE):

The Six Cities support the general objective of the Resource Sufficiency Evaluation (“RSE”), which is to ensure that participants in the EIM are adequately resourced such that no EIM entity is engaging in inappropriate “leaning” on other balancing authorities, particularly during tight supply conditions such as those experienced on various dates this past summer.  As the Department of Market Monitoring (“DMM”) has identified, western balancing authorities should coordinate regarding the prioritization of transfers during tight supply conditions, and the RSE is a component of that necessary coordination effort. 

At the same time, as the DMM has also highlighted and the January 13^th stakeholder workshop in this initiative made clear, the design and application of the RSE is highly complex and reflects a balance between various competing priorities that stakeholders, including EIM entities, may weigh differently.  The Six Cities concur that implementation of rapidly-developed changes to the RSE through this initiative may disturb this careful balance and result in unintended consequences.  In particular, the CAISO should be cautious regarding changes that may inappropriately limit – or increase existing limitations on – EIM transfers in emergencies.  Apart from possible corrections to the RSE implementation that the CAISO has identified and incremental changes that may be needed to enhance accuracy, it may be advisable for the CAISO to proceed more slowly with respect to significant RSE design changes and to place consideration of such changes on a longer stakeholder trajectory to ensure that these complex issues may be fully vetted.  The dialogue between representatives for Powerex and the CAISO relating to the need for balance between administrative complexity and accuracy in RSE elements demonstrates that these issues are not susceptible to quick and easy solutions. 

The January 14^th comments provided by PG&E in this initiative highlight several of the areas where additional data and information may be needed to comprehensively evaluate any changes that may be appropriate.  The Six Cities agree that further investigation into the areas identified in the PG&E comments may be informative.  A longer stakeholder process to address more comprehensive changes to the RSE would better accommodate any further evaluations that may be needed. 

2. Provide your organization’s feedback on the Bid Range Capacity Test for the RSE:

Please see the comments provided above in response to Question No. 1; the Six Cities do not have comments regarding this element of the RSE at this time. 

3. Provide your organization’s feedback on the Flexible Ramp Sufficiency Test (FRST) for the RSE:

Please see the comments provided above in response to Question No. 1; the Six Cities do not have comments regarding this element of the RSE at this time. 

4. Provide your organization’s feedback on the Balancing Test for the RSE:

Please see the comments provided above in response to Question No. 1; the Six Cities do not have comments regarding this element of the RSE at this time. 

5. Additional comments:

The Six Cities have no additional comments at this time. 

1. Please provide your organization’s general feedback on the Resource Sufficiency Evaluation (RSE):

Please see the attachment.

2. Provide your organization’s feedback on the Bid Range Capacity Test for the RSE:

Please see the attachment.

3. Provide your organization’s feedback on the Flexible Ramp Sufficiency Test (FRST) for the RSE:

Please see the attachment.

4. Provide your organization’s feedback on the Balancing Test for the RSE:

Please see the attachment.

5. Additional comments:

Please see the attachment.

1. Please provide your organization’s general feedback on the Resource Sufficiency Evaluation (RSE):

Vistra appreciates the CAISO taking the time to facilitate a workshop discussing the Resource Sufficiency Evaluation (“RSE”). We support the CAISO pursuing as soon as practical enhancements to the RSE to improve the accuracy of the tests as described in more detail below. On the policy question of whether policy changes are needed to improve EIM RSE by addressing “EIM transfers during stressed system conditions”, we believe more discussion is warranted before pursuing any proposals on this element. For the issues around applying “financial consequences” for failed tests, we respectfully view this as an item that does not directly impact reliability and should be considered outside of this effort.

While we are still in the process of understanding the various tools available to EIM Entity BAAs to ensure reliable operations, it is our understanding that EIM entities have access to some tools that could address the concern with “transfers during stressed system conditions” under today’s rules. For example, the processes outlined in section 11.5 of the EIM Business Practice Manual, Separation of the EIM entity, outline a path that can limit transfers out of their BAA however it would also not allow CAISO to commit or dispatch resources or allow import transfers either. On the other hand, an entity could remain in EIM footprint but use the maximum EIM transfer limit that the EIM entity BAA provides to the CAISO to limit transfers. It’s our understanding that these are initially provided and can be updated in real-time under certain conditions to limit the transfers. We would like to better understand whether Powerex’s concern is arising from limitations on the use of these existing tools. For example, is an update to the max EIM transfer limit not allowed in real-time due to stressed conditions internal to an EIM entity BAA such as nearing load loss since it is not a transmission issue. If that is the case, perhaps allowing access to max EIM transfer limit when emergency conditions arise could be appropriate for Summer 2021.

On the “financial consequences” proposal, we are concerned that Powerex’s proposal to have a zero-tolerance policy for shortages triggering a transfer limit at 0 MW that could be relaxed at $2,000/MWh is not consistent policy thinking with that adopted in Order 831 import bidding and market parameters. In that effort, the CAISO adopted a shortage threshold to differentiate its use of penalty prices between $1,000/MWh and $2,000/MWh. We believe there are complex and potentially thorny policy issues being raised that would merit a robust stakeholder process not feasible under this effort. Vistra requests the CAISO not pursue changes to the EIM rules for Summer 2021 to reflect scarcity when EIM entity areas fail their RSE but instead consider scoping it into the long-term scarcity pricing enhancement effort.

To reiterate, Vistra supports ensuring operational tools can address this type of reliability concern and encourages the CAISO to prioritize for Summer 2021 either improvements or greater flexibility to its existing tools. We believe that consideration of scarcity measures would warrant more in-depth review that is not feasible under the Summer 2021 timeline.

2. Provide your organization’s feedback on the Bid Range Capacity Test for the RSE:

Vistra supports the CAISO moving forward with practical improvements to correct the limitations in its Bid Range Capacity test calculations identified in slide 16 of the CAISO’s presentation. The CAISO identified at least two limitations in the current implementation where it is not including resource’s Pmax de-rates or Pmin re-rates in the capacity calculation and where the Net Schedule Interchange values are inaccurate. Enhancing the implementation of the RSE to correct for these limitations seems prudent and practical. Vistra encourages the CAISO to continue to look into the details of its EIM tests and identify limitations that are implementation in nature and correct as soon as practical. 

3. Provide your organization’s feedback on the Flexible Ramp Sufficiency Test (FRST) for the RSE:

4. Provide your organization’s feedback on the Balancing Test for the RSE:

None at this time.

5. Additional comments:

None at this time.

1. Please provide your organization’s general feedback on the Resource Sufficiency Evaluation (RSE):

2. Provide your organization’s feedback on the Bid Range Capacity Test for the RSE:

3. Provide your organization’s feedback on the Flexible Ramp Sufficiency Test (FRST) for the RSE:

4. Provide your organization’s feedback on the Balancing Test for the RSE:

5. Additional comments: